May 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy tool shed
by Xu, Jianpeng 18 May '12

18 May '12

Hi, I have installed the local galaxy. Right now I am interested in incorporating some new tools into our local galaxy. I am interested in some tools in the galaxy tool shed. I noticed there are some wrappers for these tools. Normally we need to make some configurations to incorporate new tools into the local galaxy. My question is that: Is there an easier way to incorporate these new tools to the local galaxy ? What is the easiest way to incorporate new tools ? Thanks a lot. ________________________________ This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments).

2 1

Using a variable in <data from_work_dir="">
by Peter van Heusden 18 May '12

18 May '12

Hi there I'm trying to wrap Bioperl's bp_genbank2gff script. This script, by default, produces an output file named the same as the basename of the input file, with .gff added. So e.g. bp_genbank2gff data.gbk produces a file named data.gbk.gff. I'm trying to use the from_work_dir attribute of the <data> tag to pick up this file. However, if I create a tool XML like: <tool id="Genbank2GFF3" name="Genbank to GFF3" version="1.0"> <description>Convert Genbank format to GFF3</description> <command interpreter="python"> bp_genbank2gff $genbank_filename </command> <inputs> <param name="genbank_filename" type="data" format="txt"> <label>Genbank sequence record(s)</label> </param> </inputs> <outputs> <data format="gff3" name="gff3_filename" from_work_dir="${genbank_filename}.gff"> </data> </outputs> <help> </help> </tool> I get an error from Galaxy indicating that it is trying to find a file name $genbank_filename.gff, I.e. the ${genbank_filename} is not expanded. Is there any way to deal with this situation? Of course the other option is to create a tiny wrapper around bp_genbank2gff that takes an output filename argument and does the renaming... Peter

3 2

Moving Data to New Instance
by Robert Chase 17 May '12

17 May '12

Hello All, We have a large amount of data on our production server that we would like to make available on our new test server. I tried to move data from our production server to our test server by copying the meta-data files and dataset files from ~/database/files to the new test server but it didn't work. Is there a way to move the datasets en mass? -Rob

2 2

Preview of large files seems not to finish
by Anthonius deBoer 16 May '12

16 May '12

Hi, I noticed in the latest installment of Galaxy-Central that there is a pretty view option for VCF files, in that it tries to show the data in nice columns. But for my VCF files it does not seem to finish loading the first MB of the file..It also no longer shows the SHOW ALL at the top, so now I have to load the file out of my browser, which is a pain... Is this just a bug or a feature? Thanks, Thon

1 0

Idea for user-based dataset subdirectories
by Josh Nielsen 16 May '12

16 May '12

Hello, Please forgive the length of this proposition as I try to explain my reasoning behind this. Let me say first of all that I understand that Galaxy is not meant to be everything to everyone and that requests for features may not suit everyone who uses Galaxy. That being said I have an idea or request that I think would be convenient for dealing with user's datasets from a file-system perspective. Galaxy has the obvious benefit and advantage (compared to manual job-submission for tools on a cluster) of providing an interface for using all the analysis tools, and the history of the operations done on your data, all in one place. However I have found that putting all the output & datasets in one directory (the files/000/ directory) on the file-system causes a problem for the users if they specifically want to interact with it *on the file-system*, and not just through the Web interface - for whatever complicated or diverse reasons. Since Galaxy runs on a cluster of its own in our environment, and we do not allow users to remote connect into it to submit manual jobs (and individually output it to their separate home directories) like we do our main cluster, it is essentially a black box beyond the GUI interface of Galaxy. That is essentially what we want except for how they can interact with the output files. The issue is that our users would like an easy means of copying their files off of the Galaxy cluster to other servers from a command line (possibly even automated by scripts). Even if we allow an FTP share of the output directory for users to do that, the common [galaxy-dist]/database/files/000/ directory clumps all of the files for all users together in one directory and uses a sequential file-naming scheme (dataset_N++) that is not easy to discriminate between as to who the owner is for each file. Is there a way that the dataset output directory locations could be designed (or set optionally?) like the FTP upload feature's expected directory structure: where the files are dropped into the corresponding subdirectory of the user who produced it? For example having under database/files/ subdirectories named according to the user's Galaxy account id (like [galaxy-dist]/database/files/jsmith, [galaxy-dist]/database/files/sparker, etc.). If they could be segregated by user it would be much easier to keep track of what datasets belong to whom on the file-system. Then I could possibly set up a read-only FTP share to the files/ directory on the cluster, from which the users could directly copy the files in their personal subdirectory to other systems, and perhaps batch download them, rather than having to rely solely on the Web interface. I understand that the way Galaxy is currently designed is that the files are just generically named (the "behind-the-scenes" handling of data is a black box) and it is the database that keeps track of which files belong to whom, and which has the metadata for more meaningful dataset/job names, etc. But a file-system hierarchy alternative would also be welcome in a heavily command-line oriented computational environment too. Would setting up a more user-representative output directory hierarchy on the file-system like that be possible? Best Regards, Josh Nielsen

3 5

⇝ IOError: [Errno 13] Permission denied: u'/home/koala2/galaxy-central/database/compiled_templates/base_panels.mako.py'
by Luciano Cosme 16 May '12

16 May '12

Hi Everyone, I was just using my local galaxy instance. I closed it up (from terminal), open IGV to look at some genes, then I restarted it from the terminal and when I tried to access it I got the error message below. I just don't know what happened in these few minutes. Any suggestion how to fix it? Thank you. Luciano URL: http://127.0.0.1:8080/ File '/home/koala2/galaxy-central/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/koala2/galaxy-central/lib/galaxy/web/controllers/root.py', line 25 in index m_c=m_c, m_a=m_a ) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/__init__.py', line 733 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/__init__.py', line 744 in fill_template_mako return template.render( **data ) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 691 in _render_context (inherit, lclcontext) = _populate_self_namespace(context, tmpl) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 637 in _populate_self_namespace ret = template.module._mako_inherit(template, context) File '/home/koala2/galaxy-central/database/compiled_templates/root/ index.mako.py', line 25 in _mako_inherit return runtime._inherit_from(context, u'/webapps/galaxy/base_panels.mako', _template_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 609 in _inherit_from ret = callable_(template, lclcontext) File '/home/koala2/galaxy-central/database/compiled_templates/webapps/galaxy/ base_panels.mako.py', line 25 in _mako_inherit return runtime._inherit_from(context, u'/base_panels.mako', _template_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 596 in _inherit_from template = _lookup_template(context, uri, calling_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 626 in _lookup_template return lookup.get_template(uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/lookup.py', line 218 in get_template return self._load(srcfile, uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/lookup.py', line 283 in _load **self.template_args) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 218 in __init__ module = self._compile_from_file(path, filename) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 245 in _compile_from_file module = imp.load_source(self.module_id, path, open(path, 'rb')) IOError: [Errno 13] Permission denied: u'/home/koala2/galaxy-central/database/compiled_templates/ base_panels.mako.py'

3 3

Latest Galaxy build gives me error on WSF extension not found
by Anthonius deBoer 16 May '12

16 May '12

Hi, In the latest Galaxy Central rev I get this error in each tool I run... Any ideas what is going on? Thanks, Thon WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'wsf': 'module' object has no attribute 'wsf'

2 1

Error loading data
by SHAUN WEBB 16 May '12

16 May '12

Hi, I have just upgrade my test version of Galaxy to the latest dist release and I am getting an error trying to load data either as an upload, a pasted entry or from UCSC. I have only altered conf files, any pointers to what could be causing this would be appreciated. The error is below: Traceback (most recent call last): File "/storage/home/galaxy/test/pre-dev/galaxy-dist/tools/data_source/upload.py", line 403, in <module> __main__() File "/storage/home/galaxy/test/pre-dev/galaxy-dist/tools/data_source/upload.py", line 378, in __main__ registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] ) AttributeError: 'Registry' object has no attribute 'load_datatypes' Thanks Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

managing external dependency eggs
by Shantanu Pavgi 16 May '12

16 May '12

Hi, At the risk of exposing of my Python ignorance, here is a doubt regarding galaxy deployment. The galaxy application manages external dependency eggs through it's custom ini specification file and eggs installation scripts. I was wondering if this could be managed through pip using a requirements.txt file. Are there any limitation of the later (pip) approach? Appreciate your comments. -- Thanks, Shantanu

2 1

Serious problems with the new Galaxy update (12 may) / bugs??
by Nikolai Vazov 16 May '12

16 May '12

Hi, I updated Galaxy following the instructions in http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_05_11 Everything went well : the update, the shed_tool (left it unchanged), database migration, reboot as --daemon Yet, when I write the url in the address line, I get : the three empty panels and the text in the middle one: ======= Not Found The resource could not be found. No route for /user/login ======= Already at boot I had an error message of type cannot import validate_username from galaxy.security.validate_user_input where actually this function validate_username is missing. I replaced the line 13 in lib/galaxy/web/controllers/user.py from from galaxy.security.validate_user_input import validate_email, validate_username, validate_password to from galaxy.security.validate_user_input import validate_email, validate_publicname, validate_password and Galaxy started. But then I only got empty panels and the error above Do you have any hints? I am sending you the paster.log at boot and a screenshot Thank you in advance Nikolay -- Nikolay Vazov, PhD Research Computing Centre - http://hpc.uio.no USIT, University of Oslo

1 0