June 2014 - galaxy-dev - lists.galaxyproject.org

Toolshed update list?
by Eric Rasche 10 Jun '14

10 Jun '14

Is there any way to access a list of repository updates in a toolshed, perhaps as an RSS feed? Alternatively, is this data available internally and it just requires writing a route to handle that query? As of now when I log into my toolshed I'm faced with a page of meaningless hashes and version numbers which I cannot mentally track for updates across multiple days/projects. Cheers, Eric -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 <tel:4046922048> esr(a)tamu.edu <mailto:esr@tamu.edu>

2 1

Tracking tool execution time?
by Jim McCusker 10 Jun '14

10 Jun '14

Is there a way to look at how long a galaxy tool takes to run? Jim

2 1

ClustalW jobs aren't working
by Malcolm Tobias 10 Jun '14

10 Jun '14

All, When running clustalW jobs, the only output we're getting is an error message that appears to show the tool isn't receiving the proper run-time parameters: Unknown option: I Value "FILE=infile_copy" invalid for option N (number expected) Unknown option: F Unknown option: I Unknown option: L Unknown option: E ... This appears to come from the rgClustalw.py wrapper: cl = ['clustalw2 -INFILE=%s -OUTFILE=%s -OUTORDER=%s -TYPE=%s -OUTPUT=%s ' % (self.iname,self.opts.output,self.opts.out_order,self.opts.dnarna,self.opts. outform)] Strangely, the log files appear to show the correct paramters: galaxy.tools WARNING 2014-06-05 10:37:43,484 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.local DEBUG 2014-06-05 10:37:43,596 (779) executing: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/001/dataset_1243.dat" -o "/home/galaxy/galaxy-dist/database/files/001/dataset_1244.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/001/dataset_1245.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" I'll post the full output from the logs below, but I was hoping someone might be able to offer some suggestions on how to debug this further. Thanks! Malcolm 10.28.56.101 - - [05/Jun/2014:10:37:42 -0500] "POST /tool_runner/index HTTP/1.1" 200 - "http://galaxy.chpc.wustl.edu/tool_runner?tool_id=clustalw" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0" galaxy.jobs DEBUG 2014-06-05 10:37:43,025 (779) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/779 galaxy.jobs.handler DEBUG 2014-06-05 10:37:43,057 (779) Dispatching to local runner galaxy.jobs DEBUG 2014-06-05 10:37:43,205 (779) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2014-06-05 10:37:43,258 (779) Job dispatched 10.28.56.101 - - [05/Jun/2014:10:37:42 -0500] "GET /history HTTP/1.1" 200 - "http://galaxy.chpc.wustl.edu/tool_runner/index" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0" galaxy.tools DEBUG 2014-06-05 10:37:43,466 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2014-06-05 10:37:43,484 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.local DEBUG 2014-06-05 10:37:43,596 (779) executing: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/001/dataset_1243.dat" -o "/home/galaxy/galaxy-dist/database/files/001/dataset_1244.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/001/dataset_1245.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" 10.28.56.101 - - [05/Jun/2014:10:37:43 -0500] "GET /api/histories/1ac3b66e11b8fe50 HTTP/1.1" 200 - "http://galaxy.chpc.wustl.edu/history" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0" galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,732 Exception caught attempting to generate viewport for dataset '972' Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py", line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_769.dat' galaxy.jobs DEBUG 2014-06-05 10:37:43,736 (779) Persisting job destination (destination id: local) galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,776 Exception caught attempting to generate viewport for dataset '985' Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py", line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_782.dat' galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,830 Exception caught attempting to generate viewport for dataset '986' Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py", line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_783.dat' galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,876 Exception caught attempting to generate viewport for dataset '987' Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py", line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_784.dat' 10.28.56.101 - - [05/Jun/2014:10:37:43 -0500] "GET /history/get_display_application_links HTTP/1.1" 200 - "http://galaxy.chpc.wustl.edu/history" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0" galaxy.jobs.runners.local DEBUG 2014-06-05 10:37:45,983 execution finished: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/001/dataset_1243.dat" -o "/home/galaxy/galaxy-dist/database/files/001/dataset_1244.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/001/dataset_1245.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" galaxy.jobs.runners DEBUG 2014-06-05 10:37:46,609 executing external set_meta script for job 779: /home/galaxy/galaxy-dist/set_metadata.sh ./database/files /home/galaxy/galaxy-dist/database/job_working_directory/000/779 . /home/galaxy/galaxy-dist/universe_wsgi.ini /home/galaxy/galaxy-dist/database/tmp/tmpsnNodx /home/galaxy/galaxy-dist/database/job_working_directory/000/779/galaxy.json /home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_in_HistoryDatasetAssociation_1448_zaFZp3,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_kwds_HistoryDatasetAssociation_1448_Rltsef,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_out_HistoryDatasetAssociation_1448_12s67j,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_results_HistoryDatasetAssociation_1448_Qucqzx,,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_override_HistoryDatasetAssociation_1448_95Vr9q /home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_in_HistoryDatasetAssociation_1447_Zh2tCn,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_kwds_HistoryDatasetAssociation_1447_UeH8AF,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_out_HistoryDatasetAssociation_1447_cZxp2H,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_results_HistoryDatasetAssociation_1447_ck1M8o,,/home/galaxy/galaxy-dist/database/job_working_directory/000/779/metadata_override_HistoryDatasetAssociation_1447_pefoLl -- Malcolm Tobias 314.362.1594

4 8

installing htseq-count from tool-shed
by Karen Chait 10 Jun '14

10 Jun '14

Hi, I am trying to install the tool htseq-count from the tool-shed. I had a problem in the first installation so I uninstalled the tool and all its dependencies (numpy, samtools 0.1.19 and pysam) from the 'manage installed tool shed repositories) and installed it again but then I get installation status 'Installed, missing tool dependencies' for htseq-count. When running this tool I get the error Fatal error: The HTSeq python package is not properly installed, contact Galaxy administrators The path set in universe_wsgi.ini: tool_dependency_dir = /home/galaxy/galaxy-dist/tool_deps in the installation log htseq/0.6.1/lparsons/htseq_count/d5edaf8dc974/INSTALLATION.log I get the error: Setup script for HTSeq: Failed to import 'numpy'. Please install numpy and then try again to install HTSeq. I reinstalled numpy but that did not help. How should I continue? Thanks, Karen

3 2

Galaxy updated botched?
by Kandalaft, Iyad 08 Jun '14

08 Jun '14

Hi everyone I was looking to upgrade galaxy to the latest verison. I did the usual hg pull; hg update;. I start up galaxy, it fetches some eggs, and then it tells me that I need a DB schema update. Fine, I ran sh db_manage.sh upgrade. All I see is a list of stack traces similar to the following. I'm using mysql and all the tables that galaxy had created are running under the myisam engine.... Hence, no foreign key constraints allowed. Why did this happen and can I simply convert all my tables to InnoDB to complete the database upgrade? Should I restore the DB from a backup? IntegrityError: (IntegrityError) (1215, 'Cannot add foreign key constraint') '\nCREATE TABLE history_dataset_collection_rating_association (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\thistory_dataset_collection_id INTEGER, \n\tuser_id INTEGER, \n\trating INTEGER, \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(history_dataset_collection_id) REFERENCES history_dataset_collection_association (id), \n\tFOREIGN KEY(user_id) REFERENCES galaxy_user (id)\n)\n\n' () (IntegrityError) (1215, 'Cannot add foreign key constraint') '\nCREATE TABLE history_dataset_collection_tag_association (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\thistory_dataset_collection_id INTEGER, \n\ttag_id INTEGER, \n\tuser_id INTEGER, \n\tuser_tname VARCHAR(255), \n\tvalue VARCHAR(255), \n\tuser_value VARCHAR(255), \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(history_dataset_collection_id) REFERENCES history_dataset_collection_association (id), \n\tFOREIGN KEY(tag_id) REFERENCES tag (id), \n\tFOREIGN KEY(user_id) REFERENCES galaxy_user (id)\n)\n\n' () Creating history_dataset_collection_tag_association table failed: (IntegrityError) (1215, 'Cannot add foreign key constraint') '\nCREATE TABLE history_dataset_collection_tag_association (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\thistory_dataset_collection_id INTEGER, \n\ttag_id INTEGER, \n\tuser_id INTEGER, \n\tuser_tname VARCHAR(255), \n\tvalue VARCHAR(255), \n\tuser_value VARCHAR(255), \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(history_dataset_collection_id) REFERENCES history_dataset_collection_association (id), \n\tFOREIGN KEY(tag_id) REFERENCES tag (id), \n\tFOREIGN KEY(user_id) REFERENCES galaxy_user (id)\n)\n\n' () Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0120_dataset_collections.py", line 179, in __create table.create() File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 593, in create checkfirst=checkfirst) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/ddl.py", line 86, in visit_table self.connection.execute(schema.CreateTable(table)) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1542, in _execute_ddl compiled File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 173, in execute self.errorhandler(self, exc, value) File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue Regards, Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228

1 0

"Resume Paused Jobs"
by Björn Grüning 07 Jun '14

07 Jun '14

Hi, can anyone confirm that "Resume Paused Jobs" on crashed tools is broken in the latest Galaxy release? Thanks, Bjoern

2 2

Docker based tools
by Kyle Ellrott 07 Jun '14

07 Jun '14

I know a few people have talked about using docker to package and distribute Galaxy tools. The most recent version now runs on all major linux distributions, so it's a bit more inclusive now (but there probably won't ever be a windows or mac version). One strategy would be for tool configs to define an option Docker archive name and possibly an alternate command line to execute if the docker archive is being used. How far has that work gotten, and what are people thinking about it? Kyle

3 2

Are there examples of tools that run processes in the background?
by Melissa Cline 06 Jun '14

06 Jun '14

Hi folks, We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications). Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time. Or is there a better approach I didn't consider? Assuming the answer is "no", then are there any tools in the toolshed that launch background processes? If so, I'd like to look at them. And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down. Is there a way to configure a process to be run when it's time to quit Galaxy? Thanks! Melissa

2 1

to put or not to put a tool suite into the toolshed
by Wolfgang Maier 06 Jun '14

06 Jun '14

Dear all, I am maintaining a (still young) suite of command line tools (written in Python) for identification of mutations in model organism genomes through whole-genome sequencing (https://sourceforge.net/projects/mimodd/) MiModD aims at geneticists that do not have much background in bioinformatics, and it's supposed to make WGS analysis for small model organism genomes (anything from yeast to fish) possible on regular PCs, so it's not a cloud/cluster solution. We do support Galaxy though through a complete set of tool wrappers for three reasons: - to provide a graphical user interface - to offer labs the possibility to install the software on one dedicated machine, but run analyses from any machine (typically with Windows installed) - to keep analysis workflows documented and reproducible. Currently, we advise users to take advantage of these features and install a local instance of Galaxy even though it will be running then only on a single desktop PC (or even just a notebook). After installation of Galaxy our software simply copies its wrapper xmls over to the tools folder and modifies the tool_conf file to integrate itself. In addition, users have to install a bit of other third-party software (samtools, snap aligner, optionally snpeff) that our code relies on. [End of lengthy introduction] So my question is: in what way could the package profit from being uploaded to a Galaxy toolshed ? I guess it would mean quite some extra work from my side since I'm not familiar with the whole procedure, so are there benefits (visibility, ease of installation, etc.) that are worth the effort ? Thanks a real lot for any feedback, Wolfgang

3 3

ToolShed for Visualisation Registry?
by Saket Choudhary 06 Jun '14

06 Jun '14

I was wondering if there is a possibility of a 'Tool'Shed for Visualisation registry plugins(custom plugins, developed by users/developers)? The motivation for this comes from a small 'tool' I wrote for visualiing fastq phred scores using box plots[Still under development] here: http://saketkc.github.io/biojs/src/test/javascript/TestPhredScoreViewer.html I am relying on d3.text to read my fastq, but I believe I could handle this in a much better way had I written it for Galaxy's Visualisation registry specifically. For e.g. I could fetch fastq scores line by line using Galaxy's API, which would make my task a bit easier. I could always keep this to my local instance of Galaxy, but a 'toolshed' or 'registry' for visualisation would help. Saket

2 1