February 2013 - galaxy-dev - lists.galaxyproject.org

Datatype display_in_upload not respected in upload1
by Nicola Soranzo 22 Feb '13

22 Feb '13

Hi all, I noticed that when I go in "Get Data" -> "Upload File from your computer" (i.e. upload1 tool) the "File Format" list contains also datatypes which have display_in_upload="false" in datatypes_conf.xml (e.g. "afg", "fli"), while the list does not contain datatypes which have no display_in_upload attribute at all (e.g. "bedstrict", "bed6"). This sounds like a bug to me, but I have not found any reference around. Thanks, Nicola -- Nicola Soranzo, Ph.D. CRS4 Bioinformatics Program Loc. Piscina Manna 09010 Pula (CA), Italy http://www.bioinformatica.crs4.it/

3 3

Inter-ToolShed repository dependencies
by Peter Cock 22 Feb '13

22 Feb '13

Hi all, I've just hit what I consider to be a bug, but may be a design choice which I don't yet understand: It appears that repository dependencies cannot refer to other Tool Sheds. As an example, I want to explore some more complicated dependency work in my Blast2GO wrapper (I'd like it to be able to download the underlying Java tool automatically). I therefore started by uploading the previous releases to the Test Tool Shed, http://toolshed.g2.bx.psu.edu/view/peterjc/blast2go --> http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go --> The Blast2GO wrapper requires the 'blastxml' format, which is declared via a repository_dependencies.xml file as follows: <?xml version="1.0"?> <repositories description="This requires the BLAST datatype definitions (e.g. the BLAST XML format).">  <repository toolshed="http://toolshed.g2.bx.psu.edu" name="blast_datatypes" owner="devteam" changeset_revision="f9a7783ed7b6" /> </repositories> Unfortunately, uploading versions of the Blast2GO wrapper with this definition (which points at the main tool shed) to the test tool shed results in this error message: repository_dependencies.xml - Repository dependencies are currently supported only within the same tool shed. Ignoring repository dependency definition for tool shed http://toolshed.g2.bx.psu.edu, name blast_datatypes, owner devteam, changeset revision f9a7783ed7b6. Given the idea of an ecosystem of Tool Sheds, with some groups maintaining their own local Tool Shed and perhaps publishing tools there, it seems likely we will need to be able to define dependencies on other tool sheds - and in particularly the main tool shed at Penn State which I see having a hub-like role. But right now, that doesn't work. Is this a short term limitation (since cross-Tool Shed dependencies are bound to be more complex to support), or a deliberate policy? Thanks, Peter

2 1

Attention users of the Blast2GO wrapper
by Peter Cock 22 Feb '13

22 Feb '13

Dear all, For anyone using my Blast2GO for pipelines wrapper: http://toolshed.g2.bx.psu.edu/view/peterjc/blast2go The newly uploaded v0.0.6 of the wrapper changes from supporting b2g4pipe v2.3.5 to supporting v2.5 instead. This is currently available here (while the old v2.3.5 release does not appear to be online any more): http://www.blast2go.com/data/blast2go/b2g4pipe_v2.5.zip Previously by default the wrapper looked for b2g4pipe v2.3.5 under location /opt/b2g4pipe/blast2go.jar It will now by default expect b24pipe v2.5 under location /opt/b2g4pipe_v2.5/blast2go.jar instead (changed so you can have both on disk at the same time during migration). Please also note that the blast2go.loc file will need to be updated to point at the updated property files. The Blast2GO property files define both the database location (e.g. the public database in Spain or your own local database) and other configurable options which have changed from v2.3.5 to v2.5. Please read the blast2go.loc.sample for details. If you need any guidance on this, I will try to advise by email, and make clarifications to the documentation if required. Regards, Peter

1 0

Questions about run_functional_test.sh
by Kahlke Tim 22 Feb '13

22 Feb '13

Hi Galaxy-developers, I'm new to galaxy and am trying to set up a full production server. First comment: Nice work, looks great, well documented!! I have a question about run_functional_test: am I right that I can run this before installing any tools/dependencies? And how long is it supposed to run? I started it 14h ago and it's still running ... Best Regards, Tim

2 2

Submitting jobs as a real user without using chown, please
by Thon de Boer 22 Feb '13

22 Feb '13

Hi, I am trying to setup my galaxy system to allow jobs to be submitted as the real user, since people want to keep an eye on their job on the cluster sometimes and they have no ideas which ones are theirs. I tried the approach on the wiki here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster?highlight=%28 submit%29%7C%28jobs%29%7C%28as%29%7C%28user%29#Submitting_Jobs_as_the_Real_U ser but unfortunately, the CHOWN command is not allowed, not even as a sudo user. Probably has to do with the fact that we run our cluster from an isilon system, which I assume is pretty typical. The job was actually successfully submitted as the intended user, so that part works, but if we can just get it to work without having to rely on chown that would be awesome. Can someone point me in the right direction? Here's the error. galaxy.jobs.runners.local DEBUG 2013-02-19 19:35:31,524 execution of external set_meta for job 148 finished galaxy.jobs DEBUG 2013-02-19 19:35:31,576 (148) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /mnt/ngs/analysis/svcgalaxy/galaxy-test/database/job_working_directory/000/1 48 svcgalaxy 10000 galaxy.jobs ERROR 2013-02-19 19:35:31,653 (148) Failed to change ownership of /mnt/ngs/analysis/svcgalaxy/galaxy-test/database/job_working_directory/000/1 48, failing Traceback (most recent call last): File "/mnt/ngs/analysis/svcgalaxy/galaxy-test/lib/galaxy/jobs/__init__.py", line 343, in finish self.reclaim_ownership() File "/mnt/ngs/analysis/svcgalaxy/galaxy-test/lib/galaxy/jobs/__init__.py", line 916, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File "/mnt/ngs/analysis/svcgalaxy/galaxy-test/lib/galaxy/jobs/__init__.py", line 902, in _change_ownership assert p.returncode == 0 AssertionError galaxy.jobs DEBUG 2013-02-19 19:35:31,722 fail(): Moved /mnt/ngs/analysis/svcgalaxy/galaxy-test/database/job_working_directory/000/1 48/galaxy_dataset_332.dat to /mnt/ngs/analysis/svcgalaxy/galaxy-test/database/files/000/dataset_332.dat galaxy.datatypes.metadata DEBUG 2013-02-19 19:35:31,924 Cleaning up external metadata files Thanks Thon

3 6

History not updating for multiple instances of Galaxy
by Neil.Burdett＠csiro.au 21 Feb '13

21 Feb '13

Hi, I have three instances of galaxy running on the same Linux box. My "/etc/apache2/sites-available/default" is: RewriteEngine on RewriteRule ^/galaxy-csiro$ /galaxy-csiro/ [R] RewriteRule ^/galaxy-csiro/static/style/(.*) /home/galaxy/galaxy-csiro/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy-csiro/static/scripts/(.*) /home/galaxy/galaxy-csiro/static/scripts/packed/$1 [L] RewriteRule ^/galaxy-csiro/static/(.*) /home/galaxy/galaxy-csiro/static/$1 [L] RewriteRule ^/galaxy-csiro/favicon.ico /home/galaxy/galaxy-csiro/static/favicon.ico [L] RewriteRule ^/galaxy-csiro/robots.txt /home/galaxy/galaxy-csiro/static/robots.txt [L] RewriteRule ^/galaxy-csiro(.*) http://localhost:9080$1 [P] RewriteRule ^/galaxy-cte$ /galaxy-cte/ [R] RewriteRule ^/galaxy-cte/static/style/(.*) /home/galaxy/galaxy-cte/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy-cte/static/scripts/(.*) /home/galaxy/galaxy-cte/static/scripts/packed/$1 [L] RewriteRule ^/galaxy-cte/static/(.*) /home/galaxy/galaxy-cte/static/$1 [L] RewriteRule ^/galaxy-cte/favicon.ico /home/galaxy/galaxy-cte/static/favicon.ico [L] RewriteRule ^/galaxy-cte/robots.txt /home/galaxy/galaxy-cte/static/robots.txt [L] RewriteRule ^/galaxy-cte(.*) http://localhost:9000$1 [P] RewriteRule ^/galaxy-Newcastle$ /galaxy-Newcastle/ [R] RewriteRule ^/galaxy-Newcastle/static/style/(.*) /home/galaxy/galaxy-Newcastle/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy-Newcastle/static/scripts/(.*) /home/galaxy/galaxy-Newcastle/static/scripts/packed/$1 [L] RewriteRule ^/galaxy-Newcastle/static/(.*) /home/galaxy/galaxy-Newcastle/static/$1 [L] RewriteRule ^/galaxy-Newcastle/favicon.ico /home/galaxy/galaxy-Newcastle/static/favicon.ico [L] RewriteRule ^/galaxy-Newcastle/robots.txt /home/galaxy/galaxy-Newcastle/static/robots.txt [L] RewriteRule ^/galaxy-Newcastle(.*) http://localhost:9010$1 [P] I can view all the instances However, when I try and "Get Data" and upload a file the history side panel remains empty. I can see that the files have been uploaded to (for example) % ls -lrt galaxy-Newcastle/database/files/000/ total 22868 -rw-rw-r-- 1 galaxy galaxy 23017819 Feb 21 13:01 dataset_1.dat -rw-rw-r-- 1 galaxy galaxy 187339 Feb 21 13:02 dataset_2.dat -rw-rw-r-- 1 galaxy galaxy 206568 Feb 21 13:03 dataset_3.dat Any ideas much appreciated... The log file state: Debug at: http://barium-rbh/galaxy-Newcastle/_debug/view/1361342321 127.0.0.1 - - [21/Feb/2013:13:16:21 +1000] "GET /galaxy-Newcastle/tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://barium-rbh/galaxy-Newcastle/root/tool_menu" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" galaxy.web.framework DEBUG 2013-02-21 13:16:28,630 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,630 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,631 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,631 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,631 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,631 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-02-21 13:16:28,631 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_create 127.0.0.1 - - [21/Feb/2013:13:16:28 +1000] "POST /galaxy-Newcastle/tool_runner/upload_async_create HTTP/1.1" 500 - "http://barium-rbh/galaxy-Newcastle/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" Debug at: http://barium-rbh/galaxy-Newcastle/_debug/view/1361342322 127.0.0.1 - - [21/Feb/2013:13:16:28 +1000] "GET /galaxy-Newcastle/tool_runner/upload_async_message HTTP/1.1" 200 - "http://barium-rbh/galaxy-Newcastle/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" 127.0.0.1 - - [21/Feb/2013:13:16:28 +1000] "GET /galaxy-Newcastle/history HTTP/1.1" 200 - "http://barium-rbh/galaxy-Newcastle/tool_runner/upload_async_message" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" galaxy.web.framework DEBUG 2013-02-21 13:16:29,261 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/root/user_get_usage 127.0.0.1 - - [21/Feb/2013:13:16:29 +1000] "POST /galaxy-Newcastle/root/user_get_usage HTTP/1.1" 500 - "http://barium-rbh/galaxy-Newcastle/history" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" Debug at: http://barium-rbh/galaxy-Newcastle/_debug/view/1361342323 galaxy.tools.actions.upload_common INFO 2013-02-21 13:16:30,841 tool upload1 created job id 4 127.0.0.1 - - [21/Feb/2013:13:16:28 +1000] "POST /galaxy-Newcastle/tool_runner/index HTTP/1.1" 200 - "http://barium-rbh/galaxy-Newcastle/tool_runner?tool_id=upload1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" 127.0.0.1 - - [21/Feb/2013:13:16:30 +1000] "GET /galaxy-Newcastle/history HTTP/1.1" 200 - "http://barium-rbh/galaxy-Newcastle/tool_runner/index" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" galaxy.web.framework DEBUG 2013-02-21 13:16:31,347 Error: this request returned None from get_history(): http://barium-rbh/galaxy-Newcastle/root/user_get_usage 127.0.0.1 - - [21/Feb/2013:13:16:31 +1000] "POST /galaxy-Newcastle/root/user_get_usage HTTP/1.1" 500 - "http://barium-rbh/galaxy-Newcastle/history" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0" Debug at: http://barium-rbh/galaxy-Newcastle/_debug/view/1361342324 galaxy.jobs.manager DEBUG 2013-02-21 13:16:33,263 (4) Job assigned to handler 'main' galaxy.jobs DEBUG 2013-02-21 13:16:38,574 (4) Working directory for job is: /home/galaxy/galaxy-Newcastle/database/job_working_directory/000/4 galaxy.jobs.handler DEBUG 2013-02-21 13:16:38,574 dispatching job 4 to local runner galaxy.jobs.handler INFO 2013-02-21 13:16:38,837 (4) Job dispatched galaxy.jobs.runners.local DEBUG 2013-02-21 13:16:39,130 executing: python /home/galaxy/galaxy-Newcastle/tools/data_source/upload.py /home/galaxy/galaxy-Newcastle /home/galaxy/galaxy-Newcastle/database/tmp/tmpQK6VjP /home/galaxy/galaxy-Newcastle/database/tmp/tmpD_ySP1 4:/home/galaxy/galaxy-Newcastle/database/job_working_directory/000/4/dataset_4_files:/home/galaxy/galaxy-Newcastle/database/files/000/dataset_4.dat galaxy.jobs.runners.local DEBUG 2013-02-21 13:16:39,549 execution finished: python /home/galaxy/galaxy-Newcastle/tools/data_source/upload.py /home/galaxy/galaxy-Newcastle /home/galaxy/galaxy-Newcastle/database/tmp/tmpQK6VjP /home/galaxy/galaxy-Newcastle/database/tmp/tmpD_ySP1 4:/home/galaxy/galaxy-Newcastle/database/job_working_directory/000/4/dataset_4_files:/home/galaxy/galaxy-Newcastle/database/files/000/dataset_4.dat galaxy.jobs DEBUG 2013-02-21 13:16:39,590 The tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs DEBUG 2013-02-21 13:16:39,900 job 4 ended So some reason the history isn't getting updated for multiple instances of Galaxy Thanks Neil

3 4

wrapper around Upload tool
by Jay Humphrey 21 Feb '13

21 Feb '13

Hi! I have just started my first project with Galaxy. I want to use the Upload tool in my workflow to download large files from FTP (Swissprot and TREMBL fasta) and use them to make a local BLAST database. Can I build another tool as a wrapper around Upload, to enter preset ftp URLs (or overrides)? Thanks -Jay Humphrey

1 0

Datasets linked into Galaxy do not work
by Sarah Diehl 21 Feb '13

21 Feb '13

Hi all, I have problems with datasets that were linked into Galaxy data libraries (as Admin: "Add datasets", select "Upload files from filesystem paths" and "Link to files without copying to Galaxy"). Those files cannot be looked at, downloaded or viewed in a genome browser. Errors in the browser are: The requested URL /datasets/126e6c4f4c2d468e/display/ was not found on this server. The requested URL /library_common/download_dataset_from_folder was not found on this server. An error occurred while accessing: http://galaxy.immunbio.mpg.de/display_application/7108f175b5be4900/igv_bam/… Read error; BinaryCodec in readmode; streamed file (filename not available) I don't see any errors in the log files. We do this kind of linking a lot and everything worked fine in the past. I need to use this feature and need it to work, due to hard disk space and data duplication. Any help is appreciated. Best regards, Sarah

2 4

help on wig file
by Alfonso Garrido-Lecca 21 Feb '13

21 Feb '13

Hello, How can I change the "chromosome" name on a wig file to "chr"? FOr whatever reason when I try feeding this file with the program i am working with (CEAS package) it gives me an error. But when I use a wig file with chr it works fine This is how my wiggle file looks like: track type=wiggle_0 name="MACS_in_Galaxy_treat_CHROMOSOME_II" description="Shifted Merged MACS tag counts for every 1 bp" variableStep chrom=CHROMOSOME_II span=1 1 3 2 4 3 5 And this is how I need it to be: track type=wiggle_0 name="MACS_in_Galaxy_treat_I" description="Shifted Merged MACS tag counts for every 1 bp" variableStep chrom=I span=1 570 1 571 1 572 1 thanks alfonso

1 0

Rendering of < and > in read-me files on Tool Shed
by Peter Cock 21 Feb '13

21 Feb '13

Hi Greg, This is probably another minor bug report for your to-do list - would you prefer I try to file Trello issues for things like this directly? Many of the README files on the Tool Shed have an XML snippet in them if they include manual install instructions, but unfortunately the less-than and greater-than signs are rendered with HTML escape codes. For example, looking at: http://toolshed.g2.bx.psu.edu/view/peterjc/venn_list The line: <tool file="plotting/venn_list.xml" /> is shown as: <tool file="plotting/venn_list.xml" /> Note this does not affect the "Browse repository tip files" view, just the main tool overview page. Thanks, Peter

2 3