April 2015 - galaxy-dev - lists.galaxyproject.org

Cloning state when installing tools.
by Carlos Lijeron 08 Apr '15

08 Apr '15

Everyone, I've been encountering problems when installing particular tools in a local Galaxy instance. One such tool is Cummerbund as you can see in link below: https://www.dropbox.com/s/bmg4qcci97ayq2k/Cloning%20State.JPG?dl=0 Would you have a suggestion on how to troubleshoot failed tool installations that do not give an output error message? Thank you all ! Carlos.

2 1

restricted data upload from server folder
by De Smet Matthias 08 Apr '15

08 Apr '15

Hey Galaxy-dev’s, I’m completely new to galaxy, but I am tasked with a small scale deployment project. I am looking for an alternate option to upload data to galaxy. All data from our sequencers is stored on a NAS, which is accessible by the galaxy server. I would like to make it so that the users can only select data from that folder, to eliminate the need to transfer data through the client to the server. Is there any way to do this? Or am I making this to hard on myself? Sincerely [cid:image001.png@01D04471.1ABB7370] Matthias De Smet Centrum voor Medische Genetica De Pintelaan 185 - 9000 Gent matthias.desmet(a)uzgent.be<mailto:matthias.desmet@uzgent.be> tel.: +32 (0)9 332 0296 http://www.cmgg.be

2 1

Installing workflows from the Tool Shed is very confusing
by Peter Cock 08 Apr '15

08 Apr '15

Hello all, I've been playing with the Tool Shed lately for publishing Galaxy workflows - and realised the download process is still very complicated, especially the final hurdle - importing the workflow in order to actually use it. For the following example I am starting from a clean state - an almost fresh Galaxy installation with NOTHING as yet installed from the Tool Shed (see end of email), and I want to install a single workflow: https://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species 1. Goto Galaxy in browser, eg http://localhost/galaxy-dev/ 2. Goto "User", "Register" 3. Create account using address added to admin_users earlier in config/galaxy.ini 4. Click "Return to the home page" 5. Since logged in as admin, can goto "Admin" settings 6. Select "Search and browse tool sheds" 7. Select "Galaxy Main Tool Shed" 8. Enter "species" in the search box 9. Select "blast_top_hit_species" 10. Note the list of dependencies 11. Click "Install into Galaxy" 12. Leave the dependency handling ticked. 13. Ponder that the section selection will apply to? The current repository has no tools but some of the dependencies do... I don't want to bundle the BLAST tools in the same section as the text manipulation tools. Go with the default (don't pick a section). 14. Click "install" 15. Wait as the following are installed: Name Description Owner Revision blast_top_hit_species Workflow to count species of top nr BLASTX hits of a transcriptome peterjc db0c1bb92308 ncbi_blast_plus NCBI BLAST+ devteam 2fe07f50a41e blast_datatypes BlastXml and other blast datatypes devteam 5482a8cd0f36 package_blast_plus_2_2_29 NCBI BLAST+ 2.2.29 (binaries only) iuc a2ec897aac2c package_biopython_1_65 Downloads and compiles version 1.65 of the Biopython package. biopython dc595937617c package_numpy_1_9 Contains a tool dependency definition that downloads and compiles version 1.9 of the the python numpy package. iuc 84e97f5cd3ab package_atlas_3_10 Contains a tool dependency definition that downloads and compiles version 3.10 of the the Automatically Tuned Linear Algebra Software. iuc 37ff27ea37ed fasta_to_tabular FASTA-to-Tabular converter devteam 9d189d08f2ad sample_seqs Sub-sample sequences files (e.g. to reduce coverage) peterjc 02c13ef1a669 unique Advanced unique lines program using GNU sort bgruening 7ce75adb93be 16. Get bored, especially if the page appears to have stalled. Try reloading the page which goes back to the "Admin" welcome. Now how do I get back to the installation status... 17. Click on "Manage installed tool shed repositories" 18. If at this point some of the dependency requirements are in error state, try a round of repairing repositories (and watch at the terminal, e.g. compiling numpy is slow). 19. Admire all the nice green "tick" icons and "Installed" entries. 20. Notice the tiny grey "cog" icon against the workflow repository, "blast_top_hit_species". 21. Read the legend, "This repository contains exported workflows", which is encouraging but there is nothing on the page hinting at *how* to actually use the new workflow. 22. Try the context menu to see if that lets you use the workflow. No. 23. At this point I fully explored the workflow menus looking for an import option, and gave up and returned to "Manage installed tool shed repositories". 23. Ignore the context menu, click on "blast_top_hit_species". 24. Click on the top right "repository actions" menu 25. Find nothing about importing a workflow. 26. Click on the "Species of top BLAST hits" entry under "Contents of this repository" 27. Observe pretty SVG image of workflow. 24. Click on the top right "repository actions" menu 25. Notice the menu now offers "import workflow to galaxy". 26. Click on "import workflow to galaxy". 26. Back at SVG with green message "Workflow Species of top BLAST hits imported successfully. " 27. Return to "Analyze Data" (or "Workflows") and finally the workflow is available (although probably only for the Galaxy administrator's personal account). 28. Cry because all the new tools are dumped in the top level tool menu (see step 13). Possible suggestions to make this easier: At step 21, include a clue in the legend? At step 22 and 25, include "Import all these workflows" in menu? At step 24, change the button to "repository/workflow actions" since it includes some actions specific to the viewed workflow? How about the common use case where the Galaxy Admin was asked to import the workflow for a specific user? Can we have an action "Import workflow for specific user"? Also I think the tool section behaviour for dependencies needs clarification (step 13/28). From a tool author perspective, I would like to be able to set the default section for each tool (see also past requests to set the order they are added to the tool panel). Regards, Peter -- This was how I setup my current galaxy development instance tracking the default dev branch on github: $ cd ~/repositories $ git clone https://github.com/galaxyproject/galaxy.git $ cd galaxy I made minor config changes, including adding myself as an admin user via the admin_users setting: $ emacs config/galaxy.ini ... I also setup Apache to serve this instance at http://localhost/galaxy-dev/ and set ToolShed path to be ~/repositories/shed_tools/ This was working fine. To hopefully reset this back to a what is almost a clean slate (for this bug report), I next removed the auto-generated XML files, data files and SQLite database, and tool shed files: $ rm -f ~/repositories/galaxy/*.xml $ rm -f ~/repositories/galaxy/config/*.xml $ rm -f ~/repositories/galaxy/database/universe.sqlite $ rm -rf ~/repositories/galaxy/database/files/* $ rm -rf ~/repositories/shed_tools/ (Dangerous - which is why I explicitly used the full paths!) I then restart Galaxy, $ cd ~/repositories/galaxy $ ./run.sh ... Note it creates a new empty SQLite database etc.

3 3

Galaxy ToolShed Update Problem
by 孙兆朕 07 Apr '15

07 Apr '15

Hi all, I want to update some of my tool plug-in and some problem shows, even if I updated my Galaxy instance to the latest. I suppose it is not related to the version of Galaxy. The problem occured when I used the administrator's panel and update my plug-in using the "Get updates" option. The next page reports that an internal Server Error has occured and the update was not successful. All the plug-in updates seems had the same problem. I checked the logs where I get the following message: CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' HTTP_HOST: '222.28.163.110' HTTP_REFERER: 'http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag….' HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' HTTP_X_FORWARDED_FOR: '222.28.170.89' HTTP_X_FORWARDED_HOST: '222.28.163.110' ORGINAL_HTTP_HOST: '127.0.0.1:8080' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…' REMOTE_ADDR: '222.28.170.89' REQUEST_METHOD: 'GET' SERVER_NAME: '127.0.0.1' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.0' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0xd888510> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0xd888fd0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0xdcf8650> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'a12a5604dce911e4a05000215e543aa8' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') wsgi process: 'Multithreaded' ------------------------------------------------------------ ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 39438) Traceback (most recent call last): File "/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 1068, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib64/python2.6/SocketServer.py", line 618, in __init__ self.finish() File "/usr/lib64/python2.6/SocketServer.py", line 661, in finish self.wfile.flush() File "/usr/lib64/python2.6/socket.py", line 303, in flush self._sock.sendall(buffer(data, write_offset, buffer_size)) error: [Errno 32] Broken pipe ---------------------------------------- 222.28.170.89 - - [07/Apr/2015:13:47:59 +0800] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changeset_revision=de11e1a921c4&latest_changeset_revision=5482a8cd0f36&latest_ctx_rev=10 HTTP/1.0" 500 - "http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag…." "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://222.28.163.110/admin_toolshed/update_to_changeset_revision?tool_shed… File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', line 133 in __call__ return self.handle_request( environ, start_response ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', line 191 in handle_request body = method( trans, **kwargs ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1954 in update_to_changeset_revision persist=True ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get' CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' HTTP_HOST: '222.28.163.110' HTTP_REFERER: 'http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag….' HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' HTTP_X_FORWARDED_FOR: '222.28.170.89' HTTP_X_FORWARDED_HOST: '222.28.163.110' ORGINAL_HTTP_HOST: '127.0.0.1:8080' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…' REMOTE_ADDR: '222.28.170.89' REQUEST_METHOD: 'GET' SERVER_NAME: '127.0.0.1' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.0' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0xd888510> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0xd888fd0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0xdcf8650> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'a58bb404dce911e481c700215e543aa8' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') wsgi process: 'Multithreaded' ------------------------------------------------------------ Could you please give me some advise to figure out where the problem is and how to fix it? Thanks, Zhaozhen

2 1

a question on data manager and data table
by Beginner TI 07 Apr '15

07 Apr '15

Hi All, I just made a new local instance, installed the fetch_all_fasta data manager, downloaded mm9 fasta. Then I noticed the following: in tool_data_table_conf.xml, it has: <table name="all_fasta" comment_char="#"> <columns>value, dbkey, name, path</columns> <file path="tool-data/all_fasta.loc" /> </table> however in shed_tool_data_table_conf.xml, it has: <table comment_char="#" name="all_fasta"> <columns>value, dbkey, name, path</columns> <file path="/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/ toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cc…" /> <tool_shed_repository><tool_shed>toolshed.g2.bx.psu.edu </tool_shed><repository_name>data_manager_fetch_genome_all_fasta</repository_name><repository_owner>devteam</repository_owner><installed_changeset_revision>cca219f2b212</installed_changeset_revision></tool_shed_repository></table> in tool-data, all_fasta.loc is empty, but in the all_fasta.loc of the shed_tool entry, it shows: mm9 mm9 Mouse July 2007 (NCBI37/mm9) (mm9) /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq/mm9.fa So if I try "Extract Genomic DNA function", I could see the parameter passed to the command line is -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data" which does not host the data. It should be at least -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq" I thought that data manager will automatically populate these loc files? Am I missing something obvious? I could manually modify tool_data_table_conf.xml to let it point to /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/ toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cc… This would work, but it is ugly. Could someone please give me a hand to fix this? Thanks, Rui

2 1

fml_gff3togtf: GTF-to-BED failes on numpy in Galaxy
by Curtis Hendrickson (Campus) 06 Apr '15

06 Apr '15

Folks I installed this very useful looking tool repository (fml_gff3togtf<https://toolshed.g2.bx.psu.edu/view/vipints/fml_gff3togtf>) to convert GFF3->BED12 based on this BioStars. https://www.biostars.org/p/85869 It installed fine, but when I run it on ftp://ftp.ncbi.nih.gov/genomes/Viruses/Human_herpesvirus_5_uid14559/NC_0062… I get Traceback (most recent call last): File "/share/apps/galaxy/test-shed-tools/toolshed.g2.bx.psu.edu/repos/vipints/fml_gff3togtf/6e589f267c14/fml_gff3togtf/gff_to_bed.py", line 18, in <module> import GFFParser File "/share/apps/galaxy/test-shed-tools/toolshed.g2.bx.psu.edu/repos/vipints/fml_gff3togtf/6e589f267c14/fml_gff3togtf/GFFParser.py", line 20, in <module> import scipy.io as sio ImportError: No module named scipy.io Anyone know the trick to get the python libraries set up correctly? (also note that this failed nightly tests on 2015-04-06 - but that looks like a lack of test data/configuration) I see some other packages (like htseq_count<https://toolshed.g2.bx.psu.edu/view/lparsons/htseq_count>) have numpy as a dependency.... Using the devteam's numpy 1.7 package: https://toolshed.g2.bx.psu.edu/view/devteam/package_numpy_1_7 Regards, Curtis Research Associate, Informatics Unit Center for Clinical and Translational Science http://bioinformatics.uab.edu University of Alabama at Birmingham

3 2

Workflow bug when *.loc entries are missing
by Peter Cock 06 Apr '15

06 Apr '15

Hi all, I was testing a new workflow which I developed on the current stable release, Galaxy 15.03, and uploaded to the Tool Shed yesterday: https://github.com/peterjc/galaxy_blast/tree/master/workflows/blast_top_hit… http://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species I then switched to my development instance running the dev branch from github, 1. Log in as an Admin 2. Install http://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species 3. Upload suggested test dataset (in workflow README file): http://nematode.net/Data/nacobbus_aberrans_transcript_assembly/N.abberans_r… 4. Click on "All Workflows" (bottom of left pane tool list) 5. Click on "Species of top BLAST hits (imported from uploaded file)" 6. On preview screen, examine step 3, blastx 7. Notice it will run against Protein BLAST database "nr" 8. Click "Run workflow" As noted in the README file, this workflow assumes the local blastdb_p.loc file includes an entry "nr" for a current/recent mirror of the NCBI "non-redundant" protein BLAST database. Initially my development galaxy had an empty blastdb_p.loc file, so I expected the workflow to spot this problem at the preview step (before ever clicking "run workflow"). Instead it proceeds and attempted to run BLASTX without no database (which of course failed): blastx -query "/mnt/galaxy/repositories/galaxy/database/files/000/dataset_81.dat" -db "" -query_gencode 1 -evalue 0.001 -out "/mnt/galaxy/repositories/galaxy/database/files/000/dataset_82.dat" -outfmt "6 qseqid sseqid pident length mismatch gapopen qstart qend sstart send evalue bitscore staxids sscinames scomnames sblastnames sskingdoms" -num_threads "${GALAXY_SLOTS:-8}" -strand both -matrix BLOSUM62 -seg yes -max_target_seqs 1 That's the bad news: It fails, but it fails with a reasonably clear error: Fatal error: Exit code 2 () BLAST Database error: Database name is required. Good news: If I update blastdb_p.loc to add the "nr" database it works. Very bad news: If I update blastdb_p.loc to add other databases (but not an entry for "nr"), the workflow defaults to the first database in blastdb_p.loc Summary: Workflow requesting key entry "xxx" from example.loc (a) example.loc empty, gets empty path, fails cleanly. (b) example.loc contains xxx, gets correct path, works. (c) example.loc non empty but lacks xxx, runs with WRONG path. Peter

4 10

changing usage indicator
by Nikhil Joshi 04 Apr '15

04 Apr '15

Hi all, Is there a way to change the "Using" indicator at the top right to be a Using/Total for the whole disk (not just Galaxy usage)? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 3

NGINX uWSGI with require_login needs uwsgi_param SCRIPT_NAME '';
by David Trudgian 02 Apr '15

02 Apr '15

Hi, When I enabled 'require_login = True' on a galaxy instance running with NGINX+uWSGI I receive internal server error pages due to a Key error for File 'lib/galaxy/web/framework/base.py', line 356 in path return self.environ['SCRIPT_NAME'] + self.environ['PATH_INFO'] KeyError: 'SCRIPT_NAME' Works fine with 'require_login = False' - I'm not sure why require_login makes it take a code path which needs SCRIPT_NAME. Anyway - the addition of... uwsgi_param SCRIPT_NAME ''; ... to the nginx.conf prevents this error. Should I try and add this to the uWSGI config snippets on the wiki? Thanks, -- David Trudgian Ph.D., Computational Scientist UT Southwestern BioHPC -- David Trudgian Ph.D., Computational Scientist UT Southwestern BioHPC ________________________________ UT Southwestern Medical Center The future of medicine, today.

2 2

April! Galactic News!
by Dave Clements 02 Apr '15

02 Apr '15

1 0

using DataManager to add custom genomes
by Nikhil Joshi 02 Apr '15

02 Apr '15

Hi all, Is it possible to use a DataManager (or anything) to allow a user (or admin) to upload their genome of choice and have it be indexed and be used as a built-in/cached genome *without* having to go on the command-line? I.e., only using the Galaxy interface? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 2

test/base/twilltestcase.py - Converting local (test-data) bam to sam failed
by Peter Cock 02 Apr '15

02 Apr '15

Hi all, I have a query about a failing tool test on the Tool Shed, where it seems Galaxy is trying to convert both the expected BAM output and the tool's BAM output into SAM for comparison - but this fails. https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-27 02:13:51 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/sa…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 40, in do_it self._verify_outputs( testdef, test_history, jobs, shed_tool_id, data_list, galaxy_interactor ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 80, in _verify_outputs galaxy_interactor.verify_output( history, jobs, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 66, in verify_output self.twill_test_case.verify_hid( outfile, hda_id=hid, attributes=attributes, dataset_fetcher=fetcher, shed_tool_id=shed_tool_id ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 2341, in verify_hid raise AssertionError( errmsg ) AssertionError: History item different than expected, difference (using diff): ( /tmp/shed_tools/toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/2d303f2e09e0/samtools_depad/test-data/sam_spec_padded.depad.bam v. /tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpExQWRD/tmpSknKqLsam_spec_padded.depad.bam ) Converting local (test-data) bam to sam failed (and same error on the second test) The same occurs on the Test Tool Shed (with a very slightly updated version of this tool), although with a more concise traceback: https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-25 08:21:59 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 67, in do_it raise e JobOutputsError: History item different than expected, difference (using diff): ( /tmp/shed_tools/testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/8e1e3a1ecad7/samtools_depad/test-data/sam_spec_padded.depad.bam v. /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpNvCGJi/tmphX3A_Fsam_spec_padded.depad.bam ) Converting local (test-data) bam to sam failed Both tests should produce a BAM file matching the provided test-data/sam_spec_padded.depad.bam example. The error comes from test/base/twilltestcase.py, p = subprocess.Popen( args='samtools view -h -o "%s" "%s"' % ( temp_local.name, local_name ), shell=True ) assert not p.wait(), 'Converting local (test-data) bam to sam failed' Running this command locally at the terminal seems to work fine, both with samtools 0.1.19 and 1.1. $ ~/bin/samtools_0.1.19 view -h -o temp.sam sam_spec_padded.depad.bam $ more temp.sam @HD VN:1.5 SO:coordinate @SQ SN:ref LN:45 ref 516 ref 1 0 45M * 0 0 AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT * r001 99 ref 7 30 8M2I4M1D3M = 37 41 TTAGATAAAGGATACTG * * 768 ref 8 30 1M * 0 0 * * CT:Z:.;Warning;Note=Ref wrong? r002 0 ref 9 30 3S6M1P1I4M * 0 0 AAAAGATAAGGATA * PT:Z:1;4;+;homopolymer r003 0 ref 9 30 5H6M * 0 0 AGCTAA * NM:i:1 r004 0 ref 16 30 6M14D5M * 0 0 ATAGCTTCAGC * r003 2064 ref 29 30 6H5M * 0 0 TAGGC * NM:i:0 r001 147 ref 37 30 9M = 7 -41 CAGCGGCAT * NM:i:1 $ ~/bin/samtools_1.1 view -h -o temp.sam sam_spec_padded.depad.bam [galaxy@ppserver test-data]$ more temp.sam @HD VN:1.5 SO:coordinate @SQ SN:ref LN:45 ref 516 ref 1 0 45M * 0 0 AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT * r001 99 ref 7 30 8M2I4M1D3M = 37 41 TTAGATAAAGGATACTG * * 768 ref 8 30 1M * 0 0 * * CT:Z:.;Warning;Note=Ref wrong? r002 0 ref 9 30 3S6M1P1I4M * 0 0 AAAAGATAAGGATA * PT:Z:1;4;+;homopolymer r003 0 ref 9 30 5H6M * 0 0 AGCTAA * NM:i:1 r004 0 ref 16 30 6M14D5M * 0 0 ATAGCTTCAGC * r003 2064 ref 29 30 6H5M * 0 0 TAGGC * NM:i:0 r001 147 ref 37 30 9M = 7 -41 CAGCGGCAT * NM:i:1 Which version of samtools is on the $PATH on the test system? Could the test system have run out of disk space again? The tests are passing on my local development instance (before and after updating to latest galaxy-central), and on TravisCI using the latest Galaxy: https://travis-ci.org/peterjc/pico_galaxy/builds/48485673 Thanks, Peter

2 5

value_from_basic - KeyError: 'base_name', was: Tests not being run on toolsheds?
by Peter Cock 02 Apr '15

02 Apr '15

Hi Dave, The following looks like a regression on the Test Tool Shed, the tests pass locally (using a recent revision), and on TravisCI using the current galaxy dev branch on GitHub: https://travis-ci.org/peterjc/pico_galaxy/builds/54870500 Problem tool: https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs This may be unrelated, but by chance this is the first time I have uploaded a tool to the Tool Shed which uses the new functionality to test the stdout/stderr strings, and more importantly it includes a test expected to fail via <test expect_failure="true" expect_exit_code="1"> Revision on my development repository: https://github.com/peterjc/pico_galaxy/commit/55ebb308b911b4acef912cc3b03f4… Test Tool Shed output from last night: Automated test environment Time tested: 2015-03-18 02:46:55 System: Linux 3.13.0-36-generic Architecture: x86_64 Python version: 2.7.6 Galaxy revision: 17050:6395e7035143 Galaxy database version: 128 Tool shed revision: 16867:0468d285f89c Tool shed database version: 25 Tool shed mercurial version: 3.2.4 Tests that failed Tool id: sample_seqs Tool version: sample_seqs Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 37, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 38, in stage_data_in_history upload_wait() File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 279, in wait while not self.__history_ready( history_id ): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 297, in __history_ready return self._state_ready( state, error_msg="History in error state." ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 356, in _state_ready raise Exception( error_msg ) Exception: History in error state. Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/__init__.py", line 158, in prepare_job job_wrapper.prepare() File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/__init__.py", line 828, in prepare tool_evaluator.set_compute_environment( compute_environment, get_special=get_special ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/evaluation.py", line 53, in set_compute_environment incoming = self.tool.params_from_strings( incoming, self.app ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/__init__.py", line 1873, in params_from_strings return params_from_strings( self.inputs, params, app, ignore_errors ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/__init__.py", line 103, in params_from_strings value = params[key].value_from_basic( value, app, ignore_errors ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/grouping.py", line 512, in value_from_basic raise e KeyError: 'base_name' requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 galaxy.datatypes.metadata: DEBUG: Cleaning up external metadata files requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents/d80a923b055ee8e9?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents/d80a923b055ee8e9/provenance?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None (All the tests for this tool seem to have failed the same way) Regards, Peter On Wed, Mar 18, 2015 at 1:46 PM, Dave Bouvier <dave(a)bx.psu.edu> wrote: > Gentlemen, > > The issue with the nightly testing was due to a stalled test run blocking > subsequent tests. I've cleared out that blockage and a manual test run > appears to have completed successfully, as should future automated test > runs. As always, feel free to let us know if you encounter any additional > inexplicable behavior. > > --Dave B. > > On 03/18/2015 07:00 AM, Peter Cock wrote: >> >> ... I can also confirm that the Test Tool Shed example I >> gave was tested overnight, although it looks like a novel failure: >> https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs >>

2 4

wrapper accepts fastqcssanger when it shouldn't
by Nikhil Joshi 02 Apr '15

02 Apr '15

Hi all, So in my wrapper for my trimming tool, sickle, I only allow fastq, fastqsanger, fastqillumina, or fastqsolexa input data. However, if a fastqsanger file's datatype has been erroneously changed to fastqcssanger, my tool will still show that file in the dropdown for the possible inputs. Why does this happen? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 1

dat quota
by Williams, Charles Houston 01 Apr '15

01 Apr '15

After reading threads. I have deleted and purged all but 33gb of my datasets and a single history. However, my quota is still 99%. What can I do to fix this issue. My user address is Charles.h.williams@vanderbilt. Edu Thanks, Chuck Charles H Williams III Graduate Student Hong Lab Department of Cell and Developmental Biology Vanderbilt University Light Hall 1155a

2 1

Trouble maintaining Galaxy config on cloud instance
by David Kovalic 01 Apr '15

01 Apr '15

Hi, I am trying to change the ftp dir (and some other modifications) vs. the standard Galaxy Cloudman config. When I modify the appropriate line in /mnt/galaxy/galaxy-app/universe_wsgi.ini on my running cloud instance: my changes disappear when I terminate/restart the cluster. When I modify universe_wsgi.ini.cloud and upload to the S3 bucket the file gets overwritten by the default when I restart the cluster... I can't find anywhere on the net which file I need to modify in order to have config changes persist. Any help would be greatly appreciated. Thanks, David

2 2

storing "set max value" in trackster for wig file display
by Hans-Rudolf Hotz 01 Apr '15

01 Apr '15

Hi We have been using trackster much more in recent times - It really has improved a lot (especially its stability) over the last few years - thank you very much. We are running into a little annoying issue: when we display wiggle files, bedgraph files, bam files in coverage mode, etc, the max value gets somehow calculated. Very often, we want to change that, ie in order to see the full heights of the peaks instead of the capped peaks. This can be easily done by changing the "set max value" at the right hand side of the track. Unfortunately, the new value does not get stored. No matter, whether we click on safe or not. So next time we wanna a look at the same visualization again, we have to set the 'max value' again. Is this a known missing feature (bug) ? I just wanna check before I create a Trello card - so far, I couldn't find in in Trello. Thanks for your help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland

2 1