April 2015 - galaxy-dev - lists.galaxyproject.org

Escaped HTML show in import-workflow red message
by Peter Cock 08 Apr '15

08 Apr '15

Hi all, I just imported a workflow and noticed the following glitch with the red message shown to the user about missing tools - the escaped HTML is show to the user (ampersand lt semi-colon etc). Specifically, I imported http://www.myexperiment.org/workflows/4601.html from Bjoern. Using the current v15.03 release, specifically: $ hg log | head changeset: 17050:6395e7035143 tag: tip parent: 17049:708cab19708d parent: 17048:28113eeee212 user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Mar 16 22:50:21 2015 -0400 summary: Merge v15.03.1 to default Peter Red message: Workflow Finding 3 genes close to each other (imported from uploaded file) imported successfully. The workflow requires the following tools that are not available in this Galaxy instance.You can likely install the required tools from one of the Galaxy tool sheds listed below. Tool name Compute, id toolshed.g2.bx.psu.edu/repos/devteam/column_maker/Add_a_column1/1.1.0, version 1.1.0 Tool name Intersect, id toolshed.g2.bx.psu.edu/repos/devteam/intersect/gops_intersect_1/0.0.1, version 0.0.1 Tool name Intersect, id toolshed.g2.bx.psu.edu/repos/devteam/intersect/gops_intersect_1/1.0.0, version 1.0.0 <a href="http://toolshed.g2.bx.psu.edu/repository/find_tools?galaxy_url=ht… main tool shed</a> <a href="http://testtoolshed.g2.bx.psu.edu/repository/find_tools?galaxy_ur… test tool shed</a>

1 1

Re: [galaxy-dev] a question on data manager and data table
by Daniel Blankenberg 08 Apr '15

08 Apr '15

Hi Rui, (cc’d back to list) One of the nice things about tool Data Tables is that it allows us to have multiple ‘.loc’ files as the source for the content for a single data table — so we can have many ‘all_fasta.loc’ files, but one ‘all_fasta’ tool data table, with merged contents, and a tool working on ‘all_fasta’ data table doesn’t know or care about the actual .loc files on disk. Then any tool that uses the ‘all_fasta’ table in an input parameter can simply call input.fields.path, to have the actual path to the fasta file passed to the tool. > 1.which file should the downloaded fasta data modify? tool_data_table_conf.xml, or shed_tool_data_table_conf.xml? Tools installed from the toolshed that include data Tables will programmatically modify the shed_ file, the other file can is for manual admin changes. > 2. which loc file is the primary one for tools to reference? the one in $GALAXYHOME/tool-data, or the one in $GALAXYHOME/tool-data//toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cca219f2b212/? If first one, then should the data manager update that one too(right now it didn't)? Tools shouldn’t be referencing .loc files directly any more, and instead should use the tool data table abstraction. > 3. For now, that tool still uses alignseq.loc, I need to manually update that file, correct? Any tool that makes direct use of a .loc file will need to have that specific file modified. > 4. How to determine if a tool is using data table is to look at the tool xml file to see if it contains reference to a data table, right? And in tool_data_table_conf.xml, it should contain data tables definition for all the possible data, right? a.) Yes, you’ll have to look at the tool.xml, although we could add another section to the ToolShed preview to include this information if it is useful. b.) tool_data_table_conf.xml should contain data tables distributed with the Galaxy distribution and any manually configured data tables. tool_data_table_conf.xml and shed_tool_data_table_conf.xml will be merged into one set of tool data tables — a tool data table does not need to be in tool_data_table_conf.xml if it is in shed_tool_data_table_conf.xml and vis-versa (but can be in both). Thanks for using Galaxy, Dan On Apr 7, 2015, at 12:47 PM, Beginner TI <ruiwang.sz(a)gmail.com> wrote: > Hi Dan, > > Thanks for the reply! > > So, what would be the expected behavior for the tool that uses data table? For example, in my case, the tool_data_table_conf.xml would be modified to reflect the update to all_fasta.loc? > > I'm a little puzzled on how this works. As you could see, after I installed that data manager and downloaded the mm9 fasta data, it modified the file shed_tool_data_table_conf.xml. This should have nothing to do with the tool Extract Genome DNA, am I right? So, my question is, > > 1.which file should the downloaded fasta data modify? tool_data_table_conf.xml, or shed_tool_data_table_conf.xml? > > 2. which loc file is the primary one for tools to reference? the one in $GALAXYHOME/tool-data, or the one in $GALAXYHOME/tool-data//toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cca219f2b212/? If first one, then should the data manager update that one too(right now it didn't)? > > 3. For now, that tool still uses alignseq.loc, I need to manually update that file, correct? > > 4. How to determine if a tool is using data table is to look at the tool xml file to see if it contains reference to a data table, right? And in tool_data_table_conf.xml, it should contain data tables definition for all the possible data, right? > > If you could please elaborate a bit, I'll really appreciate! > > Thanks, > Rui > > > > > On Tue, Apr 7, 2015 at 7:15 AM, Daniel Blankenberg <dan(a)bx.psu.edu> wrote: > Hi Rui, > > The Extract Genomic DNA Tool has not yet been updated to work with tool Data Tables. > > Additionally, this tool requires TwoBit formatted files when selecting from built-in data — when selecting a FASTA from your history, the fasta file is converted to TwoBit internally in the tool before extracting sequence chunks. > > There is a TwoBit builder Data Manager under development (https://github.com/galaxyproject/tools-devteam/tree/master/data_managers/da…) that hasn’t made it out to the ToolShed just yet, but it won’t be helpful until the the Extract Genomic DNA Tool is updated: https://trello.com/c/8unnSp7H > > > Thanks for using Galaxy, > > Dan > > > On Apr 7, 2015, at 12:16 AM, Beginner TI <ruiwang.sz(a)gmail.com> wrote: > >> Hi All, >> >> I just made a new local instance, installed the fetch_all_fasta data manager, downloaded mm9 fasta. Then I noticed the following: >> >> in tool_data_table_conf.xml, it has: >> >> <table name="all_fasta" comment_char="#"> >> <columns>value, dbkey, name, path</columns> >> <file path="tool-data/all_fasta.loc" /> >> </table> >> >> however in shed_tool_data_table_conf.xml, it has: >> >> <table comment_char="#" name="all_fasta"> >> <columns>value, dbkey, name, path</columns> >> <file path="/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cca219f2b212/all_fasta.loc" /> >> <tool_shed_repository><tool_shed>toolshed.g2.bx.psu.edu</tool_shed><repository_name>data_manager_fetch_genome_all_fasta</repository_name><repository_owner>devteam</repository_owner><installed_changeset_revision>cca219f2b212</installed_changeset_revision></tool_shed_repository></table> >> >> in tool-data, all_fasta.loc is empty, but in the all_fasta.loc of the shed_tool entry, it shows: >> >> mm9 mm9 Mouse July 2007 (NCBI37/mm9) (mm9) /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq/mm9.fa >> >> So if I try "Extract Genomic DNA function", I could see the parameter passed to the command line is >> >> -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data" >> >> which does not host the data. It should be at least >> >> -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq" >> >> I thought that data manager will automatically populate these loc files? Am I missing something obvious? I could manually modify tool_data_table_conf.xml to let it point to >> >> /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cca219f2b212/all_fasta.loc >> >> This would work, but it is ugly. Could someone please give me a hand to fix this? >> >> Thanks, >> Rui >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > >

1 0

DRMAA chmod to world-writable if to user chown fails should not be default?
by David Trudgian 08 Apr '15

08 Apr '15

Hi, Looking through the DRMAA job runner code I noticed that in lib/galaxy/jobs/__init__.py:1582 that if chown to a user in change_ownership_for_run fails then there is chmod 0777 instead. I'm thinking that this probably shouldn't happen by default, or there needs to be a big warning on the wiki? I understand the world-readable security concerns posted there - but world-writable, for whatever reason, on a shared cluster seems like a disaster waiting to happen. Maybe either warn on the wiki, make the chmod 777 a non-default option, or an option to disable it? DT ________________________________ UT Southwestern Medical Center The future of medicine, today.

2 2

Multiple outputs of unknown types
by David Kelly 08 Apr '15

08 Apr '15

Hello, I've looked through the documentation about tools that produce multiple output files, but it seems like you're required to know the data types ahead of time (or make assumptions based on input data). But what about a case where you don't know what the output datatypes will be? Is it just possible to have N output files imported and sniffed? Thanks, Davids

3 2

Re: [galaxy-dev] Galaxy ToolShed Update Problem
by Martin Čech 08 Apr '15

08 Apr '15

Hi Zhaozhen, the --config_file argument needs to point to galaxy.ini config so the script can load the database. Galaxy continues to support Python 2.6 and 2.7, I am sorry that this script required 2.7 (or you could `pip install argparse`) - we will look into improving it. Martin, Galaxy Team On Wed, Apr 8, 2015 at 4:09 AM 孙兆朕 <sunzzn502(a)163.com> wrote: > Hi Martin, > > Thanks for the advise. It helped a lot for me to figure out where the > problem is. However, there were still error messages there. > I tried to run the update_shed_config_path.py script under the /script > folder. At first it says there was no module named argparse, then I found > out it was the python version problem, which required python 2.7+. Then I > updated my python. Then there was another error said there was no module > named galaxy.model.tool_shed_install. Then I tried harder, and found out > that it was the path problem. I figured out the sys path was in the above > folder. Then I copied and pasted the update_shed_config_path.py to the > parent folder and seemed solved the problem. Then I run > the update_shed_config_path.py and the command somehow run for a while, > with the error message No module named _sqlite3. It must be the DB > versions and plug-ins, then I fixed the DB. Then finally there came the > following message: > > usage: update_shed_config_path.py [-h] --config_file CONFIG_FILE --from > BAD_FILENAME --to GOOD_FILENAME [--force] > update_shed_config_path.py: error: argument --config_file is required > > I run the command line in: python2.7 update_shed_config_path.py > I wonder if I need some more input for the command line or something? > By the way, I hope there would be some advise for the new Galaxy > administrtators to know their Python should be 2.7+ and file path issues. > When I started to deploy Galaxy last year, the requirement in the web page > was Python 2.5 or something lower than 2.6 I used. I think that message > need to be updated as well. > Thanks in advance, > > Yours, > > Zhaozhen > > > > > argparse > > > At 2015-04-07 22:43:50, "Martin Čech" <marten(a)bx.psu.edu> wrote: > > Hello Zhaozhen, > > this is a known problem. The shed_tool_conf.xml got moved along with other > configs into /config folder but unfortunately some tools stored the path > absolutely in the DB. You can use the following script (which you should > find in the Galaxy /scripts folder): > > https://github.com/galaxyproject/galaxy/blob/dev/scripts/update_shed_config… > to update your DB to have the correct path to the config. > > Sorry for the inconvenience. > > Martin, Galaxy Team > > > On Tue, Apr 7, 2015 at 10:16 AM 孙兆朕 <sunzzn502(a)163.com> wrote: > >> Hi all, >> >> I want to update some of my tool plug-in and some problem shows, even if >> I updated my Galaxy instance to the latest. I suppose it is not related to >> the version of Galaxy. >> >> The problem occured when I used the administrator's panel and update my >> plug-in using the "Get updates" option. The next page reports that an >> internal Server Error has occured and the update was not successful. All >> the plug-in updates seems had the same problem. >> >> I checked the logs where I get the following message: >> >> CGI Variables >> ------------- >> CONTENT_LENGTH: '0' >> HTTP_ACCEPT: >> 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' >> HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' >> HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' >> HTTP_CONNECTION: 'close' >> HTTP_COOKIE: >> 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' >> HTTP_HOST: '222.28.163.110' >> HTTP_REFERER: ' >> http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag… >> .' >> HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 >> (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' >> HTTP_X_FORWARDED_FOR: '222.28.170.89' >> HTTP_X_FORWARDED_HOST: '222.28.163.110' >> ORGINAL_HTTP_HOST: '127.0.0.1:8080' >> ORGINAL_REMOTE_ADDR: '127.0.0.1' >> PATH_INFO: '/admin_toolshed/update_to_changeset_revision' >> QUERY_STRING: 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ' >> REMOTE_ADDR: '222.28.170.89' >> REQUEST_METHOD: 'GET' >> SERVER_NAME: '127.0.0.1' >> SERVER_PORT: '8080' >> SERVER_PROTOCOL: 'HTTP/1.0' >> >> >> WSGI Variables >> -------------- >> application: <paste.recursive.RecursiveMiddleware object at 0xd888510> >> is_api_request: False >> paste.cookies: (<SimpleCookie: >> galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, >> 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') >> paste.expected_exceptions: [<class >> 'paste.httpexceptions.HTTPException'>] >> paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object >> at 0xd888fd0> >> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at >> 0xdcf8650> >> paste.parsed_querystring: ([('tool_shed_url', ' >> https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), >> ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), >> ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], >> 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ') >> paste.recursive.forward: <paste.recursive.Forwarder from /> >> paste.recursive.include: <paste.recursive.Includer from /> >> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from >> /> >> paste.recursive.script_name: '' >> paste.throw_errors: True >> request_id: 'a12a5604dce911e4a05000215e543aa8' >> webob._parsed_query_vars: (MultiDict([('tool_shed_url', ' >> https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), >> ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), >> ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), >> 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ') >> wsgi process: 'Multithreaded' >> ------------------------------------------------------------ >> ---------------------------------------- >> Exception happened during processing of request from ('127.0.0.1', 39438) >> Traceback (most recent call last): >> File >> "/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", >> line 1068, in process_request_in_thread >> self.finish_request(request, client_address) >> File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request >> self.RequestHandlerClass(request, client_address, self) >> File "/usr/lib64/python2.6/SocketServer.py", line 618, in __init__ >> self.finish() >> File "/usr/lib64/python2.6/SocketServer.py", line 661, in finish >> self.wfile.flush() >> File "/usr/lib64/python2.6/socket.py", line 303, in flush >> self._sock.sendall(buffer(data, write_offset, buffer_size)) >> error: [Errno 32] Broken pipe >> ---------------------------------------- >> 222.28.170.89 - - [07/Apr/2015:13:47:59 +0800] "GET >> /admin_toolshed/update_to_changeset_revision?tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> HTTP/1.0" 500 - " >> http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag…." >> "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) >> Chrome/34.0.1847.116 Safari/537.36" >> Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no >> attribute 'get' >> URL: >> http://222.28.163.110/admin_toolshed/update_to_changeset_revision?tool_shed… >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', >> line 149 in __call__ >> app_iter = self.application(environ, sr_checker) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', >> line 84 in __call__ >> return self.application(environ, start_response) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', >> line 633 in __call__ >> return self.application(environ, start_response) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', >> line 133 in __call__ >> return self.handle_request( environ, start_response ) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', >> line 191 in handle_request >> body = method( trans, **kwargs ) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/decorators.py', >> line 87 in decorator >> return func( self, trans, *args, **kwargs ) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', >> line 1954 in update_to_changeset_revision >> persist=True ) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', >> line 31 in __init__ >> metadata_dict=metadata_dict, user=None ) >> File >> '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', >> line 60 in __init__ >> self.metadata_dict = { 'shed_config_filename' : >> self.shed_config_dict.get( 'config_filename', None ) } >> AttributeError: 'NoneType' object has no attribute 'get' >> >> >> CGI Variables >> ------------- >> CONTENT_LENGTH: '0' >> HTTP_ACCEPT: >> 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' >> HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' >> HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' >> HTTP_CONNECTION: 'close' >> HTTP_COOKIE: >> 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' >> HTTP_HOST: '222.28.163.110' >> HTTP_REFERER: ' >> http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag… >> .' >> HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 >> (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' >> HTTP_X_FORWARDED_FOR: '222.28.170.89' >> HTTP_X_FORWARDED_HOST: '222.28.163.110' >> ORGINAL_HTTP_HOST: '127.0.0.1:8080' >> ORGINAL_REMOTE_ADDR: '127.0.0.1' >> PATH_INFO: '/admin_toolshed/update_to_changeset_revision' >> QUERY_STRING: 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ' >> REMOTE_ADDR: '222.28.170.89' >> REQUEST_METHOD: 'GET' >> SERVER_NAME: '127.0.0.1' >> SERVER_PORT: '8080' >> SERVER_PROTOCOL: 'HTTP/1.0' >> >> >> WSGI Variables >> -------------- >> application: <paste.recursive.RecursiveMiddleware object at 0xd888510> >> is_api_request: False >> paste.cookies: (<SimpleCookie: >> galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, >> 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') >> paste.expected_exceptions: [<class >> 'paste.httpexceptions.HTTPException'>] >> paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object >> at 0xd888fd0> >> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at >> 0xdcf8650> >> paste.parsed_querystring: ([('tool_shed_url', ' >> https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), >> ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), >> ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], >> 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ') >> paste.recursive.forward: <paste.recursive.Forwarder from /> >> paste.recursive.include: <paste.recursive.Includer from /> >> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from >> /> >> paste.recursive.script_name: '' >> paste.throw_errors: True >> request_id: 'a58bb404dce911e481c700215e543aa8' >> webob._parsed_query_vars: (MultiDict([('tool_shed_url', ' >> https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), >> ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), >> ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), >> 'tool_shed_url= >> https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changese… >> ') >> wsgi process: 'Multithreaded' >> ------------------------------------------------------------ >> >> Could you please give me some advise to figure out where the problem is >> and how to fix it? >> >> Thanks, >> >> >> Zhaozhen >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > >

1 0

Cloning state when installing tools.
by Carlos Lijeron 08 Apr '15

08 Apr '15

Everyone, I've been encountering problems when installing particular tools in a local Galaxy instance. One such tool is Cummerbund as you can see in link below: https://www.dropbox.com/s/bmg4qcci97ayq2k/Cloning%20State.JPG?dl=0 Would you have a suggestion on how to troubleshoot failed tool installations that do not give an output error message? Thank you all ! Carlos.

2 1

restricted data upload from server folder
by De Smet Matthias 08 Apr '15

08 Apr '15

Hey Galaxy-dev’s, I’m completely new to galaxy, but I am tasked with a small scale deployment project. I am looking for an alternate option to upload data to galaxy. All data from our sequencers is stored on a NAS, which is accessible by the galaxy server. I would like to make it so that the users can only select data from that folder, to eliminate the need to transfer data through the client to the server. Is there any way to do this? Or am I making this to hard on myself? Sincerely [cid:image001.png@01D04471.1ABB7370] Matthias De Smet Centrum voor Medische Genetica De Pintelaan 185 - 9000 Gent matthias.desmet(a)uzgent.be<mailto:matthias.desmet@uzgent.be> tel.: +32 (0)9 332 0296 http://www.cmgg.be

2 1

Installing workflows from the Tool Shed is very confusing
by Peter Cock 07 Apr '15

07 Apr '15

Hello all, I've been playing with the Tool Shed lately for publishing Galaxy workflows - and realised the download process is still very complicated, especially the final hurdle - importing the workflow in order to actually use it. For the following example I am starting from a clean state - an almost fresh Galaxy installation with NOTHING as yet installed from the Tool Shed (see end of email), and I want to install a single workflow: https://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species 1. Goto Galaxy in browser, eg http://localhost/galaxy-dev/ 2. Goto "User", "Register" 3. Create account using address added to admin_users earlier in config/galaxy.ini 4. Click "Return to the home page" 5. Since logged in as admin, can goto "Admin" settings 6. Select "Search and browse tool sheds" 7. Select "Galaxy Main Tool Shed" 8. Enter "species" in the search box 9. Select "blast_top_hit_species" 10. Note the list of dependencies 11. Click "Install into Galaxy" 12. Leave the dependency handling ticked. 13. Ponder that the section selection will apply to? The current repository has no tools but some of the dependencies do... I don't want to bundle the BLAST tools in the same section as the text manipulation tools. Go with the default (don't pick a section). 14. Click "install" 15. Wait as the following are installed: Name Description Owner Revision blast_top_hit_species Workflow to count species of top nr BLASTX hits of a transcriptome peterjc db0c1bb92308 ncbi_blast_plus NCBI BLAST+ devteam 2fe07f50a41e blast_datatypes BlastXml and other blast datatypes devteam 5482a8cd0f36 package_blast_plus_2_2_29 NCBI BLAST+ 2.2.29 (binaries only) iuc a2ec897aac2c package_biopython_1_65 Downloads and compiles version 1.65 of the Biopython package. biopython dc595937617c package_numpy_1_9 Contains a tool dependency definition that downloads and compiles version 1.9 of the the python numpy package. iuc 84e97f5cd3ab package_atlas_3_10 Contains a tool dependency definition that downloads and compiles version 3.10 of the the Automatically Tuned Linear Algebra Software. iuc 37ff27ea37ed fasta_to_tabular FASTA-to-Tabular converter devteam 9d189d08f2ad sample_seqs Sub-sample sequences files (e.g. to reduce coverage) peterjc 02c13ef1a669 unique Advanced unique lines program using GNU sort bgruening 7ce75adb93be 16. Get bored, especially if the page appears to have stalled. Try reloading the page which goes back to the "Admin" welcome. Now how do I get back to the installation status... 17. Click on "Manage installed tool shed repositories" 18. If at this point some of the dependency requirements are in error state, try a round of repairing repositories (and watch at the terminal, e.g. compiling numpy is slow). 19. Admire all the nice green "tick" icons and "Installed" entries. 20. Notice the tiny grey "cog" icon against the workflow repository, "blast_top_hit_species". 21. Read the legend, "This repository contains exported workflows", which is encouraging but there is nothing on the page hinting at *how* to actually use the new workflow. 22. Try the context menu to see if that lets you use the workflow. No. 23. At this point I fully explored the workflow menus looking for an import option, and gave up and returned to "Manage installed tool shed repositories". 23. Ignore the context menu, click on "blast_top_hit_species". 24. Click on the top right "repository actions" menu 25. Find nothing about importing a workflow. 26. Click on the "Species of top BLAST hits" entry under "Contents of this repository" 27. Observe pretty SVG image of workflow. 24. Click on the top right "repository actions" menu 25. Notice the menu now offers "import workflow to galaxy". 26. Click on "import workflow to galaxy". 26. Back at SVG with green message "Workflow Species of top BLAST hits imported successfully. " 27. Return to "Analyze Data" (or "Workflows") and finally the workflow is available (although probably only for the Galaxy administrator's personal account). 28. Cry because all the new tools are dumped in the top level tool menu (see step 13). Possible suggestions to make this easier: At step 21, include a clue in the legend? At step 22 and 25, include "Import all these workflows" in menu? At step 24, change the button to "repository/workflow actions" since it includes some actions specific to the viewed workflow? How about the common use case where the Galaxy Admin was asked to import the workflow for a specific user? Can we have an action "Import workflow for specific user"? Also I think the tool section behaviour for dependencies needs clarification (step 13/28). From a tool author perspective, I would like to be able to set the default section for each tool (see also past requests to set the order they are added to the tool panel). Regards, Peter -- This was how I setup my current galaxy development instance tracking the default dev branch on github: $ cd ~/repositories $ git clone https://github.com/galaxyproject/galaxy.git $ cd galaxy I made minor config changes, including adding myself as an admin user via the admin_users setting: $ emacs config/galaxy.ini ... I also setup Apache to serve this instance at http://localhost/galaxy-dev/ and set ToolShed path to be ~/repositories/shed_tools/ This was working fine. To hopefully reset this back to a what is almost a clean slate (for this bug report), I next removed the auto-generated XML files, data files and SQLite database, and tool shed files: $ rm -f ~/repositories/galaxy/*.xml $ rm -f ~/repositories/galaxy/config/*.xml $ rm -f ~/repositories/galaxy/database/universe.sqlite $ rm -rf ~/repositories/galaxy/database/files/* $ rm -rf ~/repositories/shed_tools/ (Dangerous - which is why I explicitly used the full paths!) I then restart Galaxy, $ cd ~/repositories/galaxy $ ./run.sh ... Note it creates a new empty SQLite database etc.

3 3

Galaxy ToolShed Update Problem
by 孙兆朕 07 Apr '15

07 Apr '15

Hi all, I want to update some of my tool plug-in and some problem shows, even if I updated my Galaxy instance to the latest. I suppose it is not related to the version of Galaxy. The problem occured when I used the administrator's panel and update my plug-in using the "Get updates" option. The next page reports that an internal Server Error has occured and the update was not successful. All the plug-in updates seems had the same problem. I checked the logs where I get the following message: CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' HTTP_HOST: '222.28.163.110' HTTP_REFERER: 'http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag….' HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' HTTP_X_FORWARDED_FOR: '222.28.170.89' HTTP_X_FORWARDED_HOST: '222.28.163.110' ORGINAL_HTTP_HOST: '127.0.0.1:8080' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…' REMOTE_ADDR: '222.28.170.89' REQUEST_METHOD: 'GET' SERVER_NAME: '127.0.0.1' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.0' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0xd888510> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0xd888fd0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0xdcf8650> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'a12a5604dce911e4a05000215e543aa8' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') wsgi process: 'Multithreaded' ------------------------------------------------------------ ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 39438) Traceback (most recent call last): File "/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 1068, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib64/python2.6/SocketServer.py", line 618, in __init__ self.finish() File "/usr/lib64/python2.6/SocketServer.py", line 661, in finish self.wfile.flush() File "/usr/lib64/python2.6/socket.py", line 303, in flush self._sock.sendall(buffer(data, write_offset, buffer_size)) error: [Errno 32] Broken pipe ---------------------------------------- 222.28.170.89 - - [07/Apr/2015:13:47:59 +0800] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=devteam&changeset_revision=de11e1a921c4&latest_changeset_revision=5482a8cd0f36&latest_ctx_rev=10 HTTP/1.0" 500 - "http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag…." "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://222.28.163.110/admin_toolshed/update_to_changeset_revision?tool_shed… File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', line 133 in __call__ return self.handle_request( environ, start_response ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/base.py', line 191 in handle_request body = method( trans, **kwargs ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1954 in update_to_changeset_revision persist=True ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/share/apps/galaxy/Galaxy_Test2015/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get' CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424' HTTP_HOST: '222.28.163.110' HTTP_REFERER: 'http://222.28.163.110/admin_toolshed/browse_repositories?status=done&messag….' HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36' HTTP_X_FORWARDED_FOR: '222.28.170.89' HTTP_X_FORWARDED_HOST: '222.28.163.110' ORGINAL_HTTP_HOST: '127.0.0.1:8080' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…' REMOTE_ADDR: '222.28.170.89' REQUEST_METHOD: 'GET' SERVER_NAME: '127.0.0.1' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.0' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0xd888510> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424'>, 'galaxysession=c6ca0ddb55be603a23b93349895e875da395c1164de6cfc693d5dd79dc3d9d2edcd8d527c7310424') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0xd888fd0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0xdcf8650> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'a58bb404dce911e481c700215e543aa8' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'blast_datatypes'), ('owner', 'devteam'), ('changeset_revision', 'de11e1a921c4'), ('latest_changeset_revision', '5482a8cd0f36'), ('latest_ctx_rev', '10')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=blast_datatypes&owner=de…') wsgi process: 'Multithreaded' ------------------------------------------------------------ Could you please give me some advise to figure out where the problem is and how to fix it? Thanks, Zhaozhen

2 1

a question on data manager and data table
by Beginner TI 07 Apr '15

07 Apr '15

Hi All, I just made a new local instance, installed the fetch_all_fasta data manager, downloaded mm9 fasta. Then I noticed the following: in tool_data_table_conf.xml, it has: <table name="all_fasta" comment_char="#"> <columns>value, dbkey, name, path</columns> <file path="tool-data/all_fasta.loc" /> </table> however in shed_tool_data_table_conf.xml, it has: <table comment_char="#" name="all_fasta"> <columns>value, dbkey, name, path</columns> <file path="/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/ toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cc…" /> <tool_shed_repository><tool_shed>toolshed.g2.bx.psu.edu </tool_shed><repository_name>data_manager_fetch_genome_all_fasta</repository_name><repository_owner>devteam</repository_owner><installed_changeset_revision>cca219f2b212</installed_changeset_revision></tool_shed_repository></table> in tool-data, all_fasta.loc is empty, but in the all_fasta.loc of the shed_tool entry, it shows: mm9 mm9 Mouse July 2007 (NCBI37/mm9) (mm9) /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq/mm9.fa So if I try "Extract Genomic DNA function", I could see the parameter passed to the command line is -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data" which does not host the data. It should be at least -g "/auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/mm9/seq" I thought that data manager will automatically populate these loc files? Am I missing something obvious? I could manually modify tool_data_table_conf.xml to let it point to /auto/rcf-proj/yc1/galaxy-suite/galaxy-dist/tool-data/ toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cc… This would work, but it is ugly. Could someone please give me a hand to fix this? Thanks, Rui

2 1