January 2013 - galaxy-dev - lists.galaxyproject.org

Only custom genomes for uploads in data libraries
by Carlos Borroto 29 Jan '13

29 Jan '13

Hi, I'm running latest galaxy-dist locally. When I try to upload a dataset to a data library through Admin, I only get the two custom genomes I have created for visualizations. If I use the regular upload tool under "Get data", I do get to choose any of the genomes in: tool-data/shared/ucsc/builds.txt I tried deleting the custom genomes and then the dropbox menu in the data libraries upload form is empty. Is there something I'm missing? I would like to be able to select any genome available, custom or not. Thanks, Carlos

2 4

Re: [galaxy-dev] Error trying to add tools from tool shed respository
by Waldron, Michael H 29 Jan '13

29 Jan '13

I had stopped the Galaxy server before running the update, then started it afterward. Mike Waldron Systems Specialist ITS - Research Computing Center University of North Carolina at Chapel Hill ________________________________ From: Greg Von Kuster [greg(a)bx.psu.edu] Sent: Tuesday, January 29, 2013 9:12 AM To: Waldron, Michael H Cc: galaxy-dev(a)bx.psu.edu Subject: Re: [galaxy-dev] Error trying to add tools from tool shed respository Hi Michael, Did you restart your Galaxy server after updating from Galaxy central? Your paster log shows the following: File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1201 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple However, that line no longer exists in the revision to which you've updated, and line 1201 of the admin_toolshed.py file is now: status = kwd.get( 'status', 'done' ) Greg Von Kuster On Jan 29, 2013, at 9:03 AM, Waldron, Michael H wrote: Here is the output from hg heads. [galaxy@galaxy galaxy-dist]$ hg heads changeset: 8677:22788c1262a2 tag: tip user: jeremy goecks <jeremy.goecks(a)emory.edu<mailto:jeremy.goecks@emory.edu>> date: Mon Jan 28 16:36:02 2013 -0500 summary: Language improvements for tophat wrappers. changeset: 7987:966a9b393b85 branch: add_requirement_tags_for_plink parent: 7472:06b31a61793c user: Dannon Baker <dannonbaker(a)me.com<mailto:dannonbaker@me.com>> date: Mon Aug 13 13:23:54 2012 -0400 summary: Branch close changeset: 7152:ffc20445b35e branch: split_blast2 parent: 7058:22f545c0d412 user: Dannon Baker <dannonbaker(a)me.com<mailto:dannonbaker@me.com>> date: Wed Apr 18 07:59:38 2012 -0400 summary: Branch close. changeset: 7151:c94957ac5454 branch: workflow_import_menu_api parent: 6834:1a3d01b09f7a user: Dannon Baker <dannonbaker(a)me.com<mailto:dannonbaker@me.com>> date: Wed Mar 21 09:50:44 2012 -0400 summary: Branch close for pull request. changeset: 7150:314a10ae112d branch: split_blast parent: 6757:ebf65c0b1e26 user: Dannon Baker <dannonbaker(a)me.com<mailto:dannonbaker@me.com>> date: Tue Feb 28 12:32:54 2012 -0500 summary: Close split_blast branch changeset: 7149:1327f2219423 branch: meme_requirement_on_fimo parent: 6662:8ba74c17e809 user: Daniel Blankenberg <dan(a)bx.psu.edu<mailto:dan@bx.psu.edu>> date: Fri Feb 03 11:15:29 2012 -0500 summary: close branch Mike Waldron Systems Specialist ITS - Research Computing Center University of North Carolina at Chapel Hill ________________________________ From: Greg Von Kuster [greg(a)bx.psu.edu] Sent: Tuesday, January 29, 2013 9:00 AM To: Waldron, Michael H Cc: galaxy-dev(a)bx.psu.edu<mailto:galaxy-dev@bx.psu.edu> Subject: Re: [galaxy-dev] Error trying to add tools from tool shed respository Hi Michael, The previous post you mention is outdated. The main Galaxy tool shed from which you are pulling has been tracking Galaxy dist for some time now, so you should only have to update to that. However, the communication layer between Galaxy and the tool shed is backward compatible to the December 20, 2012 Galaxy release, so whether you have updated from Galaxy central or Galaxy dist, you should not be seeing the behavior you describe. If you do, in fact, track Galaxy central though, and use the main Galaxy tool shed, you may experience some strange behavior because the the development code may include new features that are not available on the main tool shed since it tracks Galaxy dist. What is the result of typing the following in your Galaxy installation directory? hg heads Thanks, Greg Von Kuster On Jan 29, 2013, at 8:38 AM, Waldron, Michael H wrote: Users are getting the following error when trying to add a tool from the tool shed repository on our local galaxy instance. "An error occurred. See the error logs for more information. (Turn debug on to display exception reports here)" The galaxy server log shows the message included below. The main error is "ValueError: too many values to unpack ". From a similar previous post, the resolution was to use the latest code from Galaxy central. I just ran an "hg pull -u" pointing to galaxy-central (we had been pointing to galaxy-dist), but we still get the same error. Any ideas? (server log) [29/Jan/2013:08:19:05 -0400] "GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=f19bc86bac946438&changeset_revisions=2d6671b10919 HTTP/1.1" 500 - "http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?reposit…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2" Error - <type 'exceptions.ValueError'>: too many values to unpack URL: https://galaxy.its.unc.edu/admin_toolshed/prepare_for_install?tool_shed_url… File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 206 in decorator return func( self, trans, *args, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1201 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Mike Waldron Systems Specialist ITS - Research Computing Center University of North Carolina at Chapel Hill ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

5 21

Can't browse big library anymore (web and API)
by Sajoscha Sauer 29 Jan '13

29 Jan '13

Dear all, We have a really annoying problem with one of our Libraries: we can't access it anymore. We think the problem might be that the library contains too many folders/files (the error message is pasted at the end of the mail). Is there any limit for library content ? Note that we can still browse smaller libraries, so I though that we should simply split the library into smaller ones but since we can t even browse it...Any suggestions how to approach this problem? We are running a galaxy instance used by different research groups and our current setup is the following: - we have one Library per research group - we create a folder per project, under the relevant library, and add all files in these folders. Note that we add symbolic links to the real file locations (this way the large fastq files remain on owners file servers) - the files are transferred automatically to Galaxy using custom code that makes use of the galaxy API. And of course, this automated transfer now crashes as our custom code tries to read library content thru API call (to ask the user in which folder the new files should be added) PS: browsing libraries in general is really slow and gives users a bad 'user experience' (not mentioning all the extra clicks they issue because they think Galaxy did not get the request...), I am not sure why this action is that slow ( browsing small size library is also slow). I think I saw emails about this issue but I am not sure what the status is? Is there anything we could do in the code to speed this up? I am not sure this problem is related to the current post , so sorry for mixing things up. Thanks for your help! Cheers Sajoscha === error on galaxy page (nothing in the galaxy log file)

2 2

workflow Editor
by sshashi＠cdac.in 29 Jan '13

29 Jan '13

Hi, I want to know about workflow editor and how it works internally? what type of canvas editor are you using ? doest it support only python environment ? if you have any documents related with ,can i get it . Regards shashi ------------------------------------------------------------------------------------------------------------------------------- This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. -------------------------------------------------------------------------------------------------------------------------------

3 3

Error trying to add tools from tool shed respository
by Waldron, Michael H 29 Jan '13

29 Jan '13

Users are getting the following error when trying to add a tool from the tool shed repository on our local galaxy instance. "An error occurred. See the error logs for more information. (Turn debug on to display exception reports here)" The galaxy server log shows the message included below. The main error is "ValueError: too many values to unpack ". From a similar previous post, the resolution was to use the latest code from Galaxy central. I just ran an "hg pull -u" pointing to galaxy-central (we had been pointing to galaxy-dist), but we still get the same error. Any ideas? (server log) [29/Jan/2013:08:19:05 -0400] "GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=f19bc86bac946438&changeset_revisions=2d6671b10919 HTTP/1.1" 500 - "http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?reposit…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2" Error - <type 'exceptions.ValueError'>: too many values to unpack URL: https://galaxy.its.unc.edu/admin_toolshed/prepare_for_install?tool_shed_url… File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 206 in decorator return func( self, trans, *args, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1201 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Mike Waldron Systems Specialist ITS - Research Computing Center University of North Carolina at Chapel Hill

2 3

Loading a library of bam files
by Kyle Ellrott 29 Jan '13

29 Jan '13

I'm currently in the process of loading (path paste) a large library of BAM files (>10000) into the shared Data Libraries of our local galaxy installation, but I'm finding this process to be very slow. I'm doing a path paste, and not actually copying the files. I have disabled local running of 'upload1', so that it will run on the cluster, and set 'set_metadata_externally' to true. It looks like the job handlers are calling 'samtools index' directly. Looking through the code, that seems to happen in galaxy/datatypes/binary in Bam.dataset_content_needs_grooming, where it calls 'samtools index' and then waits. What would be the most efficient way to start changing the code so that this process can be done by an external script, at a deferred time out on the cluster? Kyle

3 4

Tool shed and tools with dynamically generated select list parameters
by Carlos Borroto 29 Jan '13

29 Jan '13

Hi, I'm trying to get lastz to show a locally cached reference. As I'm used to do I went ahead and copy tool-data/lastz_seqs.loc.sample(still included with galaxy-dist) to tool-data/lastz_seqs.loc and added my local references. Nothing happened. I get: galaxy.tools.parameters.dynamic_options WARNING 2013-01-28 16:42:58,203 Data table named 'lastz_seqs' is required by tool but not configured I realized there are these two files in the lastz repository: ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/lastz/0801f8207d30/lastz/tool_data_table_conf.xml.sample ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/lastz/0801f8207d30/lastz/tool-data/lastz_seqs.loc.sample I created their minus '.sample' versions in the same locations and added my local references to this lastz_seqs.loc, still nothing. I finally manually edited the main tool_data_table_conf.xml adding:  <table name="lastz_seqs" comment_char="#"> <columns>value, name, path</columns> <file path="tool-data/lastz_seqs.loc" /> </table> And that did the trick. Is this the expected way of handling these cases?. This paragraph from the wiki didn't clarify much for me: http://wiki.galaxyproject.org/InstallingRepositoriesToGalaxy "Tool shed repositories that contain tools that include dynamically generated select list parameters that refer to an entry in the tool_data_table_conf.xml file must contain a tool_data_table_conf.xml.sample file that contains the required entry for each dynamic parameter. Similarly, any index files (i.e., ~/tool-data/xxx.loc files) to which the tool_data_table_conf.xml file entries refer must be defined in xxx.loc.sample files included in the tool shed repository along with the tools. If any of these tool_data_table_conf.xml entries or any of the required xxx.loc.sample files are missing from the tool shed repository, the tools will not properly load and metadata will not be generated for the repository. This means that the tools cannot be automatically installed into a Galaxy instance." Best regards, Carlos

2 2

error uploading file with fresh installation: this request returned None from get_history()
by Dan Tenenbaum 28 Jan '13

28 Jan '13

Hi, I installed galaxy completely from scratch: rm -rf galaxy-dist && hg clone https://bitbucket.org/galaxy/galaxy-dist/ I'm not a mercurial expert but I think I have changeset 8530:a4113cc1cb5e; at least that's the first entry in "hg log". Then started galaxy like this: cd galaxy-dist ./run.sh I went to http://localhost:8080 and tried to upload a file. I never saw the file in my history, and this is what printed out in the console after I clicked "Execute" on the "Upload File" page: galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,574 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,574 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "POST /tool_runner/upload_async_create HTTP/1.1" 500 - "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" Debug at: http://localhost:8080/_debug/view/1358978130 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "GET /history HTTP/1.1" 200 - "http://localhost:8080/tool_runner/upload_async_message" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" Hope someone can help. Thanks, Dan

2 13

Galaxy with EMBOSS 5 or EMBOSS 6 ?
by Guest, Simon 28 Jan '13

28 Jan '13

Dear Galaxians, I am troubleshooting a problem with running EMBOSS infoseq on our local galaxy installation. We are running EMBOSS 6.5. There is a mismatch between the options Galaxy is passing to infoseq (-version), and what infoseq wants (-seqversion). This breaks the tool. While this would be easy to fix in this instance, I wonder if I should be using EMBOSS 5 (which is *really* old, mind you) instead of EMBOSS 6. I suspect with EMBOSS 6 there may be a whole host of similar changes to be made. Can anyone advise? If it turns out that EMBOSS 5 is actually required, perhaps someone could update the relevant info on the Galaxy dependencies page, or at least point me to the place where it says this. cheers, Simon Simon Guest Senior UNIX Technical Consultant AgResearch ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

2 3

2013 Galaxy Community Conference (GCC2013) Training Day: Please vote!
by Dave Clements 28 Jan '13

28 Jan '13

Hello all, A list of possible topics <http://bit.ly/gcc2013td> for the GCC2013 Training Day <http://bit.ly/gcc2013td> is now available at http://bit.ly/gcc2013td. Please take a few minutes to review these possibilities <http://bit.ly/gcc2013td> and *then vote for your three favorite topics* at http://bit.ly/gcc2013tdpoll* Your votes will determine not only the topics that are offered, but also which topics should be offered more than once, assigned to the larger rooms, and which ones should not be scheduled at the same time. Your vote matters. Once voting closes on 11 February, we will contact possible instructors and publish a Training Day schedule before registration opens on 22 February. *About the GCC2013 Training Day* The 2013 Galaxy Community Conference (GCC2013)<http://galaxyproject.org/GCC2013> will start on 30 June with a Training Day <http://bit.ly/gcc2013td> featuring 4 parallel tracks, each with three workshops, each of which are two hours long, for a total of twelve sessions. There will be at least one complete track (3 consecutive workshops) about using Galaxy for biological research, and at least one full track on deploying and managing Galaxy instances. As always, please let us know if you have any questions. Thanks, and hope to see you in Oslo! Dave Clements, on behalf of the GCC2013 Organizing Committee * Note: You don't have to vote for exactly three topics. You can vote for fewer topics and have each vote count for more, or you can vote for more topics and have each vote count for less. Links: http://bit.ly/gcc2013td - Training Day possible topics list http://bit.ly/gcc2013tdpoll - Vote! http://galaxyproject.org/GCC2013 - Conference home page http://bit.ly/gcc2013prom - help get the word out -- <http://galaxyproject.org/wiki/GCC2012>http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0