- galaxy-dev - lists.galaxyproject.org

Can't browse big library anymore (web and API)
by Sajoscha Sauer 29 Jan '13

29 Jan '13

Dear all, We have a really annoying problem with one of our Libraries: we can't access it anymore. We think the problem might be that the library contains too many folders/files (the error message is pasted at the end of the mail). Is there any limit for library content ? Note that we can still browse smaller libraries, so I though that we should simply split the library into smaller ones but since we can t even browse it...Any suggestions how to approach this problem? We are running a galaxy instance used by different research groups and our current setup is the following: - we have one Library per research group - we create a folder per project, under the relevant library, and add all files in these folders. Note that we add symbolic links to the real file locations (this way the large fastq files remain on owners file servers) - the files are transferred automatically to Galaxy using custom code that makes use of the galaxy API. And of course, this automated transfer now crashes as our custom code tries to read library content thru API call (to ask the user in which folder the new files should be added) PS: browsing libraries in general is really slow and gives users a bad 'user experience' (not mentioning all the extra clicks they issue because they think Galaxy did not get the request...), I am not sure why this action is that slow ( browsing small size library is also slow). I think I saw emails about this issue but I am not sure what the status is? Is there anything we could do in the code to speed this up? I am not sure this problem is related to the current post , so sorry for mixing things up. Thanks for your help! Cheers Sajoscha === error on galaxy page (nothing in the galaxy log file)

2 2

workflow Editor
by sshashi＠cdac.in 29 Jan '13

29 Jan '13

Hi, I want to know about workflow editor and how it works internally? what type of canvas editor are you using ? doest it support only python environment ? if you have any documents related with ,can i get it . Regards shashi ------------------------------------------------------------------------------------------------------------------------------- This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. -------------------------------------------------------------------------------------------------------------------------------

3 3

Error trying to add tools from tool shed respository
by Waldron, Michael H 29 Jan '13

29 Jan '13

Users are getting the following error when trying to add a tool from the tool shed repository on our local galaxy instance. "An error occurred. See the error logs for more information. (Turn debug on to display exception reports here)" The galaxy server log shows the message included below. The main error is "ValueError: too many values to unpack ". From a similar previous post, the resolution was to use the latest code from Galaxy central. I just ran an "hg pull -u" pointing to galaxy-central (we had been pointing to galaxy-dist), but we still get the same error. Any ideas? (server log) [29/Jan/2013:08:19:05 -0400] "GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=f19bc86bac946438&changeset_revisions=2d6671b10919 HTTP/1.1" 500 - "http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?reposit…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2" Error - <type 'exceptions.ValueError'>: too many values to unpack URL: https://galaxy.its.unc.edu/admin_toolshed/prepare_for_install?tool_shed_url… File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/nas02/apps/galaxy-prod/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 206 in decorator return func( self, trans, *args, **kwargs ) File '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1201 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Mike Waldron Systems Specialist ITS - Research Computing Center University of North Carolina at Chapel Hill

2 3

Loading a library of bam files
by Kyle Ellrott 28 Jan '13

28 Jan '13

I'm currently in the process of loading (path paste) a large library of BAM files (>10000) into the shared Data Libraries of our local galaxy installation, but I'm finding this process to be very slow. I'm doing a path paste, and not actually copying the files. I have disabled local running of 'upload1', so that it will run on the cluster, and set 'set_metadata_externally' to true. It looks like the job handlers are calling 'samtools index' directly. Looking through the code, that seems to happen in galaxy/datatypes/binary in Bam.dataset_content_needs_grooming, where it calls 'samtools index' and then waits. What would be the most efficient way to start changing the code so that this process can be done by an external script, at a deferred time out on the cluster? Kyle

3 4

Tool shed and tools with dynamically generated select list parameters
by Carlos Borroto 28 Jan '13

28 Jan '13

Hi, I'm trying to get lastz to show a locally cached reference. As I'm used to do I went ahead and copy tool-data/lastz_seqs.loc.sample(still included with galaxy-dist) to tool-data/lastz_seqs.loc and added my local references. Nothing happened. I get: galaxy.tools.parameters.dynamic_options WARNING 2013-01-28 16:42:58,203 Data table named 'lastz_seqs' is required by tool but not configured I realized there are these two files in the lastz repository: ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/lastz/0801f8207d30/lastz/tool_data_table_conf.xml.sample ../shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/lastz/0801f8207d30/lastz/tool-data/lastz_seqs.loc.sample I created their minus '.sample' versions in the same locations and added my local references to this lastz_seqs.loc, still nothing. I finally manually edited the main tool_data_table_conf.xml adding:  <table name="lastz_seqs" comment_char="#"> <columns>value, name, path</columns> <file path="tool-data/lastz_seqs.loc" /> </table> And that did the trick. Is this the expected way of handling these cases?. This paragraph from the wiki didn't clarify much for me: http://wiki.galaxyproject.org/InstallingRepositoriesToGalaxy "Tool shed repositories that contain tools that include dynamically generated select list parameters that refer to an entry in the tool_data_table_conf.xml file must contain a tool_data_table_conf.xml.sample file that contains the required entry for each dynamic parameter. Similarly, any index files (i.e., ~/tool-data/xxx.loc files) to which the tool_data_table_conf.xml file entries refer must be defined in xxx.loc.sample files included in the tool shed repository along with the tools. If any of these tool_data_table_conf.xml entries or any of the required xxx.loc.sample files are missing from the tool shed repository, the tools will not properly load and metadata will not be generated for the repository. This means that the tools cannot be automatically installed into a Galaxy instance." Best regards, Carlos

2 2

error uploading file with fresh installation: this request returned None from get_history()
by Dan Tenenbaum 28 Jan '13

28 Jan '13

Hi, I installed galaxy completely from scratch: rm -rf galaxy-dist && hg clone https://bitbucket.org/galaxy/galaxy-dist/ I'm not a mercurial expert but I think I have changeset 8530:a4113cc1cb5e; at least that's the first entry in "hg log". Then started galaxy like this: cd galaxy-dist ./run.sh I went to http://localhost:8080 and tried to upload a file. I never saw the file in my history, and this is what printed out in the console after I clicked "Execute" on the "Upload File" page: galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,573 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,574 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create galaxy.web.framework DEBUG 2013-01-23 13:56:11,574 Error: this request returned None from get_history(): http://localhost:8080/tool_runner/upload_async_create 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "POST /tool_runner/upload_async_create HTTP/1.1" 500 - "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" Debug at: http://localhost:8080/_debug/view/1358978130 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" 127.0.0.1 - - [23/Jan/2013:13:56:11 -0700] "GET /history HTTP/1.1" 200 - "http://localhost:8080/tool_runner/upload_async_message" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17" Hope someone can help. Thanks, Dan

2 13

Galaxy with EMBOSS 5 or EMBOSS 6 ?
by Guest, Simon 28 Jan '13

28 Jan '13

Dear Galaxians, I am troubleshooting a problem with running EMBOSS infoseq on our local galaxy installation. We are running EMBOSS 6.5. There is a mismatch between the options Galaxy is passing to infoseq (-version), and what infoseq wants (-seqversion). This breaks the tool. While this would be easy to fix in this instance, I wonder if I should be using EMBOSS 5 (which is *really* old, mind you) instead of EMBOSS 6. I suspect with EMBOSS 6 there may be a whole host of similar changes to be made. Can anyone advise? If it turns out that EMBOSS 5 is actually required, perhaps someone could update the relevant info on the Galaxy dependencies page, or at least point me to the place where it says this. cheers, Simon Simon Guest Senior UNIX Technical Consultant AgResearch ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

2 3

2013 Galaxy Community Conference (GCC2013) Training Day: Please vote!
by Dave Clements 28 Jan '13

28 Jan '13

Hello all, A list of possible topics <http://bit.ly/gcc2013td> for the GCC2013 Training Day <http://bit.ly/gcc2013td> is now available at http://bit.ly/gcc2013td. Please take a few minutes to review these possibilities <http://bit.ly/gcc2013td> and *then vote for your three favorite topics* at http://bit.ly/gcc2013tdpoll* Your votes will determine not only the topics that are offered, but also which topics should be offered more than once, assigned to the larger rooms, and which ones should not be scheduled at the same time. Your vote matters. Once voting closes on 11 February, we will contact possible instructors and publish a Training Day schedule before registration opens on 22 February. *About the GCC2013 Training Day* The 2013 Galaxy Community Conference (GCC2013)<http://galaxyproject.org/GCC2013> will start on 30 June with a Training Day <http://bit.ly/gcc2013td> featuring 4 parallel tracks, each with three workshops, each of which are two hours long, for a total of twelve sessions. There will be at least one complete track (3 consecutive workshops) about using Galaxy for biological research, and at least one full track on deploying and managing Galaxy instances. As always, please let us know if you have any questions. Thanks, and hope to see you in Oslo! Dave Clements, on behalf of the GCC2013 Organizing Committee * Note: You don't have to vote for exactly three topics. You can vote for fewer topics and have each vote count for more, or you can vote for more topics and have each vote count for less. Links: http://bit.ly/gcc2013td - Training Day possible topics list http://bit.ly/gcc2013tdpoll - Vote! http://galaxyproject.org/GCC2013 - Conference home page http://bit.ly/gcc2013prom - help get the word out -- <http://galaxyproject.org/wiki/GCC2012>http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0

Pull request for bowtie_wrappers tool
by Nicola Soranzo 28 Jan '13

28 Jan '13

Dear Galaxy DevTeam, I have made a few updates to the bowtie_wrappers tool from the Tool Shed, may someone please pull from: https://bitbucket.org/nsoranzo/bowtie_wrappers/ for the following changes: - Add fields for --nofw and --norc options also for single-end reads - Remove field for --cutoff option of bowtie-build - Do not show fields for --best and --strata options for paired-end reads - Add control to choose between -a and -k options - Show fields for --pairtries and --maxbts options only when noTryHard - Add control to choose between Maq- and SOAP-like alignment - Add range validators to all integer parameters I am not sure this mailing list is the correct place for such pull requests, if not please advise me! Thanks, Nicola -- Nicola Soranzo, Ph.D. CRS4 Bioinformatics Program Loc. Piscina Manna 09010 Pula (CA), Italy http://www.bioinformatica.crs4.it/

1 0

Uploading large files as Admin Still slow!
by Jackie Lighten 28 Jan '13

28 Jan '13

Hi, I am trying to upload 2x 2.5GB files to my data libraries (in Admin on a local instance) but it has so far been running over night and not completed. How can I speed this up? Can I upload compressed files? Can I not just copy these files directly in to $galaxy-dist/database/files/000 ? Thanks, Jack

2 3