March 2012 - galaxy-dev - lists.galaxyproject.org

debugging jobs in 'new' state
by Shantanu Pavgi 10 May '12

10 May '12

We experienced an issue where some of the galaxy jobs were sitting in the 'new' state for a quite long time. They were not waiting for cluster resources to become available, but haven't been even queued up through DRMAA. We are currently using non-debug mode and following were my observations: * No indication of new jobs in paster.log file * database/pbs script didn't contain any associated job scripts * in backend database - job table contained their galaxy job id but no command_line input was recorded Also, not all the jobs are waiting in the 'new' state. Many jobs submitted after above waiting jobs got completed successfully on the cluster. Is there any job submission logic within galaxy which is being used for submitting jobs? Any clues on how to debug this issue will be really helpful. -- Thanks, Shantanu.

3 4

R scripts for DESeq
by Jennifer Jackson 10 May '12

10 May '12

Hi Lisa, I am going to send your question over to the galaxy-dev(a)bx.psu.edu mailing list so that the development community can offer feedback. http://wiki.g2.bx.psu.edu/Support#Starting_a_technical.2C_local.2BAC8-cloud… Th galaxy-user list is primarily for tool/data usage on the Galaxy public server. http://wiki.g2.bx.psu.edu/Mailing%20Lists Thanks, Jen Galaxy team -------- Original Message -------- Subject: [galaxy-user] R scripts for DESeq Date: Tue, 21 Feb 2012 23:25:35 -0500 From: Liusong Yang <liusongyang2009(a)gmail.com> To: galaxy-user(a)lists.bx.psu.edu Hello, I am planning to put DESeq into galaxy. I am a newcomer for both Galaxy and R. I have already read all of the related discussion in this group. I also noticed that there are r_wrapper.sh and DESeq.xml added into galaxy recently under the path of tools. These lines are from r_wrapper.sh. ### Run R providing the R script in $1 as standard input and passing ### the remaining arguments on the command line I guess this means we need to give the DESeq R script to the wrapper as standard input. My question is where or what is the DESeq R scripts? I installed R and DESeq package, but I can not fund anything like DESeq.R or something similar? I feel so confusing now. Any comments or suggestions would be absolutely appreciated! Thanks Lisa ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

7 11

Migrating Workflows and Other Data to New Galaxy Instance
by Robert Chase 04 May '12

04 May '12

Hello, We are going to migrate to a new instance of the galaxy server. Our current instance contains a number of workflows and other data that we would like to have available on the new server. Is there a way to backup the workflows and then reload them on the new server? Regards, -Robert Paul Chase Channing Lab

2 5

how to get "macs" not "macs14" for Galaxy
by Sergei Manakov 01 May '12

01 May '12

Hello, I am trying to set up MACS tool on local Galaxy. Galaxy comes with it's macs-wrapper.py and macs-wrapper.xml, but it wants to use "macs" not "macs14" executable. I tried to to editing macs-wrapper.py to make it use "macs14" instead, but some options are not the same between the two, and the tool crashes. I would appreciate if someone could give me an advice on where I can get "macs" executable for 64-bit Linux. thanks, Sergei -- Sergei (Siarhei Manakou) Manakov California Institute of Technology +1 626 395 3593

2 3

Stalled upload jobs under "Admin", "Manage jobs"
by Peter Cock 30 Apr '12

30 Apr '12

Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users. e.g. Job ID User Last Update Tool State Command Line Job Runner PID/Cluster ID 2352 xxxx 21 hours ago upload1 upload None None None ... 2339 yyyy 19 hours ago upload1 upload None None None The job numbers are consecutive (2339 to 2352) and reflect a problem for a couple of hours yesterday morning. I believe this was due to the underlying file system being unmounted (without restarting Galaxy), and at the time restarting Galaxy fixed uploading files. Test jobs since then have completed normally - but these zombie jobs remain. Using the "Stop jobs" option does not clear these dead upload jobs. Restarting the Galaxy server does not clear them either. This is our production server and was running galaxy-dist, changeset 5743:720455407d1c - which I have now updated to the current release, 6621:26920e20157f - which makes no difference to these stalled jobs. Does anyone have any insight into what might be wrong, and how to get rid of these zombie tasks? Thanks, Peter

3 5

database engine pool size
by Shantanu Pavgi 18 Apr '12

18 Apr '12

I would like to get some idea about 'database_engine_option_pool_size' and 'database_engine_option_max_overflow' settings. We seem to hit both of these limits whenever a user runs a large workflow with following error: {{{ TimeoutError: QueuePool limit of size 40 overflow 50 reached, connection timed out, timeout 30 }}} These limits are not reached for regular (non-workflow) galaxy jobs. Any help on optimum values for these settings or performance tuning for workflow runs would be appreciated. Version Info: CentOS 5.7, PostgreSQL 8.4.9, Python 2.6.6, Galaxy revision 26920e20157f. -- Thanks, Shantanu

2 1

Hidden output files
by Frank Sørensen 18 Apr '12

18 Apr '12

Hi, I seem to have some problems with the visibility of tool outputs in my workflows. When I flag any given output as visible, Galaxy sometimes decides to hide it in the resulting history anyway. I can't find any systematic behavior though. It seems to be rather arbitrary weather Galaxy decides that the file should be hidden or not. I would estimate the random hiding to occur on about 50% of my outputs. Has anyone seen this before? Regards - Frank -- Frank Sørensen, B.Sc., Programmer Molecular Diagnostic Laboratory (MDL) Molekylær Medicinsk Afdeling (MOMA) Århus Universitetshospital Skejby, Brendstrupgårdsvej, 8200 Århus N Tlf. +45 7845 5363

2 5

Exporting worflow to local instal
by Muehlschlegel, Jochen D.,M.D. 18 Apr '12

18 Apr '12

Hi, I would like to download/export my workflows from the Galaxy main to a local install at my institution. When I use the URL method, I get the following error message: Data at 'http://main.g2.bx.psu.edu/u/muehlsch12/w/imported-tophat---cuffdiff-paired-… ' does not appear to be a Galaxy workflow Message: No JSON object could be decoded: line 1 column 0 (char 0) When I want to download a workflow to my local machine, I get a server error (Duplicate headers received from server). The local install does not recognize publicly published workflows, so that step won't work either. Thanks for your help, Danny The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

2 1

Re: [galaxy-dev] Proteomics category on main toolshed?
by Greg Von Kuster 18 Apr '12

18 Apr '12

Thanks Ira, Just so you know, the Galaxy functional test framework now supports testing installed tools if the repository tools include tests and test datasets are included in the repository in a test-data subdirectory. Here is the relevant section of the tool shed wiki describing this: http://wiki.g2.bx.psu.edu/Tool%20Shed#Using_Galaxy.27s_functional_test_fram… Greg On Mar 29, 2012, at 6:53 PM, Ira Cooke wrote: > Dear Greg, > > Thanks very much for that. I'll do a last proof-read of the tools and upload later on today. > > We're working on some better documentation ... to get it all working takes some extra setup steps on the part of the user. > > Ira > > > On 30/03/2012, at 9:49 AM, Greg Von Kuster wrote: > >> Hello Ira, >> >> I've added the category to the main tool shed. Thanks for the request and your upcoming contributions! >> >> Greg Von Kuster >> >> On Mar 29, 2012, at 5:33 PM, Ira Cooke wrote: >> >>> Dear list, >>> >>> I would like to add a suite of proteomics tools to the toolshed but there is currently no suitable category. >>> >>> Would it be possible to add a category called Proteomics? >>> >>> Thanks >>> Ira >> > >

3 5

Error on local galaxy using SAM-to-BAM tool on a cluster
by Carlos Borroto 12 Apr '12

12 Apr '12

Hi, I'm running into this error: "Error sorting alignments from (/tmp/5800600.1.all.q/tmpXOc5mD/tmpAZCzt_), " When using SAM-to-BAM tool on a locally install Galaxy using a SGE cluster. I'm using the last version of galaxy-dist. I'm guessing I have a problem with the configuration for the tmp folder. I have this on "universe_wsgi.ini": # Temporary files are stored in this directory. new_file_path = /home/cborroto/galaxy_dist/database/tmp But I don't see this directory being used and from the error looks like /tmp in the node is used. I wonder if this is the problem, as I don't know if there is enough space in the local /tmp directory at the nodes? I ran the same tool in a subset of the same SAM file and it ran fine. Also, I see this in the description of the tool: "This tool uses the SAMTools toolkit to produce an indexed BAM file based on a sorted input SAM file." But what I actually need is to sort a SAM file output from bwa, I haven't found any other way than to converting it to BAM. Looking at "sam_to_bam.py" I see the BAM file will also be sorted. Would it be wrong to feed an unsorted SAM file into this tool? Finally, just to be sure there is nothing wrong with the initial SAM file, I ran "samtools view ..." and "samtools sort ..." on this file manually outside of Galaxy and it ran fine. Thanks in advance, Carlos

4 5