November 2012 - galaxy-dev - lists.galaxyproject.org

Dynamic job runner status? Could I use it in combination with job splitting?
by Carlos Borroto 05 Nov '12

05 Nov '12

Hi, I've been researching the possibility of using dynamic job runner in combination with job splitting for blast jobs. My main interest is to create a rule where both the size of the query and the database are taken into consideration in order to select DRMAA and splitting options. My first question is what is the status on dynamic job runner? I found these two threads, but is not clear to me if this feature is part of galaxy-dist already: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-October/007160.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010080.html My second question is if there is any documentation other than the threads above to configure something like what I describe? In any case, there is very good information from John in these emails and I think that should get me started at least. Cheers, Carlos

2 7

Re: [galaxy-dev] Where should I put a tool's required binaries?
by Joel Rosenberg 05 Nov '12

05 Nov '12

Sorry for bumping this, but it seems like such a standard boilerplate step of installing a tool that someone muts know how. -Joel > > I installed the?bedtools tool from the public toolshed to my galaxy cluster. The jobs were failing because I hadn't installed the required binaries: > > ? ?An error occurred running this job: /opt/sge/default/spool/execd/ip-10-194-50-118/job_scripts/2: line 13: genomeCoverageBed: command not found > > Taking a closer look at the toolshed's page, I noticed there were a handful of required binaries (genomeCoverageBed,?intersectBed, etc). > > So I downloaded BEDTools and compiled all the binaries. I've put them in well organized directories using symlinks to specific versions, etc. The final executable directory containing symlinks to these binaries is "/mnt/galaxyTools/shed_tool_binaries/bin/". > > I can make a snapshot of the tools volume so that these compiled binaries are always available to me when I bring up my cluster, but how do I integrate them into the PATH in a way that lets the bedtools galaxy tool see them, but also survive cluster shutdowns? > > Thanks, > > -Joel >

3 5

aaChanges
by Scooter Willis 05 Nov '12

05 Nov '12

Trying to work with aaChanges under Human Genome Variation using an Amazon EC2 cloudman launched instance. The output is generated by Filter Pileup and has HG19 associated for the Database/build. The gene set annotations imported from USCS also have HG19. I get the following errors when selecting input data set and appears to be the same problem reported 2/2011(http://user.list.galaxyproject.org/Problems-using-aaChanges-tool-td4… Also trying to sort through what looks to be an issue with using pileup for SNP calling in the 10 column format. Column 3 is the reference base and column 4 is the consensus base. I have numerous examples where reference base = consensus base with a breakout of the number of A T C G represented as a count. For aaChanges I need an input line for each non-zero count of A T C G from the pileup report to test which of those SNPs would actually result in an amino acid change. Doesn't seem to be an easy thing to do in Galaxy. Any recommendations on how others are doing SNP calling from exome data and detecting amino acid changes? Does anyone have a published workflow they would recommend or could share? [cid:197927B4-3E74-49CB-B585-E6FA6A0E8CC4]

1 0

phylotastic hackathon (Jan 28 - Feb 1, Tucson, AZ) of possible interest to galaxy developers
by Arlin Stoltzfus 05 Nov '12

05 Nov '12

Dear galaxy developers-- I'd like to bring your attention to the call for participation included below this message. A "Phylotastic" system is a distributed tree-of-life delivery system composed of web services. Although it is not readily apparent from the presentation linked below, the project includes some work on using Galaxy as a framework for managing a phylotastic workflow. See the demos page (http://phylotastic.org/demos.html) and the "Making Galaxy Phylotastic!" screencast (http://youtu.be/kMME658xOu4 for more. I would be happy to respond to questions about the project. Please consider applying if you are excited by the project and think that you can contribute. Regards, Arlin ----- Phylogenies! Hacking! Tucson in January! An implicit promise of the Tree of Life project is that, ultimately, expert knowledge of species phylogeny will be accessible and usable by everybody. In other words, we will all be able to get the species trees we need, in a useable form, when we need them. The Phylotastic project aims to make that vision a reality, by developing a loosely coupled system of components that, in response to a user’s query (a list of taxa plus conditions), will rectify names, find suitable source trees, prune and graft to get the right species, estimate branch lengths, and return the results with metadata on sources and methods. In June 2012, the HIP (Hackathons, Interoperability, Phylogenies) working group of NESCent staged a hackathon (http://en.wikipedia.org/wiki/Hackathon) to prototype components of a Phylotastic system, and implement demos to show their potential. Thanks to 25 participants who responded to an open call (just like this one), that hackathon was an extraordinary success (see http://phylotastic.org/) Now we are gearing up for a second hackathon, to take another step toward providing computable, convenient, credible access to the Tree of Life. We aim to recruit a diverse group of participants, including students, faculty, postdocs, and staff of both genders and from different backgrounds. Only some of the seats at the hackathon will be filled by expert hackers who spend their time coding. We also need folks who can: * Provide knowledge of workflows and downstream uses of trees * Design user interfaces * Test software * Make screencasts, develop tutorials, and document user experiences Applications are now being accepted to Phylotastic 2, which will take place January 28th, 2013 to February 1st, 2013 at the BIO5 facility in warm and sunny Tucson, AZ (http://bio5.arizona.edu/about-bio5) Information on how to apply is provided below. Partial support (and, if funds allow, full support) for travel, food and lodging will be available to each successful applicant who indicates need. We particularly welcome applications from women and members of underrepresented groups. To understand how you might be able to contribute, review the slideshow (ppt format, http://bit.ly/RWRgIc, or PDF format, http://bit.ly/POaoci) consult the web site (phylotastic.org) or contact a member of the HIP leadership team (listed below). Sincerely, the HIP (Hackathons, Interoperability and Phylogenies) Leadership Team HOW TO APPLY Before 16 November, 2012 at midnight, EST, fill out the online form here http://tinyurl.com/PhyloTastic2. Note that: * The application form asks you to describe how you can contribute to Phylotastic. Remember that we are not just looking for expert coders. There are many other ways to contribute. * Support for travel, housing and meals will be available to each successful applicant who indicates need. * Please understand that your application is not a guarantee of participation in the event. Funds and space are limited: we may not be able to sponsor every individual who is qualified to participate. LEADERSHIP TEAM Karen Cranston, karen.cranston(a)nescent.org Brian Sidlauskas, brian.sidlauskas(a)oregonstate.edu Arlin Stoltzfus, arlin(a)umd.edu Mike Rosenberg mrosenb(a)asu.edu Brian O’Meara, bomeara(a)utk.edu Mark Westneat mwestneat(a)fieldmuseum.org Enrico Pontelli epontell(a)cs.nmsu.edu Rutger Vos rutgeraldo(a)gmail.com Naim Matasci nmatasci(a)iplantcollaborative.org ------- Arlin Stoltzfus (arlin(a)umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD, 20850 tel: 240 314 6208; web: www.molevol.org

2 1

Mapping tools
by Lee Hunt 05 Nov '12

05 Nov '12

Hi I'm a new Galaxy user and last week managed to map some sequences, but since Thursday the jobs won't start, even when I try re-running the one ones I did last week Lee --

1 0

Re: [galaxy-dev] Looking for recommendations: How to run galaxy workflows in batch
by Sascha Kastens 03 Nov '12

03 Nov '12

Hi Dev-Team, are you planning or maybe working on an update which enables the possiblilty to run workflows in batch mode with paired end data? Cheers, Sascha

3 3

Re: [galaxy-dev] Dataset upload fail
by Hans-Rudolf Hotz 02 Nov '12

02 Nov '12

On 11/02/2012 02:20 PM, Vladimir Yamshchikov wrote: > Someone has advised also that SQLite, which is the default option, > is not adequate for serious work. Is this something I need to > change too? Indeed, see: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server > I mean could this prevent dataset upload? I tried to intall > PostgreSQL, but it is quite messy and instructions are far > from beng adequate or even accurate. Although, using SQLite has many disadvantages, I doubt, it has a direct influence on the actual upload process. Well, it has been a while since I last installed PostgreSQL from scratch, but I never ran into troubles. And since PostgreSQL is an Open Source Project as well, there is a lot of support available, see: http://www.postgresql.org/community/ Regards, Hans-Rudolf > > Vladimir > > ----- Original Message ----- > From: "Hans-Rudolf Hotz" <hrh(a)fmi.ch> > To: "Vladimir Yamshchikov" <yaximik(a)gmail.com> > Cc: <galaxy-dev(a)lists.bx.psu.edu> > Sent: Thursday, November 01, 2012 3:18 AM > Subject: Re: [galaxy-dev] Dataset upload fail > > >> Hi Vladimir >> >> On 11/01/2012 03:46 AM, Vladimir Yamshchikov wrote: >>> Local install of Galaxy on SciLinux55. Fails to upload 5.2 GB fastq file >>> from local HD, while normally loading smaller fastq and fasta datasets >>> (less than 1 GB). Chunks of 1.2 GB size remain in */database/tmp, which >>> all represent beginning of the file that fails to upload. Several >>> attempts to upload made and several chunks of the same size are present. >>> Can I just copy the file dataset to database directory instead of >>> uploading through web interface? >>> >> >> >> Make sure you provide a directory for "library_import_dir" in the >> universe_wsgi.ini file. And copy your files to this location (or a sub >> directory of it). This will give you an extra option in the admin menu: >> 'Upload directory of files' and then use the 'Link files without copying >> to Galaxy' option. >> >> >> Alternatively, set "allow_library_path_paste" to 'True' (see also the >> comments and warnings in the universe_wsgi.ini file ) >> >> >> Regards, Hans-Rudolf >> >> >> >> >> >> >> >> >> >

1 0

Filter by quality error
by Scooter Willis 01 Nov '12

01 Nov '12

I have a new workflow setup and filter by quality on an input fastqsanger file. I get the following error message where the original elements worked correctly. The IP address is a compute node. I have one compute node running with autoscaling on. Any reason why the mnt is working? /opt/sge/default/spool/execd/ip-10-140-4-84/job_scripts/39: line 13: /mnt/galaxyTools/tools/fastx_toolkit/0.0.13/env.sh: No such file or directory /opt/sge/default/spool/execd/ip-10-140-4-84/job_scripts/39: line 13: cd: /mnt/galaxyTools/galaxy-central: No such file or directory /opt/sge/default/spool/execd/ip-10-140-4-84/job_scripts/39: line 13: /mnt/galaxyTools/galaxy-central/set_metadata.sh: No such file or directory

1 0

JBrowse direct export to Galaxy
by Erik Derohanian 01 Nov '12

01 Nov '12

Hello, I am a JBrowse Dev hoping to add the ability to export data directly from JBrowse (JavaScript) to Galaxy (without exporting from JBrowse, saving locally, opening Galaxy, loading file from drive). I have looked through the wiki page on Data Sources<http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources>, but JBrowse is client-side only. I was wondering if anybody knew of a different way to do this, or had any pointers. Thanks, Erik Derohanian

4 6

Where should I put a tool's required binaries?
by Joel Rosenberg 01 Nov '12

01 Nov '12

I installed the bedtools tool from the public toolshed to my galaxy cluster. The jobs were failing because I hadn't installed the required binaries: An error occurred running this job: /opt/sge/default/spool/execd/ip-10-194-50-118/job_scripts/2: line 13: genomeCoverageBed: command not found Taking a closer look at the toolshed's page, I noticed there were a handful of required binaries (genomeCoverageBed, intersectBed, etc). So I downloaded BEDTools and compiled all the binaries. I've put them in well organized directories using symlinks to specific versions, etc. The final executable directory containing symlinks to these binaries is "/mnt/galaxyTools/shed_tool_binaries/bin/". I can make a snapshot of the tools volume so that these compiled binaries are always available to me when I bring up my cluster, but how do I integrate them into the PATH in a way that lets the bedtools galaxy tool see them, but also survive cluster shutdowns? Thanks, -Joel

1 0