- galaxy-user - lists.galaxyproject.org

Running workflows from the command line
by Erick Antezana 28 Jun '10

28 Jun '10

Hi, I was wondering about the status of the implementation that could support/enable running wokflows (build using the Galaxy interface) from the command line. cheers, Erick

2 1

Is it possible to upload different files to different directories by Galaxy upload.py script?
by amenda lee 24 Jun '10

24 Jun '10

Hi; Is it poosible to upload files to local different directories by Galaxy upload.py script? It seems to upload all files to the same directory based on the file path in universe_wsgi.ini file. if yes, what part of upload.py needs to be edited? also is it possible to keep name of uploaded file unchanged? Thanks in advance. Amenda Lee

1 0

parallelizing an NGS mapping workflow
by Chris Berthiaume 22 Jun '10

22 Jun '10

Hello, I'd like to use Galaxy on our local beowulf cluster for NGS workflows. One typical use case we'd be replacing with Galaxy is a parallel BWA alignment of large fastq files. To distribute this across the cluster we split the fastq file into many parts, run each separately against the same reference, and then use samtools to merge the SAM output. It's not uncommon to end up with hundreds of parts after splitting. How does Galaxy handle the parallelization of large NGS mappings? I've found the tools for fastq QC, mapping, and SAM merging, but couldn't find any set of tools that would control the parallelization. This trouble ticket (http://bitbucket.org/galaxy/galaxy-central/issue/197/starting-workflows-wit…) would suggest this functionality hasn't been implemented yet, but it seems necessary for many (most?) Illumina or SOLiD runs to get a reasonable mapping turnaround time. If this is already a feature it would be great if I could be pointed to the relevant docs and maybe it could be given a more prominent place in the wiki/interface. If it's not yet a feature, is there a timeline for when it will be added? Thanks, Chris -- Chris Berthiaume Center for Environmental Genomics University of Washington

2 1

Some eggs are out of date (not really)
by Pieter Neerincx 22 Jun '10

22 Jun '10

Hi, I recently installed a new version of Galaxy on a server and now I have the problem that Galaxy keeps complaining some eggs are out of date. However this does not seem to be correct as I already have those eggs. Galaxy will try to fetch them and fails with "Removed conflicting egg...." This happens whenever I start Galaxy and even when the daemon process is not running and I simply check the status. See below for an example. Does anyone on the list know what is causing this and how I can prevent this? Cheers, Pi [root@app-prod-vh02 ~]# /nbic/prog/galaxy/run.sh --status Some eggs are out of date, attempting to fetch... Fetched http://eggs.g2.bx.psu.edu/new/Tempita/Tempita-0.1-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/Tempita-0.1-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/lrucache/lrucache-0.2-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/lrucache-0.2-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/NoseHTML/NoseHTML-0.3.1-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/NoseHTML-0.3.1-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/Beaker/Beaker-1.4-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/Beaker-1.4-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/WebError/WebError-0.8a-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/WebError-0.8a-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/twill/twill-0.9-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/twill-0.9-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/decorator/decorator-3.1.2-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/decorator-3.1.2-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/wsgiref/wsgiref-0.1.2-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/wsgiref-0.1.2-py2.4.egg Fetched http://eggs.g2.bx.psu.edu/new/nose/nose-0.11.1-py2.4.egg Removed conflicting egg: /nbic/prog/galaxy/eggs/nose-0.11.1-py2.4.egg Fetch successful. No PID file paster.pid --------------------------------------------------------------- Biomolecular Mass Spectrometry & Proteomics group Utrecht University Visiting address: H.R. Kruyt building room O607 Padualaan 8 3584 CH Utrecht The Netherlands Mail address: P.O. box 80.082 3508 TB Utrecht The Netherlands phone: +31 6 143 66 783 email: pieter.neerincx(a)gmail.com skype: pieter.online ---------------------------------------------------------------

2 1

June 21, 2010 Galaxy Development News Brief
by Greg Von Kuster 22 Jun '10

22 Jun '10

The latest news brief is available at the following link. You are strongly encouraged to upgrade to the revision detailed in this news brief as it contains important security fixes for data libraries. http://bitbucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2010_… Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Deconvoluting NGS samples with multiple barcodes
by Pip Griffin 22 Jun '10

22 Jun '10

Hi, I have a sequence file that has 454 reads for 64 barcoded samples. I also have a second 'query' which is a file with the names of all 64 samples, and the corresponding 'sample identifier sequence' (19 bp) in the following format: (AGGTTGATTGAATGGCTTA)|(GATGAAGAACGCAGAACCT) (I need to search for the forward or the reverse identifier). I want to 'join' the two queries by searching for a match in the first query with the 'sample identifying sequence' in the second query so that I end up with a copy of the first query with a new column corresponding to the sample names. But the 'join' command only returns perfect matches between columns. How can I join two queries with a partial match? (obviously only 19bp of the total result sequence will overlap with the identifying sequence) I could use the 'manipulate fastq' command, but I would have to do 64 separate steps, as far as I can tell. I would really appreciate any help with what is probably quite a simple problem! thanks very much Pip Griffin

1 0

Server Error
by Rathi Thiagarajan 17 Jun '10

17 Jun '10

Hi I am currently receiving a serve error message (see below): Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) -- Cheers, Rathi

2 1

News briefs for Galaxy distribution updates
by Greg Von Kuster 11 Jun '10

11 Jun '10

Hello Galaxy community, Based on your input at the Galaxy Developers Conference at CSHL last month, we are beginning a new process to improve clarification of the contents of new change sets that are available for upgrading your local Galaxy instances. Our initial approach to accomplish this is to send out a news brief like this one on a regular basis ( about every 2 weeks or so ) to let you know what has been committed since the last news brief. Of course, since htis is the first new brief, it'll be a bit different from those that follow in that we'll just communicate the highlights of the new features / fixes over the past 2 weeks. This process will allow you to decide on which of the following options you'd like to use when updating your local Galaxy instance. 1) Use the same process you've always used ( i.e., update at will, or whenever you see email messages come from the http://lists.bx.psu.edu/listinfo/galaxy-commits distribution list, assuming you have subscribed ) 2) Wait for the next news brief, and decide whether you want to update at that time to the specific change set to which it refers. The news brief will provide instructions for updating to a specific change set. We're treating this process as the beginning of our improved communication to you about what's available in upgrades. Please don't hesitate to provide feedback to help improve this process. So now on to our first news brief! ============================ Here are the highlights of what you will get if you perform the following upgrade - the change set was made available to the distribution on June 8, 2010: hg pull -u -r 837aabec314e ------------------ New features ------------------ 1) From the Admin perspective, the library dataset information page ( the page displayed when you click on a dataset link in a library folder ) now includes all of the metadata associated with the dataset, as well as all active links to the dataset ( the undeleted history or library items that point to this dataset's disk file ). Here is an example of a page that displays this new information. Notice that you can click on the History or Library links to see the history or library that contains the item that points to same disk file. 2) The new and vastly improved version of the Galaxy reports webapp. The Galaxy reports are not supported with the sqlite db as they are intended for Galaxy production environments, but postgres will certainly work, and mysql should work as well. The old version of the reports webapp had it's own template path defined in the reports_webapp.ini file, but the new version uses the normal galaxy template location. Due to this change, you should copy reports_webapp.ini.sample to reports_webapp.ini, and change the following setting to be what you want: port = 9001 host = 127.0.0.1 ... # database_connection = postgres:///galaxy_test?user=postgres&password=postgres To start up the reports: %sh run_reports.sh and point your browse at the URL defined by the host and port setting in your config file. The reports are greatly improved, and use the Galaxy grid framework for displaying jobs. You can use the Advanced Search feature to search for a specific job id, and filter by job state, and the various links on the grids allow for other additional filtering and sorting. The new and improved Disk space maintenance report includes a link on each displayed dataset that (when clicked) will display all of the active history items and library datasets that use that specific dataset's disk file. 3) Galaxy now allows for importing and exporting histories. Options for exporting the current history to a file or importing a history from a file are now listed in the history options menu. 4) Add a jar directory / bin which is accessible to tool configurations by specifying e.g. '${GALAXY_DATA_INDEX_DIR}/shared/jars/FOO.jar', where FOO.jar is the desired jar which has been placed in / linked to 'GALAXY_ROOT/tool-data/shared/jars' (by default; $GALAXY_DATA_INDEX_DIR can be configured to point elsewhere [value of 'tool_data_path' configuration variable in universe_wsgi.ini]). The setup.sh script has been modified to create this directory when not present. ----------------------------------------- Existing feature improvements ----------------------------------------- 1) Improvements in Search+Select boxes. 2) Add support for the GFF format to the Galaxy Operation subtract tool 3) Trackster: - Implemented drag-zooming by letting you drag a region in the top label track (where chrom position is shown). Refactored to not use zoom levels anymore. - Can manually set display modes for tracks. - Add text search to trackster dataset selection grid and fix dbkey filtering. - respect block color for bam 4) When use_remote_user = True, still display the logged in user in the User menu 5) Enhancement to the data libraries grid: display the file size of each library dataset in the grid. 6) Enable form builder to build select inputs and (2) enable users to create visualizations via a button on the saved visualizations page. -------------- New tools -------------- 1) SAM indel filter 2) CBI Rice Mart data source 3) Add tools contributed by Chungoo Park for Principal Component Analysis, Linear Discriminant Analysis, and drawing Receiver Operating Characteristic plots. ------------- Bug fixes ------------- 1) Fix autocomplete select not closing after dragging text inside, and holding arrow keys not triggering multiple times in firefox. ============================ Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Re: [galaxy-user] Upload genome index to Galaxy for Bowtie alignment?
by Kelly Vincent 10 Jun '10

10 Jun '10

Hi, It doesn't actually matter if they are the same, because unfortunately there is no means of uploading index files to Galaxy. There are other reasons, too, but the different platforms issue is one reason we did not set it up so people could upload index files. Regards, Kelly On Jun 9, 2010, at 2:43 PM, Whyte, Jeffrey wrote: > Hello Kelly, > > When I re-read your message, I noticed that you said the index > needed to be built on the same platform that Bowtie will be running > on. > > The swine index that I have was built with Bowtie on a the system > listed below. Is this compatible with the platform that Bowtie on > Galaxy is running on? > > Thanks very much, > Jeff > > Dell 1850 Dual Core Cluster > > $ uname -s -r -v -p -i -o -v > Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:56:28 EST 2006 x86_64 > x86_64 GNU/Linux > > > > ----- > Jeffrey J. Whyte, PhD > Department of Biomedical Sciences > University of Missouri-Columbia > > On Jun 8, 2010, at 2:48 PM, Kelly Vincent wrote: > >> Hello, >> >> Unfortunately, it is not possible for you to upload the actual index >> files to public Galaxy (the indexes usually need to be built on the >> same platform that Bowtie will be running on), so the only option >> when >> the indexes aren't offered built-in is to supply the fasta. However, >> as James just mentioned, we can prepare the swine index files >> ourselves and provide them as a built-in option. It will just take a >> few days to get in place. Could you let me know where I can get the >> reference? If it's not from UCSC (our standard source), it might take >> longer to get it in place. >> >> Thanks, >> Kelly >> >> >> On Tue 8 Jun 2010, at 3:42 PM, Whyte, Jeffrey wrote: >> >>> Thanks so much for the rapid response, Kelly. >>> >>> Just to clarify, if I already have the genome indexed as .ebwt files >>> and upload these, will Galaxy still take longer? We have Bowtie on >>> our Linux cluster, but I like the unified features of Galaxy. I >>> will let you know if any issues come up. >>> >>> Greatly resource! >>> >>> Jeff >>> ----- >>> Jeffrey J. Whyte, PhD >>> Department of Biomedical Sciences >>> University of Missouri-Columbia >>> >>> On Jun 8, 2010, at 2:26 PM, Kelly Vincent wrote: >>> >>>> Jeff, >>>> >>>> Yes, you can upload your swine reference. Any reference file (in >>>> fasta >>>> format) can be uploaded through Get Data. Just be aware that >>>> using a >>>> reference from your history will make Bowtie take longer than >>>> with a >>>> built-in index every time you map reads (because it has to do the >>>> indexing of that file before doing the actual mapping). Let us know >>>> if >>>> you have any issues. >>>> >>>> Regards, >>>> Kelly >>>> >>>> >>>> >>>> On Tue 8 Jun 2010, at 3:19 PM, Whyte, Jeffrey wrote: >>>> >>>>> Hello, >>>>> >>>>> On the Galaxy web interface, the option to map sequences with >>>>> Bowtie >>>>> for Illumina reads is listed under the Tools heading "NGS: >>>>> Mapping". >>>>> >>>>> At the top of the page, it states, "Will you select a reference >>>>> genome from your history or use a built-in index?" >>>>> The options are: "Use a built-in index" or "Use one from the >>>>> history". >>>>> >>>>> I have a genome index (swine) that is not available on the "built- >>>>> in" list. Can I upload this index file to Galaxy under the Get >>>>> Data >>>>> page and then select it under "Use one from the history"? >>>>> >>>>> The index file is 3 GB, so any advice is appreciated before I >>>>> try to >>>>> upload it. >>>>> >>>>> Thanks in advance, >>>>> Jeff >>>>> ----- >>>>> Jeffrey J. Whyte, PhD >>>>> Department of Biomedical Sciences >>>>> University of Missouri-Columbia >>>>> >>>>> >>>>> _______________________________________________ >>>>> galaxy-user mailing list >>>>> galaxy-user(a)lists.bx.psu.edu >>>>> http://lists.bx.psu.edu/listinfo/galaxy-user >>>> >>> >>> >> > >

1 0

Re: [galaxy-user] FASTQ to FASTA tool
by Daniel Blankenberg 10 Jun '10

10 Jun '10

Hi Omri, Thanks for reporting this issue. There were two different FASTQ to FASTA tools loaded in Galaxy. The one located under the 'Convert Formats' tool menu was only designed to handle Solexa formatted FASTQ files, which was different from the tool located under 'NGS: QC and manipulation' --> 'Generic FASTQ manipulation' --> 'FASTQ to FASTA converter' which works on any FASTQ variant. Sorry for this confusion, we've made the tool which works on any variant the default tool in both locations. You can use the tool that is now present in either location to convert your file to FASTA. Unfortunately, in this case, the re-run button will not work and you will have to manually re-select the tool from the left-hand menu. Thanks for using Galaxy, Dan On Jun 9, 2010, at 12:45 PM, Omri Finkel wrote: > Hi Dan, > > Thank you for your response. I clicked the bug link (green on my screen..) and sent out the report. I can share my workflow, but all I have done with this particular file was upload it and then try to convert it. > > Here is the error message: > An error occurred running this job: fastq_to_fasta: Invalid quality score value (char '/' ord 47 quality value -17) on line 12 > > gzip: stdout: Broken pipe > > > Cheers, > > Omri > > On Wed, Jun 9, 2010 at 7:29 PM, Daniel Blankenberg <dan(a)bx.psu.edu> wrote: > Hi Omri, > > > Can you provide an example of how this is not working? If you received a job error (red dataset) can you click on the blue bug link for that dataset and report the error? Or you can share your history with us and we can take a look. > > > Thanks for using Galaxy, > > Dan > > > On Jun 9, 2010, at 6:20 AM, Omri Finkel wrote: > >> Hello, I've been trying to convert a 454 metagenomic dataset I've downloaded from genbank SRA database, from FASTQ to FASTA. >> The operation doesn't seem to work on the Galaxy web interface. >> >> Help? >> >> thanks, >> >> Omri >> >> -- >> Omri Finkel >> The Department of Plant and Environmental Sciences >> The Hebrew University of Jerusalem >> Givat Ram 91904 >> tel: 02-6584862 >> e-mail: omri.finkel(a)mail.huji.ac.il >> >> >> _______________________________________________ >> galaxy-user mailing list >> galaxy-user(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-user > > > > > -- > Omri Finkel > The Department of Plant and Environmental Sciences > The Hebrew University of Jerusalem > Givat Ram 91904 > tel: 02-6584862 > e-mail: omri.finkel(a)mail.huji.ac.il > >

1 0