- galaxy-dev - lists.galaxyproject.org

improved "view details" button
by Assaf Gordon 12 Apr '11

12 Apr '11

Hi, Kudos on the new "View Details" button implementation, both visually nice and less intrusive tooltips. However, you know me - I can't just give praise without complaining... 1. The "View Details" tooltip (the actual tooltip) is not using JQuery, so the yellow (browser native) tooltip is showing (at least in Firefox 3.5.6 and Chrome on Linux). Other buttons show the black JQuery tooltip. 2. The "modified" date (when viewing the details of a dataset) is a bit misleading: It doesn't give the modification date of the dataset (also: what exactly is "modification" ? all datasets in Galaxy are immutable). It seems that the modification date is affected by any modification to the history. Example: - load any existing old history. - click on "view details" for a dataset: "Created" and "Modified" are in the past, or even identical. - rename the history (and don't modify any other parameter/dataset) - click on "view details" for the same dataset: "Created" stays the same. "Modified" becomes today's date. That is confusing, because the dataset was definitely not modified today. -gordon

2 2

update probelm...
by Liisa Koski 11 Apr '11

11 Apr '11

Hi I just tried to update my local installation of Galaxy but accidentally hit Ctr-C when it asked me a question added 285 changesets with 649 changes to 354 files local changed static/scripts/packed/jquery.jstore.js which remote deleted use (c)hanged version or (d)elete? interrupted! When I tried to do the update again...it says 'no changes found' But...after restarting my instance...I see that the changes did not get implemented. Is there a way to force this? Thanks in advance, Liisa

2 1

another update problem
by Ryan Golhar 11 Apr '11

11 Apr '11

I just updated my local galaxy instance using hg pull, and the history items are no longer expandable. What happened? -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information.

3 9

Question about cuffcompare tool implementation
by Assaf Gordon 11 Apr '11

11 Apr '11

Hello, I'm trying to write my own wrappers for Cufflinks,Cuffcompare,Cuffdiff, learning mostly by looking at the existing tools. I have a question regarding the multiple input/output implementation in CuffCompare: In CuffCompare, the XML can accept multiple input files (with a <repeat>), and the <output> section contains the following statements (labels removed for brevity): ========= <outputs> <data format="txt" name="transcripts_accuracy" label="" from_work_dir="cc_output" /> <data format="tabular" name="input1_tmap" label="" from_work_dir="input1.tmap" /> <data format="tabular" name="input1_refmap" label="" from_work_dir="input1.refmap"/> <data format="tabular" name="input2_tmap" label="" from_work_dir="input2.tmap"> <filter>len( input_files ) > 0</filter> </data> <data format="tabular" name="input2_refmap" label="" from_work_dir="input2.refmap"> <filter>len( input_files ) > 0</filter> </data> <data format="tabular" name="transcripts_tracking" label=""> <filter>len( input_files ) > 0</filter> </data> <data format="gtf" name="transcripts_combined" label="" from_work_dir="cc_output.combined.gtf"/> </outputs> ====== Assuming cuffcompare generates a TMAP and REFMAP file for each input file, does this mean the wrapper tool retrieves only the first and second input files' TMAP and REFMAP files ? and others are ignored ? Or is there some galaxy black magic python code that iterates multiple inputs and imports them all ? Thanks, -gordon

2 2

galaxy and Stampy
by Vincent Le Texier 11 Apr '11

11 Apr '11

Hello everyone, I would like to integrate Stampy (http://www.well.ox.ac.uk/project-stampy) in Galaxy. I couldn't find any wrapper available on the website. If someone has written a stampy wrapper and would like to share it, please contact me. Thanks in advance for your help; -- Vincent Le Texier Senior Software Engineer Synergie Lyon Cancer Leon Berard Center, Cheney C 28, rue Laennec 69373 LYON Cedex 08 +33 4 26 55 67 85 letexier(a)lyon.fnclcc.fr

2 1

Recommended Specs for Production System
by Ryan Golhar 11 Apr '11

11 Apr '11

Hi all - So, I been asked to provide specs for a production Galaxy system to support approximately 20-30 users. Most of these users are new to bioinformatics and very new to NGS. I'm targeting a user base that will use a light to moderate amount of NGS data. I've looked at the the Produce Server Wiki page stuff, but I'm curious what everyone else is using or recommends? How big of a compute cluster, how much storage, proxy/web server configurations, etc, etc. If you had to deploy a production system, based on what you know, what would you choose? Ryan -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information.

7 7

2011 Galaxy Community Conference, May 25-26, Lunteren, The Netherlands
by Dave Clements 11 Apr '11

11 Apr '11

Hello all, We are pleased to announce the *2011 Galaxy Community Conference*, being held *May 25-26 in Lunteren, The Netherlands*. The meeting will feature two full days of presentations and discussion on extending Galaxy to use new tools and data sources, deploying Galaxy at your organization, and best practices for using Galaxy to further your own and your community's research. *Link: http://galaxy.psu.edu/gcc2011/ * *Overview *This event aims to engage a broader community of developers, data producers, tool creators, and core facility and other research hub staff to become an active part of the Galaxy community. We'll cover defining resources in the Galaxy framework, increasing their visibility and making them easier to use and integrate with other resources, how to extend Galaxy to use custom data sources and custom tools, and best practices for using Galaxy in your organization. Additional topics include, but are not limited to: * Talks submitted by the Galaxy community * Integration of tools (including NGS analysis tools) and distributed job management * Deployment of Galaxy instances on local resources and on the Cloud * Management of large datasets with the Galaxy Library System * Using the Galaxy LIMS functionality at NGS sequencing facilities * Visualizing Data without leaving Galaxy * Performing reproducible research * Performing and sharing complex analyses with Workflows * An "Introduction to Galaxy" session, offered on May 24, for Galaxy newcomers. *Registration *The conference fee is €100 on or before April 24, and €120 after that. The meeting is being held at the Conference Centre De Werelt in Lunteren, The Netherlands, which is also the conference hotel. You are encouraged to register early, as space at the hotel (and at the "Intro to Galaxy" session) is limited and is likely to fill up before the conference itself does. *Link: http://galaxy.psu.edu/gcc2011/Register.html ** Abstract Submission *Abstracts are now being accepted for short oral presentations. Proposals on any topic of interest to the Galaxy community are welcome and encouraged. The abstract submission deadline is the end of February 28. *Link: http://galaxy.psu.edu/gcc2011/Abstracts.html * *Sponsors *The 2011 Galaxy Community Conference is co-sponsored by the US National Science Foundation (NSF), and the Netherlands Bioinformatics Centre (NBIC). NBIC is a collaborative institute of the bioinformatics groups in the Netherlands. Together, these groups perform cutting-edge research, develop novel tools and support platforms, create an e-science infrastructure and educate the next generations of bioinformaticians. *Links: http://www.nbic.nl/ and http://www.nsf.gov/ * We are looking forward to a great conference and hope to see you in the Netherlands! The Galaxy and NBIC Teams -- http://galaxy.psu.edu/gcc2011/ http://getgalaxy.org http://usegalaxy.org/

2 2

PostgreSQL issue: reserved keyword used as identifier
by Louise-Amélie Schmitt 09 Apr '11

09 Apr '11

Hello everyone I just met a huge problem concerning the database. I'm currently trying to transfer my data from MySQL to PostgreSQL by writing a Perl script that would do the job. Here is the issue: In the "form_definition" table, one of the field identifiers is "desc", which is a reserved SQL keyword used for ordering values. Therefore, There's currently no way of making any query of the type "INSERT INTO table_name (<identifiers list>) VALUES (<values list>);" which is a big handicap in this context, since the order of the identifiers list we dynamically retrieve is not necessarily (and seldom) the right order. Is there a way to fix this issue without blowing everything up? Cheers, L-A

4 6

April 8, 2011 Galaxy Development News Brief
by Jennifer Jackson 09 Apr '11

09 Apr '11

April 8, 2011 Galaxy Development News Brief http://bitbucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011_… Prior releases: https://bitbucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBriefs ------------ 2011_04_08 Highlights Workflow API Move Data Library Items 10 New Genomes & 29 New LiftOver Import Workflows & Histories ClustalW Multiple-Seq Alignments & Sequence Logos Major Trackster Visualization Improvements ------------ How to get this distribution https://bitbucket.org/galaxy/galaxy-dist/ new: % hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist upgrade: % hg pull -u -r 50e249442c5a ------------ Key Upcoming Galaxy Events May 24-26, 2011 Galaxy Community Conference, Lunteren, The Netherlands. http:galaxy.psu.edu/gcc2011/ April 24 is the Early registration deadline (save 20%!) http://galaxy.psu.edu/gcc2011/Register.html April 13-14, NBIC Galaxy Hackathon, Belgium. Please submit your suggestions! http://wiki.nbic.nl/index.php/NBIC_Galaxy_Hackathon_project ------------ What's New Workflow API An API for executing workflows has been added. Usage examples /scripts/api/workflow_execute.py /scripts/api/example_watch_folder.py. Move Data Library Items We have introduced a feature that enables you to move library datasets or folders (including all folder contents) to other locations within the same data library or to a different data library altogether. Galaxy administrators can perform this feature on any data library item and can move items to any data library. Users that are not Galaxy administrators must be given the modify library item permission on an item in order to move it, and they'll need the add library item permission on the desired target data library folder in order for it to be displayed in the select list of targets. Moving a single dataset To move a single dataset, select the Move this dataset option from the dataset's pop-up menu. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… The default behavior is to move items to other locations within the same data library, so you'll initially be presented with a list of valid folders from which to choose for the new target location. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… After selecting the desired folder, clicking the Move button will move the dataset to the selected folder. Moving a Folder Moving a folder is very similar to moving a single dataset - just select the Move this folder option from the folder's pop-up menu. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… To move the folder to a different data library, after selecting the Move this folder option, click the Choose another data library link on the Move data library items page. The folders select list will change to a select list of data libraries to which you are authorized to move the item. When you select a target data library, the select list will change again to display the list of folders in the selected data library to which you are authorized to move the item. Clicking the Move button will move the folder and all of it's contents to the selected target folder within the selected target data library. The target folders select list is filtered to included only valid folders to which you can move the item. For example, you cannot move a folder to one of it's own sub-folders in one step. To do this, the sub-folder must be moved outside of it's parent, and then the parent can be moved to the folder that it previously contained. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… ------------ Updated & Improved Data Content * Genomes o New & included in NGS Tools + Saccharomcyes cerevisiae: Saccharomcyes_cerevisiae_S288C_SGD2010 + Arabidopsis lyrata: Araly1 + Purple Sea Urchin: strPur3 and Spur_v2.6 + Hydra: Hydra_JCVI + Zebrafish: danRer7 + Poplar: Ptrichocarpa_156 + Chimpanzee: panTro3 + Northern White-Cheeked Gibbon: nomLeu1 + Korean Man AK1: Homo_sapiens_AK1 o New & not included in NGS Tools + Caenorhabditis remanei: caeRem2 o Existing genomes added to NGS Tools + hg19 Canonical female (no Y chromosome) + Streptococcus pneumoniae R6: 278 + Drosophila virilis: droVir3 and droVir2 * New LiftOver Files o caeRem2 --> caePb1, caeRem2 --> caeRem3, caeRem2 --> cb3, caeRem2 --> ce4, caeRem2 --> priPac1, calJac3 --> hg18, canFam2 --> monDom5, danRer6 --> danRer7, danRer7 --> fr2, danRer7 --> gasAcu1, danRer7 --> hg19, danRer7 --> mm9, danRer7 --> oryLat2, danRer7 --> panTro3, danRer7 --> tetNig2, danRer7 --> xenTro2, droVir3 --> droVir2, fr2 --> danRer7, gasAcu1 --> danRer7, hg18 --> calJac3, hg19 --> danRer7, mm9 --> danRer7, panTro3 --> danRer7, panTro3 --> hg19, ponAbe2 --> calJac3, ponAbe2 --> monDom5, strPur2 --> ci2, tetNig2 --> danRer7, xenTro2 --> danRer7 * Add Genomes to Your Instance http://bitbucket.org/galaxy/galaxy-central/wiki/NGSLocalSetup * Current Galaxy Main Genomes http://bitbucket.org/galaxy/galaxy-central/wiki/GenomeData Current Tools * Add more verbose error reporting to FASTQ Groomer tool. Provides more information to allow users to determine what is wrong with FASTQ files with invalid format. * Enhance Bowtie wrapper to accept non-Sanger variant FASTQ files. * Allow Upload Tool to function on https URLs. * Add count GFF features tool and tests: o Filter and Sort --> GFF --> Filter GFF file by feature count using simple expressions. o Tool counts the number of features in a GFF file. Note: this is different than the number of lines because a single GFF feature can often span multiple lines. * Tophat v1.2.0 support: o (a) allow indel search. o (b) max insertion and max deletion lengths. o (c) library type. * Updated gff_filter_by_feature_count tool now accepts and correctly handles all GTF, GFF, and GFF3 files. * Changes for detecting and loading BAM data type Samtools version 0.1.13 or newer produces an error condition when attempting to index an unsorted BAM file. To determine if a BAM file is sorted, we first use Samtools to check the headers. If this does not provide a definitive answer and Samtools version 0.1.13 or newer is being used, we index the BAM file to see if it produces the error. This process provides a more robust approach to determining if the BAM file is sorted. New Tools * Multiple Alignments: ClustalW multiple sequence alignment program for DNA or proteins. * Motif Tools: Sequence Logo generator for FASTA data (example: ClustalW alignment). o Both tools originated from the Community Tool Shed (see below). o The Sequence Logo tool uses Weblogo3 wrapped into Galaxy to generate a sequence logo. The input file must be a FASTA file in your current history. It is recommended for viewing multiple-sequence alignment output from the ClustalW tool. Set the ClustalW output to FASTA to create the input for this tool. A typical output looks like this: http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… Community Tools (Tool Shed) * Tuning: Clarified what is being searched in the Tool Shed. * New ClustalW wrapper (see http://www.clustal.org) for protein/dna multiple alignments based on the Galaxy ClustalW wrapper posted by Hans-Rudolf Hotz in an email on the developer list. http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003732.html * New Weblogo3 wrapper (http://weblogo.berkeley.edu) that creates sequence logos from FASTA data such as the output from a ClustalW alignment. Data Libraries * Disabled problematic eager loading on data libraries. Very large data libraries will load two to three times quicker. * Upload Improvement with example walk-through: Uploading data library datasets 1. Apply a corrected version of the patch from Peter Cock that flips the "Preserve directory structure?" setting when uploading library datasets from filesystem paths. The checkbox is now "yes" instead of the original "no", and is checked by default. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… 2. Flip the behavior of the "Copy data into Galaxy" feature when uploading library datasets (similar to 2 above) to be more clear and logical. Instead of a single checkbox, this is now a select list with clearly defined options. The default setting is to copy the files into the configured Galaxy file store. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… 3. Allow importing items from a history to replace a library dataset with a new version. Previously, you could only replace a library dataset with a new version by uploading a single file. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… Workflows * Users can now import copies of their own Workflows and Histories. Trackster * Enhancements: o Implemented a data manager for Trackster and drawing is now completely tracked and controlled. o Put show_insertions and show_differences in read track config. o Filtering now supports: + (a) score columns in BED and GFF. + (b) GTF attributes. o Added BED and GFF support to visual analytics framework, and enable GOPS intersect and subtract tools to work with visual analytics. * Improvements: o Insertions and deletions now shown in reads. o Save and restore mode for feature and read tracks. o Bug fixes for drawing feature tracks in Squish and Dense mode. o Menus and menu items now work correctly. o Remove form from navigation controls so that enter key works properly and sets the chrom/low/high. o Enable full keyboard navigation via arrow keys. * Bug fixes: o Fix issues with navigation input: + (a) arrow keys no longer perform navigation + (b) invalid chromosome names are handled well. o Fix feature track bugs so that intervals are correctly drawn as half-open. o Use gap to correctly position read connector in Pack mode. o Fix read id bug in Trackster BAM data provider. Example of Trackster visualization: Track of reads mapped using Tophat (spliced) including the transcript assembled from those reads. http://bytebucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2011… User Interface (UI) * SGD Yeast genome added as a Gbrowse display site. * IGV added as an external display application. This is not enabled on Galaxy main, but is available for local instances. CloudMan * FTP data upload enabled. http://bitbucket.org/galaxy/galaxy-central/wiki/cloud Source * Improvements in reliability and speed regarding manipulation of instance data volumes. * The value set in 'new_file_path' in universe_wsgi.ini will now be used for creation of all temporary files. This obviates the need for setting $TEMP when running data source tools on the cluster. * Datatypes: o Add a 'subclass' flag to datatype definitions in datatypes_conf.xml that allows dynamic creation of a subclass. o Allow composite datatype datasets to be populated in the Upload tool from files that were uploaded to the Galaxy FTP server. * Tool configuration enhancements: o Allow DataToolParameter to be filtered on attributes other than dbkey (dynamic options). o Allow FromParamToolOutputActionOption and ParamValueToolOutputActionOptionFilter to access attributes of parameter values. Tool Test Framework * Specifying 'ftype="bam"' as a parameter on a test's <output> tag will now cause the test framework to use 'samtools view' to convert the file to SAM. This should make it easier to debug why the test tool output differs from your baseline test output. Test framework enhancements: * Allow toolbox tests to upload a file found located in subdirectories. * Fix a bug occurring on the determination of uploaded dataset name during the handling of the removal of .gzip or .zip extension from the uploaded filenames. * When using re_match comparison method in functional tests and line counts do not match, print out first 40 lines of the history file. Admin Menu * Add a checkbox to the Create Group page that if checked will create a new Role with the same name. This provides a similar feature the the existing checkbox on the Create Role page. * However, the behavior is now changed such that new associations are created when the checkbox is checked, whereas before, only the Group or Role objects with the same name were created, but not associated with anything. Bug Fixes * Following the login link from the Logout page no longer redirects you to the Logout page once you have logged in. * The peek setting method, set_peek(), should now function consistently across datatypes descending from Text, particularly with respect to line count estimation. * Apply patch from Ry4an Brase to correct "NoneType preference on the jobs view" issue. * Don't alter the contents of a file while uploading to a data library if using one of the Upload files from file system paths or Upload a directory of files options in conjunction with the Link to files without copying into Galaxy option. This partially resolves the issue where a supposedly sorted BAM file was being resorted upon upload to a data library when using this option. A better implementation of determining whether a BAM file has been sorted (so that it does not get resorted) remains to be done. * Extract Genomic DNA tool: o Do not fix strand for GFF input. o Handle non-GFF files when interpret features is true. * Set default value for Cuffdiff's minimum alignment count parameter to reflect v0.9.3. * Enhance GFF reader to handle headers and comments and fix bug so that new feature starts when transcript_id or gene_id changes. * Suppress most logging for Cufflinks and Cuffdiff as the logging can generate output that is incompatible with UTF-8 format used by database. * Set message type to error when history deletion fails due to sharing. * Show masthead links to Published Visualizations and Published Pages only if Tracks or Pages are enabled. * Shared visualizations work again. * Fix bug in recently used tool submenu. * Change default value for Tophat coverage search to true. ------------ About Galaxy The Galaxy team is a part of BX at Penn State, and the Biology and Mathematics and Computer Science departments at Emory University. http://www.bx.psu.edu/|BX/ http://www.psu.edu/ http://www.emory.edu/ http://www.biology.emory.edu/ http://www.mathcs.emory.edu/ Galaxy is supported in part by NSF, NHGRI, the Huck Institutes of the Life Sciences, and The Institute for CyberScience at Penn State, and Emory University. http://www.nsf.gov/ http://www.genome.gov/ http://www.huck.psu.edu/ http://www.ics.psu.edu/ http://www.emory.edu/ Source http://bitbucket.org/galaxy/galaxy-central Use Galaxy! http://usegalaxy.org GalaxyProject http://galaxyproject.org Join us at Twitter #usegalaxy http://twitter.com/#!/search/galaxyproject ------------

1 0

reason/advantages for making Galaxy proxy using mod_rewrite instead of mod_proxy
by Leandro Hermida 08 Apr '11

08 Apr '11

Hi all, Sorry if this question has been asked before searched and couldn't find anyone specifically talking about this. In the Galaxy wiki under https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy there are instructions for setting up Galaxy behind a web server proxy. I wondered what are the reasons/advantages of doing a reverse proxy using all mod_rewrite directives (as shown in the instructions) vs doing a standard Apache front-end reverse proxy: ProxyRequests Off ProxyPass / http://galaxy.host.domain:8080/ ProxyPassReverse / http://galaxy.host.domain:8080/ The instructions using mod_rewrite are making a reverse proxy too since the RewriteRule directive has the [P] flag. regards, Leandro

2 3