September 2010 - galaxy-dev - lists.galaxyproject.org

Composite data types as output
by Andreu Alibés 14 Sep '10

14 Sep '10

Hi, I have a script that outputs around 20 files with fixed names. Ideally I would like to have as output a page with links to these 20 files. From the wiki I understand that Composite Datatypes should be the way to go, somehow putting all the output files in a folder. I've tried to use the AutoPrimaryComposite example but I don't manage to make it work. - How do I register the new data type? - How do I tell the data type to look for the output files in the newly created folder? Thanks, Andreu -- -------------------------------------------- *Andreu Alibés, PhD **Bioinformatics Core & EMBL-CRG Systems Biology Unit Center for Genomic Regulation* C/ Dr. Aiguader 88, 08003 Barcelona, Spain Phone: +34 93 316 0202 http://sites.google.com/site/aalibes

2 1

Looking for ideas to download multiple files
by Davide Cittaro 13 Sep '10

13 Sep '10

Hi all, I have some skilled end users that would like to download multiple files from our local galaxy data library with command line utilities (i.e. they would like to "wget" some files from histories). One way to do this is to copy the item link within the history and paste as wget argument. Unfortunately, this has to be done for every item. Another way is to select all files from the Data Library and download as archive (but you can't copy the link...) and it takes so much time when you have to download BAM files (or any other huge one)... I was looking at the source code wondering how difficult would be to add a functionality for the data library that is: For selected items "get the public link" that saves a text file with all the public links (or copies them to pasteboard, or simply displays them in another window....) so that one can use that info to script the download somewhere else (and at any time)... Any suggestion is welcome! :-) d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

3 4

Workflows are stuck and jobs are not being run.
by Brian Connolly 10 Sep '10

10 Sep '10

Hello, Jobs on our galaxy server are not being run. When a new workflow is run, only a single job in the workflow is executed. All later jobs are not run. If the server is restarted, then the 2nd job in the workflow will be executed, but all later jobs will not be run. I am logging at the debug level and see the following error message repeated many times in the logs UnboundExecutionError: Instance <Job at 0x5203290> is not bound to a Session; attribute refresh operation cannot proceed galaxy.jobs ERROR 2010-09-09 13:36:03,985 Exception in monitor_step Traceback (most recent call last): File "/labkey/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 138, in __monitor self.__monitor_step() File "/labkey/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 212, in __monitor_step log.exception( "failure running job %d" % job.id ) File "/labkey/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/attributes.py", line 158, in __get__ return self.impl.get(instance_state(instance), instance_dict(instance)) File "/labkey/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/attributes.py", line 377, in get value = callable_() File "/labkey/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/state.py", line 185, in __call__ attr.impl.key in unmodified File "/labkey/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1863, in _load_scalar_attributes "attribute refresh operation cannot proceed" % (state_str(state))) I have not been able to successfully completely run a workflow yet on this server (this is a new install) Specifics about the server are - Galaxy version: changeset: 4246:f09915c8da94 - Operating System: Ubuntu 10.04 - Python version: 2.6.5 - Database: postgres 8.4 Has anyone seen an error like this before. I have searched the mailing lists and the wiki and could not find a reference. Thank you in advance for the assistance. Brian Connolly

2 4

Local galaxy installs - merging upstream changes
by Peter 09 Sep '10

09 Sep '10

Hi all, I've just started looking at installing a local copy of Galaxy, and have registered with the mailing list. First one quick question - is there an easy way to search the Galaxy wiki (or documentation in general)? Perhaps my questions are already discussed if I knew where to look... I understand that to install Galaxy locally we just clone the mercurial repository (either galaxy-central or galaxy-dist) and then use hg to periodically update this to the latest code. i.e. There are no formal releases with a version number that come as a tar ball or similar. Naturally, once I have a copy of Galaxy, I will be making some changes to it - in particular configuration changes, for example to use PostgreSQL. Is the the workflow described here the recommend approach? http://bytebucket.org/galaxy/galaxy-central/wiki/DevConf2010/galaxy_devconf… (I found this presentation via a Google search) If I understand it, the idea is that we rename tool_conf.xml.sample to tool_conf.xml (and remove it from the .hgignore file) so that any changes made to the sample configuration will be merged into our local configuration? Alternatively, we could just take a copy and leave tool_conf.xml in the ignore file. We would then have to manually apply any changes made to the tool_conf.xml.sample on a case by case basis. Also do you recommend I make my own changes on a branch? Thanks, Peter

4 11

September 8, 2010 Galaxy Development News Brief
by Jennifer Jackson 08 Sep '10

08 Sep '10

September 8, 2010 Galaxy Development News Brief http://bitbucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2010_… Here are the highlights of what you will get if you perform the following upgrade: hg pull -u -r f09915c8da94 What's New New Multiple Sample Tracking Features * UI enhancements: checkboxes to select samples & select field to specify operations. * State change mechanism added for multiple samples and removed for individual samples in the sample_events page. * Enhanced sample searching implemented. Uses sample name, barcode and dataset name with menu items in masthead and admin menu. o Example of usage: new wild-card searches. New Custom Sorting in Grid Framework * Enhance grid framework to enable custom column sorting (default sorting unchanged). * Sort criteria are now mapped to a column, and the column defines the sorting to be done on the grid query. o Examples of usage: (a) case-insensitive sorting of text fields and (b) case-insensitive sorting for user names in published-item grids. New Data Format Conversion Tools * NGS: SAM Tools => BAM-to-SAM converts BAM format to SAM format * Convert Formats => Wig to bigWig converter New GFF Format Filtering Tools * Filter and Sort => GFF => Extract features from GFF file, Filter GFF file by attribute using simple expressions, Filter GFF file by feature count using simple expressions. * New Filter GFF file by feature count filters a GFF file using conditions based on transcripts' features counts. o Example of usage: filter for transcripts that have a minimum number of exons or transcripts that have 3' UTRs. Updated & Improved Data Libraries * Added support for the AddressField, CheckboxField, SelectField and WorkflowField form field types (in addition to the previously supported TextArea and TextField) to data library templates. * Inherited template fields now also inherit any field contents from the immediate "parent" of the current item (folder or dataset). * The information in our data library templates wiki at http:bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTemplates has been updated to reflect the new features and behavior. Sample Tracking: Email notifications * Notification of specific sample state changes. * Changes made to barcode scan AMQP handler to update sample state to also send email when required. * Notification to multiple recipients. * Request events history records notifications. * UI enhancements: o "edit request" page to allow users to change notification configurations. o "request" page to show the request type state, user & notification details. Jobs & related * Improved handling of metadata to optimize job processing. Includes a new user prompt to set metadata manually or retry auto-detection externally when using 'set_metadata_externally = True' . * Enhance wig datatype to only use first 100 data lines to determine column type metadata if dataset filesize exceeds max_optional_metadata_filesize as configured in datatypes_conf.xml. * Add 'NullToolOutputActionOption' to tool output dataset actions. This action option is used implicitly when no ToolOutputActionOption(s) have been defined. o Example of usage: a metadata value can be set to a static value on a per Tool Output basis. Installation and Management * Added support for installing Galaxy on systems without direct access to the Internet. Updated Galaxy Installation wiki (see section "How to get eggs: If your Galaxy server does not have Internet access" http:bitbucket.org/galaxy/galaxy-central/wiki/Config/Eggs. * When viewing library dataset information as an administrator, add a field that displays the path to the actual disk file that underlies the dataset. Trackster * Each track now has an Overview checkbox that will set the initial chromosome-wide view. * Visualizations that users own or are shared with can now be cloned. * New go-to feature allows users to redirect quickly to a specified chromosome or position. o Example of usage: "chr1:1-100,000,000". * Drag-zoom activated when a chromosomal position is clicked on. * Line tracks display mode can also now be activated inline. * Changing display mode from "Auto" now fetches additional detail data and displays features. Displays error messages in canvas elements if present. This also limits the number of features displayed per tile. General Improvements * Slight makeover to Galaxy tabs: (a) added published item links to 'Shared Data' tab; (b) standardized personal item lists as 'Saved XXXs'; and (c) added defaults action for Visualization tab. * Minor grid refactoring. * Don't re-sort BAM files on upload if already sorted (adapted from Brad Chapman patch). * Use image_path for Workflows and now display modal instead of relative paths. -- Galaxy Project Team http:usegalaxy.org http:bitbucket.org/galaxy/galaxy-central This project is supported in part by NSF, NHGRI, and the Huck Institutes of the Life Sciences, and The Institute for CyberScience at Penn State.

1 0

Annotation not retained after importing histories
by Davide Cittaro 08 Sep '10

08 Sep '10

Hi all, I'm trying to share some histories with other users. I've added annotations to items in histories. When users import my histories, the annotation field is blank... d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

2 3

stupid logging question
by Michael Pheasant 07 Sep '10

07 Sep '10

Using the run.sh script, and the default 'log_level = DEBUG' universe_wsgi.ini file setting gives a huge amount of output. Changing the ini to either log_level = INFO, WARNING or ERROR and i get the same output, including the INFO messages every second. I assumed that WARNING or ERROR would not give the INFO messages; is there some way to not see these? Cheers m -- Michael Pheasant Software Engineer Queensland Facility for Advanced Bioinformatics Level 6, QBP University of Queensland, QLD 4072 T: +61 (0)7 3346 2070 F: +61 (0)7 3346 2101 www.qfab.org

4 8

GMOD Europe 2010: Less than a week to go
by Dave Clements, GMOD Help Desk 07 Sep '10

07 Sep '10

Hello all, GMOD Europe 2010 (http://gmod.org/wiki/GMOD_Europe_2010) starts on Monday, 13 September, less than a week from now. This event runs for 4 days and includes: * September 2010 GMOD Meeting - has openings * GMOD Satellite Meetings - has openings * InterMine Workshop - has openings * BioMart Workshop - full (but you can get on the short waiting list) If you haven't already registered, please do so as soon as possible. This will assure you a seat and help us plan meeting logistics. See http://gmod.org/wiki/GMOD_Europe_2010 for details and a registration link. This is the last broadcast reminder I'll send out. Please let me know if you have any questions. Thanks, Dave C. -- http://gmod.org/wiki/GMOD_News http://gmod.org/wiki/GMOD_Europe_2010 http://hinv.jp/biocuration2010/ http://gmod.org/wiki/Help_Desk_Feedback

1 0

Data libraries permissions oddity -- probably user error
by Ry4an Brase 07 Sep '10

07 Sep '10

I've been setting up data libraries for researchers with their data as provided on USB drives, and the workflow is great, except that I'm doing something wrong with data library permissions and can't figure out just what. >From the admin interface I'll select "Manage data libraries" from the left bar, pick a library, and then from "library actions" select "edit permissions". On that "Manage library permissions on library xxxxxx" screen the four associated/not-associated sections have only the role that I initially associated with the data library on import, and it's always the sole item in associated, and 'not-associated' is empty. How do I add other roles to associated if not-associated is empty, and more importantly how do I let the researcher who has 'manage library permissions' pick new ones when "roles not associated" is empty for them too"? The workaround I've found is to remove every role from "roles associated" in "access library", which briefly makes every role in the system show in "roles not associated", and then I can multi-select and re-add the removed roles and the new ones. I gather from Greg's write-ups that the right usage includes smarter use of groups and non-user roles, but is it really the case that I can't associate new roles to a library's "access library" right w/o first removing all those already there? Also, is there a way for researchers to be able to add users to their group/non-user-role without and administrator having to do it for them? Thanks, -- Ry4an Brase 612-626-6575 University of Minnesota Supercomputing Institute for Advanced Computational Research http://www.msi.umn.edu

2 1

integrating genome view as a display application
by Marijn Vandevoorde 07 Sep '10

07 Sep '10

Hi all, I'm trying to integrate genomeview as a display application. The configuration of the display applications seems to be pretty straightforward, but genomeview needs multiple input files. I can of course write a cgi file that will take a bunch of get parameters and use these as input files for the java applet. so on the side of genomeview, there's not really a big problem. however, to integrate it with galaxy, i don't see how to send multiple datasets to genomeview. you can only add the "view with genomeview" link to one dataset (or to all, but all will be viewed seperately). so i came up with the idea of a new datatype, .gvl, the genomeview list. in it, basically, a list of links to the datasets, so galaxy can work with them. i could make a tool, gvl creator, to take several datasets as input and then create a really simple textfile containing one url per line, for each dataset. i think this is the way to go (if anyone has better suggestions, please tell!) but now come the real questions: most important: how do i get the public url of my datasets? all other tools use the filesystem path, can't really find an example. and secondly: any idea how i could maybe take a dynamic number of inputs? i suppose that's not possible. this is not that big a problem. i could combine multiple gvl creators and then a gvl merger or something, no biggie! all help really appreciated. and of course i'll be happy to share everything with whoever is interested afterwards :-) with kind regards, marijn -- Marijn Vandevoorde Tel:+32 (0)9 331 36 95 fax:+32 (0)9 3313809 VIB Department of Plant Systems Biology, Ghent University Technologiepark 927, 9052 Gent, BELGIUM marijn.vandevoorde(a)psb.vib-ugent.be http://www.psb.vib-ugent.be ================================================================== "If you type google into google, you can break the internet" --Jen Barber

1 1