November 2013 - galaxy-dev - lists.galaxyproject.org

rename variable #{input} as a workflow parameter
by Berner, Thomas 25 Nov '13

25 Nov '13

Hey guys, we know that we can use a variable to rename output files after their input e.g. #{input}, by using the "rename dataset" action in the workflow editor. However, we would like to use these variable as a workflow parameter too, e.g. for the "Add or Replace Groups" tool as the ID tag, so the ID would be the dataset name. Currently we have to specify read groups for 190 datasets and doing this manually would take a long time, so we would prefer to automatize it. Is this possible? Thanks for any advice. Cheers, Thomas --- Thomas Berner Julius Kühn-Institut (JKI) - Federal Research Centre for Cultivated Plants - Erwin Baur-Straße 27 06484 Quedlinburg - Germany - Phone: ++49 ( 0 ) 3946 47 562 EMail: thomas.berner(a)jki.bund.de

1 0

NCBI Blast+ "BLAST XML to tabular" tool question
by Dooley, Damion 25 Nov '13

25 Nov '13

I'm doing a 1 step generic reporting tool along the lines of the "BLAST XML to tabular" script by Peter. I was just about to ask about this line, which looked pretty much like a bug: sallseqid = ";".join(name.split(None,1)[0] for name in hit_def.split(" >")) Then I found the patch from Nov 7th 2013: https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/b… try: sallseqid = ";".join(name.split(None,1)[0] for name in hit_def.split(" >")) except IndexError as e: stop_err("Problem splitting multuple hits?\n%r\n--> %s" % (hit_def, e)) Yay! But what I've seen in recent XML output reports is that the ">" content has been changed to ">" . E.g. https://github.com/biopython/biopython/blob/master/Tests/Blast/mirna.xml <Hit> <Hit_num>66</Hit_num> <Hit_id>gi|195029385|ref|XR_047134.1|</Hit_id> <Hit_def>Drosophila grimshawi miR-7-RA (Dgri\mir-7), ncRNA >gi|195336156|ref|XR_048470.1| Drosophila sechellia miR-7-RA (Dsec\mir-7), ncRNA >gi|195585143|ref|XR_050309.1| Drosophila simulans miR-7-RA (Dsim\mir-7), ncRNA</Hit_def> <Hit_accession>XR_047134</Hit_accession> ... So perhaps a stop_err() could be avoided, if test is for ">" instead? I assume that no variants of python ElementTree.iterparse() will unescape content when returned via the iterator? Damion

2 4

FTP bug with automatically decompressed files
by David Hoover 22 Nov '13

22 Nov '13

When a file is uploaded via FTP, and the file is compressed using gzip, the file is uploaded and decompressed on the fly; however, the gzipped file is left on the FTP server, rather than being deleted as normal. Is this a bug? David Hoover Helix Systems Staff

1 0

Tab2Fasta Script
by Hagan, Richard (haganrw) 21 Nov '13

21 Nov '13

Hello, I would like to get a copy of the script that you use to convert tabular data to fasta files on the galaxy server. Would you mind sharing it with me? Thanks! Richard Hagan

2 1

Postgres database migration issue with fresh install
by Jager, Victor de 21 Nov '13

21 Nov '13

Hi all, I'm installing a new galaxy instance and run into an issue after switching to postgres. During the migration setps I get the following error galaxy.model.migrate.check INFO 2013-11-21 15:37:23,238 galaxy.model.migrate.check INFO 2013-11-21 15:37:23,238 Migrating 103 -> 104... galaxy.model.migrate.check INFO 2013-11-21 15:37:23,257 galaxy.model.migrate.check INFO 2013-11-21 15:37:23,257 Migrating 104 -> 105... galaxy.model.migrate.check INFO 2013-11-21 15:37:25,413 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Migration script to add the cleanup_event* tables. galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Migrating 105 -> 106... galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Traceback (most recent call last): File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/app.py", line 54, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options, app=self ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 66, in create_or_verify_database migrate_to_current_version( engine, db_schema ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 124, in migrate_to_current_version schema.runchange( ver, change, changeset.step ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 89, in runchange (self.version, startver)) InvalidVersionError: 103 is not 105 ./manage_db.sh upgrade seems to finish the migration. How do I ensure the database is in a healthy state? Ubuntu 12.04.3 LTS PostgreSQL 9.1 Connection via socket Installation as local galaxy user Installation started in virtual environment Victor

2 1

Write all intermediate workflow data to /tmp/ only
by Ben Gift 21 Nov '13

21 Nov '13

I'm working with a lot of data on a cluster (condor). If I save all the workflow intermediate data, as Galaxy does by default (and rightfully so), it fills the drives. How can tell Galaxy to use /tmp/ to store all intermediate data in a workflow, and keep the result? I imagine I'll have to work on how Galaxy handles jobs, but I'm hoping there is something built in for this. Thanks

4 4

Status of GATK in Galaxy
by Bjoern Gruening 21 Nov '13

21 Nov '13

Hi, I found that the GATK tools do not have the same requirement tag. For example indel-realigner has version 1.4, realigner_target_creator has version 1.3. Is that correct? What is the recent version that is required for GATK? Also realigner_target_creator uses the "gatk_picard_indexes" location file, but under tool-data is only a gatk_sorted_picard_indexes.loc file. Are there any concrete plans for the GATK wrappers. Especially to update to a newer version. Or is that not feasible because of this complicated license policy and the phone-home option ... I found the following trello card, but its a little bit outdated: https://trello.com/c/IPkT2spd Is anyone running a recent GATK version in Galaxy? Thanks! Bjoern

4 8

Lock Galaxy job submission
by Peter Cock 21 Nov '13

21 Nov '13

Hello all, It is my recollection that under the "Admin" panel, "Manage jobs" page, there used to be a toggle control to lock the submission of new jobs. This was very helpful in preparation for scheduling a system shutdown (e.g. for hardware or software updates). However, I no longer see this option on either galaxy-central or galaxy-dist. Was it removed, or am I imagining things - in which case should I file an enhancement request ;) (I've had a look in Trello and didn't find anything) Thanks, Peter

2 2

NCBI Blast+ "BLAST XML to tabular" tool question - ADDENDUM
by Dooley, Damion 20 Nov '13

20 Nov '13

Woops - I realize now findtext() must be unescaping all ">", so Peter was trying to address other non-splitting occurances of " >" as per his patch notes. But perhaps a stop_err() isn't merrited in this case? So ignore my test for ">" comment. Regards, Damion

1 0

Samtools and idxstats
by Michiel Van Bel 20 Nov '13

20 Nov '13

Hi, I would like to inquire whether anyone has attempted to implement the idxstats tool from samtools into Galaxy? The xml-file for idxstats is not present in the Galaxy source code, which led me to try and implement it myself. However, the main problem I face is that the idxstats tool silently relies on having an index file available (within the same directory) for the bam file you which to print the stats for. E.g. samtools idxstats PATH/test.bam searches for PATH/test.bam.bai and gives an error when this file is not present. And somehow I cannot model this behavior in Galaxy. A different solution would of course be to ask the author(s) of samtools to have an option available where the user can directly indicate the path to the index file. regards, Michiel PS: I've searched the mailing list archives for this problem but did not find any matches. Apologies if I somehow missed the answer. -- ================================================================== Michiel Van Bel, PhD Expert Bioinformatician Tel:+32 (0)9 331 36 95 fax:+32 (0)9 3313809 VIB Department of Plant Systems Biology, Ghent University Technologiepark 927, 9052 Gent, BELGIUM mibel(a)psb.vib-ugent.be http://www.psb.vib-ugent.be http://bioinformatics.psb.ugent.be ==================================================================

3 4