- galaxy-dev - lists.galaxyproject.org

Error while trying to extract DNA
by Alan WiIliams 27 Nov '13

27 Nov '13

Hi, I am new here. I have a interval file with the columns chr,start,end. When I run it I get 10 warnings, 1st is: Unable to fetch the sequence from '172591712' to '3209' for chrom 'chr5'. Skipped 10 invalid lines, 1st is #1, "chr5 172591712 172594921" Which doesn't make sense. Can someone help? Thanks, -- Alan Williams Life is short, art long, opportunity fleeting, experience deceptive, judgement difficult.

2 1

Compressed data files in Galaxy? (e.g. GZIP or BGZF)
by Peter Cock 27 Nov '13

27 Nov '13

Hello all, What is the current status in Galaxy for supporting compressed files? We've talked about this before, for example in addition to FASTQ, many of us have expressed a wish to work with gzipped FASTQ. I understand that some have customized their local Galaxy installations to use gzipped FASTQ as a specific data type - I'm more interested in a general file format neutral solution. Also, I'd like to be able to used BGZF (not just GZIP) because it is better for random access - see for example http://blastedbio.blogspot.com/2011/11/bgzf-blocked-bigger-better-gzip.html - and makes it much easier to break up large datafiles for sharing over a cluster (i.e. it could be exploited in the current Galaxy code for splitting large sequence files). The 11 May 2012 Galaxy Development News Brief http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-May/009757.html mentions tabix indexing - that uses bgzip, so is there something general in place yet to allow tool wrappers to say they accept not just given file formats, but different compressed versions of file formats? Ideally I'd like to be able to write an XML tool description saying a tool produced BGZF compressed tabular data, or GZIP compressed Sanger FASTQ etc. Similarly, I'd like to specify my tool accepts FASTA or gzipped FASTA (including BGZF FASTA). While for older tools if they say they accept only uncompressed FASTA, Galaxy could automatically decompress any compressed FASTA entries in my history on demand. Peter

3 4

Why does check_html only read first 100 lines?
by Robert Davidson 25 Nov '13

25 Nov '13

Hi all, I recently tried uploading an couple of xml files to my local Galaxy installation using the standard ‘Upload File (version 1.1.3)’ tool. For some files this produced the error: The uploaded file contains inappropriate HTML content. Given the files had been created by the same automated code and contained the same tags etc, I couldn't understand why one would produce this error and the other not. Finally tracking down the function check_html() in galaxy-dist/lib/galaxy/datatypes/checkers.py, I discovered that my use of the tag <metabolite> had flagged up as a likely <META > tag and produced the warning/failed upload. The reason this did not happen in every case is that check_html only reads the first 100 lines of the file and depending upon how many samples were in my dataset, my <metabolite> tag could appear before or after this cutoff. I've solved the problem by changing my xml tag names but my question is: a) why does check_html only read up to line 100? b) would it be possible to change the regular expressions in check_html so that e.g. <meta ...> would be found but e.g. <metaxxx ...> would not? Thanks for reading. Rob Dr Robert L Davidson NERC Metabolomics Facility School of Biosciences University of Birmingham Edgbaston, UK

2 1

rename variable #{input} as a workflow parameter
by Berner, Thomas 25 Nov '13

25 Nov '13

Hey guys, we know that we can use a variable to rename output files after their input e.g. #{input}, by using the "rename dataset" action in the workflow editor. However, we would like to use these variable as a workflow parameter too, e.g. for the "Add or Replace Groups" tool as the ID tag, so the ID would be the dataset name. Currently we have to specify read groups for 190 datasets and doing this manually would take a long time, so we would prefer to automatize it. Is this possible? Thanks for any advice. Cheers, Thomas --- Thomas Berner Julius Kühn-Institut (JKI) - Federal Research Centre for Cultivated Plants - Erwin Baur-Straße 27 06484 Quedlinburg - Germany - Phone: ++49 ( 0 ) 3946 47 562 EMail: thomas.berner(a)jki.bund.de

1 0

NCBI Blast+ "BLAST XML to tabular" tool question
by Dooley, Damion 25 Nov '13

25 Nov '13

I'm doing a 1 step generic reporting tool along the lines of the "BLAST XML to tabular" script by Peter. I was just about to ask about this line, which looked pretty much like a bug: sallseqid = ";".join(name.split(None,1)[0] for name in hit_def.split(" >")) Then I found the patch from Nov 7th 2013: https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/b… try: sallseqid = ";".join(name.split(None,1)[0] for name in hit_def.split(" >")) except IndexError as e: stop_err("Problem splitting multuple hits?\n%r\n--> %s" % (hit_def, e)) Yay! But what I've seen in recent XML output reports is that the ">" content has been changed to ">" . E.g. https://github.com/biopython/biopython/blob/master/Tests/Blast/mirna.xml <Hit> <Hit_num>66</Hit_num> <Hit_id>gi|195029385|ref|XR_047134.1|</Hit_id> <Hit_def>Drosophila grimshawi miR-7-RA (Dgri\mir-7), ncRNA >gi|195336156|ref|XR_048470.1| Drosophila sechellia miR-7-RA (Dsec\mir-7), ncRNA >gi|195585143|ref|XR_050309.1| Drosophila simulans miR-7-RA (Dsim\mir-7), ncRNA</Hit_def> <Hit_accession>XR_047134</Hit_accession> ... So perhaps a stop_err() could be avoided, if test is for ">" instead? I assume that no variants of python ElementTree.iterparse() will unescape content when returned via the iterator? Damion

2 4

FTP bug with automatically decompressed files
by David Hoover 22 Nov '13

22 Nov '13

When a file is uploaded via FTP, and the file is compressed using gzip, the file is uploaded and decompressed on the fly; however, the gzipped file is left on the FTP server, rather than being deleted as normal. Is this a bug? David Hoover Helix Systems Staff

1 0

Tab2Fasta Script
by Hagan, Richard (haganrw) 22 Nov '13

22 Nov '13

Hello, I would like to get a copy of the script that you use to convert tabular data to fasta files on the galaxy server. Would you mind sharing it with me? Thanks! Richard Hagan

2 1

Postgres database migration issue with fresh install
by Jager, Victor de 21 Nov '13

21 Nov '13

Hi all, I'm installing a new galaxy instance and run into an issue after switching to postgres. During the migration setps I get the following error galaxy.model.migrate.check INFO 2013-11-21 15:37:23,238 galaxy.model.migrate.check INFO 2013-11-21 15:37:23,238 Migrating 103 -> 104... galaxy.model.migrate.check INFO 2013-11-21 15:37:23,257 galaxy.model.migrate.check INFO 2013-11-21 15:37:23,257 Migrating 104 -> 105... galaxy.model.migrate.check INFO 2013-11-21 15:37:25,413 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Migration script to add the cleanup_event* tables. galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Migrating 105 -> 106... galaxy.model.migrate.check INFO 2013-11-21 15:37:25,414 Traceback (most recent call last): File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/app.py", line 54, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options, app=self ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 66, in create_or_verify_database migrate_to_current_version( engine, db_schema ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 124, in migrate_to_current_version schema.runchange( ver, change, changeset.step ) File "/mnt/data/home/NIOO/galaxy/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 89, in runchange (self.version, startver)) InvalidVersionError: 103 is not 105 ./manage_db.sh upgrade seems to finish the migration. How do I ensure the database is in a healthy state? Ubuntu 12.04.3 LTS PostgreSQL 9.1 Connection via socket Installation as local galaxy user Installation started in virtual environment Victor

2 1

Write all intermediate workflow data to /tmp/ only
by Ben Gift 21 Nov '13

21 Nov '13

I'm working with a lot of data on a cluster (condor). If I save all the workflow intermediate data, as Galaxy does by default (and rightfully so), it fills the drives. How can tell Galaxy to use /tmp/ to store all intermediate data in a workflow, and keep the result? I imagine I'll have to work on how Galaxy handles jobs, but I'm hoping there is something built in for this. Thanks

4 4

Status of GATK in Galaxy
by Bjoern Gruening 21 Nov '13

21 Nov '13

Hi, I found that the GATK tools do not have the same requirement tag. For example indel-realigner has version 1.4, realigner_target_creator has version 1.3. Is that correct? What is the recent version that is required for GATK? Also realigner_target_creator uses the "gatk_picard_indexes" location file, but under tool-data is only a gatk_sorted_picard_indexes.loc file. Are there any concrete plans for the GATK wrappers. Especially to update to a newer version. Or is that not feasible because of this complicated license policy and the phone-home option ... I found the following trello card, but its a little bit outdated: https://trello.com/c/IPkT2spd Is anyone running a recent GATK version in Galaxy? Thanks! Bjoern

4 8