- galaxy-dev - lists.galaxyproject.org

Confusion logging in (via "Workflow" or via "User")
by Peter 16 Dec '10

16 Dec '10

Hi, I'm having some trouble with our local Galaxy install with logging in / logging out. As part of this I noticed an odd inconsistency which is also present on the public Galaxy instance. Via the top ribbon "Workflow" link, I get told "You must be logged in to use Galaxy workflows." and the URL goes here - as the whole page, not the middle frame: http://main.g2.bx.psu.edu/user/login?webapp=galaxy Via the top ribbon "User", "Login" I get sent to the following URL as a new window: http://main.g2.bx.psu.edu/user/login Why is there this difference? Thanks, Peter

2 1

fastqx-toolkit related problem in Galaxy?
by Marina Gourtovaia 16 Dec '10

16 Dec '10

Hello I am using teh galaxy instance on http://main.g2.bx.psu.edu/root For an input fastq file starting with @IL14_1008:2:1:800:71/1 AAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAA + >>>>>>>>>>>>>>>>>>>><>>>>->><<>><>><< I get the following error when running FASTQ to FASTA on this input: 'An error occurred running this job: fastq_to_fasta: Invalid quality score value (char '-' ord 45 quality value -19) on line 4' When I try to use the 'Clip adapter sequences' with this input file I get the following error against the 'Library to clip' widget: 'History does not include a dataset of the required format / build' When I run the command-line version of the fastqx-toolkit tools on a Linux 64 bit machine, I have similar problems: mg8@sf-2-1-01:~/gal_data$ fastq_to_fasta -v -n -i 10000.fastq -o my.fa fastq_to_fasta: Invalid quality score value (char '-' ord 45 quality value -19) on line 4 and mg8@sf-2-1-01:~/gal_data$ fastx_clipper -i 10000.fastq -o clipped.fastq -a ACACTCTTTCCCTACACGACGCTCTTCCGATCT fastx_clipper: Invalid quality score value (char '-' ord 45 quality value -19) on line 4 Therefore, the problem seems to be with the fastqx-toolkit tools. My file is in the Sanger fastq format. Galaxy does allow me to move it to the Illumina format without problems, but that does not help with clipping the adaptors functionality Marina -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

2 2

How can I require users to log in to Galaxy?
by Peter 16 Dec '10

16 Dec '10

Hi all, We're planning to use Galaxy on two local servers. One will be restricted to those on the institute network, and here the default Galaxy behaviour is fine (anyone can use it without logging in). However, the second server will be accessible from the internet, but we only want authorised users to be able to use Galaxy on it. I don't see anything about configuring this on the security page: https://bitbucket.org/galaxy/galaxy-central/wiki/SecurityFeatures Is there some documentation I've overlooked? Thanks, Peter

2 2

Fwd: [galaxy-user] Read shuffler and code contributions
by Peter 16 Dec '10

16 Dec '10

Forwarding to the Galaxy Dev mailing list, which is probably a better place to discuss changes to the Galaxy FASTQ code. Peter ---------- Forwarded message ---------- From: Peter <peter(a)maubp.freeserve.co.uk> Date: Thu, Dec 16, 2010 at 8:50 AM Subject: Re: [galaxy-user] Read shuffler and code contributions To: Florent Angly <florent.angly(a)gmail.com> Cc: galaxy-user(a)bx.psu.edu > Hi Peter, > >> Are you asking for a tool to interleave to FASTQ or FASTA files with >> matching entries (with matching names in the same order) into one >> file which alternates forward then reverse read? > > Yes, indeed, this is what I am proposing. > >> Would you prefer it with or without error checking? > > Error checking is best. I'd agree. >> I think the scripts in velvet are fast but will fail horribly with >> bad input... note there is a simple Biopython script to do this >> included with velvet already (simple version with no error checking, >> I have written a more robust version too - it looks like I haven't >> sent it to Daniel to include in velvet though). > > I rolled my own FASTQ paired read interlacer and deinterlacer today, using > the Galaxy Python modules in lib/galaxy_utils/. I must say these modules > made it quite convenient and efficient to implement error-checking in the > (de)interlacing. You can find the scripts here if you're interested: > http://bitbucket.org/fangly/galaxy-central > I'll make the XML wrappers tomorrow and test them. Hopefully after this is > done, my changes can be pulled into the official Galaxy repository. For the deinterlacer, I previously offered to write something like that for Galaxy and was told to submit it to the Tool Shed initially (although it may be merged into the official repository at some point). See "Divide FASTQ file into paired and unpaired reads" on http://community.g2.bx.psu.edu/ for my tool. I also note you've changed the return behaviour of the Galaxy FASTQ library method get_paired_identifier - that API change could break other parts of Galaxy or 3rd party tools. Looking at that Galaxy lib, perhaps I can offer some of my code for identifying Sanger read pairs and the .f .r suffices to enhance the class fastqJoiner (look like it only does Illumina /1 and /2 right now which I think is too narrow). Peter

1 0

BLAST+ enhancements, was: blastxml to tabular bug fix
by Peter Cock 15 Dec '10

15 Dec '10

On Wed, Nov 24, 2010 at 11:02 PM, Bossers, Alex wrote: > Peter, > > a nice extra feature welcomed by myself would be to allow the > optional inclusion of the Hit_defline in the output table. In many > workflows we would need to blast, get the id from the table, use > id to get human readible name and insert/use it.... which is silly > of course since that data is available in the xml anyway. > > I don't know python and about hg changesets but I modified > your python and xml file to incorporate this (see attachement). > By default its normal blast tabular output but optionally it can > include the defline. > The hit_defline needed to be split (I hope I did it in a python > way) to eliminate multiple discriptions separated by >gi (nt > and nr) or plain semicolons for swissprot.... maybe there > are more but not sure... > > Have a look and test and maybe it will find the way in some > form into your suite. Anyway its very useful in this way to us. > > cheers > Alex Hi Alex, I'm glad to see the BLAST+ wrappers being used already, and to get positive feedback. I had a quick look at your modifications - I think it could be made more beautiful, but it looks like it would work fine. I understand the aim behind your suggested change, but I have another solution in mind. I was already planning to write another tool for splitting a column in a tabular file - e.g. splitting on the pipe character could be very useful to extract the GI number from a typical NCBI identifier string. Such a tool could also be used on the BLAST output to do what you are asking for (splitting the hit IDs), or to grab a particular word from formatted text (by spitting on spaces). I'm surprised this isn't in Galaxy already to be honest - maybe it is and I haven't found it yet ;) I'd also like to explain that I deliberately kept the provided XML to tabular functionality simple to start with - all it tried to do is recreate the default tabular output, but even that turned out to be non-trivial. I have several ideas for extension which I will try to outline here. The BLAST+ suite actually lets you ask for certain other predefined columns in the tabular output. I am wondering about offering a "full" tabular output option in the BLAST+ wrappers - this seems simpler than making the user pick and choose which columns they want. e.g. for blastp: The supported format specifiers are: qseqid means Query Seq-id qgi means Query GI qacc means Query accesion sseqid means Subject Seq-id sallseqid means All subject Seq-id(s), separated by a ';' sgi means Subject GI sallgi means All subject GIs sacc means Subject accession sallacc means All subject accessions qstart means Start of alignment in query qend means End of alignment in query sstart means Start of alignment in subject send means End of alignment in subject qseq means Aligned part of query sequence sseq means Aligned part of subject sequence evalue means Expect value bitscore means Bit score score means Raw score length means Alignment length pident means Percentage of identical matches nident means Number of identical matches mismatch means Number of mismatches positive means Number of positive-scoring matches gapopen means Number of gap openings gaps means Total number of gaps ppos means Percentage of positive-scoring matches frames means Query and subject frames separated by a '/' qframe means Query frame sframe means Subject frame Note that calculating and recording of the above will add computation cost and IO load - so keeping the default std set of columns as the default in the Galaxy wrapper makes sense to me. Potentially the BLAST XML output can be converted into this full tabular output too - I expect so but it may not be so easy. Another avenue by which to extend the BLAST+ suite is to teach Galaxy about the BLAST ASN.1 output format, and wrap the new blast_formatter application for turning ASN.1 into another BLAST output format. Regards, Peter

2 5

Split FASTQ file into paired and unpaired reads
by Peter 15 Dec '10

15 Dec '10

Hi all, I've got a Python script which divides a FASTQ file containing a mixture of paired, unpaired and orphan reads into valid pairs and single/orphan reads. Such a situation can occur after applying quality filtering to a file of paired FASTQ reads. I've also had raw data supplied in this state from a sequencing center. It works by looking at the read names, and understand the /1 and /2 convention used by Illumina, .f and .r which I understand is common, and the Sanger convention as well. The only requirement (so that only a single pass though the input is needed) is that the input be sorted such that any pairs come as consecutive entries (forward then reverse read). Is there anything like this in Galaxy already that I have missed? Would you consider merging such a tool into Galaxy? I haven't written a wrapper XML file yet, and it also currently uses Biopython for FASTQ parsing, but I could switch it to use the Galaxy FASTQ code instead. Regards, Peter

2 3

Trackster BAM file Seg Fault
by Alan Jackson 15 Dec '10

15 Dec '10

Hi, I've been trying to use the Trackster visualisation with a BAM file, and find it segfaults quite regularly, with the (somewhat unhelpful) message ./run.sh: line 47: 13782 Segmentation fault (core dumped) python ./scripts/paster.py serve universe_wsgi.ini $@ Using gdb I have gotten the following info: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x4a6b5940 (LWP 13800)] 0x00002adeee8c7937 in get_chunk_coordinates (idx=0xffffffffb0018da0, tid=0, beg=0, end=-1341883296, cnt_off=0x2aaab0016600) at samtools/bam_index.c:501 501 samtools/bam_index.c: No such file or directory. in samtools/bam_index.c However the BAM index should be getting created and stored as part of the meta data. I also notice the pysam library being used is 0.1.1, whereas they are up to 0.3.1. Any help greatly appreciated, the visualisation looks very good when it's working. I am able to send the BAM file I'm using if necessary. Cheers, --Alan

2 1

Output files using format="input"
by Peter 15 Dec '10

15 Dec '10

Hi all, I'm trying to sort out some final details for my FASTQ filter by ID script before uploading it to the tool shed. This tool takes two input files, a FASTQ file (can be any type of FASTQ) and a tabular file (which has the IDs to filter on). The output file(s) are are also FASTQ (of the same variant as the input FASTQ file). I had simply used something like this when defining my outputs: <data name="output_pos" format="fastq" label="With matched ID" /> <data name="output_neg" format="fastq" label="Without matched ID" /> However, this discards the fact that the input (and therefore my output) may have been fastqsanger or fastqillumina etc. According to the wiki http://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax I can use format="input" in this setting... but here I have TWO input files (of different formats). Although this isn't giving an error, I find this results in my output being treated as tabular: <data name="output_pos" format="input" label="With matched ID" /> <data name="output_neg" format="input" label="Without matched ID" /> I've also tried using the names of my input parameters, but that is just treated as a new file format (interestingly no error is raised): <data name="output_pos" format="input_fastq" label="With matched ID" /> <data name="output_neg" format="input_fastq" label="Without matched ID" /> How is format="input" meant to work? Is it currently restricted to the special case of only one input file? Can you specify which input file somehow, e.g. via metadata_source? Thank you, Peter

1 1

32-bit python
by David Hoover 15 Dec '10

15 Dec '10

Are there any issues with using a 32-bit version of python for Galaxy? David Hoover Helix Systems Staff http://helix.nih.gov

3 3

FASTQ to Tabular tool doesn't accept plain FASTQ
by Peter 15 Dec '10

15 Dec '10

Hi all, I just uploaded a (Sanger) FASTQ file without specifying the format. Galaxy therefore treats it as "fastq". Some tools are happy to take this as input, but not the fastq_to_tabular converter. Please consider applying the following trivial patch: --- a/tools/fastq/fastq_to_tabular.xml Wed Nov 17 16:24:12 2010 -0500 +++ b/tools/fastq/fastq_to_tabular.xml Mon Dec 13 16:52:51 2010 +0000 @@ -2,7 +2,7 @@ <description>converter</description> <command interpreter="python">fastq_to_tabular.py '$input_file' '$output_file' '${input_file.extension[len( 'fastq' ):]}'</command> <inputs> - <param name="input_file" type="data" format="fastqsanger,fastqcssanger,fastqillumina,fastqsolexa" label="FASTQ file to convert" /> + <param name="input_file" type="data" format="fastq,fastqsanger,fastqcssanger,fastqillumina,fastqsolexa" label="FASTQ file to convert" /> </inputs> <outputs> <data name="output_file" format="tabular" /> Thanks, Peter

1 1