blastxml to tabular bug fix
Hi Kanwei, Could you check and commit the following three changesets from my new branch: http://bitbucket.org/peterjc/galaxy-central/src/blastplus_nov24 i.e. these three changesets: http://bitbucket.org/peterjc/galaxy-central/changeset/8f0e5a169630 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 This address a problem found earlier today by Alex Bossers (CC'd) while testing the BLAST+ wrappers. The problem can occur when when the query sequence is masked (with XXXX characters), this broke my calculation of the identity and mismatch counts (failing an assert statement I'd included as a sanity check). The new unit test is a cut down version of his failing example. Note that in order to run the unit tests, you will need to first re-enable the tool in tools_conf.xml.sample - the converter tests does not depend on the BLAST+ binaries being installed. Thank you, Peter
Done On Wed, Nov 24, 2010 at 10:13 AM, Peter <peter@maubp.freeserve.co.uk> wrote:
Hi Kanwei,
Could you check and commit the following three changesets from my new branch:
http://bitbucket.org/peterjc/galaxy-central/src/blastplus_nov24
i.e. these three changesets:
http://bitbucket.org/peterjc/galaxy-central/changeset/8f0e5a169630 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469
This address a problem found earlier today by Alex Bossers (CC'd) while testing the BLAST+ wrappers. The problem can occur when when the query sequence is masked (with XXXX characters), this broke my calculation of the identity and mismatch counts (failing an assert statement I'd included as a sanity check). The new unit test is a cut down version of his failing example.
Note that in order to run the unit tests, you will need to first re-enable the tool in tools_conf.xml.sample - the converter tests does not depend on the BLAST+ binaries being installed.
Thank you,
Peter
Peter green box for me :) Works like a charm. hanks for the rapid fix. Alex ________________________________________ Van: p.j.a.cock@googlemail.com [p.j.a.cock@googlemail.com] namens Peter [peter@maubp.freeserve.co.uk] Verzonden: woensdag 24 november 2010 16:13 Aan: Kanwei Li CC: Bossers, Alex; Galaxy Dev Onderwerp: blastxml to tabular bug fix Hi Kanwei, Could you check and commit the following three changesets from my new branch: http://bitbucket.org/peterjc/galaxy-central/src/blastplus_nov24 i.e. these three changesets: http://bitbucket.org/peterjc/galaxy-central/changeset/8f0e5a169630 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 This address a problem found earlier today by Alex Bossers (CC'd) while testing the BLAST+ wrappers. The problem can occur when when the query sequence is masked (with XXXX characters), this broke my calculation of the identity and mismatch counts (failing an assert statement I'd included as a sanity check). The new unit test is a cut down version of his failing example. Note that in order to run the unit tests, you will need to first re-enable the tool in tools_conf.xml.sample - the converter tests does not depend on the BLAST+ binaries being installed. Thank you, Peter
Peter, a nice extra feature welcomed by myself would be to allow the optional inclusion of the Hit_defline in the output table. In many workflows we would need to blast, get the id from the table, use id to get human readible name and insert/use it.... which is silly of course since that data is available in the xml anyway. I don't know python and about hg changesets but I modified your python and xml file to incorporate this (see attachement). By default its normal blast tabular output but optionally it can include the defline. The hit_defline needed to be split (I hope I did it in a python way) to eliminate multiple discriptions separated by >gi (nt and nr) or plain semicolons for swissprot.... maybe there are more but not sure... Have a look and test and maybe it will find the way in some form into your suite. Anyway its very useful in this way to us. cheers Alex ________________________________________ Van: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu] namens Bossers, Alex [Alex.Bossers@wur.nl] Verzonden: woensdag 24 november 2010 22:18 Aan: peter@maubp.freeserve.co.uk CC: Galaxy Dev Onderwerp: Re: [galaxy-dev] blastxml to tabular bug fix Peter green box for me :) Works like a charm. hanks for the rapid fix. Alex ________________________________________ Van: p.j.a.cock@googlemail.com [p.j.a.cock@googlemail.com] namens Peter [peter@maubp.freeserve.co.uk] Verzonden: woensdag 24 november 2010 16:13 Aan: Kanwei Li CC: Bossers, Alex; Galaxy Dev Onderwerp: blastxml to tabular bug fix Hi Kanwei, Could you check and commit the following three changesets from my new branch: http://bitbucket.org/peterjc/galaxy-central/src/blastplus_nov24 i.e. these three changesets: http://bitbucket.org/peterjc/galaxy-central/changeset/8f0e5a169630 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 This address a problem found earlier today by Alex Bossers (CC'd) while testing the BLAST+ wrappers. The problem can occur when when the query sequence is masked (with XXXX characters), this broke my calculation of the identity and mismatch counts (failing an assert statement I'd included as a sanity check). The new unit test is a cut down version of his failing example. Note that in order to run the unit tests, you will need to first re-enable the tool in tools_conf.xml.sample - the converter tests does not depend on the BLAST+ binaries being installed. Thank you, Peter _______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Peter, I have tested the XML parser for blast jobs on a larger set of files. I haven't checked every detail yet but it seems to process it now without problems. Majority tested using blastp on two protein databases with over 2k queries. Another point I already mailed you off list is the ability to capture additional STDOUT and/or STDERR in a log file. Warnings make the galaxy tool run red and you have to view->save->importnew....which is cumbersome and waste of time and resources when files are over 1.5G big. These files processed without any problem though in the xml parser! Thanks and keep up the good work (many people don't realize all hassle you have to go through when wanting/making a tool available via tool_shed to others.. Alex -----Oorspronkelijk bericht----- Van: p.j.a.cock@googlemail.com [mailto:p.j.a.cock@googlemail.com] Namens Peter Verzonden: woensdag 24 november 2010 4:13 Aan: Kanwei Li CC: Bossers, Alex; Galaxy Dev Onderwerp: blastxml to tabular bug fix Hi Kanwei, Could you check and commit the following three changesets from my new branch: http://bitbucket.org/peterjc/galaxy-central/src/blastplus_nov24 i.e. these three changesets: http://bitbucket.org/peterjc/galaxy-central/changeset/8f0e5a169630 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 http://bitbucket.org/peterjc/galaxy-central/changeset/72fab21f9469 This address a problem found earlier today by Alex Bossers (CC'd) while testing the BLAST+ wrappers. The problem can occur when when the query sequence is masked (with XXXX characters), this broke my calculation of the identity and mismatch counts (failing an assert statement I'd included as a sanity check). The new unit test is a cut down version of his failing example. Note that in order to run the unit tests, you will need to first re-enable the tool in tools_conf.xml.sample - the converter tests does not depend on the BLAST+ binaries being installed. Thank you, Peter
On Thu, Dec 30, 2010 at 12:05 PM, Bossers, Alex <Alex.Bossers@wur.nl> wrote:
Peter,
I have tested the XML parser for blast jobs on a larger set of files. I haven't checked every detail yet but it seems to process it now without problems. Majority tested using blastp on two protein databases with over 2k queries.
Hi Alex, That's good news :)
Another point I already mailed you off list is the ability to capture additional STDOUT and/or STDERR in a log file. Warnings make the galaxy tool run red and you have to view->save->importnew.... which is cumbersome and waste of time and resources when files are over 1.5G big. These files processed without any problem though in the xml parser!
Sadly true, As I said off line, I found this too just before our Institute closed for Christmas, again harmless blastp warnings about Selenocysteine on stderr causing Galaxy to treat it as an failing job. Ideally this will be fixed in Galaxy: http://bitbucket.org/galaxy/galaxy-central/issue/325/allow-tool-authors-to-d... In the short term, I plan to try modifying the BLAST+ wrapper XML files to redirect stderr to stdout. That way Galaxy will show the warning in the jobs's info text. There will be lots of little tasks like this will be waiting for me back at work in the new year :(
Thanks and keep up the good work (many people don't realize all hassle you have to go through when wanting/making a tool available via tool_shed to others..
Your testing is also much appreciated, and is important as I expect every Galaxy installation will be slightly different. Regards, Peter
participants (3)
-
Bossers, Alex -
Kanwei Li -
Peter