New subject: test/base/twilltestcase.py - Converting local (test-data) bam to sam failed

27 Jan 2015


      Hi all,
I have a query about a failing tool test on the Tool Shed, where it
seems Galaxy is trying to convert both the expected BAM output
and the tool's BAM output into SAM for comparison - but this
fails.
https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad
2015-01-27 02:13:51
...
Tool id: samtools_depad
Tool version: samtools_depad
Test: test_tool_000000
(functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.1)
Stderr:
Traceback:
Traceback (most recent call last):
  File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py",
line 122, in test_tool
    self.do_it( td )
  File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py",
line 40, in do_it
    self._verify_outputs( testdef, test_history, jobs, shed_tool_id,
data_list, galaxy_interactor )
  File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py",
line 80, in _verify_outputs
    galaxy_interactor.verify_output( history, jobs, output_data,
output_testdef=output_testdef, shed_tool_id=shed_tool_id,
maxseconds=maxseconds )
  File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py",
line 66, in verify_output
    self.twill_test_case.verify_hid( outfile, hda_id=hid,
attributes=attributes, dataset_fetcher=fetcher,
shed_tool_id=shed_tool_id )
  File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py",
line 2341, in verify_hid
    raise AssertionError( errmsg )
AssertionError: History item  different than expected, difference (using diff):
( /tmp/shed_tools/toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/2d303f2e09e0/samtools_depad/test-data/sam_spec_padded.depad.bam
v. /tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpExQWRD/tmpSknKqLsam_spec_padded.depad.bam
)
Converting local (test-data) bam to sam failed
(and same error on the second test)
The same occurs on the Test Tool Shed (with a very slightly updated
version of this tool), although with a more concise traceback:
https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad
2015-01-25 08:21:59
...
Tool id: samtools_depad
Tool version: samtools_depad
Test: test_tool_000000
(functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/samtools_depad/0.0.2)
Stderr:
Traceback:
Traceback (most recent call last):
  File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
line 268, in test_tool
    self.do_it( td )
  File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py",
line 67, in do_it
    raise e
JobOutputsError: History item  different than expected, difference (using diff):
( /tmp/shed_tools/testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/8e1e3a1ecad7/samtools_depad/test-data/sam_spec_padded.depad.bam
v. /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpNvCGJi/tmphX3A_Fsam_spec_padded.depad.bam
)
Converting local (test-data) bam to sam failed
Both tests should produce a BAM file matching the provided
test-data/sam_spec_padded.depad.bam example. The error
comes from test/base/twilltestcase.py,
p = subprocess.Popen( args='samtools view -h -o "%s" "%s"' % (
temp_local.name, local_name  ), shell=True )
        assert not p.wait(), 'Converting local (test-data) bam to sam failed'
Running this command locally at the terminal seems to work fine,
both with samtools 0.1.19 and 1.1.
$ ~/bin/samtools_0.1.19 view -h -o temp.sam sam_spec_padded.depad.bam
$ more temp.sam
@HD    VN:1.5    SO:coordinate
@SQ    SN:ref    LN:45
ref    516    ref    1    0    45M    *    0    0
AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT    *
r001    99    ref    7    30    8M2I4M1D3M    =    37    41
TTAGATAAAGGATACTG    *
*    768    ref    8    30    1M    *    0    0    *    *
CT:Z:.;Warning;Note=Ref wrong?
r002    0    ref    9    30    3S6M1P1I4M    *    0    0
AAAAGATAAGGATA    *    PT:Z:1;4;+;homopolymer
r003    0    ref    9    30    5H6M    *    0    0    AGCTAA    *    NM:i:1
r004    0    ref    16    30    6M14D5M    *    0    0    ATAGCTTCAGC    *
r003    2064    ref    29    30    6H5M    *    0    0    TAGGC    *    NM:i:0
r001    147    ref    37    30    9M    =    7    -41    CAGCGGCAT
*    NM:i:1
$ ~/bin/samtools_1.1 view -h -o temp.sam sam_spec_padded.depad.bam
[galaxy@ppserver test-data]$ more temp.sam
@HD    VN:1.5    SO:coordinate
@SQ    SN:ref    LN:45
ref    516    ref    1    0    45M    *    0    0
AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT    *
r001    99    ref    7    30    8M2I4M1D3M    =    37    41
TTAGATAAAGGATACTG    *
*    768    ref    8    30    1M    *    0    0    *    *
CT:Z:.;Warning;Note=Ref wrong?
r002    0    ref    9    30    3S6M1P1I4M    *    0    0
AAAAGATAAGGATA    *    PT:Z:1;4;+;homopolymer
r003    0    ref    9    30    5H6M    *    0    0    AGCTAA    *    NM:i:1
r004    0    ref    16    30    6M14D5M    *    0    0    ATAGCTTCAGC    *
r003    2064    ref    29    30    6H5M    *    0    0    TAGGC    *    NM:i:0
r001    147    ref    37    30    9M    =    7    -41    CAGCGGCAT
*    NM:i:1
Which version of samtools is on the $PATH on the test system?
Could the test system have run out of disk space again?
The tests are passing on my local development instance (before
and after updating to latest galaxy-central), and on TravisCI using
the latest Galaxy: https://travis-ci.org/peterjc/pico_galaxy/builds/48485673
Thanks,
Peter