Hi Dave, Greg,
Based on my tool's overnight test results from the Test Tool Shed, something seems to have been broken to do with sniffing datatypes on upload.
e.g. using blastxml format, http://testtoolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr
Tool test results Automated test environment Tests that failed Tool id: blastxml_to_top_descr Tool version: blastxml_to_top_descr Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/blastxml_to_top_descr/blastxml_to_top_descr/0.0.9) Stderr: Traceback: Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1327, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1276, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'blastxml_file' to value '['blastp_four_human_vs_rhodopsin.xml']' in form 'tool_form' threw exception: cannot find value/label "blastp_four_human..odopsin.xml" in list control control: <SelectControl(blastxml_file=[])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG:
e.g. just fasta and tabular,
Tool test results Automated test environment Tests that failed Tool id: seq_primer_clip Tool version: seq_primer_clip Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/seq_primer_clip/seq_primer_clip/0.0.10) Stderr: Traceback: Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1327, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1276, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'primer_fasta' to value '['dop_primers.fasta']' in form 'tool_form' threw exception: cannot find value/label "dop_primers.fasta" in list control control: <SelectControl(primer_fasta=[])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 381 galaxy.jobs: DEBUG: (381) Working directory for job is: /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/database/job_working_directory/000/381 galaxy.jobs.handler: DEBUG: (381) Dispatching to local runner galaxy.jobs: DEBUG: (381) Persisting job destination (destination id: local) galaxy.jobs.handler: INFO: (381) Job dispatched galaxy.tools.deps: DEBUG: Building dependency shell command for dependency 'samtools' galaxy.tools.deps: WARNING: Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners.local: DEBUG: (381) executing: export GALAXY_SLOTS="1"; python /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/tools/data_source/upload.py /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpDEZCBr/tmpG8dIQ8 /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpDEZCBr/tmpMD8cHQ 404:/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/database/job_working_directory/000/381/dataset_404_files:/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpALDdhE/database/files/000/dataset_404.dat galaxy.datatypes.metadata: DEBUG: Cleaning up external metadata files galaxy.jobs.runners.local: ERROR: failure running job 381 Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG:
(snip - first test only, rest do the same).
Peter
Hi Peter,
The Amazon EC2 instance ran out of memory in last night's run which resulted in the missing test results for some repositories as well as the issue you've documented below. We're looking into the cause of this and will have it resolved (see Trello card from my previousl email).
Thanks,
Greg Von Kuster
On Oct 22, 2013, at 5:48 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
Hi Dave, Greg,
Based on my tool's overnight test results from the Test Tool Shed, something seems to have been broken to do with sniffing datatypes on upload.
e.g. using blastxml format, http://testtoolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr
Tool test results Automated test environment Tests that failed Tool id: blastxml_to_top_descr Tool version: blastxml_to_top_descr Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/blastxml_to_top_descr/blastxml_to_top_descr/0.0.9) Stderr: Traceback: Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1327, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1276, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'blastxml_file' to value '['blastp_four_human_vs_rhodopsin.xml']' in form 'tool_form' threw exception: cannot find value/label "blastp_four_human..odopsin.xml" in list control control: <SelectControl(blastxml_file=[])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG:
e.g. just fasta and tabular,
Tool test results Automated test environment Tests that failed Tool id: seq_primer_clip Tool version: seq_primer_clip Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/seq_primer_clip/seq_primer_clip/0.0.10) Stderr: Traceback: Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1327, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1276, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'primer_fasta' to value '['dop_primers.fasta']' in form 'tool_form' threw exception: cannot find value/label "dop_primers.fasta" in list control control: <SelectControl(primer_fasta=[])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 381 galaxy.jobs: DEBUG: (381) Working directory for job is: /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/database/job_working_directory/000/381 galaxy.jobs.handler: DEBUG: (381) Dispatching to local runner galaxy.jobs: DEBUG: (381) Persisting job destination (destination id: local) galaxy.jobs.handler: INFO: (381) Job dispatched galaxy.tools.deps: DEBUG: Building dependency shell command for dependency 'samtools' galaxy.tools.deps: WARNING: Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners.local: DEBUG: (381) executing: export GALAXY_SLOTS="1"; python /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/tools/data_source/upload.py /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpDEZCBr/tmpG8dIQ8 /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpDEZCBr/tmpMD8cHQ 404:/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/database/job_working_directory/000/381/dataset_404_files:/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpALDdhE/database/files/000/dataset_404.dat galaxy.datatypes.metadata: DEBUG: Cleaning up external metadata files galaxy.jobs.runners.local: ERROR: failure running job 381 Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/local.py", line 83, in queue_job preexec_fn = os.setpgrp ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1205, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG: base.twilltestcase: DEBUG:
(snip - first test only, rest do the same).
Peter
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
galaxy-dev@lists.galaxyproject.org
-
Greg Von Kuster -
Peter Cock