Problems with latest Galaxy 8167:b0c916349070
Running a CloudMan created instance on Amazon. Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070 Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented. Thanks Scooter Bowtie Stdout Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' Stderr WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Using toolshed installed Abyss and get the same error in Stdout. Stdout ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' Stderr COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrapper.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Looking at the util directory based on the expected path of the read build file galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go. -Dannon On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'
Stderr
COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrapper.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Ok that fixed Bowtie. Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it. In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other. Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4 On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel). On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
From looking at the abyss wrapper it is expecting ABYSS to be found on the
Dannon If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools. path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed. Thanks Scooter On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE:
/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_ to
ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wr ap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/uti l$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS. The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools). -Dannon On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh ar ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE:
/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_ to
ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wr ap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/uti l$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Thanks! That was the explanation I was looking for and the link for how to Do you have any suggestions for fixing the error that are still occurring after running Bowtie? Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ncbi/builds.txt' On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS.
The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools).
-Dannon
On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location > galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls > ABYSS abyss-adjtodot abyss-joindist abyss-tofastq >DistanceEst > MergePaths PathConsensus SimpleGraph > abyss2ace abyss-fac ABYSS-P AdjList KAligner > Overlap PathOverlap > abyss2afg abyss-fixmate abyss-pe Consensus > MergeContigs > ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
> Sequence file aligned. > ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ucsc/publicbuilds.txt' > ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ensembl/builds.txt' > ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ncbi/builds.txt'
Stderr > WARNING:galaxy.datatypes.registry:Error loading datatype with > extension 'blastxml': 'module' object has no attribute 'BlastXml' > WARNING:galaxy.datatypes.registry:Error appending sniffer for > datatype > 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has > no > attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
> ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ucsc/publicbuilds.txt' > ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ensembl/builds.txt' > ERROR: Unable to read builds file: [Errno 2] No such file or > directory: > > >'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >sh > ar > ed/ncbi/builds.txt'
Stderr > COMMAND FAILURE: > > >/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abys >s_ > to > > >ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_ >wr > ap > per.sh: line 9: ABYSS: command not found > WARNING:galaxy.datatypes.registry:Error loading datatype with > extension 'blastxml': 'module' object has no attribute 'BlastXml' > WARNING:galaxy.datatypes.registry:Error appending sniffer for > datatype > 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has > no > attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
> > > >galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/u >ti > l$ > ls > aliaspickler.py docutils_ext heartbeat.py json.pyc > odict.py streamball.py > aliaspickler.pyc docutils_template.txt inflection.py >lrucache.py > odict.pyc streamball.pyc > backports expressions.py inflection.pyc >lrucache.pyc > sanitize_html.py template.py > bunch.py expressions.pyc __init__.py memdump.py > sanitize_html.pyc template.pyc > bunch.pyc hash_util.py __init__.pyc >none_like.py > shed_util.py topsort.py > debugging.py hash_util.pyc json.py > none_like.pyc > shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Glad to help. The first step would be to check and see if those files actually exist (and they should). If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong. -Dannon On Nov 8, 2012, at 1:27 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Thanks! That was the explanation I was looking for and the link for how to
Do you have any suggestions for fixing the error that are still occurring after running Bowtie?
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ncbi/builds.txt'
On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS.
The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools).
-Dannon
On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
> Running a CloudMan created instance on Amazon. > > Updated using cloudman from > http://bitbucket.org/galaxy/galaxy-central > to version 8167:b0c916349070 > > Tried to run bowtie and get the following error. Searched the > message > archives an someone else reported the same problem but thought it > was > related to a new version of cufflinks. Same problem also exists when > running abyss. Given the nature of the error message not even sure > what > the actual path should be. Also for the abyss install from toolshed > the > toolshed.g2.bx.psu.edu directory doesn't exist or the six > directories > underneath. However Abyss was/is installed at the following location >> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls >> ABYSS abyss-adjtodot abyss-joindist abyss-tofastq >> DistanceEst >> MergePaths PathConsensus SimpleGraph >> abyss2ace abyss-fac ABYSS-P AdjList KAligner >> Overlap PathOverlap >> abyss2afg abyss-fixmate abyss-pe Consensus >> MergeContigs >> ParseAligns PopBubbles > > Would be great if someone could give some guidance on how to resolve > the problems. I understand if I need to do something under the > covers > but the expectation is that the one click install/updates should not > then require major command line work that isn't documented. > > Thanks > > Scooter > > > Bowtie > > Stdout > >> Sequence file aligned. >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ucsc/publicbuilds.txt' >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ensembl/builds.txt' >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ncbi/builds.txt' > > Stderr >> WARNING:galaxy.datatypes.registry:Error loading datatype with >> extension 'blastxml': 'module' object has no attribute 'BlastXml' >> WARNING:galaxy.datatypes.registry:Error appending sniffer for >> datatype >> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has >> no >> attribute 'BlastXml' > > > > Using toolshed installed Abyss and get the same error in Stdout. > > > > Stdout > >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ucsc/publicbuilds.txt' >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ensembl/builds.txt' >> ERROR: Unable to read builds file: [Errno 2] No such file or >> directory: >> >> >> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/ >> sh >> ar >> ed/ncbi/builds.txt' > > Stderr >> COMMAND FAILURE: >> >> >> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abys >> s_ >> to >> >> >> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_ >> wr >> ap >> per.sh: line 9: ABYSS: command not found >> WARNING:galaxy.datatypes.registry:Error loading datatype with >> extension 'blastxml': 'module' object has no attribute 'BlastXml' >> WARNING:galaxy.datatypes.registry:Error appending sniffer for >> datatype >> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has >> no >> attribute 'BlastXml' > > Looking at the util directory based on the expected path of the read > build file > >> >> >> >> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/u >> ti >> l$ >> ls >> aliaspickler.py docutils_ext heartbeat.py json.pyc >> odict.py streamball.py >> aliaspickler.pyc docutils_template.txt inflection.py >> lrucache.py >> odict.pyc streamball.pyc >> backports expressions.py inflection.pyc >> lrucache.pyc >> sanitize_html.py template.py >> bunch.py expressions.pyc __init__.py memdump.py >> sanitize_html.pyc template.pyc >> bunch.pyc hash_util.py __init__.pyc >> none_like.py >> shed_util.py topsort.py >> debugging.py hash_util.pyc json.py >> none_like.pyc >> shed_util.pyc topsort.pyc > > > > > > > > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/
Errors are being output to stdout The files don't exist and complicated by the /../../ where I don't know which subdirectory to follow. Attached screen shot of job failing giving the ERROR message in the web gui. I also get the same errors when attempting to run Abys which doesn't run because it isn't installed. galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ find -name publicbuilds.txt galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py debugging.py hash_util.py __init__.py lrucache.pyc odict.pyc streamball.py topsort.pyc aliaspickler.pyc docutils_ext hash_util.pyc __init__.pyc memdump.py sanitize_html.py streamball.pyc backports docutils_template.txt heartbeat.py json.py none_like.py sanitize_html.pyc template.py bunch.py expressions.py inflection.py json.pyc none_like.pyc shed_util.py template.pyc bunch.pyc expressions.pyc inflection.pyc lrucache.py odict.py shed_util.pyc topsort.py On 11/8/12 1:34 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
Glad to help. The first step would be to check and see if those files actually exist (and they should).
If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong.
-Dannon
On Nov 8, 2012, at 1:27 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Thanks! That was the explanation I was looking for and the link for how to
Do you have any suggestions for fixing the error that are still occurring after running Bowtie?
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ncbi/builds.txt'
On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS.
The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools).
-Dannon
On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other.
Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
> Unfortunately the cloud instance upgrade path requires some manual > intervention here due to tool migrations. SSH in to your instance, > edit > /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any > references to BlastXML. Save the file, restart galaxy, and you > should > be > good to go. > > -Dannon > > On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> > wrote: > >> Running a CloudMan created instance on Amazon. >> >> Updated using cloudman from >> http://bitbucket.org/galaxy/galaxy-central >> to version 8167:b0c916349070 >> >> Tried to run bowtie and get the following error. Searched the >> message >> archives an someone else reported the same problem but thought it >> was >> related to a new version of cufflinks. Same problem also exists >>when >> running abyss. Given the nature of the error message not even sure >> what >> the actual path should be. Also for the abyss install from >>toolshed >> the >> toolshed.g2.bx.psu.edu directory doesn't exist or the six >> directories >> underneath. However Abyss was/is installed at the following >>location >>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ >>>ls >>> ABYSS abyss-adjtodot abyss-joindist abyss-tofastq >>> DistanceEst >>> MergePaths PathConsensus SimpleGraph >>> abyss2ace abyss-fac ABYSS-P AdjList >>>KAligner >>> Overlap PathOverlap >>> abyss2afg abyss-fixmate abyss-pe Consensus >>> MergeContigs >>> ParseAligns PopBubbles >> >> Would be great if someone could give some guidance on how to >>resolve >> the problems. I understand if I need to do something under the >> covers >> but the expectation is that the one click install/updates should >>not >> then require major command line work that isn't documented. >> >> Thanks >> >> Scooter >> >> >> Bowtie >> >> Stdout >> >>> Sequence file aligned. >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ucsc/publicbuilds.txt' >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ensembl/builds.txt' >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ncbi/builds.txt' >> >> Stderr >>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>> datatype >>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>has >>> no >>> attribute 'BlastXml' >> >> >> >> Using toolshed installed Abyss and get the same error in Stdout. >> >> >> >> Stdout >> >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ucsc/publicbuilds.txt' >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ensembl/builds.txt' >>> ERROR: Unable to read builds file: [Errno 2] No such file or >>> directory: >>> >>> >>> >>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>a/ >>> sh >>> ar >>> ed/ncbi/builds.txt' >> >> Stderr >>> COMMAND FAILURE: >>> >>> >>> >>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab >>>ys >>> s_ >>> to >>> >>> >>> >>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys >>>s_ >>> wr >>> ap >>> per.sh: line 9: ABYSS: command not found >>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>> datatype >>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>has >>> no >>> attribute 'BlastXml' >> >> Looking at the util directory based on the expected path of the >>read >> build file >> >>> >>> >>> >>> >>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy >>>/u >>> ti >>> l$ >>> ls >>> aliaspickler.py docutils_ext heartbeat.py json.pyc >>> odict.py streamball.py >>> aliaspickler.pyc docutils_template.txt inflection.py >>> lrucache.py >>> odict.pyc streamball.pyc >>> backports expressions.py inflection.pyc >>> lrucache.pyc >>> sanitize_html.py template.py >>> bunch.py expressions.pyc __init__.py >>>memdump.py >>> sanitize_html.pyc template.pyc >>> bunch.pyc hash_util.py __init__.pyc >>> none_like.py >>> shed_util.py topsort.py >>> debugging.py hash_util.pyc json.py >>> none_like.pyc >>> shed_util.pyc topsort.pyc >> >> >> >> >> >> >> >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >
That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc) Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ? -Dannon On Nov 8, 2012, at 1:41 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Errors are being output to stdout
The files don't exist and complicated by the /../../ where I don't know which subdirectory to follow. Attached screen shot of job failing giving the ERROR message in the web gui. I also get the same errors when attempting to run Abys which doesn't run because it isn't installed.
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ find -name publicbuilds.txt galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py debugging.py hash_util.py __init__.py lrucache.pyc odict.pyc streamball.py topsort.pyc aliaspickler.pyc docutils_ext hash_util.pyc __init__.pyc memdump.py sanitize_html.py streamball.pyc backports docutils_template.txt heartbeat.py json.py none_like.py sanitize_html.pyc template.py bunch.py expressions.py inflection.py json.pyc none_like.pyc shed_util.py template.pyc bunch.pyc expressions.pyc inflection.pyc lrucache.py odict.py shed_util.pyc topsort.py
On 11/8/12 1:34 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
Glad to help. The first step would be to check and see if those files actually exist (and they should).
If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong.
-Dannon
On Nov 8, 2012, at 1:27 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Thanks! That was the explanation I was looking for and the link for how to
Do you have any suggestions for fixing the error that are still occurring after running Bowtie?
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ncbi/builds.txt'
On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS.
The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools).
-Dannon
On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> wrote:
> Ok that fixed Bowtie. > > Any thoughts on expected state of installing Abyss from toolshed. It > indicates no requirements so my expectation is that installing from > toolshed should install the appropriate abyss package in a location > the > tool is expecting to find it. > > In looking at the toolshed looks like two wrappers have been > configured. > One by msjeon and the other edward-kirton. I did the install for > msjeon. > Will try the other. > > Is the abyss install located at > /mnt/galaxyTools/tools/abyss/1.2.5/bin > from prior testing and should be deleted. Current release version of > abyss > as of May 30 2012 is Abyss 1.3.4 > > > > > On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote: > >> Unfortunately the cloud instance upgrade path requires some manual >> intervention here due to tool migrations. SSH in to your instance, >> edit >> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any >> references to BlastXML. Save the file, restart galaxy, and you >> should >> be >> good to go. >> >> -Dannon >> >> On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> >> wrote: >> >>> Running a CloudMan created instance on Amazon. >>> >>> Updated using cloudman from >>> http://bitbucket.org/galaxy/galaxy-central >>> to version 8167:b0c916349070 >>> >>> Tried to run bowtie and get the following error. Searched the >>> message >>> archives an someone else reported the same problem but thought it >>> was >>> related to a new version of cufflinks. Same problem also exists >>> when >>> running abyss. Given the nature of the error message not even sure >>> what >>> the actual path should be. Also for the abyss install from >>> toolshed >>> the >>> toolshed.g2.bx.psu.edu directory doesn't exist or the six >>> directories >>> underneath. However Abyss was/is installed at the following >>> location >>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ >>>> ls >>>> ABYSS abyss-adjtodot abyss-joindist abyss-tofastq >>>> DistanceEst >>>> MergePaths PathConsensus SimpleGraph >>>> abyss2ace abyss-fac ABYSS-P AdjList >>>> KAligner >>>> Overlap PathOverlap >>>> abyss2afg abyss-fixmate abyss-pe Consensus >>>> MergeContigs >>>> ParseAligns PopBubbles >>> >>> Would be great if someone could give some guidance on how to >>> resolve >>> the problems. I understand if I need to do something under the >>> covers >>> but the expectation is that the one click install/updates should >>> not >>> then require major command line work that isn't documented. >>> >>> Thanks >>> >>> Scooter >>> >>> >>> Bowtie >>> >>> Stdout >>> >>>> Sequence file aligned. >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ucsc/publicbuilds.txt' >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ensembl/builds.txt' >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ncbi/builds.txt' >>> >>> Stderr >>>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>>> datatype >>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>> has >>>> no >>>> attribute 'BlastXml' >>> >>> >>> >>> Using toolshed installed Abyss and get the same error in Stdout. >>> >>> >>> >>> Stdout >>> >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ucsc/publicbuilds.txt' >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ensembl/builds.txt' >>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>> directory: >>>> >>>> >>>> >>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>> a/ >>>> sh >>>> ar >>>> ed/ncbi/builds.txt' >>> >>> Stderr >>>> COMMAND FAILURE: >>>> >>>> >>>> >>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab >>>> ys >>>> s_ >>>> to >>>> >>>> >>>> >>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys >>>> s_ >>>> wr >>>> ap >>>> per.sh: line 9: ABYSS: command not found >>>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>>> datatype >>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>> has >>>> no >>>> attribute 'BlastXml' >>> >>> Looking at the util directory based on the expected path of the >>> read >>> build file >>> >>>> >>>> >>>> >>>> >>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy >>>> /u >>>> ti >>>> l$ >>>> ls >>>> aliaspickler.py docutils_ext heartbeat.py json.pyc >>>> odict.py streamball.py >>>> aliaspickler.pyc docutils_template.txt inflection.py >>>> lrucache.py >>>> odict.pyc streamball.pyc >>>> backports expressions.py inflection.pyc >>>> lrucache.pyc >>>> sanitize_html.py template.py >>>> bunch.py expressions.pyc __init__.py >>>> memdump.py >>>> sanitize_html.pyc template.pyc >>>> bunch.pyc hash_util.py __init__.pyc >>>> none_like.py >>>> shed_util.py topsort.py >>>> debugging.py hash_util.pyc json.py >>>> none_like.pyc >>>> shed_util.pyc topsort.pyc >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >
<default.png>
From looking at the abyss wrapper it is expecting ABYSS to be found on
Ok that directory exists but public builds.txt has a .sample at the end. ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc$ ls builds.txt builds.txt.sample publicbuilds.txt.sample ucsc_build_sites.txt.new builds.txt.buildbot manual_builds.txt ucsc_build_sites.txt ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg tip Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy changeset: 8167:b0c916349070 tag: tip user: clements date: Wed Oct 31 23:08:09 2012 -0700 summary: Update docstrings so they no longer generate warnings in Sphix. ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg st Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy M run.sh M tool-data/shared/ucsc/ucsc_build_sites.txt ? paster.log ? paster.log.16_06 ? paster.log.16_46 ? paster.log.17_29 ? paster.log.21_01 ? paster.log.22_24 ? paster.log.22_38 ? paster.log.22_42 ? paster.pid ? static/images/cloud.gif ? static/images/cloud_text.png ? tool-data/HYPHY ? tool-data/shared/ucsc/ucsc_build_sites.txt.new ? tool_data_table_conf.xml.cloud ? tool_data_table_conf.xml_DEF ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg diff Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy diff -r b0c916349070 run.sh --- a/run.sh Wed Oct 31 23:08:09 2012 -0700 +++ b/run.sh Thu Nov 08 19:01:34 2012 +0000 @@ -1,8 +1,9 @@ #!/bin/sh +export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx24-amd64/libdrmaa.so.1.0 cd `dirname $0` -python ./scripts/check_python.py +python -ES ./scripts/check_python.py [ $? -ne 0 ] && exit 1 SAMPLES=" @@ -17,10 +18,7 @@ tool_sheds_conf.xml.sample openid_conf.xml.sample universe_wsgi.ini.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample tool-data/shared/igv/igv_build_sites.txt.sample tool-data/shared/rviewer/rviewer_build_sites.txt.sample tool-data/*.sample @@ -36,20 +34,16 @@ fi done -if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then - python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR" -fi - # explicitly attempt to fetch eggs before running FETCH_EGGS=1 for arg in "$@"; do [ "$arg" = "--stop-daemon" ] && FETCH_EGGS=0; break done if [ $FETCH_EGGS -eq 1 ]; then - python ./scripts/check_eggs.py -q + python -ES ./scripts/check_eggs.py -q if [ $? -ne 0 ]; then echo "Some eggs are out of date, attempting to fetch..." - python ./scripts/fetch_eggs.py + python -ES ./scripts/fetch_eggs.py if [ $? -eq 0 ]; then echo "Fetch successful." else @@ -68,8 +62,8 @@ fi for server in $servers; do echo "Handling $server with log file $server.log..." - python ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@ + python -ES ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@ done else - python ./scripts/paster.py serve universe_wsgi.ini $@ + python -ES ./scripts/paster.py serve universe_wsgi.ini $@ fi diff -r b0c916349070 tool-data/shared/ucsc/ucsc_build_sites.txt --- a/tool-data/shared/ucsc/ucsc_build_sites.txt Wed Oct 31 23:08:09 2012 -0700 +++ b/tool-data/shared/ucsc/ucsc_build_sites.txt Thu Nov 08 19:01:34 2012 +0000 @@ -1,5 +1,5 @@ #Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn -main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2 +main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2,hg19 #Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn archaea http://archaea.ucsc.edu/cgi-bin/hgTracks? alkaEhrl_MLHE_1,shewW318,idioLoih_L2TR,sulSol1,erwiCaro_ATROSEPTICA,symbTher_IAM14863,moorTher_ATCC39073,therFusc_YX,methHung1,bradJapo,therElon,shewPutrCN32,pediPent_ATCC25745,mariMari_MCS10,nanEqu1,baciSubt,chlaTrac,magnMagn_AMB_1,chroViol,ralsSola,acidCryp_JF_5,erytLito_HTCC2594,desuVulg_HILDENBOROUG,pyrAer1,sulfToko1,shewANA3,paraSp_UWE25,geobKaus_HTA426,rhizEtli_CFN_42,uncuMeth_RCI,candBloc_FLORIDANUS,deinRadi,yersPest_CO92,saccEryt_NRRL_2338,rhodRHA1,candCars_RUDDII,burkMall_ATCC23344,eschColi_O157H7,burk383,psycIngr_37,rhodSpha_2_4_1,wolbEndo_OF_DROSOPHIL,burkViet_G4,propAcne_KPA171202,enteFaec_V583,campJeju_81_176,acidJS42,heliPylo_26695,pseuHalo_TAC125,chroSale_DSM3043,methVann1,archFulg1,neisMeni_Z2491_1,fusoNucl,vermEise_EF01_2,anabVari_ATCC29413,tropWhip_TW08_27,heliHepa,acinSp_ADP1,anapMarg_ST_MARIES,natrPhar1,haheChej_KCTC_2396,therPetr_RKU_1,neisGono_FA1090_1,colwPsyc_34H,desuPsyc_LSV54,hyphNept_ATCC15444,vibrChol1,deinGeot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_TULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDNIH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00 #Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn On 11/8/12 1:55 PM, "Dannon Baker" <dannonbaker@me.com<mailto:dannonbaker@me.com>> wrote: That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc) Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ? -Dannon On Nov 8, 2012, at 1:41 PM, Scooter Willis <HWillis@scripps.edu<mailto:HWillis@scripps.edu>> wrote: Errors are being output to stdout The files don't exist and complicated by the /../../ where I don't know which subdirectory to follow. Attached screen shot of job failing giving the ERROR message in the web gui. I also get the same errors when attempting to run Abys which doesn't run because it isn't installed. galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ find -name publicbuilds.txt galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py debugging.py hash_util.py __init__.py lrucache.pyc odict.pyc streamball.py topsort.pyc aliaspickler.pyc docutils_ext hash_util.pyc __init__.pyc memdump.py sanitize_html.py streamball.pyc backports docutils_template.txt heartbeat.py json.py none_like.py sanitize_html.pyc template.py bunch.py expressions.py inflection.py json.pyc none_like.pyc shed_util.py template.pyc bunch.pyc expressions.pyc inflection.pyc lrucache.py odict.py shed_util.pyc topsort.py On 11/8/12 1:34 PM, "Dannon Baker" <dannonbaker@me.com<mailto:dannonbaker@me.com>> wrote: Glad to help. The first step would be to check and see if those files actually exist (and they should). If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong. -Dannon On Nov 8, 2012, at 1:27 PM, Scooter Willis <HWillis@scripps.edu<mailto:HWillis@scripps.edu>> wrote: Thanks! That was the explanation I was looking for and the link for how to Do you have any suggestions for fixing the error that are still occurring after running Bowtie? Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ncbi/builds.txt' On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com<mailto:dannonbaker@me.com>> wrote: You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS. The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools). -Dannon On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu<mailto:HWillis@scripps.edu>> wrote: Dannon If I install abyss to the galaxy tools volume and use cloud admin to do a snapshot I assume I then need to remove and then add compute nodes? From the EC2 management console I see two 15GiB EBS volumes from the same snapshot and I have a master and one compute node running. The other approach is that the compute nodes have a NFS mount back into the galaxy_tools volume. Trying to make sure I understand the need to restart the compute nodes after an install of third party tools. the path via a .sh file that gets called. Where would I update the path to a particular installed tool such that it will be picked up for each compute node that gets started? I couldn't find any examples of doing a third party install with discussion about how that install would persist to future compute nodes. The missing element for me at the moment is where to make path related changes that will be picked up by compute nodes. I could hard code the path in the .sh file used by the wrapper. Just trying to figure out how to use the software as designed. Thanks Scooter On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com<mailto:dannonbaker@me.com>> wrote: Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user. You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own. The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel). On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu<mailto:HWillis@scripps.edu>> wrote: Ok that fixed Bowtie. Any thoughts on expected state of installing Abyss from toolshed. It indicates no requirements so my expectation is that installing from toolshed should install the appropriate abyss package in a location the tool is expecting to find it. In looking at the toolshed looks like two wrappers have been configured. One by msjeon and the other edward-kirton. I did the install for msjeon. Will try the other. Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin from prior testing and should be deleted. Current release version of abyss as of May 30 2012 is Abyss 1.3.4 On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com<mailto:dannonbaker@me.com>> wrote: Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go. -Dannon On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu<mailto:HWillis@scripps.edu>> wrote: Running a CloudMan created instance on Amazon. Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070 Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented. Thanks Scooter Bowtie Stdout Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ncbi/builds.txt' Stderr WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Using toolshed installed Abyss and get the same error in Stdout. Stdout ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat a/ sh ar ed/ncbi/builds.txt' Stderr COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab ys s_ to ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys s_ wr ap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Looking at the util directory based on the expected path of the read build file galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy /u ti l$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ <default.png>
Ah ha! This is the problem -- your run.sh is incorrect due to what I'm assuming was a failed merge. Here's a correct copy, note the inclusion of the extra .sample files. Modify or replace your run.sh with this (which is from the most recent cloud images), restart galaxy, and you're good to go. On Nov 8, 2012, at 2:03 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Ok that directory exists but public builds.txt has a .sample at the end.
ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc$ ls builds.txt builds.txt.sample publicbuilds.txt.sample ucsc_build_sites.txt.new builds.txt.buildbot manual_builds.txt ucsc_build_sites.txt
ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg tip Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy changeset: 8167:b0c916349070 tag: tip user: clements date: Wed Oct 31 23:08:09 2012 -0700 summary: Update docstrings so they no longer generate warnings in Sphix.
ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg st Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy M run.sh M tool-data/shared/ucsc/ucsc_build_sites.txt ? paster.log ? paster.log.16_06 ? paster.log.16_46 ? paster.log.17_29 ? paster.log.21_01 ? paster.log.22_24 ? paster.log.22_38 ? paster.log.22_42 ? paster.pid ? static/images/cloud.gif ? static/images/cloud_text.png ? tool-data/HYPHY ? tool-data/shared/ucsc/ucsc_build_sites.txt.new ? tool_data_table_conf.xml.cloud ? tool_data_table_conf.xml_DEF ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg diff Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy diff -r b0c916349070 run.sh --- a/run.sh Wed Oct 31 23:08:09 2012 -0700 +++ b/run.sh Thu Nov 08 19:01:34 2012 +0000 @@ -1,8 +1,9 @@ #!/bin/sh +export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx24-amd64/libdrmaa.so.1.0 cd `dirname $0` -python ./scripts/check_python.py +python -ES ./scripts/check_python.py [ $? -ne 0 ] && exit 1 SAMPLES=" @@ -17,10 +18,7 @@ tool_sheds_conf.xml.sample openid_conf.xml.sample universe_wsgi.ini.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample tool-data/shared/igv/igv_build_sites.txt.sample tool-data/shared/rviewer/rviewer_build_sites.txt.sample tool-data/*.sample @@ -36,20 +34,16 @@ fi done -if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then - python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR" -fi - # explicitly attempt to fetch eggs before running FETCH_EGGS=1 for arg in "$@"; do [ "$arg" = "--stop-daemon" ] && FETCH_EGGS=0; break done if [ $FETCH_EGGS -eq 1 ]; then - python ./scripts/check_eggs.py -q + python -ES ./scripts/check_eggs.py -q if [ $? -ne 0 ]; then echo "Some eggs are out of date, attempting to fetch..." - python ./scripts/fetch_eggs.py + python -ES ./scripts/fetch_eggs.py if [ $? -eq 0 ]; then echo "Fetch successful." else @@ -68,8 +62,8 @@ fi for server in $servers; do echo "Handling $server with log file $server.log..." - python ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@ + python -ES ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@ done else - python ./scripts/paster.py serve universe_wsgi.ini $@ + python -ES ./scripts/paster.py serve universe_wsgi.ini $@ fi diff -r b0c916349070 tool-data/shared/ucsc/ucsc_build_sites.txt --- a/tool-data/shared/ucsc/ucsc_build_sites.txt Wed Oct 31 23:08:09 2012 -0700 +++ b/tool-data/shared/ucsc/ucsc_build_sites.txt Thu Nov 08 19:01:34 2012 +0000 @@ -1,5 +1,5 @@ #Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn -main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2 +main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2,hg19 #Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn archaea http://archaea.ucsc.edu/cgi-bin/hgTracks? alkaEhrl_MLHE_1,shewW318,idioLoih_L2TR,sulSol1,erwiCaro_ATROSEPTICA,symbTher_IAM14863,moorTher_ATCC39073,therFusc_YX,methHung1,bradJapo,therElon,shewPutrCN32,pediPent_ATCC25745,mariMari_MCS10,nanEqu1,baciSubt,chlaTrac,magnMagn_AMB_1,chroViol,ralsSola,acidCryp_JF_5,erytLito_HTCC2594,desuVulg_HILDENBOROUG,pyrAer1,sulfToko1,shewANA3,paraSp_UWE25,geobKaus_HTA426,rhizEtli_CFN_42,uncuMeth_RCI,candBloc_FLORIDANUS,deinRadi,yersPest_CO92,saccEryt_NRRL_2338,rhodRHA1,candCars_RUDDII,burkMall_ATCC23344,eschColi_O157H7,burk383,psycIngr_37,rhodSpha_2_4_1,wolbEndo_OF_DROSOPHIL,burkViet_G4,propAcne_KPA171202,enteFaec_V583,campJeju_81_176,acidJS42,heliPylo_26695,pseuHalo_TAC125,chroSale_DSM3043,methVann1,archFulg1,neisMeni_Z2491_1,fusoNucl,vermEise_EF01_2,anabVari_ATCC29413,tropWhip_TW08_27,heliHepa,acinSp_ADP1,anapMarg_ST_MARIES,natrPhar1,haheChej_KCTC_2396,therPetr_RKU_1,neisGono_FA1090_1,colwPsyc_34H,desuPsyc_LSV54,hyphNept_ATCC15444,vibrChol1,deinGeot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_TULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDNIH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00 #Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn
On 11/8/12 1:55 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc)
Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ?
-Dannon
On Nov 8, 2012, at 1:41 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Errors are being output to stdout The files don't exist and complicated by the /../../ where I don't know which subdirectory to follow. Attached screen shot of job failing giving the ERROR message in the web gui. I also get the same errors when attempting to run Abys which doesn't run because it isn't installed. galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ find -name publicbuilds.txt galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py debugging.py hash_util.py __init__.py lrucache.pyc odict.pyc streamball.py topsort.pyc aliaspickler.pyc docutils_ext hash_util.pyc __init__.pyc memdump.py sanitize_html.py streamball.pyc backports docutils_template.txt heartbeat.py json.py none_like.py sanitize_html.pyc template.py bunch.py expressions.py inflection.py json.pyc none_like.pyc shed_util.py template.pyc bunch.pyc expressions.pyc inflection.pyc lrucache.py odict.py shed_util.pyc topsort.py On 11/8/12 1:34 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
Glad to help. The first step would be to check and see if those files actually exist (and they should). If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong. -Dannon On Nov 8, 2012, at 1:27 PM, Scooter Willis <HWillis@scripps.edu> wrote:
Thanks! That was the explanation I was looking for and the link for how to Do you have any suggestions for fixing the error that are still occurring after running Bowtie? Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share d/ ncbi/builds.txt' On 11/8/12 12:44 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS. The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools). -Dannon On Nov 8, 2012, at 12:09 PM, Scooter Willis <HWillis@scripps.edu> wrote: > Dannon > If I install abyss to the galaxy tools volume and use cloud admin to > do > a > snapshot I assume I then need to remove and then add compute nodes? > From > the EC2 management console I see two 15GiB EBS volumes from the same > snapshot and I have a master and one compute node running. The other > approach is that the compute nodes have a NFS mount back into the > galaxy_tools volume. Trying to make sure I understand the need to > restart > the compute nodes after an install of third party tools. > From looking at the abyss wrapper it is expecting ABYSS to be found on > the > path via a .sh file that gets called. Where would I update the path > to a > particular installed tool such that it will be picked up for each > compute > node that gets started? I couldn't find any examples of doing a third > party install with discussion about how that install would persist to > future compute nodes. The missing element for me at the moment is > where > to > make path related changes that will be picked up by compute nodes. I > could > hard code the path in the .sh file used by the wrapper. Just trying to > figure out how to use the software as designed. > Thanks > Scooter > On 11/8/12 11:49 AM, "Dannon Baker" <dannonbaker@me.com> wrote: >> Neither abyss wrapper in the toolshed installs binaries (you'd see a >> tool_dependencies.xml in the repository), that's left up to the end >> user. >> You might contact the authors of the wrappers via the toolshed to see >> if >> they were going to add dependency management and had a newer version, >> but >> otherwise you'll have to install binaries on your own. The cloud >> volumes >> do have a version of the Abyss binaries preinstalled, but it's >> certainly >> possible to add a newer version to your particular instance and >> persist >> the changes to the tool volume (see the cloud admin panel). >> On Nov 8, 2012, at 11:20 AM, Scooter Willis <HWillis@scripps.edu> >> wrote: >>> Ok that fixed Bowtie. >>> Any thoughts on expected state of installing Abyss from toolshed. It >>> indicates no requirements so my expectation is that installing from >>> toolshed should install the appropriate abyss package in a location >>> the >>> tool is expecting to find it. >>> In looking at the toolshed looks like two wrappers have been >>> configured. >>> One by msjeon and the other edward-kirton. I did the install for >>> msjeon. >>> Will try the other. >>> Is the abyss install located at >>> /mnt/galaxyTools/tools/abyss/1.2.5/bin >>> from prior testing and should be deleted. Current release version of >>> abyss >>> as of May 30 2012 is Abyss 1.3.4 >>> On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote: >>>> Unfortunately the cloud instance upgrade path requires some manual >>>> intervention here due to tool migrations. SSH in to your instance, >>>> edit >>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any >>>> references to BlastXML. Save the file, restart galaxy, and you >>>> should >>>> be >>>> good to go. >>>> -Dannon >>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> >>>> wrote: >>>>> Running a CloudMan created instance on Amazon. >>>>> Updated using cloudman from >>>>> http://bitbucket.org/galaxy/galaxy-central >>>>> to version 8167:b0c916349070 >>>>> Tried to run bowtie and get the following error. Searched the >>>>> message >>>>> archives an someone else reported the same problem but thought it >>>>> was >>>>> related to a new version of cufflinks. Same problem also exists >>>>> when >>>>> running abyss. Given the nature of the error message not even sure >>>>> what >>>>> the actual path should be. Also for the abyss install from >>>>> toolshed >>>>> the >>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six >>>>> directories >>>>> underneath. However Abyss was/is installed at the following >>>>> location >>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ >>>>>> ls >>>>>> ABYSS abyss-adjtodot abyss-joindist abyss-tofastq >>>>>> DistanceEst >>>>>> MergePaths PathConsensus SimpleGraph >>>>>> abyss2ace abyss-fac ABYSS-P AdjList >>>>>> KAligner >>>>>> Overlap PathOverlap >>>>>> abyss2afg abyss-fixmate abyss-pe Consensus >>>>>> MergeContigs >>>>>> ParseAligns PopBubbles >>>>> Would be great if someone could give some guidance on how to >>>>> resolve >>>>> the problems. I understand if I need to do something under the >>>>> covers >>>>> but the expectation is that the one click install/updates should >>>>> not >>>>> then require major command line work that isn't documented. >>>>> Thanks >>>>> Scooter >>>>> Bowtie >>>>> Stdout >>>>>> Sequence file aligned. >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ucsc/publicbuilds.txt' >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ensembl/builds.txt' >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ncbi/builds.txt' >>>>> Stderr >>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>>>>> datatype >>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>>>> has >>>>>> no >>>>>> attribute 'BlastXml' >>>>> Using toolshed installed Abyss and get the same error in Stdout. >>>>> Stdout >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ucsc/publicbuilds.txt' >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ensembl/builds.txt' >>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or >>>>>> directory: >>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat >>>>>> a/ >>>>>> sh >>>>>> ar >>>>>> ed/ncbi/builds.txt' >>>>> Stderr >>>>>> COMMAND FAILURE: >>>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab >>>>>> ys >>>>>> s_ >>>>>> to >>>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys >>>>>> s_ >>>>>> wr >>>>>> ap >>>>>> per.sh: line 9: ABYSS: command not found >>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with >>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml' >>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for >>>>>> datatype >>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object >>>>>> has >>>>>> no >>>>>> attribute 'BlastXml' >>>>> Looking at the util directory based on the expected path of the >>>>> read >>>>> build file >>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy >>>>>> /u >>>>>> ti >>>>>> l$ >>>>>> ls >>>>>> aliaspickler.py docutils_ext heartbeat.py json.pyc >>>>>> odict.py streamball.py >>>>>> aliaspickler.pyc docutils_template.txt inflection.py >>>>>> lrucache.py >>>>>> odict.pyc streamball.pyc >>>>>> backports expressions.py inflection.pyc >>>>>> lrucache.pyc >>>>>> sanitize_html.py template.py >>>>>> bunch.py expressions.pyc __init__.py >>>>>> memdump.py >>>>>> sanitize_html.pyc template.pyc >>>>>> bunch.pyc hash_util.py __init__.pyc >>>>>> none_like.py >>>>>> shed_util.py topsort.py >>>>>> debugging.py hash_util.pyc json.py >>>>>> none_like.pyc >>>>>> shed_util.pyc topsort.pyc >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> http://lists.bx.psu.edu/ <default.png>
I removed two BlastXML references and restarted galaxy and a new compute node. This problem still exists and is found at stdout. Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ ncbi/builds.txt' On 11/8/12 11:00 AM, "Dannon Baker" <dannonbaker@me.com> wrote:
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations. SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML. Save the file, restart galaxy, and you should be good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <HWillis@scripps.edu> wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles
Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap per.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
Looking at the util directory based on the expected path of the read build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Dannon Baker -
Scooter Willis