Problem running ncbi_makeblastdb as real user in cluster
Hello, I am setting up a local galaxy installation and using our local SGE cluster to run the jobs as real users. One of the first few tools I tested was the ncbi makebkastdb tool. When I try to run it the history for that particular job is immediately shown as follows: 0: (unnamed dataset) Failed to retrieve dataset information. An error occurred with this dataset:/hasattr(): attribute name must be string/ In the stdout of the job I was getting the following error: Fatal error: Matched on Error: Error: NCBI C++ Exception: "/usr/local/src/ncbiblast+/2.2.27/c++/src/objtools/blast/seqdb_writer/build_db.cpp", line 979: Error: ncbi::s_CreateDirectories() - Failed to create directory 'dataset_38_files' I finally figured out that the makedb wrapper was using $outfile.extra_files_path instead of $outfile.files_path. I fixed the wrapper and restarted the instance. The job now runs ok and reports successful completion in the stdout. But, the history for this particular job is still shown as explained above. Can somebody help me figure out what is going wrong in my setup? Here are some of the relevant logs: ==> web1.log <== 128.192.203.31 - - [19/Mar/2013:15:14:35 -0400] "POST /tool_runner/index HTTP/1.1" 200 - "http://galaxy.qbcg.uga.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos/..." "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22" ==> web0.log <== 128.192.203.31 - - [19/Mar/2013:15:14:36 -0400] "GET /history HTTP/1.1" 200 - "http://galaxy.qbcg.uga.edu/tool_runner/index" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22" galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,161 Error in history API at listing contents with history d413a19dec13d11e, hda e89067bb68bee7a0: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,316 Error in history API at listing contents with history d413a19dec13d11e, hda ba03619785539f8c: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,316 Error in history API at listing contents with history d413a19dec13d11e, hda cbbbf59e8f08c98c: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,317 Error in history API at listing contents with history d413a19dec13d11e, hda 964b37715ec9bd22: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,317 Error in history API at listing contents with history d413a19dec13d11e, hda 1fad1eaf5f4f1766: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,318 Error in history API at listing contents with history d413a19dec13d11e, hda 2fdbd5c5858e78fb: hasattr(): attribute name must be string ==> manager.log <== galaxy.jobs.manager DEBUG 2013-03-19 15:14:40,856 (42) Job assigned to handler 'handler0' ==> handler0.log <== galaxy.jobs DEBUG 2013-03-19 15:14:41,790 (42) Working directory for job is: /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy.jobs.rules.200_rules DEBUG 2013-03-19 15:14:41,791 toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/0.0.1 galaxy.jobs.handler DEBUG 2013-03-19 15:14:41,792 dispatching job 42 to drmaa runner galaxy.jobs.handler INFO 2013-03-19 15:14:41,882 (42) Job dispatched galaxy.tools DEBUG 2013-03-19 15:14:42,068 Building dependency shell command for dependency 'makeblastdb' galaxy.tools WARNING 2013-03-19 15:14:42,068 Failed to resolve dependency on 'makeblastdb', ignoring galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,453 (42) submitting file /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/pbs/galaxy_42.sh galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,453 (42) command is: makeblastdb -version &> /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_42; makeblastdb -out /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/dataset_41_files/blastdb -hash_index -in " /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/files/000/dataset_30.dat " -title "wert" -dbtype nucl; cd /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist; /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/set_metadata.sh ./database/files /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 . /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/universe_wsgi.ini /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/tmp/tmpxi116L /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy.json /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_in_HistoryDatasetAssociation_40_4YjnIJ,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_kwds_HistoryDatasetAssociation_40_fWHejS,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_out_HistoryDatasetAssociation_40_BwBD1y,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_results_HistoryDatasetAssociation_40_0IjMAn,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy_dataset_41.dat,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_override_HistoryDatasetAssociation_40_pXhQCd galaxy.jobs DEBUG 2013-03-19 15:14:42,456 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 raj76 1083 galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,766 (42) submitting with credentials: raj76 [uid: 740] galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,774 (42) Job script for external submission is: /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/pbs/42.jt_json galaxy.jobs.runners.drmaa INFO 2013-03-19 15:14:43,198 (42) queued as 2246095 galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:43,420 (42/2246095) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:54,444 (42/2246095) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:15:04,614 (42/2246095) job left DRM queue with following message: code 18: The job specified by the 'jobid' does not exist. galaxy.jobs DEBUG 2013-03-19 15:15:04,648 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy 1234 galaxy.jobs DEBUG 2013-03-19 15:15:06,988 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy 1234 galaxy.jobs DEBUG 2013-03-19 15:15:07,833 finish(): Moved /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy_dataset_41.dat to /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/files/000/dataset_41.dat galaxy.tools DEBUG 2013-03-19 15:15:09,546 Error opening galaxy.json file: [Errno 2] No such file or directory: '/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy.json' galaxy.jobs DEBUG 2013-03-19 15:15:09,626 job 42 ended galaxy.datatypes.metadata DEBUG 2013-03-19 15:15:09,626 Cleaning up external metadata files galaxy.objectstore CRITICAL 2013-03-19 15:15:09,665 /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 delete error [Errno 39] Directory not empty: '/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42' ==> web1.log <== sqlalchemy.pool.QueuePool.0x...4350 WARNING 2013-03-19 15:16:14,801 Error closing cursor: current transaction is aborted, commands ignored until end of transaction block Thanks, -Raj
Il giorno mar, 19/03/2013 alle 15.20 -0400, Raj Ayyampalayam ha scritto:
One of the first few tools I tested was the ncbi makebkastdb tool. When I try to run it the history for that particular job is immediately shown as follows:
0: (unnamed dataset) Failed to retrieve dataset information. An error occurred with this dataset:/hasattr(): attribute name must be string/
Hi Raj, this error in the history should be fixed by this patch: https://bitbucket.org/nsoranzo/peterjc-galaxy-central-tools2/commits/35163cb... I have sent a pull request to Peter (the maintainer), hopefully he will release soon a new version of blast_datatypes with this fix. Best, Nicola
On Wed, Mar 20, 2013 at 10:08 AM, Nicola Soranzo <soranzo@crs4.it> wrote:
Il giorno mar, 19/03/2013 alle 15.20 -0400, Raj Ayyampalayam ha scritto:
One of the first few tools I tested was the ncbi makebkastdb tool. When I try to run it the history for that particular job is immediately shown as follows:
0: (unnamed dataset) Failed to retrieve dataset information. An error occurred with this dataset:/hasattr(): attribute name must be string/
Hi Raj, this error in the history should be fixed by this patch:
https://bitbucket.org/nsoranzo/peterjc-galaxy-central-tools2/commits/35163cb...
I have sent a pull request to Peter (the maintainer), hopefully he will release soon a new version of blast_datatypes with this fix.
Best, Nicola
The patch just removes the MetadataElement call - is that wise? Looking at the code, I don't see a recent API change regarding the name: https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/... Is there perhaps a universe_wsgi.ini setting which might be involved, since I've not seen this error locally? Peter
Il giorno mer, 20/03/2013 alle 12.41 +0000, Peter Cock ha scritto:
On Wed, Mar 20, 2013 at 10:08 AM, Nicola Soranzo <soranzo@crs4.it> wrote:
Il giorno mar, 19/03/2013 alle 15.20 -0400, Raj Ayyampalayam ha scritto:
One of the first few tools I tested was the ncbi makebkastdb tool. When I try to run it the history for that particular job is immediately shown as follows:
0: (unnamed dataset) Failed to retrieve dataset information. An error occurred with this dataset:/hasattr(): attribute name must be string/
Hi Raj, this error in the history should be fixed by this patch:
https://bitbucket.org/nsoranzo/peterjc-galaxy-central-tools2/commits/35163cb...
I have sent a pull request to Peter (the maintainer), hopefully he will release soon a new version of blast_datatypes with this fix.
Best, Nicola
The patch just removes the MetadataElement call - is that wise?
Hi Peter, my question instead is, what are they for? With 'name' they have no meaning.
Looking at the code, I don't see a recent API change regarding the name:
https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/...
Is there perhaps a universe_wsgi.ini setting which might be involved, since I've not seen this error locally?
$ cd lib/galaxy/datatypes/ $ grep 'MetadataElement(' *.py|grep -v name returns nothing, so the 'name' parameter is really mandatory. Best, Nicola
On Wed, Mar 20, 2013 at 1:52 PM, Nicola Soranzo <soranzo@crs4.it> wrote:
Il giorno mer, 20/03/2013 alle 12.41 +0000, Peter Cock ha scritto:
The patch just removes the MetadataElement call - is that wise?
Hi Peter, my question instead is, what are they for? With 'name' they have no meaning.
Looking at the code, I don't see a recent API change regarding the name:
https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/...
Is there perhaps a universe_wsgi.ini setting which might be involved, since I've not seen this error locally?
$ cd lib/galaxy/datatypes/ $ grep 'MetadataElement(' *.py|grep -v name
returns nothing, so the 'name' parameter is really mandatory.
Best, Nicola
Good question - I'd have to ask Edward what he thought this did, but you seem to be right that as the code stands this metadata element is rather pointless. (I'm still puzzled why we don't see the error here though). I'll apply your patch and test it locally... Peter
On Wed, Mar 20, 2013 at 1:55 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Mar 20, 2013 at 1:52 PM, Nicola Soranzo <soranzo@crs4.it> wrote:
Il giorno mer, 20/03/2013 alle 12.41 +0000, Peter Cock ha scritto:
The patch just removes the MetadataElement call - is that wise?
Hi Peter, my question instead is, what are they for? With 'name' they have no meaning.
Looking at the code, I don't see a recent API change regarding the name:
https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/...
Is there perhaps a universe_wsgi.ini setting which might be involved, since I've not seen this error locally?
$ cd lib/galaxy/datatypes/ $ grep 'MetadataElement(' *.py|grep -v name
returns nothing, so the 'name' parameter is really mandatory.
Best, Nicola
Good question - I'd have to ask Edward what he thought this did, but you seem to be right that as the code stands this metadata element is rather pointless.
(I'm still puzzled why we don't see the error here though).
I'll apply your patch and test it locally...
Peter
Nicola - It looks good here, uploaded to the ToolShed as v0.0.15 http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes Raj - Could you apply the update from the ToolShed and confirm if that fixes the problem for you? Edward - have you received my direct email off list? Thanks, Peter
Hi Guys, Thanks for the quick response. I installed the latest blast datatypes from the toolshed and that fixed the issue. Peter, can you also fix the $outfile.extra_files_path to $outfile.files_path on line 5 in the makedb wrapper xml file. Thanks, -Raj On Wednesday, March 20, 2013 10:42:43 AM, Peter Cock wrote:
On Wed, Mar 20, 2013 at 1:55 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Mar 20, 2013 at 1:52 PM, Nicola Soranzo <soranzo@crs4.it> wrote:
Il giorno mer, 20/03/2013 alle 12.41 +0000, Peter Cock ha scritto:
The patch just removes the MetadataElement call - is that wise?
Hi Peter, my question instead is, what are they for? With 'name' they have no meaning.
Looking at the code, I don't see a recent API change regarding the name:
https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/...
Is there perhaps a universe_wsgi.ini setting which might be involved, since I've not seen this error locally?
$ cd lib/galaxy/datatypes/ $ grep 'MetadataElement(' *.py|grep -v name
returns nothing, so the 'name' parameter is really mandatory.
Best, Nicola
Good question - I'd have to ask Edward what he thought this did, but you seem to be right that as the code stands this metadata element is rather pointless.
(I'm still puzzled why we don't see the error here though).
I'll apply your patch and test it locally...
Peter
Nicola - It looks good here, uploaded to the ToolShed as v0.0.15 http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes
Raj - Could you apply the update from the ToolShed and confirm if that fixes the problem for you?
Edward - have you received my direct email off list?
Thanks,
Peter
-- Bio-informatics consultant QBCG (http://qbcg.uga.edu) 706-542-6092 706-583-0442
On Wed, Mar 20, 2013 at 3:43 PM, Raj Ayyampalayam <raj76@uga.edu> wrote:
Hi Guys,
Thanks for the quick response. I installed the latest blast datatypes from the toolshed and that fixed the issue.
Great - thanks for letting us know so promptly.
Peter, can you also fix the $outfile.extra_files_path to $outfile.files_path on line 5 in the makedb wrapper xml file.
What do you mean 'fix' it? It should be using extra_files_path shouldn't it? This is a composite datatype in Galaxy. Peter
On Wed, Mar 20, 2013 at 3:47 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Mar 20, 2013 at 3:43 PM, Raj Ayyampalayam <raj76@uga.edu> wrote:
Hi Guys,
Thanks for the quick response. I installed the latest blast datatypes from the toolshed and that fixed the issue.
Great - thanks for letting us know so promptly.
Peter, can you also fix the $outfile.extra_files_path to $outfile.files_path on line 5 in the makedb wrapper xml file.
What do you mean 'fix' it? It should be using extra_files_path shouldn't it? This is a composite datatype in Galaxy.
Are you perhaps running jobs as real user? That might explain when this difference is important: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-April/009579.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-March/013889.html Thanks, Peter
participants (3)
-
Nicola Soranzo -
Peter Cock -
Raj Ayyampalayam