Catch problems preparing cluster job scripts
Hi all, I've run into some file permissions problems as part of using the same mapped directory on both the Galaxy server and our cluster. In the process I wrote the following patch which fixes the following bug - where Galaxy seems to leave the job in the pending state: galaxy.jobs INFO 2011-07-29 14:06:46,170 job 30 dispatched galaxy.jobs.runners.drmaa ERROR 2011-07-29 14:06:46,582 Uncaught exception queueing job Traceback (most recent call last): File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 114, in run_next self.queue_job( obj ) File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 164, in queue_job os.chmod( jt.remoteCommand, 0750 ) OSError: [Errno 1] Operation not permitted: '/data/galaxy/galaxy-central/database/pbs/galaxy_30.sh' The jobs was left stuck in the grey pending state. It looks like this exception should have been caught and the job put into an error state as in this patch: https://bitbucket.org/peterjc/galaxy-central/changeset/c5fa48633c0b This is currently the one and only change on this branch: https://bitbucket.org/peterjc/galaxy-central/src/job_scripts Please could this be reviewed and applied to the trunk. Thanks, Peter
On Fri, Jul 29, 2011 at 2:22 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I've run into some file permissions problems as part of using the same mapped directory on both the Galaxy server and our cluster. In the process I wrote the following patch which fixes the following bug - where Galaxy seems to leave the job in the pending state:
galaxy.jobs INFO 2011-07-29 14:06:46,170 job 30 dispatched galaxy.jobs.runners.drmaa ERROR 2011-07-29 14:06:46,582 Uncaught exception queueing job Traceback (most recent call last): File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 114, in run_next self.queue_job( obj ) File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 164, in queue_job os.chmod( jt.remoteCommand, 0750 ) OSError: [Errno 1] Operation not permitted: '/data/galaxy/galaxy-central/database/pbs/galaxy_30.sh'
The jobs was left stuck in the grey pending state. It looks like this exception should have been caught and the job put into an error state as in this patch:
https://bitbucket.org/peterjc/galaxy-central/changeset/c5fa48633c0b
This is currently the one and only change on this branch:
https://bitbucket.org/peterjc/galaxy-central/src/job_scripts
Please could this be reviewed and applied to the trunk.
Thanks,
Peter
Should I file a bug with this patch? Peter
Hi Peter, Yes, if you wanted to create a ticket in bitbucket for this patch that would be great. It is the best way to help us to keep track/prioritize. Please just submit, but don't change status from "new" or assign to anyone in particular. Do feel free to note whether a ticket is intended as a bug (fix), enhancement, or proposal. Ideally, all submitted patches (from anyone) should go there. Then, the creator can email galaxy-dev with the link and a description to let our team and the other users know about it. http://bitbucket.org/galaxy/galaxy-central/issues When a tool is added to the Tool Shed, an email to galaxy-dev is also welcomed, to let the community know about what's new or to gather feedback. We encourage tool developers to submit first to the Tool Shed any stand-alone tools, then request if the tool might be appropriate for the core build (if they want and use is broad enough) as an enhancement. http://galaxyproject.org/Tool%20Shed To remind others (readers of this email), bitbucket is not intended to replace the mailing lists for suspected bugs/bug reports, questions, or general discussion. http://galaxyproject.org/Support The galaxy-dev list has become a very active community and we really appreciate all of your input Peter! Take care, Jen Galaxy team On 8/4/11 2:51 AM, Peter Cock wrote:
On Fri, Jul 29, 2011 at 2:22 PM, Peter Cock<p.j.a.cock@googlemail.com> wrote:
Hi all,
I've run into some file permissions problems as part of using the same mapped directory on both the Galaxy server and our cluster. In the process I wrote the following patch which fixes the following bug - where Galaxy seems to leave the job in the pending state:
galaxy.jobs INFO 2011-07-29 14:06:46,170 job 30 dispatched galaxy.jobs.runners.drmaa ERROR 2011-07-29 14:06:46,582 Uncaught exception queueing job Traceback (most recent call last): File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 114, in run_next self.queue_job( obj ) File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 164, in queue_job os.chmod( jt.remoteCommand, 0750 ) OSError: [Errno 1] Operation not permitted: '/data/galaxy/galaxy-central/database/pbs/galaxy_30.sh'
The jobs was left stuck in the grey pending state. It looks like this exception should have been caught and the job put into an error state as in this patch:
https://bitbucket.org/peterjc/galaxy-central/changeset/c5fa48633c0b
This is currently the one and only change on this branch:
https://bitbucket.org/peterjc/galaxy-central/src/job_scripts
Please could this be reviewed and applied to the trunk.
Thanks,
Peter
Should I file a bug with this patch?
Peter
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support
Hi Jen, On Fri, Aug 5, 2011 at 8:27 PM, Jennifer Jackson <jen@bx.psu.edu> wrote:
Hi Peter,
Yes, if you wanted to create a ticket in bitbucket for this patch that would be great. It is the best way to help us to keep track/prioritize. Please just submit, but don't change status from "new" or assign to anyone in particular. Do feel free to note whether a ticket is intended as a bug (fix), enhancement, or proposal.
Done, https://bitbucket.org/galaxy/galaxy-central/issue/639/catch-problems-prepari...
To remind others (readers of this email), bitbucket is not intended to replace the mailing lists for suspected bugs/bug reports, questions, or general discussion. http://galaxyproject.org/Support
Could you clarify that bit on the wiki that reads "If you think you've seen a bug - please, report it to the Galaxy Development List rather than the issue tracker", since this seems to requesting email first rather than filing a bug. Peter
Hi Peter, On 8/9/11 2:55 AM, Peter Cock wrote:
Hi Jen,
<more>
To remind others (readers of this email), bitbucket is not intended to replace the mailing lists for suspected bugs/bug reports, questions, or general discussion. http://galaxyproject.org/Support
Could you clarify that bit on the wiki that reads "If you think you've seen a bug - please, report it to the Galaxy Development List rather than the issue tracker", since this seems to requesting email first rather than filing a bug.
Yes, emailing to the galaxy-dev@bx.psu.edu mailing list is the preferred way to vet out a potential bug with local or cloud installs Once a problem is confirmed, then we can work together to decide if it can simply be fixed right away (and by whom) or if it is longer-term issue that would be best tracked by opening a ticket. Thanks for asking - we are working on updating the wiki in this area over the next few weeks, so are happy to work together to come up with good & clear communication plan. I'll definitely use your comments to clarify some of these statements. Thanks as always Peter! Jen Galaxy team
Peter
-- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support
participants (2)
-
Jennifer Jackson
-
Peter Cock