Hi Dev,
I am running a local instance Galaxy for developing a variant calling pipeline. I ran into this issue after configuring and installing GATK2 on my local when running the GATK *Realigner Target Creator on the BAM file generated by Mark duplicates.* *Can some suggest how to resolve this issue or please point to how to debug this issue from the GATK2 wrapper? BTW, I follow the instruction by ensuring that my GATK3 environment was set as stated in the Galaxy GATk2 readme.txt.*
Below here was the issue I encountered;
##### ERROR ------------------------------------------------------------------------------------------ ##### ERROR A USER ERROR has occurred (version 2.0-7-g7323e98): ##### ERROR The invalid arguments or inputs must be corrected before the GATK can proceed ##### ERROR Please do not post this error to the GATK forum ##### ERROR ##### ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments. ##### ERROR Visit our website and forum for extensive documentation and answers to ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk ##### ERROR ##### ERROR MESSAGE: Argument with name 'num_cpu_threads_per_data_thread' isn't defined. ##### ERROR ------------------------------------------------------------------------------------------
On Mon, May 29, 2017 at 6:44 PM, Peter Cock p.j.a.cock@googlemail.com wrote:
Thanks, pull request submitted:
https://github.com/galaxyproject/galaxy/pull/4121
Peter
On Mon, May 29, 2017 at 5:23 PM, Saravanaraj Ayyampalayam raj76@uga.edu wrote:
Peter,
Here is the git diff output that shows the changes I made:
diff --git a/lib/galaxy/jobs/runners/__init__.py
b/lib/galaxy/jobs/runners/__init__.py
index ebf4859..6ee51c8 100644 --- a/lib/galaxy/jobs/runners/__init__.py +++ b/lib/galaxy/jobs/runners/__init__.py @@ -419,7 +419,7 @@ class JobState( object ): job_name += '_%s' % self.job_wrapper.tool.old_id if self.job_wrapper.user: job_name += '_%s' % self.job_wrapper.user
self.job_name = ''.join( map( lambda x: x if x in (
string.letters + string.digits + '_' ) else '_', job_name ) )
self.job_name = ''.join( map( lambda x: x if x in (
string.ascii_letters + string.digits + '_' ) else '_', job_name ) )
@staticmethod def default_job_file( files_dir, id_tag ):
diff --git a/lib/galaxy/jobs/runners/drmaa.py b/lib/galaxy/jobs/runners/
drmaa.py
index 7c08984..5c5dc62 100644 --- a/lib/galaxy/jobs/runners/drmaa.py +++ b/lib/galaxy/jobs/runners/drmaa.py @@ -394,7 +394,7 @@ class DRMAAJobRunner( AsynchronousJobRunner ): job_name += '_%s' % job_wrapper.tool.old_id if external_runjob_script is None: job_name += '_%s' % job_wrapper.user
job_name = ''.join( x if x in ( string.letters + string.digits
- '_' ) else '_' for x in job_name )
job_name = ''.join( x if x in ( string.ascii_letters +
string.digits + '_' ) else '_' for x in job_name )
if self.restrict_job_name_length: job_name = job_name[:self.restrict_job_name_length] return job_name
-Raj
On May 29, 2017, at 11:33 AM, Peter Cock p.j.a.cock@googlemail.com
wrote:
Thanks Saravanaraj,
You should not need to change the locale, after all many Galaxy servers
and
their users will be labelling their files in non-English with "special" characters - but perhaps en_US.UTF-8 would be safer in your case as it seems that few people use Galaxy with en_US.iso885915 otherwise this would have been reported before?
Can you tell us both places you changed string.letters to
string.ascii_letters
please? I could then submit this as a pull request to get Galaxy itself
updated
for future release.
Peter
On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam <
raj76@uga.edu> wrote:
Peter,
You are correct. Changing string.letters to string.ascii_letters fixed
it. I had to change it in two places.
My locale is :
LANG=en_US.iso885915 LC_CTYPE="en_US.iso885915" LC_NUMERIC="en_US.iso885915" LC_TIME="en_US.iso885915" LC_COLLATE="en_US.iso885915" LC_MONETARY="en_US.iso885915" LC_MESSAGES="en_US.iso885915" LC_PAPER="en_US.iso885915" LC_NAME="en_US.iso885915" LC_ADDRESS="en_US.iso885915" LC_TELEPHONE="en_US.iso885915" LC_MEASUREMENT="en_US.iso885915" LC_IDENTIFICATION="en_US.iso885915" LC_ALL=
I will talk to our sysadmin and see if I can change the locale to
en_US.UTF-8.
Thank you very much. -Raj
On May 29, 2017, at 10:58 AM, Peter Cock p.j.a.cock@googlemail.com
wrote:
Curious, my new guess is this is position 52 a temporary string from this expression:
string.letters + string.digits + '_'
This would fit with it being something unusual about the locale of your Galaxy server, and my suggested hack may work - replace string.letters with string.ascii_letters
Separately at the Linux command line of the server running Galaxy, and on a cluster node, what does the command "locale" give? e.g. I get:
$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=
Peter
On Mon, May 29, 2017 at 3:43 PM, Saravanaraj Ayyampalayam <
raj76@uga.edu> wrote:
Peter,
The job name looks OK. Here is the output of the debug statements:
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making
job_name, tag was '15132', plan to use u'g15132_fastq_to_fasta_ python_raj76@uga.edu' with underscores replacements
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making
job_name, character 0 in proposed job_name u'g' is 103
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making
job_name, character 1 in proposed job_name u'1' is 49
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making
job_name, character 2 in proposed job_name u'5' is 53
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making
job_name, character 3 in proposed job_name u'1' is 49
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making
job_name, character 4 in proposed job_name u'3' is 51
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making
job_name, character 5 in proposed job_name u'2' is 50
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making
job_name, character 6 in proposed job_name u'_' is 95
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making
job_name, character 7 in proposed job_name u'f' is 102
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making
job_name, character 8 in proposed job_name u'a' is 97
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making
job_name, character 9 in proposed job_name u's' is 115
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making
job_name, character 10 in proposed job_name u't' is 116
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making
job_name, character 11 in proposed job_name u'q' is 113
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making
job_name, character 12 in proposed job_name u'_' is 95
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 13 in proposed job_name u't' is 116
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 14 in proposed job_name u'o' is 111
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 15 in proposed job_name u'_' is 95
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 16 in proposed job_name u'f' is 102
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 17 in proposed job_name u'a' is 97
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making
job_name, character 18 in proposed job_name u's' is 115
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 19 in proposed job_name u't' is 116
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 20 in proposed job_name u'a' is 97
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 21 in proposed job_name u'_' is 95
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 22 in proposed job_name u'p' is 112
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 23 in proposed job_name u'y' is 121
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making
job_name, character 24 in proposed job_name u't' is 116
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making
job_name, character 25 in proposed job_name u'h' is 104
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making
job_name, character 26 in proposed job_name u'o' is 111
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making
job_name, character 27 in proposed job_name u'n' is 110
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making
job_name, character 28 in proposed job_name u'_' is 95
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making
job_name, character 29 in proposed job_name u'r' is 114
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making
job_name, character 30 in proposed job_name u'a' is 97
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making
job_name, character 31 in proposed job_name u'j' is 106
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making
job_name, character 32 in proposed job_name u'7' is 55
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making
job_name, character 33 in proposed job_name u'6' is 54
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making
job_name, character 34 in proposed job_name u'@' is 64
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making
job_name, character 35 in proposed job_name u'u' is 117
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making
job_name, character 36 in proposed job_name u'g' is 103
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making
job_name, character 37 in proposed job_name u'a' is 97
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making
job_name, character 38 in proposed job_name u'.' is 46
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making
job_name, character 39 in proposed job_name u'e' is 101
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making
job_name, character 40 in proposed job_name u'd' is 100
galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making
job_name, character 41 in proposed job_name u'u' is 117
galaxy.jobs.runners ERROR 2017-05-29 10:37:21,431 (15132) Unhandled
exception calling queue_job
Traceback (most recent call last): File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/__init__.py", line 104, in run_next
method(arg) File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 132, in queue_job
job_name = self._job_name(job_wrapper) File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 400, in _job_name
job_name = ''.join( x if x in ( string.letters + string.digits +
'_' ) else '_' for x in job_name )
File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 400, in <genexpr>
job_name = ''.join( x if x in ( string.letters + string.digits +
'_' ) else '_' for x in job_name )
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position
52: ordinal not in range(128)
There is no sign of the 0xa6 at position 52 (the job name is only 42
chars long) in the job name.
Thanks! -Raj
> On May 29, 2017, at 5:50 AM, Peter Cock p.j.a.cock@googlemail.com
wrote:
> > Time for more debugging, try inserting something like this inserted > the line above into > /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py
> (having backed up the file): > > log.debug( 'Making job_name, tag was %r, plan to use %r with > underscores replacements' % (galaxy_id_tag, job_name)) > > And if that doesn't help, check all the characters: > > for i, letter in enumerate(job_name): log.debug( 'Making job_name, > character %i in proposed job_name %r is %i' % (i, letter, > ord(letter))) > > My hunch is that the locale of the Galaxy server differs from that
on
> the cluster, meaning DRMAA is stricter about allowed characters > because Python's string.letters would be whatever is allowed on the > Galaxy server's locale. > > If I am right, a crude fix would be switching to using > string.ascii_letters, i.e. > > job_name = ''.join( x if x in ( string.ascii_letters +
string.digits +
> '_' ) else '_' for x in job_name ) > > Peter > > On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam <
raj76@uga.edu> wrote:
>> Hello, >> >> I updated to the latest version (17.05) galaxy code. I was
updating from a very old version.
>> I fixed all the issues with except one. I am getting the following
error when I submit a job.
>> >> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131)
Unhandled exception calling queue_job
>> Traceback (most recent call last): >> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/__init__.py", line 104, in run_next
>> method(arg) >> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 132, in queue_job
>> job_name = self._job_name(job_wrapper) >> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 397, in _job_name
>> job_name = ''.join( x if x in ( string.letters + string.digits +
'_' ) else '_' for x in job_name )
>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-
dist/lib/galaxy/jobs/runners/drmaa.py", line 397, in <genexpr>
>> job_name = ''.join( x if x in ( string.letters + string.digits +
'_' ) else '_' for x in job_name )
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in
position 52: ordinal not in range(128)
>> >> I spent a day trying to debug this and didn’t get anywhere. The
job_name looks OK.
>> >> I would really appreciate some help from the community. >> >> Thanks! >> -Raj >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
galaxy-dev@lists.galaxyproject.org