31 May
2017
31 May
'17
3:36 a.m.
Hi Dev, I am running a local instance Galaxy for developing a variant calling pipeline. I ran into this issue after configuring and installing GATK2 on my local when running the GATK *Realigner Target Creator on the BAM file generated by Mark duplicates.* *Can some suggest how to resolve this issue or please point to how to debug this issue from the GATK2 wrapper? BTW, I follow the instruction by ensuring that my GATK3 environment was set as stated in the Galaxy GATk2 readme.txt.* Below here was the issue I encountered; ##### ERROR ------------------------------------------------------------------------------------------ ##### ERROR A USER ERROR has occurred (version 2.0-7-g7323e98): ##### ERROR The invalid arguments or inputs must be corrected before the GATK can proceed ##### ERROR Please do not post this error to the GATK forum ##### ERROR ##### ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments. ##### ERROR Visit our website and forum for extensive documentation and answers to ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk ##### ERROR ##### ERROR MESSAGE: Argument with name 'num_cpu_threads_per_data_thread' isn't defined. ##### ERROR ------------------------------------------------------------------------------------------ On Mon, May 29, 2017 at 6:44 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote: > Thanks, pull request submitted: > > https://github.com/galaxyproject/galaxy/pull/4121 > > Peter > > > On Mon, May 29, 2017 at 5:23 PM, Saravanaraj Ayyampalayam <raj76@uga.edu> > wrote: > > Peter, > > > > Here is the git diff output that shows the changes I made: > > > > diff --git a/lib/galaxy/jobs/runners/__init__.py > b/lib/galaxy/jobs/runners/__init__.py > > index ebf4859..6ee51c8 100644 > > --- a/lib/galaxy/jobs/runners/__init__.py > > +++ b/lib/galaxy/jobs/runners/__init__.py > > @@ -419,7 +419,7 @@ class JobState( object ): > > job_name += '_%s' % self.job_wrapper.tool.old_id > > if self.job_wrapper.user: > > job_name += '_%s' % self.job_wrapper.user > > - self.job_name = ''.join( map( lambda x: x if x in ( > string.letters + string.digits + '_' ) else '_', job_name ) ) > > + self.job_name = ''.join( map( lambda x: x if x in ( > string.ascii_letters + string.digits + '_' ) else '_', job_name ) ) > > > > @staticmethod > > def default_job_file( files_dir, id_tag ): > > diff --git a/lib/galaxy/jobs/runners/drmaa.py b/lib/galaxy/jobs/runners/ > drmaa.py > > index 7c08984..5c5dc62 100644 > > --- a/lib/galaxy/jobs/runners/drmaa.py > > +++ b/lib/galaxy/jobs/runners/drmaa.py > > @@ -394,7 +394,7 @@ class DRMAAJobRunner( AsynchronousJobRunner ): > > job_name += '_%s' % job_wrapper.tool.old_id > > if external_runjob_script is None: > > job_name += '_%s' % job_wrapper.user > > - job_name = ''.join( x if x in ( string.letters + string.digits > + '_' ) else '_' for x in job_name ) > > + job_name = ''.join( x if x in ( string.ascii_letters + > string.digits + '_' ) else '_' for x in job_name ) > > if self.restrict_job_name_length: > > job_name = job_name[:self.restrict_job_name_length] > > return job_name > > > > -Raj > > > >> On May 29, 2017, at 11:33 AM, Peter Cock <p.j.a.cock@googlemail.com> > wrote: > >> > >> Thanks Saravanaraj, > >> > >> You should not need to change the locale, after all many Galaxy servers > and > >> their users will be labelling their files in non-English with > >> "special" characters - > >> but perhaps en_US.UTF-8 would be safer in your case as it seems that few > >> people use Galaxy with en_US.iso885915 otherwise this would have been > >> reported before? > >> > >> Can you tell us both places you changed string.letters to > string.ascii_letters > >> please? I could then submit this as a pull request to get Galaxy itself > updated > >> for future release. > >> > >> Peter > >> > >> On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam < > raj76@uga.edu> wrote: > >>> Peter, > >>> > >>> You are correct. Changing string.letters to string.ascii_letters fixed > it. I had to change it in two places. > >>> > >>> My locale is : > >>> > >>> LANG=en_US.iso885915 > >>> LC_CTYPE="en_US.iso885915" > >>> LC_NUMERIC="en_US.iso885915" > >>> LC_TIME="en_US.iso885915" > >>> LC_COLLATE="en_US.iso885915" > >>> LC_MONETARY="en_US.iso885915" > >>> LC_MESSAGES="en_US.iso885915" > >>> LC_PAPER="en_US.iso885915" > >>> LC_NAME="en_US.iso885915" > >>> LC_ADDRESS="en_US.iso885915" > >>> LC_TELEPHONE="en_US.iso885915" > >>> LC_MEASUREMENT="en_US.iso885915" > >>> LC_IDENTIFICATION="en_US.iso885915" > >>> LC_ALL= > >>> > >>> I will talk to our sysadmin and see if I can change the locale to > en_US.UTF-8. > >>> > >>> Thank you very much. > >>> -Raj > >>> > >>> > >>>> On May 29, 2017, at 10:58 AM, Peter Cock <p.j.a.cock@googlemail.com> > wrote: > >>>> > >>>> Curious, my new guess is this is position 52 a temporary string from > >>>> this expression: > >>>> > >>>> string.letters + string.digits + '_' > >>>> > >>>> This would fit with it being something unusual about the locale of > >>>> your Galaxy server, and my suggested hack may work - replace > >>>> string.letters with string.ascii_letters > >>>> > >>>> Separately at the Linux command line of the server running Galaxy, and > >>>> on a cluster node, what does the command "locale" give? e.g. I get: > >>>> > >>>> $ locale > >>>> LANG=en_US.UTF-8 > >>>> LC_CTYPE="en_US.UTF-8" > >>>> LC_NUMERIC="en_US.UTF-8" > >>>> LC_TIME="en_US.UTF-8" > >>>> LC_COLLATE="en_US.UTF-8" > >>>> LC_MONETARY="en_US.UTF-8" > >>>> LC_MESSAGES="en_US.UTF-8" > >>>> LC_PAPER="en_US.UTF-8" > >>>> LC_NAME="en_US.UTF-8" > >>>> LC_ADDRESS="en_US.UTF-8" > >>>> LC_TELEPHONE="en_US.UTF-8" > >>>> LC_MEASUREMENT="en_US.UTF-8" > >>>> LC_IDENTIFICATION="en_US.UTF-8" > >>>> LC_ALL= > >>>> > >>>> > >>>> Peter > >>>> > >>>> On Mon, May 29, 2017 at 3:43 PM, Saravanaraj Ayyampalayam < > raj76@uga.edu> wrote: > >>>>> Peter, > >>>>> > >>>>> The job name looks OK. Here is the output of the debug statements: > >>>>> > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making > job_name, tag was '15132', plan to use u'g15132_fastq_to_fasta_ > python_raj76@uga.edu' with underscores replacements > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making > job_name, character 0 in proposed job_name u'g' is 103 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making > job_name, character 1 in proposed job_name u'1' is 49 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making > job_name, character 2 in proposed job_name u'5' is 53 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making > job_name, character 3 in proposed job_name u'1' is 49 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making > job_name, character 4 in proposed job_name u'3' is 51 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making > job_name, character 5 in proposed job_name u'2' is 50 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making > job_name, character 6 in proposed job_name u'_' is 95 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making > job_name, character 7 in proposed job_name u'f' is 102 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making > job_name, character 8 in proposed job_name u'a' is 97 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making > job_name, character 9 in proposed job_name u's' is 115 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making > job_name, character 10 in proposed job_name u't' is 116 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making > job_name, character 11 in proposed job_name u'q' is 113 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making > job_name, character 12 in proposed job_name u'_' is 95 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 13 in proposed job_name u't' is 116 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 14 in proposed job_name u'o' is 111 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 15 in proposed job_name u'_' is 95 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 16 in proposed job_name u'f' is 102 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 17 in proposed job_name u'a' is 97 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,426 Making > job_name, character 18 in proposed job_name u's' is 115 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 19 in proposed job_name u't' is 116 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 20 in proposed job_name u'a' is 97 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 21 in proposed job_name u'_' is 95 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 22 in proposed job_name u'p' is 112 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 23 in proposed job_name u'y' is 121 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,427 Making > job_name, character 24 in proposed job_name u't' is 116 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making > job_name, character 25 in proposed job_name u'h' is 104 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making > job_name, character 26 in proposed job_name u'o' is 111 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making > job_name, character 27 in proposed job_name u'n' is 110 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making > job_name, character 28 in proposed job_name u'_' is 95 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making > job_name, character 29 in proposed job_name u'r' is 114 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making > job_name, character 30 in proposed job_name u'a' is 97 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making > job_name, character 31 in proposed job_name u'j' is 106 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making > job_name, character 32 in proposed job_name u'7' is 55 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making > job_name, character 33 in proposed job_name u'6' is 54 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making > job_name, character 34 in proposed job_name u'@' is 64 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making > job_name, character 35 in proposed job_name u'u' is 117 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making > job_name, character 36 in proposed job_name u'g' is 103 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making > job_name, character 37 in proposed job_name u'a' is 97 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making > job_name, character 38 in proposed job_name u'.' is 46 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making > job_name, character 39 in proposed job_name u'e' is 101 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making > job_name, character 40 in proposed job_name u'd' is 100 > >>>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making > job_name, character 41 in proposed job_name u'u' is 117 > >>>>> galaxy.jobs.runners ERROR 2017-05-29 10:37:21,431 (15132) Unhandled > exception calling queue_job > >>>>> Traceback (most recent call last): > >>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/__init__.py", line 104, in run_next > >>>>> method(arg) > >>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 132, in queue_job > >>>>> job_name = self._job_name(job_wrapper) > >>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 400, in _job_name > >>>>> job_name = ''.join( x if x in ( string.letters + string.digits + > '_' ) else '_' for x in job_name ) > >>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 400, in <genexpr> > >>>>> job_name = ''.join( x if x in ( string.letters + string.digits + > '_' ) else '_' for x in job_name ) > >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position > 52: ordinal not in range(128) > >>>>> > >>>>> > >>>>> There is no sign of the 0xa6 at position 52 (the job name is only 42 > chars long) in the job name. > >>>>> > >>>>> Thanks! > >>>>> -Raj > >>>>> > >>>>> > >>>>>> On May 29, 2017, at 5:50 AM, Peter Cock <p.j.a.cock@googlemail.com> > wrote: > >>>>>> > >>>>>> Time for more debugging, try inserting something like this inserted > >>>>>> the line above into > >>>>>> /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py > >>>>>> (having backed up the file): > >>>>>> > >>>>>> log.debug( 'Making job_name, tag was %r, plan to use %r with > >>>>>> underscores replacements' % (galaxy_id_tag, job_name)) > >>>>>> > >>>>>> And if that doesn't help, check all the characters: > >>>>>> > >>>>>> for i, letter in enumerate(job_name): log.debug( 'Making job_name, > >>>>>> character %i in proposed job_name %r is %i' % (i, letter, > >>>>>> ord(letter))) > >>>>>> > >>>>>> My hunch is that the locale of the Galaxy server differs from that > on > >>>>>> the cluster, meaning DRMAA is stricter about allowed characters > >>>>>> because Python's string.letters would be whatever is allowed on the > >>>>>> Galaxy server's locale. > >>>>>> > >>>>>> If I am right, a crude fix would be switching to using > >>>>>> string.ascii_letters, i.e. > >>>>>> > >>>>>> job_name = ''.join( x if x in ( string.ascii_letters + > string.digits + > >>>>>> '_' ) else '_' for x in job_name ) > >>>>>> > >>>>>> Peter > >>>>>> > >>>>>> On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam < > raj76@uga.edu> wrote: > >>>>>>> Hello, > >>>>>>> > >>>>>>> I updated to the latest version (17.05) galaxy code. I was > updating from a very old version. > >>>>>>> I fixed all the issues with except one. I am getting the following > error when I submit a job. > >>>>>>> > >>>>>>> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131) > Unhandled exception calling queue_job > >>>>>>> Traceback (most recent call last): > >>>>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/__init__.py", line 104, in run_next > >>>>>>> method(arg) > >>>>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 132, in queue_job > >>>>>>> job_name = self._job_name(job_wrapper) > >>>>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 397, in _job_name > >>>>>>> job_name = ''.join( x if x in ( string.letters + string.digits + > '_' ) else '_' for x in job_name ) > >>>>>>> File "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy- > dist/lib/galaxy/jobs/runners/drmaa.py", line 397, in <genexpr> > >>>>>>> job_name = ''.join( x if x in ( string.letters + string.digits + > '_' ) else '_' for x in job_name ) > >>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in > position 52: ordinal not in range(128) > >>>>>>> > >>>>>>> I spent a day trying to debug this and didn’t get anywhere. The > job_name looks OK. > >>>>>>> > >>>>>>> I would really appreciate some help from the community. > >>>>>>> > >>>>>>> Thanks! > >>>>>>> -Raj > >>>>>>> > >>>>>>> ___________________________________________________________ > >>>>>>> Please keep all replies on the list by using "reply all" > >>>>>>> in your mail client. To manage your subscriptions to this > >>>>>>> and other Galaxy lists, please use the interface at: > >>>>>>> https://lists.galaxyproject.org/ > >>>>>>> > >>>>>>> To search Galaxy mailing lists use the unified search at: > >>>>>>> http://galaxyproject.org/search/ > >>>>> > >>> > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ > -- Ajayi Olabode O. University of the western cape South Africa National Bioinformatics Institute(Msc In Progress) Department of Computer Science Computer Science (B.sc & Hons) SAP ERP, MCSE,MCTS,MCP,MCSA Mechanical Engineering E-mail: boraton2010@gmail.com Mobile Phone: 0791024568 "You are the future. Believe in yourself, in your greatness... Become all you are capable of and more. Dare to dream. Dare to innovate. Dare to Lead" -- Disclaimer - This e-mail is subject to UWC policies and e-mail disclaimer published on our website at: https://www.uwc.ac.za/Pages/emaildisclaimer.aspx