Hi Peter,

FYI: just found on the galaxy documentation a paragraph that even
recommends running upload on the cluster:

https://galaxyproject.org/admin/config/performance/cluster/

"""
If your cluster nodes have Internet access (NAT is okay) and you want to
run the data source tools (upload, ucsc, etc.) on the cluster (doing so
is highly recommended), set new_file_path in galaxy.ini to a directory
somewhere in your shared filesystem:
"""

Cheers,
Matthias


On 10.07.2017 11:11, Matthias Bernt wrote:
> Dear Peter,
>
> my job_conf.xml is attached.
>
>
> Best,
> Matthuas
>
> On 08.07.2017 18:00, galaxy-dev-request@lists.galaxyproject.org wrote:
>> Send galaxy-dev mailing list submissions to
>>     galaxy-dev@lists.galaxyproject.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>     https://lists.galaxyproject.org/listinfo/galaxy-dev
>> or, via email, send a message with subject or body 'help' to
>>     galaxy-dev-request@lists.galaxyproject.org
>>
>> You can reach the person managing the list at
>>     galaxy-dev-owner@lists.galaxyproject.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of galaxy-dev digest..."
>>
>>
>> HEY!  This is important!  If you reply to a thread in a digest, please
>> 1. Change the subject of your response from "Galaxy-dev Digest Vol
>> ..." to the original subject for the thread.
>> 2. Strip out everything else in the digest that is not part of the
>> thread you are responding to.
>>
>> Why?
>> 1. This will keep the subject meaningful.  People will have some idea
>> from the subject line if they should read it or not.
>> 2. Not doing this greatly increases the number of emails that match
>> search queries, but that aren't actually informative.
>>
>> Today's Topics:
>>
>>     1. Re: any introduce for how Mako and JavaScript work in    galaxy?
>>        (Daniel Blankenberg)
>>     2. Re: ustacks - bad interpreter (Björn Grüning)
>>     3. Re: Improved DRMAAJobRunner (Peter van Heusden) (Matthias Bernt)
>>     4. Re: Improved DRMAAJobRunner (Peter van Heusden)
>>        (Peter van Heusden)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Thu, 6 Jul 2017 22:36:27 -0400
>> From: Daniel Blankenberg <dan.blankenberg@gmail.com>
>> To: Steven Shen <ishenweiyan@gmail.com>
>> Cc: galaxy-dev@lists.galaxyproject.org
>> Subject: Re: [galaxy-dev] any introduce for how Mako and JavaScript
>>     work in    galaxy?
>> Message-ID:
>>     <CAE9nRFg+Edj2=3+K=6+FZCxj5=hb4toGW+Lm++E+SD1+J2+vbw@mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi Steven,
>>
>> For a quick first pass, have you taken a look at https://github.com/
>> galaxyproject/galaxy/blob/dev/client/README.md?
>>
>>
>> Thanks for using Galaxy,
>>
>> Dan
>>
>> On Thu, Jul 6, 2017 at 10:11 PM, Steven Shen <ishenweiyan@gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I want to make some change for my local galaxy web, actually I can edit
>>> galaxy_path/static/scripts/bundled/*.js directly.
>>>
>>> It seems mako templates are used for galaxy, as well as many JavaScript
>>> files, but I don't know how they work. So is there any introduce for how
>>> mako and JavaScript files working in galaxy?
>>>
>>> Thank you very much.
>>>
>>> Steven
>>>
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>    https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>    http://galaxyproject.org/search/
>>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20170706/8fbd5cbe/attachment-0001.html>
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Sat, 8 Jul 2017 11:31:31 +0200
>> From: Björn Grüning <bjoern.gruening@gmail.com>
>> To: David Meltzer <David.Meltzer@glasgow.ac.uk>, Nate Coraor
>>     <nate@bx.psu.edu>
>> Cc: "galaxy-dev@lists.galaxyproject.org"
>>     <galaxy-dev@lists.galaxyproject.org>
>> Subject: Re: [galaxy-dev] ustacks - bad interpreter
>> Message-ID: <a5a7d281-6040-f65e-e674-6cb2e9343af4@gmail.com>
>> Content-Type: text/plain; charset=utf-8
>>
>> Hi,
>>
>> please also make sure you have a recent conda version. This 80 character
>> length smalls a lot like an old conda restriction.
>>
>> Cheers,
>> Bjoern
>>
>> Am 07.07.2017 um 18:09 schrieb David Meltzer:
>>> Good afternoon,
>>>
>>>
>>> Thank you for that! I will look in to it!
>>>
>>>
>>> Best Regards,
>>>
>>> David Jacob Meltzer
>>>
>>>
>>> *From: *Nate Coraor <nate@bx.psu.edu>
>>> *Date: *Friday, July 7, 2017 at 5:02 PM
>>> *To: *David Meltzer <David.Meltzer@glasgow.ac.uk>
>>> *Cc: *Yvan Le Bras <yvan.le-bras@mnhn.fr>,
>>> "galaxy-dev@lists.galaxyproject.org"
>>> <galaxy-dev@lists.galaxyproject.org>
>>> *Subject: *Re: [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>> Hi David,
>>>
>>>
>>> You may want to have a look at the first line of:
>>>
>>>
>>> /home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a13db6458b9361ca87c93be9feed656f9938ebcc435642f3a0a3/bin/stacks_summary.py
>>>
>>>
>>>
>>> The shell is claiming it is:
>>>
>>>
>>> #!/home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a1
>>>
>>>
>>>
>>> When it should be:
>>>
>>>
>>> #!/home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a13db6458b9361ca87c93be9feed656f9938ebcc435642f3a0a3/bin/python
>>>
>>>
>>>
>>> However, it could be that the line is correct and the output was
>>> truncated, or the shell has a length limit for the shebang (although the
>>> truncated string is only 79 characters, I'd be surprised if the limit
>>> for your shell was that short).
>>>
>>>
>>> --nate
>>>
>>>
>>> On Fri, Jul 7, 2017 at 8:21 AM, David Meltzer
>>> <David.Meltzer@glasgow.ac.uk <mailto:David.Meltzer@glasgow.ac.uk>>
>>> wrote:
>>>
>>>      Good afternoon,
>>>
>>>
>>>      I will do my best!
>>>
>>>
>>>      Best Regards,
>>>      David
>>>
>>>
>>>      *From: *Yvan Le Bras <yvan.le-bras@mnhn.fr
>>>      <mailto:yvan.le-bras@mnhn.fr>>
>>>      *Date: *Friday, July 7, 2017 at 1:06 PM
>>>
>>>
>>>      *To: *David Meltzer <David.Meltzer@glasgow.ac.uk
>>>      <mailto:David.Meltzer@glasgow.ac.uk>>,
>>>      "galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>"
>>>      <galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>>
>>>      *Subject: *RE : [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>>      No problem, I was not writing my question clearly...
>>>
>>>
>>>      I suspect a pb with the stacks_summary related script... Can you
>>>      investigate it, I unfortunately can't work on it for now. .. Maybe
>>>      ""just"" a missing python interpreter invocation somewhere. .. when
>>>      executing stacks_summary. ...
>>>
>>>
>>>
>>>      -------- Message d'origine --------
>>>      De : David Meltzer <David.Meltzer@glasgow.ac.uk
>>>      <mailto:David.Meltzer@glasgow.ac.uk>>
>>>      Date : 07/07/2017 13:47 (GMT+01:00)
>>>      À : Yvan Le Bras <yvan.le-bras@mnhn.fr
>>>      <mailto:yvan.le-bras@mnhn.fr>>, galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>
>>>      Objet : Re: [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>>      Good afternoon,
>>>
>>>
>>>      I apologize. I misunderstood the question. Yes, stacks_summary has
>>>      been installed correctly as have stacks and velvet.
>>>
>>>
>>>      Additionally, the tool is outputting three items into the history -
>>>
>>>
>>>      Summary from Stacks – red (error)
>>>
>>>      Ustacks.log – red (error)
>>>
>>>      Stacks from Data – green (successful)
>>>
>>>
>>>      Best Regards,
>>>
>>>      David Jacob Meltzer
>>>
>>>
>>>      *From: *galaxy-dev <galaxy-dev-bounces@lists.galaxyproject.org
>>>      <mailto:galaxy-dev-bounces@lists.galaxyproject.org>> on behalf of
>>>      David Meltzer <David.Meltzer@glasgow.ac.uk
>>>      <mailto:David.Meltzer@glasgow.ac.uk>>
>>>      *Date: *Friday, July 7, 2017 at 12:31 PM
>>>      *To: *Yvan Le Bras <yvan.le-bras@mnhn.fr
>>>      <mailto:yvan.le-bras@mnhn.fr>>, "galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>"
>>>      <galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>>
>>>      *Subject: *Re: [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>>      Good afternoon,
>>>
>>>
>>>      I am sorry. I don’t see a tool called stacks_summary installed
>>> on my
>>>      system or available on the toolshed.
>>>
>>>
>>>      Best Regards,
>>>
>>>      David
>>>
>>>
>>>      *From: *Yvan Le Bras <yvan.le-bras@mnhn.fr
>>>      <mailto:yvan.le-bras@mnhn.fr>>
>>>      *Date: *Friday, July 7, 2017 at 12:08 PM
>>>      *To: *David Meltzer <David.Meltzer@glasgow.ac.uk
>>>      <mailto:David.Meltzer@glasgow.ac.uk>>,
>>>      "galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>"
>>>      <galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>>
>>>      *Subject: *RE : [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>>      Hi David,
>>>
>>>
>>>      A pb with stacks_summary apparently... and ustacks is working well
>>>      apparently... Can you check you have stacks_summary well
>>> installed ?
>>>      I think it is but there is a pb using it...
>>>
>>>
>>>      Cheers,
>>>
>>>
>>>      Yvan
>>>
>>>
>>>
>>>
>>>      Envoyé depuis mon appareil Samsung
>>>
>>>
>>>
>>>      -------- Message d'origine --------
>>>      De : David Meltzer <David.Meltzer@glasgow.ac.uk
>>>      <mailto:David.Meltzer@glasgow.ac.uk>>
>>>      Date : 07/07/2017 12:38 (GMT+01:00)
>>>      À : galaxy-dev@lists.galaxyproject.org
>>>      <mailto:galaxy-dev@lists.galaxyproject.org>
>>>      Objet : [galaxy-dev] ustacks - bad interpreter
>>>
>>>
>>>      Good morning,
>>>
>>>
>>>      I have a user attempting to run ustacks. They are receiving the
>>>      following error
>>>
>>>
>>>      Fatal error: Exit code 126 (Error in Stacks execution)
>>>
>>>
>>> /home/big_galaxy/galaxy/database/jobs_directory/079/79363/tool_script.sh:
>>>
>>>
>>> /home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a13db6458b9361ca87c93be9feed656f9938ebcc435642f3a0a3/bin/stacks_summary.py:
>>>
>>>
>>> /home/big_galaxy/galaxy/tool_dependencies/_conda/envs/mulled-v1-8e6d35eac718a1:
>>>
>>>      bad interpreter: No such file or directory
>>>
>>>
>>>      The tool produces an additional output on the bug report –
>>>
>>>
>>>      ustacks parameters selected:
>>>
>>>        Sample ID: 1
>>>
>>>        Min depth of coverage to create a stack: 3
>>>
>>>        Max distance allowed between stacks: 2
>>>
>>>        Max distance allowed to align secondary reads: 4
>>>
>>>        Max number of stacks allowed per de novo locus: 3
>>>
>>>        Deleveraging algorithm: disabled
>>>
>>>        Removal algorithm: enabled
>>>
>>>        Model type: SNP
>>>
>>>        Alpha significance level for model: 0.05
>>>
>>>        Gapped alignments: disabled
>>>
>>>      Parsing stacks_inputs/ZU-6.fq
>>>
>>>      Loading RAD-Tags...done
>>>
>>>      Loaded 933120 RAD-Tags.
>>>
>>>        Inserted 256463 elements into the RAD-Tags hash map.
>>>
>>>        0 reads contained uncalled nucleotides that were modified.
>>>
>>>      65509 initial stacks were populated; 190954 stacks were set
>>> aside as
>>>      secondary reads.
>>>
>>>      Initial coverage mean: 10.9276; Std Dev: 25.3864; Max: 4650
>>>
>>>      Deleveraging trigger: 36; Removal trigger: 62
>>>
>>>      Calculating distance for removing repetitive stacks.
>>>
>>>        Distance allowed between stacks: 1; searching with a k-mer length
>>>      of 35 (36 k-mers per read); 1 k-mer hits required.
>>>
>>>      Removing repetitive stacks.
>>>
>>>        Removed 993 stacks.
>>>
>>>        64677 stacks remain for merging.
>>>
>>>      Post-Repeat Removal, coverage depth Mean: 10.2655; Std Dev:
>>> 9.03472;
>>>      Max: 62
>>>
>>>      Calculating distance between stacks...
>>>
>>>        Distance allowed between stacks: 2; searching with a k-mer length
>>>      of 23 (48 k-mers per read); 2 k-mer hits required.
>>>
>>>      Merging stacks, maximum allowed distance: 2 nucleotide(s)
>>>
>>>        64677 stacks merged into 57327 loci; deleveraged 0 loci;
>>>      blacklisted 189 loci.
>>>
>>>      After merging, coverage depth Mean: 11.0747; Std Dev: 9.82823;
>>> Max: 91
>>>
>>>      Merging remainder radtags
>>>
>>>        217265 remainder sequences left to merge.
>>>
>>>        Distance allowed between stacks: 4; searching with a k-mer length
>>>      of 13 (58 k-mers per read); 6 k-mer hits required.
>>>
>>>        Matched 26211 remainder reads; unable to match 191054
>>> remainder reads.
>>>
>>>      After remainders merged, coverage depth Mean: 11.5347; Std Dev:
>>>      9.95581; Max: 93
>>>
>>>      Calling final consensus sequences, invoking SNP-calling model...
>>>
>>>      Number of utilized reads: 742066
>>>
>>>      Writing loci, SNPs, and alleles to 'stacks_outputs/'...
>>>
>>>        Refetching sequencing IDs from stacks_inputs/ZU-6.fq... read
>>>      933120 sequence IDs.
>>>
>>>      done.
>>>
>>>      ustacks is done.
>>>
>>>
>>>      I am not sure what is causing the bad interpreter: No such file or
>>>      directory. The tools are correctly installed along with its
>>>      requisite conda packages.
>>>
>>>
>>>      Any help would be greatly appreciated.
>>>
>>>
>>>      Best Regards,
>>>
>>>
>>>      David
>>>
>>>
>>>      ___________________________________________________________
>>>      Please keep all replies on the list by using "reply all"
>>>      in your mail client.  To manage your subscriptions to this
>>>      and other Galaxy lists, please use the interface at:
>>>        https://lists.galaxyproject.org/
>>>
>>>      To search Galaxy mailing lists use the unified search at:
>>>        http://galaxyproject.org/search/
>>>
>>>
>>>
>>>
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>    https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>    http://galaxyproject.org/search/
>>>
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Sat, 08 Jul 2017 11:56:50 +0200
>> From: Matthias Bernt <m.bernt@ufz.de>
>> To: galaxy-dev@lists.galaxyproject.org
>> Subject: Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
>> Message-ID: <fb33e9595421.5960c882@ufz.de>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi Peter,
>>
>>> Code looks interesting. This is for Univa Grid Engine?
>>
>> In its current state its tailored for the Univa Grid Engine. But the
>> drmaa library code that is based on job_info() and wait() should run
>> on all grid engines.
>> For the command line based code changes might be necessary for the
>> different grid engines. (Currently there is only a small bug, because
>> wait() is currently
>> called twice for finished jobs: in the repeated polling and then in
>> the final check. The second call won't work which causes a fallback to
>> qacct. I will fix this
>> soon.)
>> I guess that the output of qstat and qacct will be different. But I
>> think this can be configured one just needs a way to get the info
>> which grid engine is running.
>>
>>> In terms of the upload jobs, are those not designed to be run as
>>> 'local' jobs and not with the 'real user' setting?
>>
>> Sounds reasonable, but this is not what is happening on my
>> installation of galaxy. Any idea where I could start to find the problem.
>>
>> Best,
>> Matthias
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20170708/a72251d4/attachment-0001.html>
>>
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Sat, 08 Jul 2017 13:52:39 +0000
>> From: Peter van Heusden <pvh@sanbi.ac.za>
>> To: galaxy-dev@lists.galaxyproject.org
>> Subject: Re: [galaxy-dev] Improved DRMAAJobRunner (Peter van Heusden)
>> Message-ID:
>>     <CAK1reXg1w4VkVvWAmDaPseVGQ93XScn1ipF8T8ibJyTyTPr8XQ@mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> On Sat, 8 Jul 2017 at 11:56 Matthias Bernt <m.bernt@ufz.de> wrote:
>>
>>> Hi Peter,
>>>
>>>> Code looks interesting. This is for Univa Grid Engine?
>>>
>>> In its current state its tailored for the Univa Grid Engine. But the
>>> drmaa
>>> library code that is based on job_info() and wait() should run on all
>>> grid
>>> engines.
>>> For the command line based code changes might be necessary for the
>>> different grid engines. (Currently there is only a small bug, because
>>> wait() is currently
>>> called twice for finished jobs: in the repeated polling and then in the
>>> final check. The second call won't work which causes a fallback to
>>> qacct. I
>>> will fix this
>>> soon.)
>>> I guess that the output of qstat and qacct will be different. But I
>>> think
>>> this can be configured one just needs a way to get the info which grid
>>> engine is running.
>>>
>>>> In terms of the upload jobs, are those not designed to be run as
>>>> 'local'
>>> jobs and not with the 'real user' setting?
>>>
>>> Sounds reasonable, but this is not what is happening on my
>>> installation of
>>> galaxy. Any idea where I could start to find the problem.
>>>
>>> I'm not an expert on this, but what does your job_conf.xml look like?
>>
>> Peter
>>
>>> Best,
>>> Matthias ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>    https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>    http://galaxyproject.org/search/
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20170708/99ebb014/attachment-0001.html>
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> galaxy-dev mailing list
>> galaxy-dev@lists.galaxyproject.org
>> https://lists.galaxyproject.org/listinfo/galaxy-dev
>>
>> To search Galaxy mailing lists use the unified search at:
>>    http://galaxyproject.org/search/mailinglists/
>>
>> ------------------------------
>>
>> End of galaxy-dev Digest, Vol 133, Issue 6
>> ******************************************
>>
>

--

-------------------------------------------
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone +49 341 235 482296,
m.bernt@ufz.de, www.ufz.de

Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
-------------------------------------------