Re: [galaxy-dev] unable to stob jobs

15 Aug 2014


      Fantastic - thanks for the update! I hope it is okay - I am cc'ing the
-dev list so if people search on this problem in the future and search
the can see what the problem turned out to be and how to fix it. Enjoy
your job killing.

-John

On Fri, Aug 15, 2014 at 12:47 PM, Michael Mason
<MMason@benaroyaresearch.org> wrote:
...
Hi John,
Just an update on the hypothesis. Our server guy figured out that the nfs
memory setting was such that Galaxy used all the memory. He believes this
resulted in no memory for jobs to be killed. He changed the setting so
that there is some leftover memory when I am using Galaxy at an all out
pace and I was able to kill jobs this morning.
Thanks again.
Mike
On 8/15/14 9:32 AM, "John Chilton" <jmchilton@gmail.com> wrote:
...
Very interesting - whenever you get a chance I would try to kill
simple jobs when Galaxy is not under load to verify the problem is
related to Galaxy's job load. If you still have problems I would then
try again to get that stack trace.
Good luck,
-John
On Wed, Aug 13, 2014 at 6:46 PM, Michael Mason
<MMason@benaroyaresearch.org> wrote:
...
Hi John,
We actually went ahead and recovered an archived instance from Friday.
This should should be the same build though. Because of the archived
instance I believe we lost the stack trace but I'll check with the IT
folks.
Thanks for your help. Below is the tip call. BTW we are using Galaxy
with
fastq's from a Fluidigm's C1 machine. This means we often run 100-200
libraries on a single flow cell. The resulting 100-200 libraries tend to
tax Galaxy. But it tends to handle it though this may be the root cause
of
our difficulty killing jobs.
tip
13753:d3b1f484c4b6bbb3daa50fa167eef97a384890b3
latest_2014.06.02
13742:8a863a311a6c9f14b302799bffcf94df9186fef7
release_2014.06.02
13712:7e257c7b10badb65772b1528cb61d58175a42e47
latest_2014.04.14
13085:68a8b0397947c732b28207d465d3f3c4e2a7a8a0
release_2014.04.14
13064:9e53251b0b7e93b9563008a2b112f2e815a04bbc
release_2014.02.10
12440:5e605ed6069fe4c5ca9875e95e91b2713499e8ca
release_2013.11.04
11218:26f58e05aa1068761660681583821e21e6cbf7ab
release_2013.08.12
10392:1ae95b3aa98d1ccf15b243ac3ce6a895eb7efc53
release_2013.06.03
9943:524f246ca85395082719ae7a6ff72260d7ad5612
security_2013.04.08
9292:2cc8d10988e03257dc7b97f8bb332c7df745d1dd
release_2013.04.01
9231:75f09617abaadbc8cc732bb8ee519decaeb56ea7
release_2013.02.08
8794:1c717491139269651bb59687563da9410b84c65d
release_2013.01.13
8530:a4113cc1cb5eaa68091c9a73375f00555b66dd11
On 8/13/14 1:39 PM, "John Chilton" <jmchilton@gmail.com> wrote:
...
When was the last time you updated Galaxy - we fixed a bug that would
be causing this behavior several releases ago.
Otherwise - additional information would help - like are you using the
DRMAA job runner or the newer specialized Slurm job runner and are
there any details in the Galaxy log that might be helpful? I would
expect to see a stack trace of some kind if there are problems like
this. If there are no stack traces and you have an update-to-date
Galaxy - perhaps checking the SLURM logs for errors might also provide
additional insight.
-John
On Tue, Aug 12, 2014 at 12:10 PM, Michael Mason
<MMason@benaroyaresearch.org> wrote:
...
Hello all,
I am running a Galaxy instance on slurm. I am unable to stop jobs via
the
admin "Manage Jobs" window. Via postgres, there are no jobs in the job
table. Any thoughts?
I realize this may not be dev but I am at a loss of what to do.
Thanks
Mike
________________________________
--CONFIDENTIALITY NOTICE--: The information contained in this email is
intended for the exclusive use of the addressee and may contain
confidential
information. If you are not the intended recipient, you are hereby
notified
that any form of dissemination of this communication is strictly
prohibited.
www.benaroyaresearch.org
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
________________________________
--CONFIDENTIALITY NOTICE--: The information contained in this email is
intended for the exclusive use of the addressee and may contain
confidential information. If you are not the intended recipient, you are
hereby notified that any form of dissemination of this communication is
strictly prohibited. www.benaroyaresearch.org
________________________________
--CONFIDENTIALITY NOTICE--: The information contained in this email is intended for the exclusive use of the addressee and may contain confidential information. If you are not the intended recipient, you are hereby notified that any form of dissemination of this communication is strictly prohibited. www.benaroyaresearch.org

John Chilton

tags

participants (1)