Hi,
I was just looking at what the galaxy program was doing using top and it is using all the memory! I've rebooted the machine with 90GB RAM and it remained up and running using 57GB until I logged into galaxy and looked to see if there were any running jobs in the admin panel. The memory usages nudged up past 90GB and then died. So it's definitely a memory problem.
Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University
w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.uk t: 01970 62 2807
________________________________ From: galaxy-dev galaxy-dev-bounces@lists.galaxyproject.org on behalf of Martin Vickers [mjv08] mjv08@aber.ac.uk Sent: 16 July 2015 08:48 PM To: Nate Coraor Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] ./run.sh segfault
Hi Nate,
Thanks for the reply. In syslog I'm getting;
Jul 16 20:36:41 galaxy kernel: [ 117.123921] Out of memory: Kill process 1390 (python) score 986 or sacrifice child Jul 16 20:36:41 galaxy kernel: [ 117.124087] Killed process 1390 (python) total-vm:43496348kB, anon-rss:32611892kB, file-rss:1800kB (END)
It's a 32GB VM. I could increase it but I wouldn't expect 32GB to be too little. I've attached the full syslog.
Dr. Martin Vickers
Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University
w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.uk t: 01970 62 2807
________________________________ From: Nate Coraor nate@bx.psu.edu Sent: 16 July 2015 04:36 PM To: Martin Vickers [mjv08] Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] ./run.sh segfault
Hi Martin,
Is there anything in the syslog?
--nate
On Thu, Jul 16, 2015 at 11:26 AM, Martin Vickers <mjv08@aber.ac.ukmailto:mjv08@aber.ac.uk> wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi All,
I have a weird issue that's just cropped up. After a new install of galaxy (checked out on Monday from github) on a ubuntu vm, using postgres rather than sqlite as well as a few other production recommendations, I started playing around with the Data Libraries functionality. I linked a bunch of fastq.gz files into galaxy (around 150 in total) and everything was working fine. I went home and the next day, it was down.
I tried to start it up as usual (using an init.d script), it worked for less than a minute and then disappeared again. So I tried running it as the galaxy user using ./run.sh and I get a seg fault;
Starting server in PID 23173. serving on http://144.124.110.39:8080 Segmentation fault
Tried again with strace
Starting server in PID 23552. serving on http://144.124.110.39:8080 [{WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL}], 0, NULL) = 23552 - --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=23552, si_status=SIGKILL, si_utime=1590, si_stime=1930} --- rt_sigreturn() = 23552 write(2, "Killed\n", 7Killed ) = 7 read(10, "", 8192) = 0 exit_group(137) = ? +++ exited with 137 +++
I can't see anything odd in the log file and I've turned debugging on in galaxy.ini. I'm at a bit of a loss. Does anyone know what might be causing it?
Cheers,
- -- Dr. Martin Vickers
Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University
w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.ukmailto:mjv08@aber.ac.uk t: 01970 62 2807
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
