Hi, I was just looking at what the galaxy program was doing using top and it is using all the memory! I've rebooted the machine with 90GB RAM and it remained up and running using 57GB until I logged into galaxy and looked to see if there were any running jobs in the admin panel. The memory usages nudged up past 90GB and then died. So it's definitely a memory problem. Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.uk t: 01970 62 2807 ________________________________ From: galaxy-dev <galaxy-dev-bounces@lists.galaxyproject.org> on behalf of Martin Vickers [mjv08] <mjv08@aber.ac.uk> Sent: 16 July 2015 08:48 PM To: Nate Coraor Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] ./run.sh segfault Hi Nate, Thanks for the reply. In syslog I'm getting; Jul 16 20:36:41 galaxy kernel: [ 117.123921] Out of memory: Kill process 1390 (python) score 986 or sacrifice child Jul 16 20:36:41 galaxy kernel: [ 117.124087] Killed process 1390 (python) total-vm:43496348kB, anon-rss:32611892kB, file-rss:1800kB (END) It's a 32GB VM. I could increase it but I wouldn't expect 32GB to be too little. I've attached the full syslog. Dr. Martin Vickers Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.uk t: 01970 62 2807 ________________________________ From: Nate Coraor <nate@bx.psu.edu> Sent: 16 July 2015 04:36 PM To: Martin Vickers [mjv08] Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] ./run.sh segfault Hi Martin, Is there anything in the syslog? --nate On Thu, Jul 16, 2015 at 11:26 AM, Martin Vickers <mjv08@aber.ac.uk<mailto:mjv08@aber.ac.uk>> wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I have a weird issue that's just cropped up. After a new install of galaxy (checked out on Monday from github) on a ubuntu vm, using postgres rather than sqlite as well as a few other production recommendations, I started playing around with the Data Libraries functionality. I linked a bunch of fastq.gz files into galaxy (around 150 in total) and everything was working fine. I went home and the next day, it was down. I tried to start it up as usual (using an init.d script), it worked for less than a minute and then disappeared again. So I tried running it as the galaxy user using ./run.sh and I get a seg fault; Starting server in PID 23173. serving on http://144.124.110.39:8080 Segmentation fault Tried again with strace Starting server in PID 23552. serving on http://144.124.110.39:8080 [{WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL}], 0, NULL) = 23552 - --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=23552, si_status=SIGKILL, si_utime=1590, si_stime=1930} --- rt_sigreturn() = 23552 write(2, "Killed\n", 7Killed ) = 7 read(10, "", 8192) = 0 exit_group(137) = ? +++ exited with 137 +++ I can't see anything odd in the log file and I've turned debugging on in galaxy.ini. I'm at a bit of a loss. Does anyone know what might be causing it? Cheers, - -- Dr. Martin Vickers Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University w: http://www.martin-vickers.co.uk/ e: mjv08@aber.ac.uk<mailto:mjv08@aber.ac.uk> t: 01970 62 2807 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) iQEcBAEBAgAGBQJVp80YAAoJEHa0a8GkKQgIGlIH/1VfAPbs/5ApDBdyoOV5qf1y oCOv93IojARyfI0ksSjF8NRNzw5fNp1R8AzZzomaR3SOUkBuZutre600sy0azTZw E6gjxtMuvaMyEsOTXtToVarVJT0wTG8+5DJRIYLxtYZm7kvbZK0WuzrN2zDT6663 Rnm7zI/zBpTAyp6uXwgmz0x5gpH6KFwRcEHEbU3JWy6nj1zithJShwYPlBuhT5IB OaPwOKflcZpZ8NBTEGsh038JrkU+eE50a9aEjQ2m/DpfM/TN9ujgEFm1dyy/iQS7 ewwQUpWJDkA/u0ZX602dsNdV2LvGuKVVMEHiQ25zaUQZ/iGTwKBQsFM2LlDybgA= =jzYG -----END PGP SIGNATURE----- ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
