Jan;
Thanks for getting back with all the detailed information. I dug into
this further and understand what is happening:
- tools/data_source/upload.py calls
lib/galaxy/datatypes/sniff.py:stream_to_file
- stream_to_file uses pythons tempfile module
- tempfile defaults to using /tmp
- As large files stream in the temporary space fills up, causing the
issue you are seeing.
The best way to work around this is to have the galaxy user on Amazon
export TMPDIR to point at a temporary directory on /mnt/galaxyData
instead of the root filesystem.
I'm hoping that Enis or Dannon might be able to help out with the best
place to set this in CloudMan to avoid the issue, I've cc'ed them in.
If you want to manually fix it to get some work done, you could create a
directory /mnt/galaxyData/tmp and then symlink /tmp there:
ln -s /mnt/galaxyData/tmp /tmp
Hope this helps and we can come up with a more permanent fix. Thanks
again,
Brad
Thanks Brad,
Sorry it has taken me so long to respond. I had a meeting sort of day. I started another
instance to recreate what is happening to me in more detail. I hope this helps. I tried
to color code things so you could follow everything more easily.
This is what I have when I start up Galaxy through BioCloudCentral before I import any
files...
Get cloud support with Ubuntu Advantage Cloud Guest
http://www.ubuntu.com/business/services/cloud
ubuntu@ip-10-44-78-218:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 20G 13G 6.1G 68% /
udev 8.4G 4.0K 8.4G 1% /dev
tmpfs 3.4G 644K 3.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 8.4G 0 8.4G 0% /run/shm
/dev/xvdb 404G 201M 383G 1% /mnt
After starting to import data (pasted
https://s3.amazonaws.com/thunnus/BluefinAQ30_shuffled.fastq), it starts filling up
immediately
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 20G 19G 0 100% /
udev 8.4G 4.0K 8.4G 1% /dev
tmpfs 3.4G 660K 3.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 8.4G 0 8.4G 0% /run/shm
/dev/xvdb 404G 201M 383G 1% /mnt
/dev/xvdg1 700G 654G 47G 94% /mnt/galaxyIndices
/dev/xvdg2 10G 1.7G 8.4G 17% /mnt/galaxyTools
/dev/xvdg3 500G 81M 500G 1% /mnt/galaxyData