Jan;
I appreciate the help, I am in no way a competent UNIX/Linux user. I just have 2 shuffled fastq files in an S3 bucket-I pasted the URL for these into Galaxy's upload file URL/Text box and that seems to be where the trouble started. On my first try loading the data files, I got a message saying there was no disk space.
I'm trying to reproduce here but can't seem to figure out how the disk is filling up. A new filesystem should look like:
$ df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 20G 13G 6.1G 68% / udev 3.7G 4.0K 3.7G 1% /dev tmpfs 1.5G 660K 1.5G 1% /run none 5.0M 0 5.0M 0% /run/lock none 3.7G 0 3.7G 0% /run/shm /dev/xvdb 414G 201M 393G 1% /mnt /dev/xvdg1 700G 654G 47G 94% /mnt/galaxyIndices /dev/xvdg2 10G 1.7G 8.4G 17% /mnt/galaxyTools /dev/xvdg3 5.0G 81M 5.0G 2% /mnt/galaxyData
with 6Gb of free space on '/'. If you pasted a file, like:
https://s3.amazonaws.com/chapmanb/example.fastq
into the text box, it should show up as:
/mnt/galaxyData/files/000/dataset_1.dat
in the /mnt/galaxyData mount, which has all of the space for storing files. So I'm not sure exactly what is filling up your root partition. Could you try checking a couple of places and see if any of the filesystem usage is different from what we expect:
$ du -sh /home/ubuntu/ 361M /home/ubuntu/
$ sudo du -sh /var/log/ 3.1M /var/log/
I'm cc'ing Enis in case he has any other ideas where we might be accidentally filling up the root filesystem.
Thanks again for the feedback and sorry about the problems, Brad
I tried again, and it worked. Then, when I tried to run velveth, it got hung up again with the no disk space issue. I browsed around in the /dev/xvda1 file system and there were quite a few files (I pasted them below). I assumed they were files relating to BloudBioLinux and I was not sure if I could get rid of any of the files. It was nothing I put there. I started the instance using BioCloudCentral. Maybe it would be better to just start Galaxy the long way (choosing a public AMI under launch instance...)?
ubuntu@ip-10-44-117-85:/$ ls bin etc initrd.img lib64 mnt proc sbin sys var boot export lib lost+found opt root selinux tmp vmlinuz dev home lib32 media pkg run srv usr
Thanks again, Jan
On Mar 6, 2012, at 8:47 PM, Brad Chapman wrote:
Jan; Glad to hear you got Galaxy running successfully. It sounds like everything is good to go once we sort out the disk space issue.
However, when I try to use NX to get the virtual desktop going I get the message usr/bin/nxserver: line 381: echo: write error: No space left on device.
Using the df -h command, I get:
Filesystem Size Used Avail Use% Mounted on /dev/xvda1 20G 19G 0 100% / udev 8.4G 4.0K 8.4G 1% /dev tmpfs 3.4G 660K 3.4G 1% /run none 5.0M 0 5.0M 0% /run/lock none 8.4G 0 8.4G 0% /run/shm /dev/xvdb 404G 202M 383G 1% /mnt /dev/xvdg1 700G 654G 47G 94% /mnt/galaxyIndices /dev/xvdg2 10G 1.7G 8.4G 17% /mnt/galaxyTools /dev/xvdg3 200G 11G 190G 6% /mnt/galaxyData
So, I guess my question as a new user is: How do I point Galaxy and CloudBioLinux to all of this unused space?
By default CloudMan will put files into /mnt/galaxyData. However, as you noticed the main filesystem got filled up at some point. Could this have happened while transferring files over from S3? Are there files in your home directory that you could delete or move to /mnt/galaxyData to free up space?
CloudBioLinux and CloudMan shouldn't put a large number of files in the root directory, but when the root filesystem is full it's going to be very unhappy. Once you manually clear up some room there hopefully things will run smoother.
If that doesn't help let us know and we can dig into it further. Thanks, Brad