Using data_fabfile.py to update data from s3
Hi, I am trying to use install_data_s3 (data_fabfile.py) to get data from the cloudbiolinux bucket into a virtualbox VM, as per the instructions at usegalaxy.org/vm . When I ran fab -f data_fabfile.py -H localhost install_data_s3:config/galaxy_default_biodata.yaml the first time, it worked. But when I try running it a second time, it dies with [localhost] run: echo 'index phix /mnt/galaxyIndices/genomes/phiX174/phix/seq/phix.fa >> sam_fa_indices.loc [localhost] out: /bin/bash: sam_fa_indices.loc: Permission denied This means that I can't update /opt/galaxy/pkg/cloudbiolinux/config/galaxy_default_biodata.yaml and run install_data_s3 again to get more genomes. The first time I ran it I only included phiX (just to test it, and because that's the default). fab fails the second time whether I edit the yaml file or not. I found a file called sam_fa_indices.loc under /mnt/galaxyTools/galaxy-central/tool-data and sure enough it has write permissions off unless you are user galaxy (not galaxyadmin, which you are at that point in the instructions). What am I doing wrong? Is this list the right venue for these questions? I also noticed a couple of minor things in earlier stages of the instructions: sudo mount /dev/sdb /mnt/galaxyData and sudo mount /dev/sdb /mnt/galaxyIndices are telling the user to mount the same device on both; sdb is only illustrative but should probably be sdc or something. Also those `sudo mount` commands mount galaxyData and galaxyIndices as root-only, so running fab will fail with permissions errors even the first time. I changed the permissions on /mnt/galaxyIndices to get round this, just to try it out, but in general this might have been a bad thing to do! Finally: something I don't understand happens if I try to redirect the stdout and/or stderr from fab into a file. So it's hard to keep records of what happened each time I ran it. If anyone understands what's going on there that would be helpful too! Sorry for what are probably basic questions, I'm new to python fabric. Thanks, Clare -- E: sloc@unimelb.edu.au P: 03 903 53357 M: 0414 854 759
Clare;
I am trying to use install_data_s3 (data_fabfile.py) to get data from the cloudbiolinux bucket into a virtualbox VM, as per the instructions at usegalaxy.org/vm .
When I ran fab -f data_fabfile.py -H localhost install_data_s3:config/galaxy_default_biodata.yaml the first time, it worked.
But when I try running it a second time, it dies with [localhost] run: echo 'index phix /mnt/galaxyIndices/genomes/phiX174/phix/seq/phix.fa >> sam_fa_indices.loc [localhost] out: /bin/bash: sam_fa_indices.loc: Permission denied
The fabric script is running standard shell commands, so all of the normal unix permissions will apply. As you mention, if the permissions to /mnt/galaxyTools/galaxy-central/tool-data/sam_fa_indices.loc do not allow access to galaxyadmin you'll need to adjust that from the commandline. Reading the instructions, the 'galaxyadmin' and 'galaxy' user suggestions are a bit confusing. My suggestion it to run the install_data_s3 command as the 'galaxy' user, and do: 'sudo chmod -R galaxy /directory/name' on any directories with permission issues. Since you later run the server as 'galaxy', this will ensure that user has proper permissions. Hope this helps, Brad
participants (2)
-
Brad Chapman
-
Clare Sloggett