Few questions about first time out with local install
Hey guys, I just recently got several inquiries from users who would like to set up a local galaxy instance here at our university. I setup a server and got it basically running, the only modifications I have made thus far, are opening the website outside of localhost, and having it communicate with our central postgres server. I have two questions I am hoping someone can answer, I have spent some time on a few blogs and through the archive and haven't found an exact answer. My first question: Is there a way to make the galaxy server retrieve data from a local storage set, instead of uploading newly through the web interface? Http seems like an awfully inefficient way to transmit sequence data. If I mounted an nfs directory full of sequence to the server could I load and analyze it, and what options would I do to config this? My second question: When setting up a cluster do most people make the galaxy server a head and farm out to compute nodes, or do you send jobs off to a remote cluster entirely? It seems like the options in the universe_wsgi.ini file calls a local daemon for cluster management, but I am a bit confused. I hope I haven't wasted anybodies time with these questions. This tool looks amazing and our researchers are keen to get going at it. AC
On Thu, Jun 21, 2012 at 8:54 AM, Alan V. Cowles <alan.cowles@duke.edu> wrote:
My first question: Is there a way to make the galaxy server retrieve data from a local storage set, instead of uploading newly through the web interface? Http seems like an awfully inefficient way to transmit sequence data. If I mounted an nfs directory full of sequence to the server could I load and analyze it, and what options would I do to config this?
I would highly recommend to take a look at the admin documentation on Galaxy's Wiki[1]. You do have several options to upload data from a local file system and share it with your users using Data Libraries[2]. You can also configure your Galaxy to allow users to use FTP to upload data[3], this can also be easily adapted to any other method by which your users can place the data in the specify directory in Galaxy's config( I personally prefer using scp or rsync over ssh, as all my users have local unix account in the server.) [1]http://wiki.g2.bx.psu.edu/Admin [2]http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files [3]http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP
My second question: When setting up a cluster do most people make the galaxy server a head and farm out to compute nodes, or do you send jobs off to a remote cluster entirely? It seems like the options in the universe_wsgi.ini file calls a local daemon for cluster management, but I am a bit confused.
My Galaxy server happens to run on my SGE head node, but as long as the node running Galaxy has qsub abilities and ideally share a file system with the cluster, you should be fine. There is a way around the shared file system but I haven't used so far. You can find documentation about both options in Galaxy's Wiki[4]. [4]http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster Hope it helps, Carlos
Thanks Carlos, I will give it a look over and see what works best for my situation. I think for my first demo setup, I am going to create an SGE environment in my ESX cluster, and just test out the main galaxy server / SGE head, with 3-4 baby nodes. AC On 06/21/2012 11:25 AM, Carlos Borroto wrote:
On Thu, Jun 21, 2012 at 8:54 AM, Alan V. Cowles <alan.cowles@duke.edu> wrote:
My first question: Is there a way to make the galaxy server retrieve data from a local storage set, instead of uploading newly through the web interface? Http seems like an awfully inefficient way to transmit sequence data. If I mounted an nfs directory full of sequence to the server could I load and analyze it, and what options would I do to config this?
I would highly recommend to take a look at the admin documentation on Galaxy's Wiki[1]. You do have several options to upload data from a local file system and share it with your users using Data Libraries[2]. You can also configure your Galaxy to allow users to use FTP to upload data[3], this can also be easily adapted to any other method by which your users can place the data in the specify directory in Galaxy's config( I personally prefer using scp or rsync over ssh, as all my users have local unix account in the server.)
[1]http://wiki.g2.bx.psu.edu/Admin [2]http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files [3]http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP
My second question: When setting up a cluster do most people make the galaxy server a head and farm out to compute nodes, or do you send jobs off to a remote cluster entirely? It seems like the options in the universe_wsgi.ini file calls a local daemon for cluster management, but I am a bit confused.
My Galaxy server happens to run on my SGE head node, but as long as the node running Galaxy has qsub abilities and ideally share a file system with the cluster, you should be fine. There is a way around the shared file system but I haven't used so far. You can find documentation about both options in Galaxy's Wiki[4].
[4]http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster
Hope it helps, Carlos
participants (2)
-
Alan V. Cowles -
Carlos Borroto