Hello all, I recently migrated the location of the Galaxy file_path directory (~/database/files/) along with the temp, job_working_directory, and pbs directory locations (all under ~/database/) to a new storage system mount point and I properly updated the change in the universe_wsgi.ini file but have run into some problems with managing our datasets. I can run Galaxy jobs just fine but I cannot view or download the datasets in the output of the jobs after they have run - when it tries to open it from the new path - and I get an actual browser error when trying to fetch the dataset. On a hunch I created a symbolic link named after the old path in the / directory pointing to the new location (and pointed file_path in the universe_wsgi.ini file back to it - which looks exactly like the old directory path), and sure enough this works as a (temporary) fix to view/download the files. However doing that seems to, in turn, mess with the file uploading function which will fail every time with a "OSError: [Errno 2] No such file or directory" message. So I'm caught between two broken configurations depending on the path I point to (the new path or the symlink alternative). I guess in the latter situation it does not like traversing a symbolic link (which is the only explanation that I can think of). The root of the problem needs to be fixed though in that Galaxy seems to be "remembering" the old path from somewhere (the Galaxy database?) such that it will not let me simply migrate the data to a new location and change the path variables in universe_wsgi.ini. Any suggestions on the proper changes that need to be made to fix this permanently? Thanks, Josh Nielsen
Okay, I solved half of the problem. The upload job was consistently being submitted to the one compute node in our cluster that I forgot to create the symlink on that points to the new location. I can now upload files. This means I am fully functional now, which is good, however I still prefer to do away with the symlink approach altogether and get Galaxy to work directly with the new directory path. Any suggestions for getting that to work? Thanks! On Mon, Sep 17, 2012 at 3:50 PM, Josh Nielsen <jnielsen@hudsonalpha.com>wrote:
Hello all,
I recently migrated the location of the Galaxy file_path directory (~/database/files/) along with the temp, job_working_directory, and pbs directory locations (all under ~/database/) to a new storage system mount point and I properly updated the change in the universe_wsgi.ini file but have run into some problems with managing our datasets. I can run Galaxy jobs just fine but I cannot view or download the datasets in the output of the jobs after they have run - when it tries to open it from the new path - and I get an actual browser error when trying to fetch the dataset. On a hunch I created a symbolic link named after the old path in the / directory pointing to the new location (and pointed file_path in the universe_wsgi.ini file back to it - which looks exactly like the old directory path), and sure enough this works as a (temporary) fix to view/download the files. However doing that seems to, in turn, mess with the file uploading function which will fail every time with a "OSError: [Errno 2] No such file or directory" message. So I'm caught between two broken configurations depending on the path I point to (the new path or the symlink alternative). I guess in the latter situation it does not like traversing a symbolic link (which is the only explanation that I can think of). The root of the problem needs to be fixed though in that Galaxy seems to be "remembering" the old path from somewhere (the Galaxy database?) such that it will not let me simply migrate the data to a new location and change the path variables in universe_wsgi.ini. Any suggestions on the proper changes that need to be made to fix this permanently?
Thanks, Josh Nielsen
Hi Josh, Are you using XSendFile or X-Accel-Redirect, by any chance? --nate On Sep 17, 2012, at 5:18 PM, Josh Nielsen wrote:
Okay, I solved half of the problem. The upload job was consistently being submitted to the one compute node in our cluster that I forgot to create the symlink on that points to the new location. I can now upload files. This means I am fully functional now, which is good, however I still prefer to do away with the symlink approach altogether and get Galaxy to work directly with the new directory path. Any suggestions for getting that to work?
Thanks!
On Mon, Sep 17, 2012 at 3:50 PM, Josh Nielsen <jnielsen@hudsonalpha.com> wrote: Hello all,
I recently migrated the location of the Galaxy file_path directory (~/database/files/) along with the temp, job_working_directory, and pbs directory locations (all under ~/database/) to a new storage system mount point and I properly updated the change in the universe_wsgi.ini file but have run into some problems with managing our datasets. I can run Galaxy jobs just fine but I cannot view or download the datasets in the output of the jobs after they have run - when it tries to open it from the new path - and I get an actual browser error when trying to fetch the dataset. On a hunch I created a symbolic link named after the old path in the / directory pointing to the new location (and pointed file_path in the universe_wsgi.ini file back to it - which looks exactly like the old directory path), and sure enough this works as a (temporary) fix to view/download the files. However doing that seems to, in turn, mess with the file uploading function which will fail every time with a " OSError: [Errno 2] No such file or directory" message.
So I'm caught between two broken configurations depending on the path I point to (the new path or the symlink alternative). I guess in the latter situation it does not like traversing a symbolic link (which is the only explanation that I can think of). The root of the problem needs to be fixed though in that Galaxy seems to be "remembering" the old path from somewhere (the Galaxy database?) such that it will not let me simply migrate the data to a new location and change the path variables in universe_wsgi.ini. Any suggestions on the proper changes that need to be made to fix this permanently?
Thanks, Josh Nielsen
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Yes I am, and recently I thought to check my httpd.conf and it still has a "XSendFilePath /panfs/galaxy_data" setting, where "/panfs" is the obsolete directory path, and figured that was the problem, I just haven't changed it yet. Thanks for the suggestion to look! I'll let you know if changing that doesn't fix it. -Josh On Mon, Oct 1, 2012 at 11:33 AM, Nate Coraor <nate@bx.psu.edu> wrote:
Hi Josh,
Are you using XSendFile or X-Accel-Redirect, by any chance?
--nate
On Sep 17, 2012, at 5:18 PM, Josh Nielsen wrote:
Okay, I solved half of the problem. The upload job was consistently being submitted to the one compute node in our cluster that I forgot to create the symlink on that points to the new location. I can now upload files. This means I am fully functional now, which is good, however I still prefer to do away with the symlink approach altogether and get Galaxy to work directly with the new directory path. Any suggestions for getting that to work?
Thanks!
On Mon, Sep 17, 2012 at 3:50 PM, Josh Nielsen <jnielsen@hudsonalpha.com> wrote: Hello all,
I recently migrated the location of the Galaxy file_path directory (~/database/files/) along with the temp, job_working_directory, and pbs directory locations (all under ~/database/) to a new storage system mount point and I properly updated the change in the universe_wsgi.ini file but have run into some problems with managing our datasets. I can run Galaxy jobs just fine but I cannot view or download the datasets in the output of the jobs after they have run - when it tries to open it from the new path - and I get an actual browser error when trying to fetch the dataset. On a hunch I created a symbolic link named after the old path in the / directory pointing to the new location (and pointed file_path in the universe_wsgi.ini file back to it - which looks exactly like the old directory path), and sure enough this works as a (temporary) fix to view/download the files. However doing that seems to, in turn, mess with the file uploading function which will fail every time with a " OSError: [Errno 2] No such file or directory" message.
So I'm caught between two broken configurations depending on the path I point to (the new path or the symlink alternative). I guess in the latter situation it does not like traversing a symbolic link (which is the only explanation that I can think of). The root of the problem needs to be fixed though in that Galaxy seems to be "remembering" the old path from somewhere (the Galaxy database?) such that it will not let me simply migrate the data to a new location and change the path variables in universe_wsgi.ini. Any suggestions on the proper changes that need to be made to fix this permanently?
Thanks, Josh Nielsen
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Josh Nielsen -
Nate Coraor