Moving this to galaxy-dev list. Just wanted to share my notes on both upload options. Although both ftp_upload_dir and 'user library import' options help in getting data into the galaxy there are some differences in both options: 1. ftp upload option deletes original copies of files after galaxy import, whereas the user library option doesn't delete original copies from the file system 2. ftp option doesn't provide an option to 'sym-link' to files without copying them in galaxy, which 'user library' option supports 3. ftp option scans configured location recursively and displays all files in the web interface, whereas with 'user import' option you will need to organize files in sub-directories 4. ftp option requires galaxy application user to have full permissions over 'ftp_upload_dir', whereas 'user import' will work even with read-execute permissions (execute - for scanning the dirs) for the galaxy application user. Both upload options are useful and have certain advantages-disadvantages depending on your use case. And actually you don't need a FTP server setup for transferring files into ftp_upload_dir location as long as you take care of file system permissions. I think the galaxy server treats 'ftp_upload_dir' location as FTP staging area - where files can be dropped off, then galaxy imports these files in it's application specific naming convention and deletes original copies of these imported files from 'ftp_upload_dir' location. If one takes care of appropriate file permissions for the galaxy application user and user-depositing files then 'ftp_upload_dir' location can be used without setting up FTP server. You can drop of these files using any other transfer protocol as well - e.g. we have used SCP. However, these file permission issues can be handled easily using a FTP server program and hence FTP server option is definitely a recommended approach. -- Shantanu. On Dec 13, 2011, at 10:55 AM, Yury V Bukhman wrote:

Perhaps someone else will find this of interest. As it turned out, we didn't need FTP for a local server. Instead, one can set an option in universe_wsgi.ini (or universe_wsgi.webapp.ini if you are running multiple web server processes) to allow regular users to upload a set of files from a file system into a data library. Thus we have set up a drop box on a shared drive, with a subfolder for each user, where they can put their files, and gave each user their own data library in Galaxy where those files can be uploaded. The drop box folder is mounted on the Galaxy server.

The option in universe_wsgi.webapp.ini is: user_library_import_dir = /mnt/dropbox

It seems to work...

Yury

On 11/14/2011 5:25 PM, Chorny, Ilya wrote:

...

FTP server.

Ilya

-----Original Message----- From: galaxy-user-bounces@lists.bx.psu.edu [mailto:galaxy-user-bounces@lists.bx.psu.edu] On Behalf Of Yury V Bukhman Sent: Monday, November 14, 2011 3:01 PM To: galaxy-user@lists.bx.psu.edu Subject: [galaxy-user] enabling regular users to upload large data volumes to a local Galaxy server

Hi,

we are running a local Galaxy server, administered by a bioinformatics core group. Our end users increasingly come to us with sets of large NGS files that they can't upload to Galaxy on their own through a web browser. We copy their data to a Galaxy filesystem and upload into data libraries from there using the admin interface. However, the users would prefer to be able get their data onto the server on their own. What's the best solution to that? Should we set up FTP upload? Are there other tricks? Any advice would be appreciated.

Thanks.

Yury

-- Yury V. Bukhman, Ph.D. Associate Scientist, Bioinformatics Group Leader Great Lakes Bioenergy Research Center University of Wisconsin - Madison 445 Henry Mall, Rm. 513 Madison, WI 53706, USA Phone: 608-890-2680 Fax: 608-890-2427 Email: ybukhman@glbrc.wisc.edu

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- Yury V. Bukhman, Ph.D. Associate Scientist, Bioinformatics Group Leader Great Lakes Bioenergy Research Center University of Wisconsin - Madison 445 Henry Mall, Rm. 513 Madison, WI 53706, USA Phone: 608-890-2680 Fax: 608-890-2427 Email: ybukhman@glbrc.wisc.edu

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/