Hello Alessandro,

I misunderstood the issue you were raising in your initial email, but now understand based on the information you've supplied below.  

When you use the "Upload a directory of files" option to upload files to a data library, all of the files in the directory are uploaded (there is currently no way to choose which files in the directory to upload and which files to ignore).  So in order to keep from uploading files more than once, the files that have been previously uploaded will need to be removed from the subdirectory.  This same behavior applies to the "Upload files from filesystem paths" option if the path is pointing to a subdirectory of files.

The ability to select which files to upload and which to ignore is an enhancement that we'll add to our development roadmap, but it may be a while before we can get to it.

Sorry for my initial misunderstanding on this, as well as the inconvenience this may cause.

Greg Von Kuster


On Oct 14, 2011, at 11:21 AM, alessandro albiero wrote:

Dear Greg,

many thank for your help. I follow all your instruction but my problem is always the same.
Attache you can find 3 images where you can see the problem.

In the figure 1, I'm uploading the files from my folder using sym link. As you can see in the figure 2 there is my file (all_homo.fa).
After that I added a new file in my folder and when I upload again this folder, all the files are reuploaded. See the figure 3, here you can see that the file all_homo.fa is present 2 times and there is also a new sam file.

I'd like to avoid the duplications and upload only the new file not previously uploaded.

Probably I'm doing something wrong but I am not able to understand what.

All the best

Alessandro

2011/10/12 Greg Von Kuster <greg@bx.psu.edu>
Allesandro,

Are your users making the correct selection in the Copy data into Galaxy? select list?  If they choose "Link to files without copying into Galaxy" as shown below, you should see the behavior you want.  The default selection is "Copy files into Galaxy", so if they leave the default selection, copies of the files will be made.

<upload.tiff>

On Oct 12, 2011, at 12:10 PM, alessandro albiero wrote:

Dear Greg,

thank you for your answer. I followed the wiki but I did not solve my problem.
I setted these parameters on the universe_wsgi.ini:

user_library_import_dir = /u/galaxy/userimport

Then I created the /u/galaxy/userimport directory with a series of sub-dirs named as the login users. 
I done this to allow our user to load into galaxy their  files. In this case every user put its data in its dir on /u/galaxy/userimport (or using symlinks) and then load them on galaxy.

My problem is that galaxy duplicate the files; for example:

1) User "A" place a file "FILE1" on its dir and it load the dir on galaxy. On galaxy he sees "FILE1"
2) The user "A" place another file in its dir ("FILE2") and re-load the dir on galaxy. On galaxy he sees "FILE1", "FILE1", "FILE2" 

It seems that galaxy re-loads all the files of the user dir without checking the duplications.
We would like to avoid this duplications.

We found that removing the old files from the user directory solves the problem. Is there a simpler way to do this (without removing the files) ?


All the best

Alessandro



2011/10/11 Greg Von Kuster <greg@bx.psu.edu>
Hello Alessandro,

You can upload files from filesystem paths to a Galaxy data library without copying the files into Galaxy's configured file directory.  This approach should provide what you're looking to do.  See our wiki at http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files for all of the details.

Greg Von Kuster

On Oct 11, 2011, at 6:45 AM, alessandro albiero wrote:

Dear all,

I have a problem with galaxy and I have already tried to write on ther user list....
My problem concern the upload on galaxy of new files.

We are setting up a galaxy istance on our server (HPC Cluster with torque and PBS) for internal use.
Our problem is that we manage only NGS data (SOLiD and Illumina Data) that are already stored on the server. 
We tried to set up the user directory in the universe_wgsi.ini file to allow to any user tu upload their local files, but adding new file and re-uploading the dir we found duplicated files (it seems that galaxy re-upload every time all the files ignoring if some are already present). This method is OK for us but how can we avoid the file duplications ?

In a presentation found on web (blankenberg_pittsburg_2001), we saw that there is the possibility to automatically upload the sequenced reads on Galaxy (and let them available to all our user). Where can we find information for doing this ?


Thank in advance

All the best 

Alessandro 




--
******************************
Alessandro Albiero
Bioinformatics Lab
CRIBI-Padua University
Viale G. Colombo, 3
35121 Padova-Italy

http://genomics.cribi.unipd.it
http://genomics.cribi.unipd.it/index.php/User:Albiero
Phone. +39. 049 827-6165
Fax.   +39. 049 827-6159
********************************


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Greg Von Kuster
Galaxy Development Team





Greg Von Kuster
Galaxy Development Team




<Fig_1.png><Fig_2.png><Fig_3.png>

Greg Von Kuster
Galaxy Development Team
greg@bx.psu.edu