Hi Nate, I am afraid there is still an issue while trying to upload a directory of files (containing ZIP files). My ZIP files are not erased anymore from my "user_library_import_dir". But I still get the message: "Job error (click name for more info)" and after clicking on any of those files, I get: Message: Uploaded by: erick@mydomain.com Date uploaded: 2010-03-31 Build: ? Miscellaneous information: The uploaded file contains inappropriate content error Any suggestions? thanks in advance, Erick p.s. my changeset: 3560:4c95f1a101f1 On 23 March 2010 17:45, Nate Coraor <nate@bx.psu.edu> wrote:

Hi Erick,

Thanks for reporting this, the deletion bug has been fixed in changeset 3556:8f6fa47b21c4.

--nate

Erick Antezana wrote:

...

Hi,

It seems there is a (serious) bug while "Adding datasets" via the "Upload directory of files" (with or without copying the data into Galaxy (3545:2590120aed68), i.e. (un)tick the 'No' box) if the files are in ZIP format. Actually the files get erased from the filesystem.

In the "information" column, I get:

       Job error (click name for more info)

Then, after clicking on the file name:

      Information about 2010.fastq.zip       Message:

      Uploaded by: erick@mydomain.com       Date uploaded: 2010-03-23       Build: ?       Miscellaneous information: The uploaded file contains inappropriate content       error

We have no problems with adding unzipped files...

thanks, Erick _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Erick Antezana wrote:

Hi Nate,

I am afraid there is still an issue while trying to upload a directory of files (containing ZIP files). My ZIP files are not erased anymore from my "user_library_import_dir". But I still get the message:

"Job error (click name for more info)"

Hi Erick, Do you have tools that work directly with unmodified ZIP files? We don't have support for unpacking ZIP files during upload yet (this is the cause of the "inappropriate content" message), although work on this is in progress. Thanks, --nate

and after clicking on any of those files, I get:

Message:

Uploaded by: erick@mydomain.com

Date uploaded: 2010-03-31

Build: ?

Miscellaneous information: The uploaded file contains inappropriate content

error

Any suggestions?

thanks in advance, Erick p.s. my changeset: 3560:4c95f1a101f1

On 23 March 2010 17:45, Nate Coraor <nate@bx.psu.edu> wrote:

...

Hi Erick,

Thanks for reporting this, the deletion bug has been fixed in changeset 3556:8f6fa47b21c4.

--nate

Erick Antezana wrote:

...

Hi,

It seems there is a (serious) bug while "Adding datasets" via the "Upload directory of files" (with or without copying the data into Galaxy (3545:2590120aed68), i.e. (un)tick the 'No' box) if the files are in ZIP format. Actually the files get erased from the filesystem.

In the "information" column, I get:

Job error (click name for more info)

Then, after clicking on the file name:

Information about 2010.fastq.zip Message:

Uploaded by: erick@mydomain.com Date uploaded: 2010-03-23 Build: ? Miscellaneous information: The uploaded file contains inappropriate content error

We have no problems with adding unzipped files...

thanks, Erick _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Hmm, this is a tricky case. I can think of three possible solutions. A "don't unzip" checkbox on the upload form (but this may be confusing to people who aren't aware that only a few specialized tools would be able to deal with the zipped files). Let Galaxy auto-detect the file type based on extension, if it's actually a zip file. Like Python eggs and self-extracting exes, it could be a zip file which has been renamed to some new extension. Force users to select the correct file type from the drop-down, or else if it's detected as a zip it's always unpacked. This is the way we did it for a few older tools. --nate Erick Antezana wrote:

Hi Nate,

actually, I wanted to develop a couple of tools (to be further used from within a workflow) that could take as input a ZIP-formatted file and then process it with other tools. The motivation was due to two data providers that sent out data in ZIP :-( and not GZ... So we couldn't ask our users to UNZIP them (lots of files) and afterwards to GZIP them... Anyway, I manage to get a *compromise* with our Galaxy users and we will use GZ-formatted files (thus the ZIP ones will become GZ...).

thanks, Erick

On 31 March 2010 18:32, Nate Coraor <nate@bx.psu.edu> wrote:

...

Erick Antezana wrote:

...

Hi Nate,

I am afraid there is still an issue while trying to upload a directory of files (containing ZIP files). My ZIP files are not erased anymore from my "user_library_import_dir". But I still get the message:

"Job error (click name for more info)" Hi Erick,

Do you have tools that work directly with unmodified ZIP files? We don't have support for unpacking ZIP files during upload yet (this is the cause of the "inappropriate content" message), although work on this is in progress.

Thanks, --nate

...

and after clicking on any of those files, I get:

Message:

Uploaded by: erick@mydomain.com

Date uploaded: 2010-03-31

Build: ?

Miscellaneous information: The uploaded file contains inappropriate content

error

Any suggestions?

thanks in advance, Erick p.s. my changeset: 3560:4c95f1a101f1

On 23 March 2010 17:45, Nate Coraor <nate@bx.psu.edu> wrote:

...

Hi Erick,

Thanks for reporting this, the deletion bug has been fixed in changeset 3556:8f6fa47b21c4.

--nate

Erick Antezana wrote:

...

Hi,

It seems there is a (serious) bug while "Adding datasets" via the "Upload directory of files" (with or without copying the data into Galaxy (3545:2590120aed68), i.e. (un)tick the 'No' box) if the files are in ZIP format. Actually the files get erased from the filesystem.

In the "information" column, I get:

Job error (click name for more info)

Then, after clicking on the file name:

Information about 2010.fastq.zip Message:

Uploaded by: erick@mydomain.com Date uploaded: 2010-03-23 Build: ? Miscellaneous information: The uploaded file contains inappropriate content error

We have no problems with adding unzipped files...

thanks, Erick _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Hi Erick, I've added these suggestions to an existing ticket here: http://bitbucket.org/galaxy/galaxy-central/issue/280/allow-zip-file-uploads --nate Erick Antezana wrote:

Nate,

I personally like the second scenario and 3rd one you'd proposed.

Erick

On 1 April 2010 14:01, Nate Coraor <nate@bx.psu.edu> wrote:

...

Hmm, this is a tricky case. I can think of three possible solutions.

A "don't unzip" checkbox on the upload form (but this may be confusing to people who aren't aware that only a few specialized tools would be able to deal with the zipped files).

Let Galaxy auto-detect the file type based on extension, if it's actually a zip file. Like Python eggs and self-extracting exes, it could be a zip file which has been renamed to some new extension.

Force users to select the correct file type from the drop-down, or else if it's detected as a zip it's always unpacked. This is the way we did it for a few older tools.

--nate

Erick Antezana wrote:

...

Hi Nate,

actually, I wanted to develop a couple of tools (to be further used from within a workflow) that could take as input a ZIP-formatted file and then process it with other tools. The motivation was due to two data providers that sent out data in ZIP :-( and not GZ... So we couldn't ask our users to UNZIP them (lots of files) and afterwards to GZIP them... Anyway, I manage to get a *compromise* with our Galaxy users and we will use GZ-formatted files (thus the ZIP ones will become GZ...).

thanks, Erick

On 31 March 2010 18:32, Nate Coraor <nate@bx.psu.edu> wrote:

...

Erick Antezana wrote:

...

Hi Nate,

I am afraid there is still an issue while trying to upload a directory of files (containing ZIP files). My ZIP files are not erased anymore from my "user_library_import_dir". But I still get the message:

"Job error (click name for more info)" Hi Erick,

Do you have tools that work directly with unmodified ZIP files? We don't have support for unpacking ZIP files during upload yet (this is the cause of the "inappropriate content" message), although work on this is in progress.

Thanks, --nate

...

and after clicking on any of those files, I get:

Message:

Uploaded by: erick@mydomain.com

Date uploaded: 2010-03-31

Build: ?

Miscellaneous information: The uploaded file contains inappropriate content

error

Any suggestions?

thanks in advance, Erick p.s. my changeset: 3560:4c95f1a101f1

On 23 March 2010 17:45, Nate Coraor <nate@bx.psu.edu> wrote:

...

Hi Erick,

Thanks for reporting this, the deletion bug has been fixed in changeset 3556:8f6fa47b21c4.

--nate

Erick Antezana wrote: > Hi, > > It seems there is a (serious) bug while "Adding datasets" via the > "Upload directory of files" (with or without copying the data into > Galaxy (3545:2590120aed68), i.e. (un)tick the 'No' box) if the files > are in ZIP format. Actually the files get erased from the filesystem. > > In the "information" column, I get: > > Job error (click name for more info) > > Then, after clicking on the file name: > > Information about 2010.fastq.zip > Message: > > Uploaded by: erick@mydomain.com > Date uploaded: 2010-03-23 > Build: ? > Miscellaneous information: The uploaded file contains > inappropriate content > error > > We have no problems with adding unzipped files... > > thanks, > Erick > _______________________________________________ > galaxy-user mailing list > galaxy-user@lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

Hello Peter, You'll need to uncomment the following lines in the ~/lib/galaxy/admin.py controller (lines 92-94 in the latest rev): #operations.append( grids.GridOperation( "Delete", condition=( lambda item: not item.deleted ), allow_multiple=True ) ) #operations.append( grids.GridOperation( "Undelete", condition=( lambda item: item.deleted and not item.purged ), allow_multiple=True ) ) #operations.append( grids.GridOperation( "Purge", condition=( lambda item: item.deleted and not item.purged ), allow_multiple=True ) ) The buttons will then be displayed on the manage users page in the admin view. Be aware that "deleting a user" only marks the user as deleted ( the deleted column is True ) in the database, keeping the state of everything associated with the user, allowing for undeletion if desired. If you want to go further, you can "Purge the user" after you have deleted their account. Here is what happens when you purge the user's account: # This method should only be called for a User that has previously been deleted. # We keep the User in the database ( marked as purged ), and stuff associated # with the user's private role in case we want the ability to unpurge the user # some time in the future. # Purging a deleted User deletes all of the following: # - History where user_id = User.id # - HistoryDatasetAssociation where history_id = History.id # - Dataset where HistoryDatasetAssociation.dataset_id = Dataset.id # - UserGroupAssociation where user_id == User.id # - UserRoleAssociation where user_id == User.id EXCEPT FOR THE PRIVATE ROLE # Purging Histories and Datasets must be handled via the cleanup_datasets.py script Again, the user's record remains in the database, but you can manually remove it and the private role using sql if you wish ( we do not advise this, but it's not hard to do). The user's private role is in the role table where the type is private and the name is the user's account / email address. On Jun 3, 2010, at 6:04 AM, Pieter Neerincx wrote:

Hi,

In my univers_wsgi.ini I have:

# Can an admin user delete user accounts? allow_user_deletion = True

But when I go the the admin web interface -> security -> manage users, I cannot delete users. There is only a "Create New User" button at the top and a "for selected users: Reset Password" button at the bottom... Am I missing something?

Cheers,

Pi

--------------------------------------------------------------- Biomolecular Mass Spectrometry & Proteomics group Utrecht University

Visiting address: H.R. Kruyt building room O607 Padualaan 8 3584 CH Utrecht The Netherlands

Mail address: P.O. box 80.082 3508 TB Utrecht The Netherlands

phone: +31 6 143 66 783 email: pieter.neerincx@gmail.com skype: pieter.online ---------------------------------------------------------------

_______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Greg Von Kuster Galaxy Development Team greg@bx.psu.edu

Hi Peter, On Jun 4, 2010, at 7:25 AM, Pieter Neerincx wrote:

Dear Greg,

On Jun 3, 2010, at 2:51 PM, Greg Von Kuster wrote:

...

Hello Peter,

You'll need to uncomment the following lines in the ~/lib/galaxy/admin.py controller (lines 92-94 in the latest rev):

#operations.append( grids.GridOperation( "Delete", condition=( lambda item: not item.deleted ), allow_multiple=True ) ) #operations.append( grids.GridOperation( "Undelete", condition=( lambda item: item.deleted and not item.purged ), allow_multiple=True ) ) #operations.append( grids.GridOperation( "Purge", condition=( lambda item: item.deleted and not item.purged ), allow_multiple=True ) )

The buttons will then be displayed on the manage users page in the admin view. Be aware that "deleting a user" only marks the user as deleted ( the deleted column is True ) in the database, keeping the state of everything associated with the user, allowing for undeletion if desired.

Thanks for the info! Is there any particular reason why this is disabled by default?

This is disabled by default because: - it is very rarely needed ( if ever ), and probably should never be used in production instances - We have a TODO item in the code to add an enhancement to be able to check the configuration setting before displaying the buttons, but it is low on the priority list for now ( again, because this feature is questionable in production instances ).

I know Galaxy's philosophy is to save everything you do to make sure research is reproducible, but sometimes it can be really handy to delete stuff. I was trying to debug an issue with user account registration forms and therefore created dozens of accounts to see if the issue was gone after fiddling with my Galaxy and Apache configs. And I really really want to delete those phony test accounts as they clutter up the lists for sharing histories and workflows with colleagues :).

If debugging, it may be better to use a Galaxy instance that is not your production instance where the database can easily be dropped and recreated as necessary.

...

If you want to go further, you can "Purge the user" after you have deleted their account. Here is what happens when you purge the user's account:

# This method should only be called for a User that has previously been deleted. # We keep the User in the database ( marked as purged ), and stuff associated # with the user's private role in case we want the ability to unpurge the user # some time in the future. # Purging a deleted User deletes all of the following: # - History where user_id = User.id # - HistoryDatasetAssociation where history_id = History.id # - Dataset where HistoryDatasetAssociation.dataset_id = Dataset.id # - UserGroupAssociation where user_id == User.id # - UserRoleAssociation where user_id == User.id EXCEPT FOR THE PRIVATE ROLE # Purging Histories and Datasets must be handled via the cleanup_datasets.py script

Again, the user's record remains in the database, but you can manually remove it and the private role using sql if you wish ( we do not advise this, but it's not hard to do). The user's private role is in the role table where the type is private and the name is the user's account / email address.

To reduce the amount of manual maintenance work on a Galaxy install it would be nice if functionality for purging users would eventually be added to the cleanup scripts just like for the datasets and histories...

We'll consider this ( e.g., if a user has been deleted for X number of days, they can be purged ). However, this will probably be fairly low on the priority list as well.

Thanks,

Pi

...

On Jun 3, 2010, at 6:04 AM, Pieter Neerincx wrote:

...

Hi,

In my univers_wsgi.ini I have:

# Can an admin user delete user accounts? allow_user_deletion = True

But when I go the the admin web interface -> security -> manage users, I cannot delete users. There is only a "Create New User" button at the top and a "for selected users: Reset Password" button at the bottom... Am I missing something?

Cheers,

Pi

--------------------------------------------------------------- Biomolecular Mass Spectrometry & Proteomics group Utrecht University

Visiting address: H.R. Kruyt building room O607 Padualaan 8 3584 CH Utrecht The Netherlands

Mail address: P.O. box 80.082 3508 TB Utrecht The Netherlands

phone: +31 6 143 66 783 email: pieter.neerincx@gmail.com skype: pieter.online ---------------------------------------------------------------

_______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Greg Von Kuster Galaxy Development Team greg@bx.psu.edu

------------------------------------------------------------- mobile: +31 6 143 66 783 e-mail: pieter.neerincx@gmail.com skype: pieter.online -------------------------------------------------------------

_______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user

Greg Von Kuster Galaxy Development Team greg@bx.psu.edu