I tried to clone a large history (334 active datasets) with very big files (more than half are FASTQ/FASTA files, of 400MB to 1.5GB size).
The python process was busy for 17 minutes, and of course I got an HTTP 504 "proxy timeout" error.
The cloning actually succeeded (after 17 minutes, the history appeared in my histories list), but I was wondering if there's a way to make this faster - because it effectively disables one of the two web-service processes that I have (luckily, I have two now, thanks yesterday's help).
I think I noticed that external set_metadata process was called for some/all of the the cloned datasets. Is it possible to disable set-metadata when cloning? the datasets already have a metadata. (Or I could be mistaken and the external set-metadata was not related to the cloning?).
user: Greg Von Kuster <greg(a)bx.psu.edu>
date: Wed Jan 27 13:34:53 2010 -0500
Some tweaks to the way roles are derived from the access permission on an item.
lib/galaxy/security/__init__.py | 6 ++++--
lib/galaxy/web/controllers/library_common.py | 14 +++++++++++---
2 files changed, 15 insertions(+), 5 deletions(-)
diffs (46 lines):
diff -r 13e217878147 -r e8b17a1cfdf9 lib/galaxy/security/__init__.py
--- a/lib/galaxy/security/__init__.py Wed Jan 27 12:43:11 2010 -0500
+++ b/lib/galaxy/security/__init__.py Wed Jan 27 13:34:53 2010 -0500
@@ -138,13 +138,15 @@
return map( operator.getitem, intermed, ( -1, ) * len( intermed ) )
roles = set()
- # If a library has roles associated with the LIBRARY_ACCESS permission, we need to start with them.
+ # If item has roles associated with the access permission, we need to start with them.
access_roles = item.get_access_roles( trans )
for role in access_roles:
roles.add( role )
# Each role potentially has users. We need to find all roles that each of those users have.
for ura in role.users:
- roles.add( ura.role )
+ user = ura.user
+ for ura2 in user.roles:
+ roles.add( ura2.role )
# Each role also potentially has groups which, in turn, have members ( users ). We need to
# find all roles that each group's members have.
for gra in role.groups:
diff -r 13e217878147 -r e8b17a1cfdf9 lib/galaxy/web/controllers/library_common.py
--- a/lib/galaxy/web/controllers/library_common.py Wed Jan 27 12:43:11 2010 -0500
+++ b/lib/galaxy/web/controllers/library_common.py Wed Jan 27 13:34:53 2010 -0500
@@ -489,10 +489,18 @@
msg=util.sanitize_text( msg ),
messagetype='error' ) )
lddas.append( ldda )
- # If the library is public all roles are legitimate, but if the library is restricted, only those
- # roles associated with the LIBRARY_ACCESS permission are legitimate.
library = trans.sa_session.query( trans.app.model.Library ).get( trans.security.decode_id( library_id ) )
- roles = trans.app.security_agent.get_legitimate_roles( trans, library )
+ # If access to the dataset is restricted, then use the roles associated with the DATASET_ACCESS permission to
+ # determine the legitimate roles. If the dataset is public, see if access to the library is restricted. If
+ # it is, use the roles associated with the LIBRARY_ACCESS permission to determine the legitimate roles. If both
+ # the dataset and the library are public, all roles are legitimate. All of the datasets will have the same
+ # permissions at this point.
+ ldda = lddas
+ if trans.app.security_agent.dataset_is_public( ldda.dataset ):
+ # The dataset is public, so check access to the library
+ roles = trans.app.security_agent.get_legitimate_roles( trans, library )
+ roles = trans.app.security_agent.get_legitimate_roles( trans, ldda.dataset )
if params.get( 'update_roles_button', False ):
current_user_roles = trans.get_current_user_roles()
if cntrller=='library_admin' or ( trans.app.security_agent.can_manage_library_item( current_user_roles, ldda ) and \
I noticed there's a trick to accelerate file downloads for nginx ("X_accel_redirect"),
The following patch adds the same feature for Apache's xsendfile.
To enable it:
1. Download, Compile, Install mod_xsendfile
2. Make sure your apache configuration has the correct "LoadModule" for xsendfile.
3. In the apache's configuration relevant to galaxy, add the two directives:
ReWriteRule ^/galaxy$ /galaxy/ [R]
RewriteRule ^/galaxy(.*) http://127.0.0.1:8080$1 [P]
<Location "/galaxy" >
allow from all
4. In "universe_wsgi.ini", add the following parameter:
apache_xsendfile = true
If all worked well, whenever a user downloads a dataset (via the "save" link), the actual file transfer will be delegated to back apache.
Comments are welcomed,