- galaxy-user - lists.galaxyproject.org

Ultradeep sequencing analysis with Galaxy
by rocco.piazza 15 Jul '09

15 Jul '09

2 1

Galaxy user list
by Lisette Hörnblad 15 Jul '09

15 Jul '09

Hi, I'm not using Galaxy right now and I therefore would like to exit the user-list so that I don't get e-mails regarding Galaxy any more. Thank you very much! Best Regards Lisette

1 0

Re: [galaxy-user] RBAC and libraries
by Ido M. Tamir 15 Jul '09

15 Jul '09

On Tuesday 14 July 2009 17:42:16 Greg Von Kuster wrote: > Ido M. Tamir wrote: > > On Tuesday 14 July 2009 15:25:47 you wrote: > >> Hello Ido, > >> > All roles are required because of the complication of derived > permissions for datasets that are produced from running jobs on > different input datasets. Take, for example, two datasets, DatasetA, > and DatasetB. Access to DatasetA requires a user to be associated with > RoleA, and access to DatasetB with RoleB. It follows that DatasetAB ( > created by running a tool with DatasetA and DatasetB as inputs ) should > have both RoleA and RoleB associated for the access permission. > However, if ANY associated role was sufficient, this would allow users > who are only associated either RoleA OR RoleB to view data which was > originally not accessible to them. That is an important point. But if this is the best solution I am not sure yet - first I would like to get my data in, then I can think on the effects of combining them. Could you show me how I could solve the problem I presented in my previous e-mail without creating ad-hoc roles for every user combination? thank your very much, ido

2 2

Re: [galaxy-user] RBAC and libraries]
by Greg Von Kuster 14 Jul '09

14 Jul '09

Ido M. Tamir wrote: > On Tuesday 14 July 2009 15:25:47 you wrote: > >> Hello Ido, >> >> We're sorry to hear that you have encountered problems in working with >> dataset libraries and security in Galaxy. Please see my explanations >> below, and let me know if more clarification is needed or if you come >> across behavior that differs from that described. >> > > Unfortunately I am a bit slow and need more clarification. The behavior > does not differ from what you described and there I think lies my problem. > > >> 1) manage permissions: Role members can manage the roles associated with >> this dataset >> 2) access: Role members can import this dataset into their history for >> analysis. NOTE: Users must have every role associated with the access >> permission on this dataset in order to access it >> > > It is this NOTE that I don't understand. In every other case you allow one > of each role. In this case you require that it fits all of the roles. > All roles are required because of the complication of derived permissions for datasets that are produced from running jobs on different input datasets. Take, for example, two datasets, DatasetA, and DatasetB. Access to DatasetA requires a user to be associated with RoleA, and access to DatasetB with RoleB. It follows that DatasetAB ( created by running a tool with DatasetA and DatasetB as inputs ) should have both RoleA and RoleB associated for the access permission. However, if ANY associated role was sufficient, this would allow users who are only associated either RoleA OR RoleB to view data which was originally not accessible to them. > >> The "library dataset" permission "access" is a restrict permission. If >> no roles are associated with it, the library dataset is considered >> public, and all users can access it in the library. However, if a role >> is associated with the "access" permission on the library dataset, a >> user must have that role in order to access the dataset in the library. >> If more than 1 role is associated with the "access" permission, then >> a user must have all roles in order to access it. >> > But why do you model this permission like this - requiring all roles? > See above explanation > >>> You require that the user should have all of the roles that are currently >>> associated with the library. This makes simple things difficult and >>> complicated things impossible. >>> >> This is not necessarily the case - for example, user can have >> permission to add a library item, but not manage library permissions. >> What behavior are you seeing that leads you to this conclusion? >> > > I actually only talk about the "access" permission. Here the user has > to have all the roles associated with him (should have pointed this > out more). > > >> Do you mean user/group ( Galaxy has no concept that would differentiate >> a user from a bioinformatician )? It depends on how you set up your >> groups of users. You could have 1 group with all users, and associate >> that group with a role, then associate that role with each "library" >> permission. >> > But then as soon as I add a new user, that should not have access to the > datasets all the other users have, this policy scheme breaks down. > > Maybe I can explain what I intend in an example - and then you can > tell me how I should actually do this: > > users (I simplify and say no user is in more than one group): > userName group > user1 group1 > user2 group2 > user3 bioinfo > user4* group4 > > user4 is a collaborator of user1 - he should have access to the data > of user1 - not to the rest of group1 data. > > Dataset owner > A user1 > B user2 > > I would normally solve my policy problem by creating the following roles: > role members > r_u1 user1* > r_u2 user2* > r_u3 user3* > r_u4 user4* > r_g1 group1** > r_g2 group2** > r_g4 group4** > r_b user3 > > *I think galaxy automatically creates roles from users or allows association > of libraries with users, so this might not be necessary. > Galaxy automatically creates a "private" role for each user, but this is the only role that is automatically created. > **I think galaxy automatically adds the roles of the groups one is member of > to the users roles, so I don't add user1 to r_u1 etc... > > Then I would associate the following privileges with the datasets > Dataset modify access > A r_u1,r_b r_u1,r_g1,r_b,r_u4 > B r_u2,r_b r_u2,r_g2,r_b > > Galaxy does not allow me to grant these access privileges, because > there is no user having all the roles. > It would be very simple to solve, if you would model the access permission > like you model all the other permissions. > Hopefully my comments above clarify why this is necessary. The roles associated with the access permission on a dataset are "and", not "or". Otherwise data security quickly breaks down on datasets derived from others. I agree that it's tricky, but ensuring data is only accessible by the desired users was our highest priority in implementing RBAC. > > Thank you very much for you time, > ido > > > >

1 0

Standalone galaxy server and user management
by Andreas Kuntzagk 14 Jul '09

14 Jul '09

Hi, we want to use a standalone Galaxy server here and integrate that with our Sun Grinengine cluster. While I could find information on how to do that I also want to know if it's possible to use our local user database (NIS based) to manage the galaxy users and make their galaxy scripts running with their userid. Could somebody tell me if that is a) easy b) hard c) impossible regards, Andreas

3 6

Re: [galaxy-user] how to clear job queue?
by R 14 Jul '09

14 Jul '09

thanks for help! the error log as below: --------------------------------------------------------- galaxy.jobs.runners.local INFO 2009-07-13 17:30:21,924 starting workers galaxy.jobs.runners.local DEBUG 2009-07-13 17:30:21,945 5 workers ready galaxy.jobs.runners.local INFO 2009-07-13 17:30:21,950 starting workers galaxy.jobs.runners.local DEBUG 2009-07-13 17:30:21,969 5 workers ready galaxy.jobs.schedulingpolicy.roundrobin INFO 2009-07-13 17:30:21,992 RoundRobin policy: initialized galaxy.jobs INFO 2009-07-13 17:30:21,992 job scheduler policy is galaxy.jobs.schedulingpolicy.roundrobin:UserRoundRobin galaxy.jobs INFO 2009-07-13 17:30:22,005 job manager started galaxy.jobs DEBUG 2009-07-13 17:30:22,202 no runner: 326 is still in new state, adding to the jobs queue galaxy.jobs DEBUG 2009-07-13 17:30:22,202 no runner: 330 is still in new state, adding to the jobs queue Traceback (most recent call last): File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/web/buildapp.py", line 56, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/app.py", line 66, in __init__ self.job_manager = jobs.JobManager( self ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 33, in __init__ self.job_queue = JobQueue( app, self.dispatcher ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 110, in __init__ self.__check_jobs_at_startup() File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 127, in __check_jobs_at_startup job_wrapper = JobWrapper( job, self.app.toolbox.tools_by_id[ job.tool_id ], self ) KeyError: u'xslt_converter_16' ---------------------------------------------------------- --- On Mon, 7/13/09, Nate Coraor <nate(a)bx.psu.edu> wrote: From: Nate Coraor <nate(a)bx.psu.edu> Subject: Re: [galaxy-user] how to clear job queue? To: "R" <riververy(a)yahoo.com> Cc: "galaxy" <galaxy-user(a)bx.psu.edu> Date: Monday, July 13, 2009, 9:36 AM R wrote: > I accidentally shut down my galaxy server in the middle of an job execution, after that I try to restart my galaxy server, it always shows the error that a job is in new state, and I can't start my galaxy server at all. I guess I have to clear that job out of the queue, so my galaxy server can start. > > Does anyone know how to clear the job queue? or any idea how to fix the problem? Hello, Can you provide us with any relevant terminal output or errors? Galaxy attempts to manage new/queued/running jobs upon startup and should not hang or crash during this process. --nate

2 1

how to clear the job queue
by R 14 Jul '09

14 Jul '09

I accidentally shut down my galaxy server in the middle of an job execution, after that I try to restart my galaxy server, it always shows the error that a job is in new state, and I can't start my galaxy server at all. I guess I have to clear that job out of the queue, so my galaxy server can start. Does anyone know how to clear the job queue? or any idea how to fix the problem? the error log as below: --------------------------------------------------------- galaxy.jobs.runners.local INFO 2009-07-13 17:30:21,924 starting workers galaxy.jobs.runners.local DEBUG 2009-07-13 17:30:21,945 5 workers ready galaxy.jobs.runners.local INFO 2009-07-13 17:30:21,950 starting workers galaxy.jobs.runners.local DEBUG 2009-07-13 17:30:21,969 5 workers ready galaxy.jobs.schedulingpolicy.roundrobin INFO 2009-07-13 17:30:21,992 RoundRobin policy: initialized galaxy.jobs INFO 2009-07-13 17:30:21,992 job scheduler policy is galaxy.jobs.schedulingpolicy.roundrobin:UserRoundRobin galaxy.jobs INFO 2009-07-13 17:30:22,005 job manager started galaxy.jobs DEBUG 2009-07-13 17:30:22,202 no runner: 326 is still in new state, adding to the jobs queue galaxy.jobs DEBUG 2009-07-13 17:30:22,202 no runner: 330 is still in new state, adding to the jobs queue Traceback (most recent call last): File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/web/buildapp.py", line 56, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/app.py", line 66, in __init__ self.job_manager = jobs.JobManager( self ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 33, in __init__ self.job_queue = JobQueue( app, self.dispatcher ) File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 110, in __init__ self.__check_jobs_at_startup() File "/media/Applications/work/spring2009/galaxy/galaxy3/lib/galaxy/jobs/__init__.py", line 127, in __check_jobs_at_startup job_wrapper = JobWrapper( job, self.app.toolbox.tools_by_id[ job.tool_id ], self ) KeyError: u'xslt_converter_16'

1 0

RBAC and libraries
by Ido M. Tamir 14 Jul '09

14 Jul '09

Hi, a) I have problems with your conception of roles in your RBAC scheme. Normally in RBAC[1] a subject can execute a transaction if the currently active roles of the user allow this. You require that the user should have all of the roles that are currently associated with the library. This makes simple things difficult and complicated things impossible. My policy should be: each user can do everything with his own dataset. the group of a user can access (view and push it to history) the library. a group of bioinformaticians can do everything with allmost all libraries. How would you implement this in your sheme which requires "all of the roles"? Would I have to create a new role for each combination of user/group/bioinformaticians? If you would require just "one of the roles", you would simply add 1) the user as a role to the lib (for everything) 2) the group the user is in as a role to the lib (for pushing to history + view if this is implemented) 3) the bioinformatician group as a role to the lib (for everything) b) An important permission would be to be able to view the item. I played around (maybe not enough) but could not find a way to really hide an item (library, folder, dataset) from the view. c) The little arrow pointing down to bring up the context-menu is very small. It would be helpful if you could maybe make it bigger or turn it into a button. thank you very much, Ido [1] http://csrc.nist.gov/groups/SNS/rbac/documents/Role_Based_Access_Control-19…

2 2

August 2009 GMOD Meeting
by Dave Clements, GMOD Help Desk 14 Jul '09

14 Jul '09

Hello all, The next GMOD meeting will be held 6-7 August, at the University of Oxford, in Oxford, United Kingdom. Registration is now open. Space is available on a first come, first served basis and there is room for 55 attendees. The meeting cost is £50. See http://gmod.org/wiki/August_2009_GMOD_Meeting to register As with previous GMOD meetings, this meeting will have a mixture of project, component, and user talks. The agenda is driven by attendee suggestions, and you are encouraged to add your suggestions now (see http://gmod.org/wiki/August_2009_GMOD_Meeting#Agenda_Suggestions) For examples of what happens at a GMOD meeting, see the writeups of the January 2009, July 2008, or any other previous meeting (see http://gmod.org/wiki/Meetings) GMOD meetings are an excellent way to meet other GMOD developers and users and to learn (and affect) what's coming in the project. Please join us in Oxford this August, Dave Clements GMOD Help Desk Note: Unless you have applied to and been admitted to the Summer School, don't you dare register for it. The registration web site will let you do this, but bureaucratic hellishness will ensue. -- * Learn more about GMOD at: ISMB/ECCB: http://www.iscb.org/ismbeccb2009/ (BioMart, Chado, Galaxy, InterMine) * Please keep responses on the list! * Was this helpful? Let us know at http://gmod.org/wiki/Help_Desk_Feedback

1 1

Handling large files in Galaxy
by Brad Chapman 13 Jul '09

13 Jul '09

Hi all; I've recently gotten a local Galaxy install up and running for our group. We do a lot of short read sequencing analysis and are looking at Galaxy as a framework to present the data and custom analyses associated with it. One of our main interests is scaling the presentation to large fastq and alignment files. Specifically, we have a case where we'd like to make a large ~300Gb alignment file available to users to query and retrieve sections of alignments corresponding to genomic coordinates. We have a custom C++ program that does this, and would like to plug it in through the tools interface. We'd ideally like to use the Library permissions interface to make this available to certain users. Would anyone be able to offer some advice about the best way to handle this? The standard upload, history, analyze would not be ideal since this large file would be copied around. We've brainstormed 3 different ways to approach this: - Have "special" uploaded files which are actually symlinks to the original file and do not get copied. This looks relatively difficult on my initial assessment. - Pass the logged in user to the C++ program and embed the logic of finding the right file within the external tool. Here we would need some advice about if it were possible to pass the current user through the tools interface. - The hack solution: upload a file that is actually just a link reference to the desired file, and this file gets passed to the external tool. The tool then can read the tiny file, know what large file to access, and proceed from there. This would involve some new datatype integration to handle the hack. I am still relatively uninitiated in the Galaxy way, so could use some advice on if any of these solutions are more likely to work smoothly then others. Generally, what sort of approach is Galaxy taking towards increasingly massive files? Is anyone else doing something similar? Thanks for any thoughts, Brad

2 1