Workflows with multiple editors
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm working with a number of people in my department to develop a single workflow that will be used for a course we teach. So far I've found that "sharing" a workflow with individual users/roles isn't very optimal. They email me with changes they'd like to happen, or they create their own copies which immediately branch off and don't help everyone else who might need them. As such, I'd like to propose changes and would like community feedback on my proposed changes so when I create the trello card/maybe try my hand at implementing them, they aren't completely specific to my use case. In my mind, the 'optimal' version of managing workflows would be something along the following lines: Top priorities: - - workflows can be shared as "write" or "read only" with multiple users/roles. - This would allow there to be a single copy of a workflow in cases where multiple people might make changes to a single workflow - For "read only", this allows the previous/traditional model of sharing a workflow and the receiver being able to run it as is, or to copy and modify it. Note: For my case at least, realtime multiple person editing is not needed. If a banner shows up which reads "someone started editing before you, either ask them to save + leave or save as a copy" would work in my case. Wishlist features: - - workflows are treated as VCS repositories - Note that full merge/branching probably not really necessary for this case (that would be really nice, but probably too much of a nightmare to code...) - - You can see previous iterations of workflows. - - Workflows can be "owned" by either a user or a role - users could create a workflow and transfer ownership to a role - role owned workflows are delete-able (or maybe require some X% of users in that role to confirmation deletion?) - This would sort of work like a group owned VCS repository, where multiple users can create workflows of interest to a group of people (role) and have them, by default, be available. Anyone have input on this proposal? Cheers, Eric - -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr@tamu.edu rasche.eric@yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJScWXTAAoJEMqDXdrsMcpVFH4QAMO0V4tSJGzM3PGGIwdonY0v nyuHTCJJ5qliWuapUJUK9E0UQBblnaJYooLjh2l1vgo95JaXdHEAC9NVecQLH8J2 ey5cu1xODf6JmBGlicRGxQWlLZmo9U3R4T/OhHco40I239aTpuSDPcwcBJUm1cAE czz+/2pgdCpkJqYQJ9EE956TAYdyPBYFvSrIO7ebWkSvC5He40qca0x06tgq14uR Dta+8QH8brVYX1G+TAgdh3hGlLUwIlPMiZsWOEMcwSUtyXP3I7OKx/XIOy3ZYVTu Py/XlvNlZ87yiaiwDSxjxR9qaDeGQ3J8CRCG9mqWu8O5Y9QKkyrC410I+hZH5lE+ my28z7l5rK0i05pR6o8euASJ7D1eFHEDYmBmN6j3qRcOgzzdup2oQpm6TA7ue1sd NAS4rnW7QGmfNj+9xNSQhDWSj7CeIUeHXbDe55suBX3JofCNg8Qq6uYzPd7KGj7n jvWp8tGnpwbgTmvDPlR03gJiXMZld+gGwKrlCyn1kdBkfI1r5l0UfoaIUE+8HDm3 wXviQ8GU7seYU5bJFK798owVNawjY1H3JenzIYt1XvF1z53Uc39x58Q9lxXbVfSe KgiqCfF0YuvVtOczGTAQ57qCFwlpKg2ra+X+HtAZyMDNMsb3ANgfogTqsB+7HYvF czq2QJHq9mBC5N9JLhRo =conD -----END PGP SIGNATURE-----
Hi Eric, I've being pondering this too - it can easily get very complicated with multiple copies of a workflow with minor variations. I agree that "read only" sharing makes sense as a short term goal, but I see two variants being useful here - an read only snapshot copy from the instant it was shared (where the shared version is locked forever but may of course optionally be copied by the receiver, and the copy edited), and as an always up to date live pointer (in case the original owner revises it - then all the users get the changes immediately). Some of your wish list ideas also sound very good, but may not be practical - not everyone will understand a VCS approach to managing workflows :( Regards, Peter On Wed, Oct 30, 2013 at 8:02 PM, Eric Rasche <rasche.eric@yandex.ru> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm working with a number of people in my department to develop a single workflow that will be used for a course we teach. So far I've found that "sharing" a workflow with individual users/roles isn't very optimal. They email me with changes they'd like to happen, or they create their own copies which immediately branch off and don't help everyone else who might need them.
As such, I'd like to propose changes and would like community feedback on my proposed changes so when I create the trello card/maybe try my hand at implementing them, they aren't completely specific to my use case.
In my mind, the 'optimal' version of managing workflows would be something along the following lines:
Top priorities:
- - workflows can be shared as "write" or "read only" with multiple users/roles. - This would allow there to be a single copy of a workflow in cases where multiple people might make changes to a single workflow - For "read only", this allows the previous/traditional model of sharing a workflow and the receiver being able to run it as is, or to copy and modify it.
Note:
For my case at least, realtime multiple person editing is not needed. If a banner shows up which reads "someone started editing before you, either ask them to save + leave or save as a copy" would work in my case.
Wishlist features:
- - workflows are treated as VCS repositories - Note that full merge/branching probably not really necessary for this case (that would be really nice, but probably too much of a nightmare to code...) - - You can see previous iterations of workflows. - - Workflows can be "owned" by either a user or a role - users could create a workflow and transfer ownership to a role - role owned workflows are delete-able (or maybe require some X% of users in that role to confirmation deletion?) - This would sort of work like a group owned VCS repository, where multiple users can create workflows of interest to a group of people (role) and have them, by default, be available.
Anyone have input on this proposal?
Cheers, Eric
- -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr@tamu.edu rasche.eric@yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBAgAGBQJScWXTAAoJEMqDXdrsMcpVFH4QAMO0V4tSJGzM3PGGIwdonY0v nyuHTCJJ5qliWuapUJUK9E0UQBblnaJYooLjh2l1vgo95JaXdHEAC9NVecQLH8J2 ey5cu1xODf6JmBGlicRGxQWlLZmo9U3R4T/OhHco40I239aTpuSDPcwcBJUm1cAE czz+/2pgdCpkJqYQJ9EE956TAYdyPBYFvSrIO7ebWkSvC5He40qca0x06tgq14uR Dta+8QH8brVYX1G+TAgdh3hGlLUwIlPMiZsWOEMcwSUtyXP3I7OKx/XIOy3ZYVTu Py/XlvNlZ87yiaiwDSxjxR9qaDeGQ3J8CRCG9mqWu8O5Y9QKkyrC410I+hZH5lE+ my28z7l5rK0i05pR6o8euASJ7D1eFHEDYmBmN6j3qRcOgzzdup2oQpm6TA7ue1sd NAS4rnW7QGmfNj+9xNSQhDWSj7CeIUeHXbDe55suBX3JofCNg8Qq6uYzPd7KGj7n jvWp8tGnpwbgTmvDPlR03gJiXMZld+gGwKrlCyn1kdBkfI1r5l0UfoaIUE+8HDm3 wXviQ8GU7seYU5bJFK798owVNawjY1H3JenzIYt1XvF1z53Uc39x58Q9lxXbVfSe KgiqCfF0YuvVtOczGTAQ57qCFwlpKg2ra+X+HtAZyMDNMsb3ANgfogTqsB+7HYvF czq2QJHq9mBC5N9JLhRo =conD -----END PGP SIGNATURE----- ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Peter, Please bear with me, I'm mostly tossing these ideas out for my benefit/to spawn discussion on how one might implement this. This was originally a short email before I figured out how much work would be involved... Upon cursory inspection I see that workflows are stored as lines in the database. Were it to be easily stored in a VCS, all database workflow steps would probably have to be migrated to disk. In the toolsheds workflows are stored as JSON, so it'd just be a matter of dumping existing internal workflows to disk. On disk it would be stored in a "workflow" folder, with a set of numbered sub-folders. Each subfolder would store a VCS repo. These numbered sub-folders would be referenced in the database. Additionally, there would be a "cache" folder in the workflow folder, to store the current/active version of the workflow, per workflow, per user. Continuing just to toss ideas out, I could see implementing this in a VCS while hiding as much of that as possible away from the user. How about this: - - Sharing creates a tag in the vcs (your permanent, immutable version) - - If it was shared to them with edit capabilities, a branch is created to allow them to make their changes and eventually merge back. - - Upon save, we'd ask them if they'd like to add a short message describing their changes (by default it'd read "{UserRealName} made modifications" or something like that so they could just click through if they didn't want to write a message) - - In the edit view, there'd be an additional button which reads "Merge these changes back into {Sharer'sRealName}'s copy of this workflow" which would execute a merge (I guess it could create a merge request but that seems like a bit much) - - In the workflow list we would now have an area to select previous revisions of workflows and open those up in the edit window. - - selecting a previous version of the workflow would set the user's active workflow (somewhere in DB) to that revision. Keeping track of which workflow was "active" might be a bit difficult... I would imagine it would work like so: - - workflows are separated from users. - - the DB table of "stored_workflow" would be modified to store which repository (in the workflow/NNN/ folder), as well as the current head - - a table would be created, mapping users to specific revisions of a workflow. (workflow_user_revision) [user_id,workflow_id,revision] - maybe revision would be either revision hash or "HEAD" to say they always want the latest version? - - sharing a workflow (read/write) would add a new row in the workflow_user_revision, with an updated user_id, and a copy of the that user's current revision hash (because that user would be sharing that specific version). - - sharing a workflow (read only) would do pretty much the same. This made sense in my mind at least...does this seem like a reasonable way to implement this? Do you (or anyone else) have comments to make on this proposal? (Mind you, this is just me thinking out loud for the time being...I have a lot of other galaxy modifications to get through first for my organisation (native LDAP integration, script ACLs)) On 10/30/2013 05:11 PM, Peter Cock wrote:
Hi Eric,
I've being pondering this too - it can easily get very complicated with multiple copies of a workflow with minor variations.
I agree that "read only" sharing makes sense as a short term goal, but I see two variants being useful here - an read only snapshot copy from the instant it was shared (where the shared version is locked forever but may of course optionally be copied by the receiver, and the copy edited), and as an always up to date live pointer (in case the original owner revises it - then all the users get the changes immediately).
Some of your wish list ideas also sound very good, but may not be practical - not everyone will understand a VCS approach to managing workflows :(
Regards,
Peter
On Wed, Oct 30, 2013 at 8:02 PM, Eric Rasche <rasche.eric@yandex.ru> wrote: I'm working with a number of people in my department to develop a single workflow that will be used for a course we teach. So far I've found that "sharing" a workflow with individual users/roles isn't very optimal. They email me with changes they'd like to happen, or they create their own copies which immediately branch off and don't help everyone else who might need them.
As such, I'd like to propose changes and would like community feedback on my proposed changes so when I create the trello card/maybe try my hand at implementing them, they aren't completely specific to my use case.
In my mind, the 'optimal' version of managing workflows would be something along the following lines:
Top priorities:
- workflows can be shared as "write" or "read only" with multiple users/roles. - This would allow there to be a single copy of a workflow in cases where multiple people might make changes to a single workflow - For "read only", this allows the previous/traditional model of sharing a workflow and the receiver being able to run it as is, or to copy and modify it.
Note:
For my case at least, realtime multiple person editing is not needed. If a banner shows up which reads "someone started editing before you, either ask them to save + leave or save as a copy" would work in my case.
Wishlist features:
- workflows are treated as VCS repositories - Note that full merge/branching probably not really necessary for this case (that would be really nice, but probably too much of a nightmare to code...) - You can see previous iterations of workflows. - Workflows can be "owned" by either a user or a role - users could create a workflow and transfer ownership to a role - role owned workflows are delete-able (or maybe require some X% of users in that role to confirmation deletion?) - This would sort of work like a group owned VCS repository, where multiple users can create workflows of interest to a group of people (role) and have them, by default, be available.
Anyone have input on this proposal?
Cheers, Eric
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Cheers, Eric -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iQIcBAEBAgAGBQJScpCLAAoJEMqDXdrsMcpV8usQAKxwj7m6zpsiI5Yj54Yzr5dU LsPKUriNcvzCxNWXvjzvXjywzulNQtWCBIpiecHhK8kuXkc+QlLkJ/bHNvaCyS7U 99haARombM2+3vQ4UF3kn7q4OgoFOe7OM+N6g9cxvsyrf+jC2m7k/HD5rfN+WPtG xIMwGfROg6eSWrB3jJJtFEprRDntTT9H94RYEi0XfRtsv4KEAMsQv4gXtk7AdjDg 4HfdLZPkQ7GV/SxLopwj/halMIiKk7E3Kb762UI3Rf9IFPBUEftXKDbfIzrriBb/ k/Ya3VhQEK9GRwnzcZbVY7Z/yTHFq8FvaBkFOniY+bnHPqcoPknFKWkvnTGry3B5 4y18KvTnLYc/2EEk4MeHq9ErQr7XMH0YAc027vKrEFBIzyhs6xwLyY4Ud2skrw0J H7bKY2RpXICkumMmeVEn6n2cOJYT1rexHlRld/OJYEc2hHfSNEi3t9lZ8NK+Ljhx U4eAt/uMVl437gWRV/0NJlKdT/+lNLB4plvmswXdSvG2yQsaFbfCFYbbwcaKFUMA Hd9j9lJwVj076j0UzM3NITSgoxRB2oOIoAVccGjRTEShBvNCL9KuggoFosWQf28i CTOYB8YNw8ud9rBt9HrTVDWmqowC+azEYBgavyY6ZoMpF1IvFQnA8Lqx0PHFFRdd Zq/KWDhf37aGw5pF/QoH =OAgc -----END PGP SIGNATURE-----
Hey Eric, Thanks for thinking through some of those details. I don't know if this is an extension of that or an alternative to that - but the Galaxy tool shed has some support for workflows and stored these in a VCS repository (mercurial). Augmenting that to support a sort of one-click (or couple click) export from Galaxy to the tool shed was something on my radar, but going further and providing tracking, branching, etc... over time of the workflow as you have outlined here could make a lot of sense. Echoing his comments here, I know Peter is not a fan of tool shed repositories being implemented as mercurial repositories. Continuing to make progress on hiding this fact from users while allowing leveraging of more advanced features as you described here should help that decision bear more fruit going forward. The other feature requests related to allowing multiple editors - I think this is a great idea. I think histories, pages, and workflows all need to support more sophisticated permissions. I have created Trello cards related to these requests here, feel free to comment on these to flush out the details: https://trello.com/c/3blHqhxb https://trello.com/c/YvKxFOdd Thanks again for the suggestions, -John On Thu, Oct 31, 2013 at 12:16 PM, Eric Rasche <rasche.eric@yandex.ru> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Peter,
Please bear with me, I'm mostly tossing these ideas out for my benefit/to spawn discussion on how one might implement this. This was originally a short email before I figured out how much work would be involved...
Upon cursory inspection I see that workflows are stored as lines in the database. Were it to be easily stored in a VCS, all database workflow steps would probably have to be migrated to disk. In the toolsheds workflows are stored as JSON, so it'd just be a matter of dumping existing internal workflows to disk.
On disk it would be stored in a "workflow" folder, with a set of numbered sub-folders. Each subfolder would store a VCS repo. These numbered sub-folders would be referenced in the database. Additionally, there would be a "cache" folder in the workflow folder, to store the current/active version of the workflow, per workflow, per user.
Continuing just to toss ideas out, I could see implementing this in a VCS while hiding as much of that as possible away from the user. How about this:
- - Sharing creates a tag in the vcs (your permanent, immutable version) - - If it was shared to them with edit capabilities, a branch is created to allow them to make their changes and eventually merge back. - - Upon save, we'd ask them if they'd like to add a short message describing their changes (by default it'd read "{UserRealName} made modifications" or something like that so they could just click through if they didn't want to write a message) - - In the edit view, there'd be an additional button which reads "Merge these changes back into {Sharer'sRealName}'s copy of this workflow" which would execute a merge (I guess it could create a merge request but that seems like a bit much) - - In the workflow list we would now have an area to select previous revisions of workflows and open those up in the edit window. - - selecting a previous version of the workflow would set the user's active workflow (somewhere in DB) to that revision.
Keeping track of which workflow was "active" might be a bit difficult... I would imagine it would work like so:
- - workflows are separated from users. - - the DB table of "stored_workflow" would be modified to store which repository (in the workflow/NNN/ folder), as well as the current head - - a table would be created, mapping users to specific revisions of a workflow. (workflow_user_revision) [user_id,workflow_id,revision] - maybe revision would be either revision hash or "HEAD" to say they always want the latest version? - - sharing a workflow (read/write) would add a new row in the workflow_user_revision, with an updated user_id, and a copy of the that user's current revision hash (because that user would be sharing that specific version). - - sharing a workflow (read only) would do pretty much the same.
This made sense in my mind at least...does this seem like a reasonable way to implement this? Do you (or anyone else) have comments to make on this proposal?
(Mind you, this is just me thinking out loud for the time being...I have a lot of other galaxy modifications to get through first for my organisation (native LDAP integration, script ACLs))
On 10/30/2013 05:11 PM, Peter Cock wrote:
Hi Eric,
I've being pondering this too - it can easily get very complicated with multiple copies of a workflow with minor variations.
I agree that "read only" sharing makes sense as a short term goal, but I see two variants being useful here - an read only snapshot copy from the instant it was shared (where the shared version is locked forever but may of course optionally be copied by the receiver, and the copy edited), and as an always up to date live pointer (in case the original owner revises it - then all the users get the changes immediately).
Some of your wish list ideas also sound very good, but may not be practical - not everyone will understand a VCS approach to managing workflows :(
Regards,
Peter
On Wed, Oct 30, 2013 at 8:02 PM, Eric Rasche <rasche.eric@yandex.ru> wrote: I'm working with a number of people in my department to develop a single workflow that will be used for a course we teach. So far I've found that "sharing" a workflow with individual users/roles isn't very optimal. They email me with changes they'd like to happen, or they create their own copies which immediately branch off and don't help everyone else who might need them.
As such, I'd like to propose changes and would like community feedback on my proposed changes so when I create the trello card/maybe try my hand at implementing them, they aren't completely specific to my use case.
In my mind, the 'optimal' version of managing workflows would be something along the following lines:
Top priorities:
- workflows can be shared as "write" or "read only" with multiple users/roles. - This would allow there to be a single copy of a workflow in cases where multiple people might make changes to a single workflow - For "read only", this allows the previous/traditional model of sharing a workflow and the receiver being able to run it as is, or to copy and modify it.
Note:
For my case at least, realtime multiple person editing is not needed. If a banner shows up which reads "someone started editing before you, either ask them to save + leave or save as a copy" would work in my case.
Wishlist features:
- workflows are treated as VCS repositories - Note that full merge/branching probably not really necessary for this case (that would be really nice, but probably too much of a nightmare to code...) - You can see previous iterations of workflows. - Workflows can be "owned" by either a user or a role - users could create a workflow and transfer ownership to a role - role owned workflows are delete-able (or maybe require some X% of users in that role to confirmation deletion?) - This would sort of work like a group owned VCS repository, where multiple users can create workflows of interest to a group of people (role) and have them, by default, be available.
Anyone have input on this proposal?
Cheers, Eric
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Cheers, Eric -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux)
iQIcBAEBAgAGBQJScpCLAAoJEMqDXdrsMcpV8usQAKxwj7m6zpsiI5Yj54Yzr5dU LsPKUriNcvzCxNWXvjzvXjywzulNQtWCBIpiecHhK8kuXkc+QlLkJ/bHNvaCyS7U 99haARombM2+3vQ4UF3kn7q4OgoFOe7OM+N6g9cxvsyrf+jC2m7k/HD5rfN+WPtG xIMwGfROg6eSWrB3jJJtFEprRDntTT9H94RYEi0XfRtsv4KEAMsQv4gXtk7AdjDg 4HfdLZPkQ7GV/SxLopwj/halMIiKk7E3Kb762UI3Rf9IFPBUEftXKDbfIzrriBb/ k/Ya3VhQEK9GRwnzcZbVY7Z/yTHFq8FvaBkFOniY+bnHPqcoPknFKWkvnTGry3B5 4y18KvTnLYc/2EEk4MeHq9ErQr7XMH0YAc027vKrEFBIzyhs6xwLyY4Ud2skrw0J H7bKY2RpXICkumMmeVEn6n2cOJYT1rexHlRld/OJYEc2hHfSNEi3t9lZ8NK+Ljhx U4eAt/uMVl437gWRV/0NJlKdT/+lNLB4plvmswXdSvG2yQsaFbfCFYbbwcaKFUMA Hd9j9lJwVj076j0UzM3NITSgoxRB2oOIoAVccGjRTEShBvNCL9KuggoFosWQf28i CTOYB8YNw8ud9rBt9HrTVDWmqowC+azEYBgavyY6ZoMpF1IvFQnA8Lqx0PHFFRdd Zq/KWDhf37aGw5pF/QoH =OAgc -----END PGP SIGNATURE----- ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Eric Rasche -
John Chilton -
Peter Cock