Resolved toolshed mercurial authentication issue, public name change issue remains
Hi Greg, I finally got to the bottom of the reasons for my abort: authorization failed message when trying to push tool updates into my local toolshed using hg command line (rather than web interface), having given up for a while, and just come back to it. Since my authentication agent (apache LDAP module using corporate Active Directory) was providing bare usernames to the Toolshed, I had been relying on the remote_user_maildomain feature to turn this into an email address. However, it seems this function is not triggered when using hg command line access, so I had to change my rewrite rule which sets up the apache REMOTE_USER in my apache conf file for the toolshed, from this: RequestHeader set REMOTE_USER %{RU}e to this: RequestHeader set REMOTE_USER %{RU}e@agresearch.co.nz Now I can push updates directly from hg, and it's much more convenient. I didn't have to change anything in that toolshed middleware (although some temporary log messages I added there were the key for me solving the problem). However, that bug remains where I can continue to change my public name in the web user interface, even after I have created repos. I know not to do this. If there's more information you want from me to try to track down what is going wrong here, just say what you need, and I will endeavour to provide it. But for me, this is now a minor issue, so don't do anything just on my account. Thanks for your help. cheers, Simon
-----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, 6 September 2013 1:00 p.m. To: Guest, Simon Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hello Simon,
The tool shed middleware basically consists of 2 files in ~/lib/galaxy/webapps/tool_shed/framework/middleware: hg.py which handles authentication when pushing from the command line. The remoteuser.py file handles remote user authentication, but I cannot guarantee it properly handles hg push from the command line since I don't have an environment set up that way. That would be the place to take a look though, and if you discover problems let me know and we'll apply a patch.
Thanks very much,
Greg Von Kuster
On Sep 5, 2013, at 4:53 PM, "Guest, Simon" <Simon.Guest@agresearch.co.nz> wrote:
Hi Greg,
I just changed the public name on the web interface, where you click on the User dropdown, and choose Public Name. Something about my environment let it happen, when it apparently should have not been possible.
I suspect the reason lies in some mismatch between my apache user authentication and toolshed userids. See my previous email for the details of what I've been doing with that. I don't understand the connection between userids (email addresses) used for authentication, and public names, which appear to be used by the mercurial server inside the toolshed (by inference from the public name appearing in the allow_push in the toolshed repo). Is there a layer of toolshed middleware that maps between the two?
I am going to investigate other ways of doing the authentication, as I don't think passing in REMOTE_USER from apache is working (for me). I would like to know how others are doing that.
cheers, Simon
-----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, 6 September 2013 1:17 a.m. To: Guest, Simon Cc: Dave Bouvier; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hello Simon,
On Sep 4, 2013, at 5:13 PM, "Guest, Simon" <Simon.Guest@agresearch.co.nz> wrote:
Hi Dave, Greg,
Thanks for your reply.
I'm running a recently checked out stable branch. hg log shows this tip: changeset: 10473:c42567f43aa7 tag: tip user: greg date: Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy.
I wonder if this problem is related to my more fundamental problem of being unable to get mercurial push authentication working properly. I care much more about that, as that's blocking me (whereas the problem I reported has an easy work-around).
Even though you've found a work-around to the problem that results when you change your public user name in the tool shed after you have created a repository that uses it, it is extremely important that we get a fix ffor the issue committed to the tool shed code. Can you let us know how you went about changing your public user name? The only way we've been able to do it is by manually updaing the database table record using a sql statement. If you have discovered another way to do it, please let us know the steps we can take to reproduce it so that we can get a fix as soon as possible. If this is, in fact, a bug, then it is a very critical one.
Thanks,
Greg Von Kuster
Here is more information about my environment.
I'm using apache for digest authentication, with the htdigest file containing email addresses for usernames (so it matches what the toolshed wants). It passes it through to the toolshed using REMOTE_USER. (We have something like this working well for our main Galaxy instance.) I can access the toolshed just fine via the web interface, and I can do e.g. hg outgoing on the command line against a toolshed repo to see what needs pushing. But when I try hg push, I get abort: authorization failed.
Looking inside the hgrc file inside my toolshed repo, I see it has allow_push = simon-guest (i.e. my public name, not my email address). This isn't going to match the REMOTE_USER passed in from apache, which is an email address. So I think to get this working with my authentication framework, public name has to match the email address. (I tried hacking the allow_push in hgrc to be the email address, but that didn't work.)
Is anyone else using a toolshed behind apache with authentication handled by apache, like this? Should I expect hg push to work in this configuration, or am I outside the bounds of what has been tested so far?
Thanks for your help, much appreciated.
cheers, Simon
-----Original Message----- From: Dave Bouvier [mailto:dave@bx.psu.edu] Sent: Thursday, 5 September 2013 1:07 a.m. To: Greg Von Kuster Cc: Guest, Simon; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Simon,
I am unable to reproduce this behavior on my local tool shed installation, which revision are you running?
--Dave B.
Hello Simin,
Bjoern is correct - if this is possible (which it looks like it is from your traceback), then a bug has been introduced somewhat recently. We'll take a look at this today and get a fix committed. Thanks for reporting this.
Greg Von Kuster
On Sep 4, 2013, at 3:17 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
> Hi Simon, > > you are right. Public usernames, or usernames associated with a > repository can't be changed. If you can that is a bug, please > fill a trello card with it. The reason is to guarantee reproducibility. > If anyone has installed your repository the triple > (username,revision,toolshed) is saved to reinstall that > repository at any time, if needed. > > Cheers, > Bjoern > >>> It looks like it's trying to use the public name in the repo >>> path, which isn't going to work very well if public names can >>> be changed. >>> >>> Changing my public name back to what it was before made it >>> start working again. >> >> To reply to my own post, I see now that when you create a user, >> a comment on that page says: "You cannot change your public name after you have created a repository in this tool shed." >> >> Does this mean: "If you change your public name after you have >> created a repository in this tool shed, those repositories won't >> work any more" or some such ;-). Or is there actually some >> code to stop the user changing their public name in this case, >> which would imply my toolshed is somehow broken in a way unique to me? >> (I'm currently struggling to push tools into my toolshed using >> mercurial command line, through my apache front end, with the >> various layers of authentication, without it saying abort: >> authorization failed.) >> >> cheers, >> Simon >> >> >> ================================================================ >> == >> == >> === >> Attention: The information contained in this message and/or >> attachments from AgResearch Limited is intended only for the >> persons or entities to which it is addressed and may contain >> confidential and/or privileged material. Any review, >> retransmission, dissemination or other use of, or taking of any >> action in reliance upon, this information by persons or entities >> other than the intended recipients is prohibited by AgResearch >> Limited. If you have received this message in error, please >> notify
On 09/04/2013 06:57 AM, Greg Von Kuster wrote: the sender immediately.
>> ================================================================ >> == >> == >> === >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this and >> other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and > other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
==================================================================== == = Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ==================================================================== == =
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
====================================================================== = Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ====================================================================== =
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================
Am Sonntag, den 15.09.2013, 22:01 +0000 schrieb Guest, Simon:
Hi Greg,
I finally got to the bottom of the reasons for my abort: authorization failed message when trying to push tool updates into my local toolshed using hg command line (rather than web interface), having given up for a while, and just come back to it.
Since my authentication agent (apache LDAP module using corporate Active Directory) was providing bare usernames to the Toolshed, I had been relying on the remote_user_maildomain feature to turn this into an email address. However, it seems this function is not triggered when using hg command line access, so I had to change my rewrite rule which sets up the apache REMOTE_USER in my apache conf file for the toolshed, from this:
RequestHeader set REMOTE_USER %{RU}e
to this:
RequestHeader set REMOTE_USER %{RU}e@agresearch.co.nz
Hi Simon, thanks Simon. Bjoern
Now I can push updates directly from hg, and it's much more convenient. I didn't have to change anything in that toolshed middleware (although some temporary log messages I added there were the key for me solving the problem).
However, that bug remains where I can continue to change my public name in the web user interface, even after I have created repos. I know not to do this. If there's more information you want from me to try to track down what is going wrong here, just say what you need, and I will endeavour to provide it. But for me, this is now a minor issue, so don't do anything just on my account.
Thanks for your help.
cheers, Simon
-----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, 6 September 2013 1:00 p.m. To: Guest, Simon Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hello Simon,
The tool shed middleware basically consists of 2 files in ~/lib/galaxy/webapps/tool_shed/framework/middleware: hg.py which handles authentication when pushing from the command line. The remoteuser.py file handles remote user authentication, but I cannot guarantee it properly handles hg push from the command line since I don't have an environment set up that way. That would be the place to take a look though, and if you discover problems let me know and we'll apply a patch.
Thanks very much,
Greg Von Kuster
On Sep 5, 2013, at 4:53 PM, "Guest, Simon" <Simon.Guest@agresearch.co.nz> wrote:
Hi Greg,
I just changed the public name on the web interface, where you click on the User dropdown, and choose Public Name. Something about my environment let it happen, when it apparently should have not been possible.
I suspect the reason lies in some mismatch between my apache user authentication and toolshed userids. See my previous email for the details of what I've been doing with that. I don't understand the connection between userids (email addresses) used for authentication, and public names, which appear to be used by the mercurial server inside the toolshed (by inference from the public name appearing in the allow_push in the toolshed repo). Is there a layer of toolshed middleware that maps between the two?
I am going to investigate other ways of doing the authentication, as I don't think passing in REMOTE_USER from apache is working (for me). I would like to know how others are doing that.
cheers, Simon
-----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, 6 September 2013 1:17 a.m. To: Guest, Simon Cc: Dave Bouvier; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hello Simon,
On Sep 4, 2013, at 5:13 PM, "Guest, Simon" <Simon.Guest@agresearch.co.nz> wrote:
Hi Dave, Greg,
Thanks for your reply.
I'm running a recently checked out stable branch. hg log shows this tip: changeset: 10473:c42567f43aa7 tag: tip user: greg date: Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy.
I wonder if this problem is related to my more fundamental problem of being unable to get mercurial push authentication working properly. I care much more about that, as that's blocking me (whereas the problem I reported has an easy work-around).
Even though you've found a work-around to the problem that results when you change your public user name in the tool shed after you have created a repository that uses it, it is extremely important that we get a fix ffor the issue committed to the tool shed code. Can you let us know how you went about changing your public user name? The only way we've been able to do it is by manually updaing the database table record using a sql statement. If you have discovered another way to do it, please let us know the steps we can take to reproduce it so that we can get a fix as soon as possible. If this is, in fact, a bug, then it is a very critical one.
Thanks,
Greg Von Kuster
Here is more information about my environment.
I'm using apache for digest authentication, with the htdigest file containing email addresses for usernames (so it matches what the toolshed wants). It passes it through to the toolshed using REMOTE_USER. (We have something like this working well for our main Galaxy instance.) I can access the toolshed just fine via the web interface, and I can do e.g. hg outgoing on the command line against a toolshed repo to see what needs pushing. But when I try hg push, I get abort: authorization failed.
Looking inside the hgrc file inside my toolshed repo, I see it has allow_push = simon-guest (i.e. my public name, not my email address). This isn't going to match the REMOTE_USER passed in from apache, which is an email address. So I think to get this working with my authentication framework, public name has to match the email address. (I tried hacking the allow_push in hgrc to be the email address, but that didn't work.)
Is anyone else using a toolshed behind apache with authentication handled by apache, like this? Should I expect hg push to work in this configuration, or am I outside the bounds of what has been tested so far?
Thanks for your help, much appreciated.
cheers, Simon
-----Original Message----- From: Dave Bouvier [mailto:dave@bx.psu.edu] Sent: Thursday, 5 September 2013 1:07 a.m. To: Greg Von Kuster Cc: Guest, Simon; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Simon,
I am unable to reproduce this behavior on my local tool shed installation, which revision are you running?
--Dave B.
On 09/04/2013 06:57 AM, Greg Von Kuster wrote: > Hello Simin, > > Bjoern is correct - if this is possible (which it looks like it is > from your traceback), then a bug has been introduced somewhat recently. We'll take a look at this today and get a fix committed. Thanks for reporting this. > > Greg Von Kuster > > On Sep 4, 2013, at 3:17 AM, Bjoern Gruening > <bjoern.gruening@gmail.com> wrote: > >> Hi Simon, >> >> you are right. Public usernames, or usernames associated with a >> repository can't be changed. If you can that is a bug, please >> fill a trello card with it. The reason is to guarantee reproducibility. >> If anyone has installed your repository the triple >> (username,revision,toolshed) is saved to reinstall that >> repository at any time, if needed. >> >> Cheers, >> Bjoern >> >>>> It looks like it's trying to use the public name in the repo >>>> path, which isn't going to work very well if public names can >>>> be changed. >>>> >>>> Changing my public name back to what it was before made it >>>> start working again. >>> >>> To reply to my own post, I see now that when you create a user, >>> a comment on that page says: "You cannot change your public name after you have created a repository in this tool shed." >>> >>> Does this mean: "If you change your public name after you have >>> created a repository in this tool shed, those repositories won't >>> work any more" or some such ;-). Or is there actually some >>> code to stop the user changing their public name in this case, >>> which would imply my toolshed is somehow broken in a way unique to me? >>> (I'm currently struggling to push tools into my toolshed using >>> mercurial command line, through my apache front end, with the >>> various layers of authentication, without it saying abort: >>> authorization failed.) >>> >>> cheers, >>> Simon >>> >>> >>> ================================================================ >>> == >>> == >>> === >>> Attention: The information contained in this message and/or >>> attachments from AgResearch Limited is intended only for the >>> persons or entities to which it is addressed and may contain >>> confidential and/or privileged material. Any review, >>> retransmission, dissemination or other use of, or taking of any >>> action in reliance upon, this information by persons or entities >>> other than the intended recipients is prohibited by AgResearch >>> Limited. If you have received this message in error, please >>> notify the sender immediately. >>> ================================================================ >>> == >>> == >>> === >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this and >>> other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this and >> other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and > other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >
==================================================================== == = Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ==================================================================== == =
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
====================================================================== = Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ====================================================================== =
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Björn Grüning -
Guest, Simon