Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock
For a simple patch like yours, just pasting and sending your diff to the list works well:
%hg diff ... For bigger patches/enhancements, a bitbucket fork + pull request is highly encouraged.
Best, J.
On Aug 23, 2012, at 4:47 AM, James wrote:
Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi All,
diff -r ba56d4746f7a tools/data_source/upload.py --- a/tools/data_source/upload.py Fri Aug 03 13:31:48 2012 -0400 +++ b/tools/data_source/upload.py Fri Aug 24 01:26:36 2012 +1200 @@ -357,6 +357,7 @@ else: sniff.convert_newlines( dp ) shutil.move( dp, os.path.join( files_path, name ) ) + os.chmod(files_path +"/"+name, "0644") # Move the dataset to its "real" path shutil.move( dataset.primary_file, output_path ) # Write the job info
Fixes permissions not been created for filenames correctly so an apache instance can access the extra files.
Cheers James. Research Assistant Biochemistry Department University of Otago New Zealand
Maybe I'm just doing it wrong… but I find it unusually difficult to put together a coherent pull request without doing a dedicated repo clone. maybe somebody who knows a good way to submit pull requests could populate this wiki page?
http://wiki.g2.bx.psu.edu/Develop/Mercurial
brad On Aug 23, 2012, at 8:55 AM, Jeremy Goecks jeremy.goecks@emory.edu wrote:
For a simple patch like yours, just pasting and sending your diff to the list works well:
%hg diff ...
For bigger patches/enhancements, a bitbucket fork + pull request is highly encouraged.
Best, J.
On Aug 23, 2012, at 4:47 AM, James wrote:
Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Brad Langhorst langhorst@neb.com 978-380-7564
Hi All my bad,
Error in email. the fix was else: sniff.convert_newlines( dp ) shutil.move( dp, os.path.join( files_path, name ) ) + # My fix jboocock # + os.chmod(files_path +"/" + name, 0644)
tab was off
Cheers James. On 8/24/2012 1:28 AM, James wrote:
Hi All,
diff -r ba56d4746f7a tools/data_source/upload.py --- a/tools/data_source/upload.py Fri Aug 03 13:31:48 2012 -0400 +++ b/tools/data_source/upload.py Fri Aug 24 01:26:36 2012 +1200 @@ -357,6 +357,7 @@ else: sniff.convert_newlines( dp ) shutil.move( dp, os.path.join( files_path, name ) )
os.chmod(files_path +"/"+name, "0644") # Move the dataset to its "real" path shutil.move( dataset.primary_file, output_path ) # Write the job infoFixes permissions not been created for filenames correctly so an apache instance can access the extra files.
Cheers James. Research Assistant Biochemistry Department University of Otago New Zealand
I don't know that difficult is the right word, I would call the process heavy handed and time consuming, especially relative to similar git workflows. I have heard the Galaxy developers say on multiple occasions the prefer pull requests coming using a dedicated clone though and this makes a lot of sense because branches stick around forever in mercurial. So I now use dedicated repositories for each of my pull requests.
I am still trying to optimize my workflow, but here is a high level overview of my current process. We have a local version of Galaxy (lets call it msi-galaxy), if we want to push changes back I generally create a dedicated clone of galaxy-central on bitbucket for this change (say galaxy-central-feature1) and then export the desired revision/revisions as a patch from our local branch and import it in my feature clone.
% cd msi-galaxy % hg export -r rev1 -r rev2 > /tmp/feature1.patch % cd .. % hg clone ssh://hg@bitbucket.org/jmchilton/galaxy-central-feature1 % cd galaxy-central-feature1 % hg import /tmp/feature1.patch % hg push
Create pull request.
Other tricks I have learned:
Use ssh keys for bitbucket interactions instead of https, it seems considerably faster.
You can add --no-commit to the import and then manually commit. This can let you map multiple changesets from your local branch to the feature clone (a poor man's rebasing I guess).
The longest part of the process is the original clone down of the feature clones. So I have just started reusing old clones for this.
- Rename galaxy-central-old-feature to galaxy-central-new-feature on bitbucket. - % cd galaxy-central-old-feature - % hg pull ssh://hg@bitbucket.org/galaxy/galaxy-central # catch up with galaxy-central - Make changes - % hg push ssh://hg@bitbucket.org/jmchilton/galaxy-central-new-feature - Create pull request.
The downside of this cheat is that previously accept pull requests now have source repository names associated with them in bitbucket, like this https://bitbucket.org/galaxy/galaxy-central/pull-request/44/allow-usage-of-d....
-John
------------------------------------------------ John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223
On Thu, Aug 23, 2012 at 8:29 AM, Langhorst, Brad Langhorst@neb.com wrote:
Maybe I'm just doing it wrong… but I find it unusually difficult to put together a coherent pull request without doing a dedicated repo clone. maybe somebody who knows a good way to submit pull requests could populate this wiki page?
http://wiki.g2.bx.psu.edu/Develop/Mercurial
brad On Aug 23, 2012, at 8:55 AM, Jeremy Goecks jeremy.goecks@emory.edu wrote:
For a simple patch like yours, just pasting and sending your diff to the list works well:
%hg diff ...
For bigger patches/enhancements, a bitbucket fork + pull request is highly encouraged.
Best, J.
On Aug 23, 2012, at 4:47 AM, James wrote:
Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Brad Langhorst langhorst@neb.com 978-380-7564
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi John,
Thank you, this is really useful!
I don't understand the purpose of renaming the repository to galaxy-central-new-feature - what does this achieve? I'm pretty new to mercurial and more familiar with git, so that could be why I'm confused.
Thanks, Clare
On 24 August 2012 00:07, John Chilton chilton@msi.umn.edu wrote:
I don't know that difficult is the right word, I would call the process heavy handed and time consuming, especially relative to similar git workflows. I have heard the Galaxy developers say on multiple occasions the prefer pull requests coming using a dedicated clone though and this makes a lot of sense because branches stick around forever in mercurial. So I now use dedicated repositories for each of my pull requests.
I am still trying to optimize my workflow, but here is a high level overview of my current process. We have a local version of Galaxy (lets call it msi-galaxy), if we want to push changes back I generally create a dedicated clone of galaxy-central on bitbucket for this change (say galaxy-central-feature1) and then export the desired revision/revisions as a patch from our local branch and import it in my feature clone.
% cd msi-galaxy % hg export -r rev1 -r rev2 > /tmp/feature1.patch % cd .. % hg clone ssh://hg@bitbucket.org/jmchilton/galaxy-central-feature1 % cd galaxy-central-feature1 % hg import /tmp/feature1.patch % hg push
Create pull request.
Other tricks I have learned:
Use ssh keys for bitbucket interactions instead of https, it seems considerably faster.
You can add --no-commit to the import and then manually commit. This can let you map multiple changesets from your local branch to the feature clone (a poor man's rebasing I guess).
The longest part of the process is the original clone down of the feature clones. So I have just started reusing old clones for this.
- Rename galaxy-central-old-feature to galaxy-central-new-feature on bitbucket.
- % cd galaxy-central-old-feature
- % hg pull ssh://hg@bitbucket.org/galaxy/galaxy-central # catch up
with galaxy-central
- Make changes
- % hg push ssh://hg@bitbucket.org/jmchilton/galaxy-central-new-feature
- Create pull request.
The downside of this cheat is that previously accept pull requests now have source repository names associated with them in bitbucket, like this https://bitbucket.org/galaxy/galaxy-central/pull-request/44/allow-usage-of-d....
-John
John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223
On Thu, Aug 23, 2012 at 8:29 AM, Langhorst, Brad Langhorst@neb.com wrote:
Maybe I'm just doing it wrong… but I find it unusually difficult to put together a coherent pull request without doing a dedicated repo clone. maybe somebody who knows a good way to submit pull requests could populate this wiki page?
http://wiki.g2.bx.psu.edu/Develop/Mercurial
brad On Aug 23, 2012, at 8:55 AM, Jeremy Goecks jeremy.goecks@emory.edu wrote:
For a simple patch like yours, just pasting and sending your diff to the list works well:
%hg diff ...
For bigger patches/enhancements, a bitbucket fork + pull request is highly encouraged.
Best, J.
On Aug 23, 2012, at 4:47 AM, James wrote:
Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Brad Langhorst langhorst@neb.com 978-380-7564
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Great, I am glad it was useful.
Renaming the repository isn't needed, especially if you just have one pull request open at a time. But there have been times I have had maybe 4 or 5 open pull requests at once, so then it helps to keep track of which repository maps to which feature or bug fix. If I logged into bitbucket and saw galaxy-central-1,galaxy-central-2, galaxy-central-3, that wouldn't be as informative.
-John
On Tue, Aug 28, 2012 at 8:11 PM, Clare Sloggett sloc@unimelb.edu.au wrote:
Hi John,
Thank you, this is really useful!
I don't understand the purpose of renaming the repository to galaxy-central-new-feature - what does this achieve? I'm pretty new to mercurial and more familiar with git, so that could be why I'm confused.
Thanks, Clare
On 24 August 2012 00:07, John Chilton chilton@msi.umn.edu wrote:
I don't know that difficult is the right word, I would call the process heavy handed and time consuming, especially relative to similar git workflows. I have heard the Galaxy developers say on multiple occasions the prefer pull requests coming using a dedicated clone though and this makes a lot of sense because branches stick around forever in mercurial. So I now use dedicated repositories for each of my pull requests.
I am still trying to optimize my workflow, but here is a high level overview of my current process. We have a local version of Galaxy (lets call it msi-galaxy), if we want to push changes back I generally create a dedicated clone of galaxy-central on bitbucket for this change (say galaxy-central-feature1) and then export the desired revision/revisions as a patch from our local branch and import it in my feature clone.
% cd msi-galaxy % hg export -r rev1 -r rev2 > /tmp/feature1.patch % cd .. % hg clone ssh://hg@bitbucket.org/jmchilton/galaxy-central-feature1 % cd galaxy-central-feature1 % hg import /tmp/feature1.patch % hg push
Create pull request.
Other tricks I have learned:
Use ssh keys for bitbucket interactions instead of https, it seems considerably faster.
You can add --no-commit to the import and then manually commit. This can let you map multiple changesets from your local branch to the feature clone (a poor man's rebasing I guess).
The longest part of the process is the original clone down of the feature clones. So I have just started reusing old clones for this.
- Rename galaxy-central-old-feature to galaxy-central-new-feature on bitbucket.
- % cd galaxy-central-old-feature
- % hg pull ssh://hg@bitbucket.org/galaxy/galaxy-central # catch up
with galaxy-central
- Make changes
- % hg push ssh://hg@bitbucket.org/jmchilton/galaxy-central-new-feature
- Create pull request.
The downside of this cheat is that previously accept pull requests now have source repository names associated with them in bitbucket, like this https://bitbucket.org/galaxy/galaxy-central/pull-request/44/allow-usage-of-d....
-John
John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223
On Thu, Aug 23, 2012 at 8:29 AM, Langhorst, Brad Langhorst@neb.com wrote:
Maybe I'm just doing it wrong… but I find it unusually difficult to put together a coherent pull request without doing a dedicated repo clone. maybe somebody who knows a good way to submit pull requests could populate this wiki page?
http://wiki.g2.bx.psu.edu/Develop/Mercurial
brad On Aug 23, 2012, at 8:55 AM, Jeremy Goecks jeremy.goecks@emory.edu wrote:
For a simple patch like yours, just pasting and sending your diff to the list works well:
%hg diff ...
For bigger patches/enhancements, a bitbucket fork + pull request is highly encouraged.
Best, J.
On Aug 23, 2012, at 4:47 AM, James wrote:
Hi All,
Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database.
Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.)
Cheers James Boocock ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Brad Langhorst langhorst@neb.com 978-380-7564
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
--
Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759
galaxy-dev@lists.galaxyproject.org
-
Clare Sloggett -
James -
Jeremy Goecks -
John Chilton -
Langhorst, Brad