- galaxy-dev - lists.galaxyproject.org

tool of installing tool shed repositories
by ngsflow＠hygenomics.com 04 Nov '13

04 Nov '13

Hi, there: I'm currently trying to migrate all repositories of main tool shed on http://toolshed.g2.bx.psu.edu/ to local, but got some problems. I'm wondering whether there is an existing tool can automatically do the job? thanks rgds, Ray ngsflow(a)hygenomics.com

1 0

Installing toolshed repositories with set_environment dependency
by S.D. Hiltemann 02 Nov '13

02 Nov '13

I updated my Galaxies to the latest version today, and when I try to install a toolshed repository which defines an environment variable via the <set_environment> tags in the tool_dependencies.xml file, I get an error (see below). tool_dependencies.xml: <?xml version="1.0"?> <tool_dependency> <set_environment version="1.0"> <environment_variable name="CONDEL_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable> </set_environment> </tool_dependency> (before the update these repositories installed without problems) ..doing the same as an action within a package dependency does still work, so for now I have just changed the tools I needed to set the environment this way, e.g.: <tool_dependency> <package name="condel" version="1"> <install version="1.0"> <actions> <action type="set_environment"> <environment_variable name="CONDEL_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable> </action> </actions> </install> <readme> Set condel environment variable </readme> </package> </tool_dependency> Is this the preferred way of doing it, and should I change all my tools this way? or should both ways still be working? Saskia Error Traceback: View as: Interactive<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> | Text<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> | XML<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> (full)<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> ⇝ NameError: global name 'env_var_elem' is not defined URL: http://galaxy.trait-ctmm.cloudlet.sara.nl/admin_toolshed/manage_tool_depend… Module weberror.evalexception.middleware:364 in respond [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> body = method( trans, **kwargs ) Module galaxy.web.framework:229 in decorator [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:757 in manage_tool_dependencies [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> self.initiate_tool_dependency_installation( trans, tool_dependencies_for_installation ) Module galaxy.web.framework:229 in decorator [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:433 in initiate_tool_dependency_installation [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> tool_dependencies=tool_dependencies ) Module tool_shed.util.common_install_util:486 in handle_tool_dependencies [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> env_var_name = env_var_elem.get( 'name', None ) NameError: global name 'env_var_elem' is not defined ________________________________ From: S.D. Hiltemann Sent: Friday, August 02, 2013 11:19 AM To: galaxy-dev(a)lists.bx.psu.edu Subject: Toolshed repository update error I am working on putting a tool into the toolshed. I have a bash script wrapper. I uploaded the first version fine, but when I wanted to upload a revision, some unwanted characters are inserted around the revised function of my code (see bottom) ..the inserted "<<<<<< local" etc strings are causing my script to fail of course. How can I prevent this from happening? Saskia snippet of the code: if [[ ! -s $rfile ]] then dummycol=${addcols:2} outputcol=${dummycol//",B."/" "} <<<<<<< local echo -e "${col_chr_name}\t${col_start_name}\t${col_end_name}\t${col_ref_name}\t${col_obs_name}\t$outputcol" > $rfile cat $rfile ======= numcommas=`echo "$addcols" | grep -o "," | wc -l` echo "numcolums: $numcommas" awk 'BEGIN{FS="\t";OFS="\t"}{ if(FNR==1) print $0,"'"$outputcol"'"; else{ printf $0 for(i=0;i<="'"$numcommas"'"+1;i++) printf "\t" printf "\n" } }END{}' $ofile > tempofile mv tempofile $ofile return >>>>>>> other fi

6 17

Re: [galaxy-dev] Working with setup_ruby_environment tag
by Ira Cooke 02 Nov '13

02 Nov '13

Hi Bjoern, I've just done some more testing of my tool with a fresh install and a fresh checkout of galaxy. After install of this tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 I have an env.sh that looks like GEM_PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2:$GEM_PATH; export GEM_PATH PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2/bin:$PATH; export PATH Which seems to be enough for me to run the executables in my gem. So I must be wrong about GEM_HOME needing to be set ... sorry for the false alarm there. The thing that I'm now having trouble with is that all the other set_environment tags in my tool definition are ignored. In my tool_dependencies.xml I have <action type="set_environment"> <environment_variable action="prepend_to" name="PATH">$INSTALL_DIR/bin</environment_variable> <environment_variable action="set_to" name="GEM_HOME">$INSTALL_DIR/</environment_variable> <environment_variable action="set_to" name="PROTK_BLAST_ROOT">FROMPATH</environment_variable> </action> Admittedly some of that stuff is redundant ( I now realise ) ... but other stuff (eg PROTK_BLAST_ROOT ) is important ... but it gets ignored. Is this intended? Seems like I should be able to use the set_environment action as well as the setup_ruby_environment action in the same tool definition? Hope you can clarify. Thanks Ira On 31/10/2013, at 6:06 AM, Ira Cooke <mudflatsoftware(a)gmail.com> wrote: > Hi Bjoern, > > I saw the same SO post ... but it says that GEM_PATH can be multiple paths but one of them should be GEM_HOME. Under your system (I think) GEM_HOME remains unset? > > Anyway ... I'll test again to double check that this is required for my tool to work. > > Cheers > Ira > > > On 31 Oct 2013, at 6:00 am, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: > >> Hi Ira, >> >>> Hi Bjoern, >>> >>> I'll test, >> >> thanks! >> >>> but I wonder if the problem is that GEM_PATH is set instead of GEM_HOME. I think GEM_HOME needs to be set also. It's zoo of environment variables out there :)!! >> >> Mh according to >> >> http://stackoverflow.com/questions/11277227/whats-the-difference-between-ge… >> >> GEM_PATH will contain GEM_HOME and GEM_HOME is used for installation, >> and I use it for installation of GEMs, See the code. If you think >> GEM_HOME is necessary to export I will do so. Otherwise I would not >> export to much. >> >> Thanks! >> Bjoern >> >>> Also ... I think GEM_HOME should be set using "set_to" not "prepend_to" ... >>> >>> I am using the latest galaxy-central (default branch). >>> >>> Cheers >>> Ira >>> >>> >>> >>> >>> On 31 Oct 2013, at 12:33 am, Bjoern Gruening <bjoern.gruening(a)gmail.com> wrote: >>> >>>> Hi Ira, >>>> >>>> that is strange, because PATH and GEM_HOME should be set automatically >>>> for you. >>>> >>>> Please have a look at >>>> https://github.com/bgruening/galaxytools/blob/master/orphan_tool_dependenci… >>>> http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_ruby2_bioruby_1_4 >>>> >>>> That definition results on my test environment in: >>>> >>>> GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH >>>> >>>> And the code for that is here: >>>> https://bitbucket.org/galaxy/galaxy-central/src/77d58fdd1c2e0dfde3742762092… >>>> >>>> Are you using the latest galaxy central? >>>> >>>> Regarding RUBYLIB, I think we need to set it to specify the right ruby >>>> version, because many can be installed in parallel. Some holds for >>>> RUBYHOME or? >>>> >>>> I see your point in having it set automatically and came with the >>>> attached patch. Here I need to assume that package_ruby_x_x will have >>>> RUBYLIB and RUBYHOME specified, but I think that is a save assumption. >>>> Nevertheless, we should document it. >>>> >>>> With that new patch you will end up with >>>> >>>> RUBYLIB=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/; export RUBYLIB >>>> RUBYHOME=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63; export RUBYHOME >>>> GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH >>>> >>>> >>>> Thanks very much for your feedback. I do not have any ruby repository so >>>> I really appropriate your testing! >>>> Bjoern >>>> >>>> >>>> >>>>> Hi Bjoern, >>>>> >>>>> Ross just gave me some pointers and I realised that I could just set these environment variables myself by doing >>>>> >>>>> <action type="set_environment"> >>>>> <environment_variable action="prepend_to" name="PATH">$INSTALL_DIR/bin</environment_variable> >>>>> <environment_variable action="set_to" name="GEM_HOME">$INSTALL_DIR/</environment_variable> >>>>> </action> >>>>> >>>>> In my protk tool definition. >>>>> >>>>> RUBYLIB isn't set ... hopefully this will not be an issue since I guess that's mostly just needed for Gem installation/compilation. >>>>> >>>>> This part works now :) >>>>> >>>>> Thanks >>>>> Ira >>>>> >>>>> >>>>> On 30/10/2013, at 3:51 PM, Ira Cooke <iracooke(a)gmail.com> wrote: >>>>> >>>>>> Hi Bjoern, >>>>>> >>>>>> Changing the title to reflect the proper topic >>>>>> >>>>>> I see now that the gem executables are installed under $INSTALLDIR/bin so I added this to the environment for my protk install. >>>>>> >>>>>> That got me to the point where my executable would run ... but now I'm seeing that in my env.sh file I don't really have my ruby environment setup properly (ie RUBYLIB and GEM_HOME are not set). This resulted in my tool running but it couldn't find my gem. >>>>>> >>>>>> So I tried adding ruby explicitly as a tool dependency for my downstream tool (this seems wrong though ... since it is required by protk ... and if I include protk I should get it automatically right?). After doing this though I was getting an error on installing protk about "List index out of range". >>>>>> >>>>>> Just to summarize, I can't see how to setup my dependencies properly so that the executables installed by protk get run with the correct environment. >>>>>> >>>>>> The repository I'm trying to install is >>>>>> >>>>>> http://testtoolshed.g2.bx.psu.edu/view/iracooke/omssa >>>>>> >>>>>> I'm sure I've just got muddled somewhere ... but it feels like I'm just messing around and getting it wrong. I'd rather do it in the way you envisioned. >>>>>> >>>>>> If you could give me some pointers on what I've done wrong it would help alot >>>>>> >>>>>> Thanks >>>>>> Ira >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 29/10/2013, at 8:14 PM, Bjoern Gruening <bjoern.gruening(a)gmail.com> wrote: >>>>>> >>>>>>> >>>>>>>> Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. >>>>>>> >>>>>>> No problem! >>>>>>> >>>>>>>> I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then >>>>>>> >>>>>>> Sure, thanks for working on that! >>>>>>> >>>>>>> Here is the promised trello card: https://trello.com/c/l9ZlGa8R >>>>>>> Ciao, >>>>>>> Bjoern >>>>>>> >>>>>>>> Cheers >>>>>>>> Ira >>>>>>>> >>>>>>>> >>>>>>>>> On 29 Oct 2013, at 7:17 pm, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: >>>>>>>>> >>>>>>>>> Hi Ira, >>>>>>>>> >>>>>>>>> as far as I can see you missed the <actions> tag. That results in no log >>>>>>>>> message at all, since no action type is executed. Galaxy should probably >>>>>>>>> give an error here. Or better validate that during tool upload. I will >>>>>>>>> fill a Trello Card for it. >>>>>>>>> >>>>>>>>> The following code snipped worked for me: >>>>>>>>> >>>>>>>>> <?xml version="1.0"?> >>>>>>>>> <tool_dependency> >>>>>>>>> <package name="protk" version="1.2.5"> >>>>>>>>> <install version="1.0"> >>>>>>>>> <actions> >>>>>>>>> <action type="setup_ruby_environment"> >>>>>>>>> <repository changeset_revision="a0494c6e1c63" >>>>>>>>> name="package_ruby_2_0" owner="bgruening" >>>>>>>>> toolshed="http://testtoolshed.g2.bx.psu.edu"> >>>>>>>>> <package name="ruby" version="2.0" /> >>>>>>>>> </repository> >>>>>>>>> <package>protk=1.2.6.pre1</package> >>>>>>>>> </action> >>>>>>>>> </actions> >>>>>>>>> </install> >>>>>>>>> <readme> >>>>>>>>> Installs version 1.2.5 of the protk rubygem from rubygems.org >>>>>>>>> </readme> >>>>>>>>> </package> >>>>>>>>> </tool_dependency> >>>>>>>>> >>>>>>>>> Keep in mind that the repository tag should be filled. If you have a >>>>>>>>> actions and action tag it will be filled automatically by the TS during >>>>>>>>> upload. But better to check :) >>>>>>>>> >>>>>>>>> To populate the RUBY_HOME env var ... do you think its better to set it >>>>>>>>> via the installation automatically? My idea was to just "require" ruby >>>>>>>>> in the tool? If you want to set it in your protk definition you need to >>>>>>>>> use the $ENV[] construct. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Bjoern >>>>>>>>> >>>>>>>>>> Hi Bjoern, >>>>>>>>>> >>>>>>>>>> Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. >>>>>>>>>> >>>>>>>>>> I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. >>>>>>>>>> >>>>>>>>>> Cheers >>>>>>>>>> Ira >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 29/10/2013, at 10:06 AM, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ira, >>>>>>>>>>> >>>>>>>>>>> I can reproduce that error here, will try to study it tomorrow more >>>>>>>>>>> deeply. For the meantime can you try to remove the following lines? >>>>>>>>>>> >>>>>>>>>>> <action type="set_environment_for_install"> >>>>>>>>>>> <repository name="package_ruby_2_0" owner="bgruening"> >>>>>>>>>>> <package name="ruby" version="2.0" /> >>>>>>>>>>> </repository> >>>>>>>>>>> </action> >>>>>>>>>>> >>>>>>>>>>> It's not necessary. >>>>>>>>>>> Will get back to you tomorrow. >>>>>>>>>>> Bjoern >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the "Installing" phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). >>>>>>>>>>>> >>>>>>>>>>>> The longer version of my problem is below. >>>>>>>>>>>> >>>>>>>>>>>> In my case I am struggling with the following tool >>>>>>>>>>>> >>>>>>>>>>>> http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 >>>>>>>>>>>> >>>>>>>>>>>> Which uses the new "setup_ruby_environment" tag ... so the tool should do the following >>>>>>>>>>>> >>>>>>>>>>>> (1) Install the dependency .. Ruby 2.0 ... which it does successfully >>>>>>>>>>>> (2) Install my protk rubygem from rubygems.org .. which it gets stuck doing. >>>>>>>>>>>> >>>>>>>>>>>> The last few lines from my galaxy server log during the failed install are; >>>>>>>>>>>> >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:26 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:30 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdQaLI7O >>>>>>>>>>>> ruby version 2.0 installed in /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63 >>>>>>>>>>>> [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdJYyhO0 >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:34 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:47:22 +0000] "POST /admin_toolshed/manage_repositories HTTP/1.1" 302 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:37 +0000] "GET /admin_toolshed/monitor_repository_installation?tool_shed_repository_ids=f597429621d6eb2b&tool_shed_repository_ids=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> With a normal tool install I would see the last line of this output repeat lots of times as installation is monitored ... but for some reason in this case monitoring just shuts down and I don't see any further updates. >>>>>>>>>>>> >>>>>>>>>>>> If I recreate the situation on the command-line (eg by manually setting the RUBYLIB and GEM_HOME environment variables appropriately I can install my gem without any errors. >>>>>>>>>>>> >>>>>>>>>>>> More details on my setup: >>>>>>>>>>>> >>>>>>>>>>>> I'm using a vagrant VM for testing which is based on Ubuntu precise64 ... with a minimal number of packages installed ... >>>>>>>>>>>> (My VM is documented in full here: https://bitbucket.org/iracooke/protk_vagrant/src/0ef8d86cc3dcec4ee3a9a83edf… ) >>>>>>>>>>>> >>>>>>>>>>>> I'm testing based on the current default branch of galaxy-central .... which I regularly tear down and re-clone from scratch ... always with the same result. >>>>>>>>>>>> >>>>>>>>>>>> Any help or pointers to how to resolve this much appreciated. >>>>>>>>>>>> >>>>>>>>>>>> Cheers >>>>>>>>>>>> Ira >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> <add_more_env_vars_to_ruby_install_environment.patch> >>> >> >> >> >

2 1

Re: [galaxy-dev] Tool XML form building bug with <options from_data_table="..."> ?
by damion＠learningpoint.ca 01 Nov '13

01 Nov '13

John, thanks for following up. I myself abashedly sent out a followup mail which maybe didn't get to this list (from the other galaxy-user(a)lists.bx.pu.edu list). Yes indeed, I'd closed my one-liner options tag. Problem fixed now. Regards, Damion ---------------------------------------- From: "John Chilton" <chilton(a)msi.umn.edu> Sent: 01 November 2013 09:48 To: Cc: "Galaxy Dev" <galaxy-dev(a)bx.psu.edu>, damion(a)learningpoint.ca Subject: Re: [galaxy-dev] Tool XML form building bug with <options from_data_table="..."> ? Hey Damion, Because this wasn't explicitly shown in your example e-mail, can you verify for me that you are putting the filter elements inside of the options element and not inside of the param element? -John

1 0

Tool XML form building bug with <options from_data_table="..."> ?
by Jennifer Jackson 01 Nov '13

01 Nov '13

Moving this over to galaxy-dev(a)bx.psu.edu, the best list for tool design questions. Best! Jen, Galaxy team On 10/30/13 2:24 PM, damion(a)learningpoint.ca wrote: > Has anyone run into this? I'm building a general-purpose filter > control on my galaxy tool xml template for enabling numeric fields to > be filtered by > < etc. parameters - in a user friendly way. I have a > select list <para> driven by a data table: > ... > <param name="filter_column" type="select" label="Col"> > <options from_data_table="bccdc_blast_fields" /> > </param> > ... > This works fine in building a list of fields to select from. Then I add > <filter type="sort_by" column="1"/> > <filter type="add_value" name="TESTESTTEST" value="WHWHWHW" /> > > but nothing happens, sort remains incorrect and no extra value. I try > these same filters on a previous <param> in form that is driven > by <options from_file="bccdc_blast_bins.loc"> and they work fine. > > I also tried applying <fiilter type="static_value" ...> to no avail on > from_data_table options tag, but from from_file options, no problem. > > So I start to think there's a bug whereby NO filters work on <options > from_data_table="bccdc_blast_fields" /> input? I've surveyed the > galaxy.tools.parameters.dynamic_options > python code, but can't see a decision point there in which > from_data_table work vs from_file choice is made; is it in another > script file? > > I'm using a BioLinux 2012 install that includes Galaxy. > > Help appreciated! > > Regards, > > Damion Dooley > BC Centre for Disease Control > Vancouver, BC > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ -- Jennifer Hillman-Jackson http://galaxyproject.org

2 1

Workflows with multiple editors
by Eric Rasche 01 Nov '13

01 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm working with a number of people in my department to develop a single workflow that will be used for a course we teach. So far I've found that "sharing" a workflow with individual users/roles isn't very optimal. They email me with changes they'd like to happen, or they create their own copies which immediately branch off and don't help everyone else who might need them. As such, I'd like to propose changes and would like community feedback on my proposed changes so when I create the trello card/maybe try my hand at implementing them, they aren't completely specific to my use case. In my mind, the 'optimal' version of managing workflows would be something along the following lines: Top priorities: - - workflows can be shared as "write" or "read only" with multiple users/roles. - This would allow there to be a single copy of a workflow in cases where multiple people might make changes to a single workflow - For "read only", this allows the previous/traditional model of sharing a workflow and the receiver being able to run it as is, or to copy and modify it. Note: For my case at least, realtime multiple person editing is not needed. If a banner shows up which reads "someone started editing before you, either ask them to save + leave or save as a copy" would work in my case. Wishlist features: - - workflows are treated as VCS repositories - Note that full merge/branching probably not really necessary for this case (that would be really nice, but probably too much of a nightmare to code...) - - You can see previous iterations of workflows. - - Workflows can be "owned" by either a user or a role - users could create a workflow and transfer ownership to a role - role owned workflows are delete-able (or maybe require some X% of users in that role to confirmation deletion?) - This would sort of work like a group owned VCS repository, where multiple users can create workflows of interest to a group of people (role) and have them, by default, be available. Anyone have input on this proposal? Cheers, Eric - -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr(a)tamu.edu rasche.eric(a)yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJScWXTAAoJEMqDXdrsMcpVFH4QAMO0V4tSJGzM3PGGIwdonY0v nyuHTCJJ5qliWuapUJUK9E0UQBblnaJYooLjh2l1vgo95JaXdHEAC9NVecQLH8J2 ey5cu1xODf6JmBGlicRGxQWlLZmo9U3R4T/OhHco40I239aTpuSDPcwcBJUm1cAE czz+/2pgdCpkJqYQJ9EE956TAYdyPBYFvSrIO7ebWkSvC5He40qca0x06tgq14uR Dta+8QH8brVYX1G+TAgdh3hGlLUwIlPMiZsWOEMcwSUtyXP3I7OKx/XIOy3ZYVTu Py/XlvNlZ87yiaiwDSxjxR9qaDeGQ3J8CRCG9mqWu8O5Y9QKkyrC410I+hZH5lE+ my28z7l5rK0i05pR6o8euASJ7D1eFHEDYmBmN6j3qRcOgzzdup2oQpm6TA7ue1sd NAS4rnW7QGmfNj+9xNSQhDWSj7CeIUeHXbDe55suBX3JofCNg8Qq6uYzPd7KGj7n jvWp8tGnpwbgTmvDPlR03gJiXMZld+gGwKrlCyn1kdBkfI1r5l0UfoaIUE+8HDm3 wXviQ8GU7seYU5bJFK798owVNawjY1H3JenzIYt1XvF1z53Uc39x58Q9lxXbVfSe KgiqCfF0YuvVtOczGTAQ57qCFwlpKg2ra+X+HtAZyMDNMsb3ANgfogTqsB+7HYvF czq2QJHq9mBC5N9JLhRo =conD -----END PGP SIGNATURE-----

3 3

browse directories on galaxy server
by David Hoover 01 Nov '13

01 Nov '13

Is there any desire to create a way in galaxy to browse local directories on the galaxy server itself? Not the client machine running the web browwser, but the machine on which galaxy is running. I've created a tool that allows a user to upload a single file from their own personal directory on the galaxy server into galaxy directly, without going through Apache. However, this is very limiting. It would be very helpful to be able to browse directories that are deemed accessible to the galaxy user, and before I begin trying to develop anything I'd like to know if there is any support for such a feature. David Hoover Helix Systems Staff National Institutes of Health

3 2

User activations
by jordi＠biociphers.org 01 Nov '13

01 Nov '13

Hi all, Recently I have updated our test galaxy server in order to check some new features like user email activation using emails. I saw that in the admin page http://wiki.galaxyproject.org/Admin/UserAccounts . But I download the last version and the conf values that are supose to be in the universe_wsgi.ini are not present ( at least in the sample file). Do you know if this is a future feature or there is anything I need to do for testing it? thanks everyone for your help Jordi

2 1

November 2013 Galaxy Update Newsletter
by Dave Clements 31 Oct '13

31 Oct '13

Hello all, The November 2013 Galaxy Update is out<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11> : *Highlights:* - Two new public Galaxy servers<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#New_Public_Servers> : CoSSci: Complex Social Science Gateway<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#CoSSci:_Complex_Social_…> (which has *nothing* to do with biology), and BioCiphers Lab Galaxy<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#BioCiphers_Lab_Galaxy> . - 53 new papers<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#New_Papers>, including "Expanding roles in a library-based bioinformatics service program: a case study," "DDBJ Read Annotation Pipeline: A Cloud Computing-Based Pipeline for High-Throughput Analysis of Next-Generation Sequencing Data," and "Ten Simple Rules for Reproducible Computational Research" - Who's hiring<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#Who.27s_Hiring> - Upcoming Events<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#Other_Events>, including - Save these dates! GCC2014: June 30 - July 2, Baltimore<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#GCC2014:_June_30_-_July…> - Galaxy Day, December 4, Paris<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#Galaxy_Day.2C_December_…> - UC Davis Bioinformatics Boot Camps<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#UC_Davis_Bioinformatics…> - Lifeportal launched at the University of Oslo<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#Lifeportal_at_the_Unive…> - Tool Shed contributions<http://wiki.galaxyproject.org/GalaxyUpdates/2013_11#Tool_Shed_Contributions> If you have anything you would like to see in the next *Galaxy Update<http://wiki.galaxyproject.org/GalaxyUpdates> *, please let us know. Dave Clements and the Galaxy Team <http://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0

Running functional tests is too slow
by Peter Cock 31 Oct '13

31 Oct '13

Hi all, I'd like to echo Florent's concern that run_functional_tests.sh is too slow, and that this discourages Tool Authors from adding more tests to their tools: https://trello.com/c/wL21d2do/1017-functional-tests-take-too-long (And encourage people to vote up that issue ;) ) Florent has identified one clear bottleneck is the creation of a fresh SQLite database each time, upon which the growing number of schema migration calls must be performed. Could we not cache an empty but up to date copy of this database? i.e. Something like this: 1. run_functional_tests.sh starts 2. If it exists, copy empty_test_sqlite_database.db, otherwise create new test SQLite database as now. 3. Check the schema version. 4. If the temporary SQlite database is out of date, run the migration, and then save a copy of this now up to date database as empty_test_sqlite_database.db, 5. run the tests Perhaps the empty SQLite database could even be cached in BitBucket too, for a faster first test run with a clean checkout? Separately, running the tests themselves seems overly slow - can anything be tweaked here? For example, is there any point executing the external set_meta script in the test environment? Regards, Peter P.S. On a related note, my achievement last Friday was to get TravisCI doing continuous integration testing of a GitHub repository of Galaxy tools: https://travis-ci.org/peterjc/galaxy_blast https://github.com/peterjc/galaxy_blast For those not familiar with this system, the idea is that via a special .travis.yml configuration file, each time new commits are pushed to GitHub, the latest code is checked out and tested on a virtual machine. i.e. Continuous in the sense of running the tests after each change, rather than just once a night. Right now the tests for the BLAST+ suite and associated tool wrappers like Blast2GO takes from 15 to 20 minutes, which I feel could be much improved.

2 5