November 2013 - galaxy-dev - lists.galaxyproject.org

galaxy cant find binaries in
by tomasz jankowski 05 Nov '13

05 Nov '13

Hi, I supose it is simple misteake but i'm realy close to got mad because dealing with it since friday... so I'm writing to list... I dont know why galaxy cant find samtools ininstallation dir. It seems that all was done acordingly to admin manual... what I can see after sending bam file is Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found I thought that galaxy doesntread env.sh file but in log file I have galaxy.tools DEBUG 2013-11-04 23:07:22,556 Building dependency shell command for dependency 'samtools' galaxy.jobs.runners.local DEBUG 2013-11-04 23:07:22,774 (35) executing: PACKAGE_BASE=/opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19; export PACKAGE_BASE; . /opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/env.sh; all log is in attached file I have fresh galaxy stable version of galaxy system centos 6.4 is runing on virtualmachine (vbox) Linux galaxy 2.6.32-358.23.2.el6.x86_64 #1 SMP Wed Oct 16 18:37:12 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux main galaxy instalation dir is /opt/galaxy/galaxy-dist tools dependency directory in universe_wsgi.ini is tool_dependency_dir = tools_dependences [galaxy@galaxy galaxy-dist]$ ls -alh /opt/galaxy/galaxy-dist/tools_dependences/samtools/ total 16K drwxrwx--- 4 galaxy galaxy 4.0K Nov 4 22:31 . drwxrwxr-x 3 galaxy galaxy 4.0K Nov 4 22:25 .. drwxrwx--- 3 galaxy galaxy 4.0K Nov 4 23:04 0.1.19 lrwxrwxrwx 1 galaxy galaxy 6 Nov 4 22:31 default -> 0.1.19 drwxrwx--- 6 galaxy galaxy 4.0K Nov 4 22:04 samtools-0.1.19 [galaxy@galaxy galaxy-dist]$ ls -alh /opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/bin/samtools -rwxrwx--- 1 galaxy galaxy 1.5M Nov 4 22:06 /opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/bin/samtools [galaxy@galaxy galaxy-dist]$ cat /opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/env.sh PATH="/opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/bin:$PATH" export PATH When I put /opt/galaxy/galaxy-dist/tools_dependences/samtools/0.1.19/bin to /home/galaxy/.bash_profile galaxy CAN see the samtools binaries... could you tell me what I'm doing wrong? tj

2 1

Question regarding walltime exceeded not being correctly reported via the WebUI
by Daniel Patrick Sullivan 05 Nov '13

05 Nov '13

Hi, Galaxy Developers, I have what I hops is somewhat of a basic question regarding Galaxy's interaction with a pbs job cluster and information reported via the webUI. Basically, in certain situations, the walltime of a specific job is exceeded. This is of course to be expected and all fine and understandeable. My problem is that the information is not being relayed back to the end user via the Galaxy web UI, which causes confusion in our Galaxy user community. Basically the Torque scheduler generates the following message when a walltime is exceeded: 11/04/2013 08:39:45;000d;PBS_Server.30621;Job;163.sctest.cri.uchicago.edu;preparing to send 'a' mail for job 163.sctest.cri.uchicago.edu to s.cri.galaxy(a)crigalaxy-test.uchicago.edu (Job exceeded its walltime limit. Job was aborted 11/04/2013 08:39:45;0009;PBS_Server.30621;Job;163.sctest.cri.uchicago.edu;job exit status -11 handled Now, my problem is that this status -11 return code is not being correctly handled by Galaxy. What happens is that Galaxy throws an exception, specificially: 10.135.217.178 - - [04/Nov/2013:08:39:42 -0500] "GET /api/histories/90240358ebde1489 HTTP/1.1" 200 - " https://crigalaxy-test.uchicago.edu/history" "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" galaxy.jobs.runners.pbs DEBUG 2013-11-04 08:39:46,137 (2150/ 163.sctest.cri.uchicago.edu) PBS job state changed from R to C galaxy.jobs.runners.pbs ERROR 2013-11-04 08:39:46,139 (2150/ 163.sctest.cri.uchicago.edu) PBS job failed: Unknown error: -11 galaxy.jobs.runners ERROR 2013-11-04 08:39:46,139 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File "/group/galaxy_test/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 60, in run_next method(arg) File "/group/galaxy_test/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 561, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' After this exception occurs, the Galaxy job status via the Web UI is still reported as "Job is currently running". It appears that the job will remain in this state (from the end users perspective) indefinitely. Has anybody seen this issue before? I noticed that return code -11 does not exist in /group/galaxy_test/galaxy-dist/lib/galaxy/jobs/runners/pbs.py under the JOB_EXIT_STATUS dictionary. I tried adding an entry for this, however when I do the exception changes to: galaxy.jobs.runners.pbs ERROR 2013-11-04 10:02:17,274 (2151/ 164.sctest.cri.uchicago.edu) PBS job failed: job walltime exceeded galaxy.jobs.runners ERROR 2013-11-04 10:02:17,275 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File "/group/galaxy_test/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 60, in run_next method(arg) File "/group/galaxy_test/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 562, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' I am wondering if this is a bug or if it is just because I am using a newer version of TORQUE (I am using TORQUE 4.2.2). In terms of Galaxy, I am using: [s.cri.galaxy@crigalaxy-test galaxy-dist]$ hg parents changeset: 10408:6822f41bc9bb branch: stable parent: 10393:d05bf67aefa6 user: Dave Bouvier <dave(a)bx.psu.edu> date: Mon Aug 19 13:06:17 2013 -0400 summary: Fix for case where running functional tests might overwrite certain files in database/files. [s.cri.galaxy@crigalaxy-test galaxy-dist]$ Does anybody know how I could fix this such that walltime exceeded messages are correctly reporeted via the Galaxy web UI for TORQUE 4.2.2? Thank you so much for your input and guidance, and for the ongoing development of Galaxy. Dan Sullivan

2 1

Galaxy dropping jobs?
by Nikolai Vazov 05 Nov '13

05 Nov '13

Hi, I discovered a weird issue in the job behaviour : Galaxy is running a long job on a cluster (more than 24h), about 15 hours later it misses the connection with SLURM on the cluster and throws the following message : [root@galaxy-prod01 galaxy-dist]# grep 3715200 paster.log galaxy.jobs.runners.drmaa INFO 2013-10-30 10:51:54,149 (555) queued as 3715200 galaxy.jobs.runners.drmaa DEBUG 2013-10-30 10:51:55,149 (555/3715200) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-10-30 10:52:13,516 (555/3715200) state change: job is running galaxy.jobs.runners.drmaa INFO 2013-10-31 03:29:33,090 (555/3715200) job left DRM queue with following message: code 1: slurm_load_jobs error: Unable to contact slurm controller (connect failure),job_id: 3715200 Is there a timeout in Galaxy for contacting slurm? Yet, the job is still running properly on the cluster ... Thanks for help, it's really urgent :) Nikolay -- Nikolay Vazov, PhD Research Computing Centre - http://hpc.uio.no USIT, University of Oslo

2 2

Retrieve list of published workflows from API
by Philip Mabon 04 Nov '13

04 Nov '13

Is there a way to interact with the published workflows section of Galaxy using only the API? Only workflows that I imported or constructed myself show up using the get_workflows from bioblend. Thanks!

2 1

Nov 04, 2013 Galaxy Distribution
by Jennifer Jackson 04 Nov '13

04 Nov '13

1 0

Re: [galaxy-dev] [galaxy-user] Inquiring
by Nate Coraor 04 Nov '13

04 Nov '13

Hi Yan, I've moved this discussion to the galaxy-dev list since it pertains to a local installation of Galaxy. Responses to your questions follow, in-line. Yan Luo wrote: > Dear Sir, > > (1)We installed Galaxy, but recently the user can't registered and got the > following error, how can we fix it? > > Sever error > An error occurred. See the error logs for more information.(To turn debug on > to display ...). Since debug = False in universe_wsgi.ini, you should be able to find a more detailed error message in the log file. If starting Galaxy with: % sh run.sh --daemon The default log file is 'paster.log' in Galaxy's root directory. > (2) Could you please let me know if there is any command to stop galaxy? If starting with the --daemon flag (as above), you can use: % sh run.sh --stop-daemon If running in the foreground, you can use Ctrl-C to terminate the process. There is a recent bug whereby Ctrl-C is ineffective on some platforms under Python 2.6 - in this case you will have to kill/pkill the process manually. We are working on a fix for the latter. > (3) If I reset universe_wsgi.ini file and want to set an administrator > user(I can add a line in the above file), how can I get the password? Should > I stop galaxy(See question 2) first? then run "./setup.sh" and "./run.sh". setup.sh would have only been necessary prior to running Galaxy the first time, however, this step has recently been removed. If you are referencing documentation that still refers to setup.sh, please let us know so we can update it - I did notice this was still on the "Production Server" page, so I removed it from there. You no longer need to run setup.sh at all. > (4) If I run "setup.sh", will a new file "universe_wsgi.ini" be generated? > if I want to change this file,should I edit it before "run.sh" and after > "setup.sh". Is it right? setup.sh and its replacements in run.sh and the Galaxy application itself never overwrite files, they only create files from sample files if they do not exist. > (5) I read some of your docs, command "sh setup.sh"(sh run.sh) and > "./setup.sh"(./run.sh), which one is correct under Linux? Both syntaxes are effectively the same in most cases. --nate > > Looking forward to hearing from you. > > Best Wises, > > Yan Luo, Ph.D. > NIH > <http://int.ask.com/web?siteid=10000861&webqsrc=999&l=dis&q=By%20the%20way,> > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

4 7

Fw: Re: tool of installing tool shed repositories
by ngsflow＠hygenomics.com 04 Nov '13

04 Nov '13

ngsflow(a)hygenomics.com From: ngsflow(a)hygenomics.com Date: 2013-11-04 17:53 To: Björn Grüning Subject: Re: Re: [galaxy-dev] tool of installing tool shed repositories hi Bjoern: I am trying to bulk import repositories, by developing some tools following the thought similar to those in toolshed tools test. I just want to make sure whether there are some tools already exist. rgds, Ray ngsflow(a)hygenomics.com From: Björn Grüning Date: 2013-11-04 16:46 To: ngsflow(a)hygenomics.com CC: galaxy-dev Subject: Re: [galaxy-dev] tool of installing tool shed repositories Hi Ray, there is some work in that direction to easily import and export repositories. The export feature is already integrated and should help you. You will end up with a tarball with all informations about that repository and the import should be easier. If you want to work on the import part, I think your work is more than welcome! Cheers, Bjoern > Hi, there: > > I'm currently trying to migrate all repositories of main tool shed on http://toolshed.g2.bx.psu.edu/ to local, but got some problems. > > I'm wondering whether there is an existing tool can automatically do the job? > > thanks > > rgds, > Ray > > > ______________________________________________________________________ > ngsflow(a)hygenomics.com > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/

1 0

tool of installing tool shed repositories
by ngsflow＠hygenomics.com 04 Nov '13

04 Nov '13

Hi, there: I'm currently trying to migrate all repositories of main tool shed on http://toolshed.g2.bx.psu.edu/ to local, but got some problems. I'm wondering whether there is an existing tool can automatically do the job? thanks rgds, Ray ngsflow(a)hygenomics.com

1 0

Installing toolshed repositories with set_environment dependency
by S.D. Hiltemann 02 Nov '13

02 Nov '13

I updated my Galaxies to the latest version today, and when I try to install a toolshed repository which defines an environment variable via the <set_environment> tags in the tool_dependencies.xml file, I get an error (see below). tool_dependencies.xml: <?xml version="1.0"?> <tool_dependency> <set_environment version="1.0"> <environment_variable name="CONDEL_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable> </set_environment> </tool_dependency> (before the update these repositories installed without problems) ..doing the same as an action within a package dependency does still work, so for now I have just changed the tools I needed to set the environment this way, e.g.: <tool_dependency> <package name="condel" version="1"> <install version="1.0"> <actions> <action type="set_environment"> <environment_variable name="CONDEL_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable> </action> </actions> </install> <readme> Set condel environment variable </readme> </package> </tool_dependency> Is this the preferred way of doing it, and should I change all my tools this way? or should both ways still be working? Saskia Error Traceback: View as: Interactive<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> | Text<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> | XML<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> (full)<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> ⇝ NameError: global name 'env_var_elem' is not defined URL: http://galaxy.trait-ctmm.cloudlet.sara.nl/admin_toolshed/manage_tool_depend… Module weberror.evalexception.middleware:364 in respond [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> body = method( trans, **kwargs ) Module galaxy.web.framework:229 in decorator [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:757 in manage_tool_dependencies [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> self.initiate_tool_dependency_installation( trans, tool_dependencies_for_installation ) Module galaxy.web.framework:229 in decorator [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:433 in initiate_tool_dependency_installation [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> tool_dependencies=tool_dependencies ) Module tool_shed.util.common_install_util:486 in handle_tool_dependencies [http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/media/plus.jpg] <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> view<http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> >> <http://galaxy.trait-ctmm.cloudlet.sara.nl/_debug/view/1382536593#> env_var_name = env_var_elem.get( 'name', None ) NameError: global name 'env_var_elem' is not defined ________________________________ From: S.D. Hiltemann Sent: Friday, August 02, 2013 11:19 AM To: galaxy-dev(a)lists.bx.psu.edu Subject: Toolshed repository update error I am working on putting a tool into the toolshed. I have a bash script wrapper. I uploaded the first version fine, but when I wanted to upload a revision, some unwanted characters are inserted around the revised function of my code (see bottom) ..the inserted "<<<<<< local" etc strings are causing my script to fail of course. How can I prevent this from happening? Saskia snippet of the code: if [[ ! -s $rfile ]] then dummycol=${addcols:2} outputcol=${dummycol//",B."/" "} <<<<<<< local echo -e "${col_chr_name}\t${col_start_name}\t${col_end_name}\t${col_ref_name}\t${col_obs_name}\t$outputcol" > $rfile cat $rfile ======= numcommas=`echo "$addcols" | grep -o "," | wc -l` echo "numcolums: $numcommas" awk 'BEGIN{FS="\t";OFS="\t"}{ if(FNR==1) print $0,"'"$outputcol"'"; else{ printf $0 for(i=0;i<="'"$numcommas"'"+1;i++) printf "\t" printf "\n" } }END{}' $ofile > tempofile mv tempofile $ofile return >>>>>>> other fi

6 17

Re: [galaxy-dev] Working with setup_ruby_environment tag
by Ira Cooke 01 Nov '13

01 Nov '13

Hi Bjoern, I've just done some more testing of my tool with a fresh install and a fresh checkout of galaxy. After install of this tool http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 I have an env.sh that looks like GEM_PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2:$GEM_PATH; export GEM_PATH PATH=/home/vagrant/tool_dependencies/protk/1.2.5/iracooke/package_protk_1_2_5/1100ac98dfd2/bin:$PATH; export PATH Which seems to be enough for me to run the executables in my gem. So I must be wrong about GEM_HOME needing to be set ... sorry for the false alarm there. The thing that I'm now having trouble with is that all the other set_environment tags in my tool definition are ignored. In my tool_dependencies.xml I have <action type="set_environment"> <environment_variable action="prepend_to" name="PATH">$INSTALL_DIR/bin</environment_variable> <environment_variable action="set_to" name="GEM_HOME">$INSTALL_DIR/</environment_variable> <environment_variable action="set_to" name="PROTK_BLAST_ROOT">FROMPATH</environment_variable> </action> Admittedly some of that stuff is redundant ( I now realise ) ... but other stuff (eg PROTK_BLAST_ROOT ) is important ... but it gets ignored. Is this intended? Seems like I should be able to use the set_environment action as well as the setup_ruby_environment action in the same tool definition? Hope you can clarify. Thanks Ira On 31/10/2013, at 6:06 AM, Ira Cooke <mudflatsoftware(a)gmail.com> wrote: > Hi Bjoern, > > I saw the same SO post ... but it says that GEM_PATH can be multiple paths but one of them should be GEM_HOME. Under your system (I think) GEM_HOME remains unset? > > Anyway ... I'll test again to double check that this is required for my tool to work. > > Cheers > Ira > > > On 31 Oct 2013, at 6:00 am, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: > >> Hi Ira, >> >>> Hi Bjoern, >>> >>> I'll test, >> >> thanks! >> >>> but I wonder if the problem is that GEM_PATH is set instead of GEM_HOME. I think GEM_HOME needs to be set also. It's zoo of environment variables out there :)!! >> >> Mh according to >> >> http://stackoverflow.com/questions/11277227/whats-the-difference-between-ge… >> >> GEM_PATH will contain GEM_HOME and GEM_HOME is used for installation, >> and I use it for installation of GEMs, See the code. If you think >> GEM_HOME is necessary to export I will do so. Otherwise I would not >> export to much. >> >> Thanks! >> Bjoern >> >>> Also ... I think GEM_HOME should be set using "set_to" not "prepend_to" ... >>> >>> I am using the latest galaxy-central (default branch). >>> >>> Cheers >>> Ira >>> >>> >>> >>> >>> On 31 Oct 2013, at 12:33 am, Bjoern Gruening <bjoern.gruening(a)gmail.com> wrote: >>> >>>> Hi Ira, >>>> >>>> that is strange, because PATH and GEM_HOME should be set automatically >>>> for you. >>>> >>>> Please have a look at >>>> https://github.com/bgruening/galaxytools/blob/master/orphan_tool_dependenci… >>>> http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_ruby2_bioruby_1_4 >>>> >>>> That definition results on my test environment in: >>>> >>>> GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH >>>> >>>> And the code for that is here: >>>> https://bitbucket.org/galaxy/galaxy-central/src/77d58fdd1c2e0dfde3742762092… >>>> >>>> Are you using the latest galaxy central? >>>> >>>> Regarding RUBYLIB, I think we need to set it to specify the right ruby >>>> version, because many can be installed in parallel. Some holds for >>>> RUBYHOME or? >>>> >>>> I see your point in having it set automatically and came with the >>>> attached patch. Here I need to assume that package_ruby_x_x will have >>>> RUBYLIB and RUBYHOME specified, but I think that is a save assumption. >>>> Nevertheless, we should document it. >>>> >>>> With that new patch you will end up with >>>> >>>> RUBYLIB=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/; export RUBYLIB >>>> RUBYHOME=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63; export RUBYHOME >>>> GEM_PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1:$GEM_PATH; export GEM_PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/bioruby/1.4/bgruening/package_ruby2_bioruby_1_4/0a30368657b1/bin:$PATH; export PATH >>>> PATH=/home/gruening/projects/code/galaxy-central/tool_deps/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH >>>> >>>> >>>> Thanks very much for your feedback. I do not have any ruby repository so >>>> I really appropriate your testing! >>>> Bjoern >>>> >>>> >>>> >>>>> Hi Bjoern, >>>>> >>>>> Ross just gave me some pointers and I realised that I could just set these environment variables myself by doing >>>>> >>>>> <action type="set_environment"> >>>>> <environment_variable action="prepend_to" name="PATH">$INSTALL_DIR/bin</environment_variable> >>>>> <environment_variable action="set_to" name="GEM_HOME">$INSTALL_DIR/</environment_variable> >>>>> </action> >>>>> >>>>> In my protk tool definition. >>>>> >>>>> RUBYLIB isn't set ... hopefully this will not be an issue since I guess that's mostly just needed for Gem installation/compilation. >>>>> >>>>> This part works now :) >>>>> >>>>> Thanks >>>>> Ira >>>>> >>>>> >>>>> On 30/10/2013, at 3:51 PM, Ira Cooke <iracooke(a)gmail.com> wrote: >>>>> >>>>>> Hi Bjoern, >>>>>> >>>>>> Changing the title to reflect the proper topic >>>>>> >>>>>> I see now that the gem executables are installed under $INSTALLDIR/bin so I added this to the environment for my protk install. >>>>>> >>>>>> That got me to the point where my executable would run ... but now I'm seeing that in my env.sh file I don't really have my ruby environment setup properly (ie RUBYLIB and GEM_HOME are not set). This resulted in my tool running but it couldn't find my gem. >>>>>> >>>>>> So I tried adding ruby explicitly as a tool dependency for my downstream tool (this seems wrong though ... since it is required by protk ... and if I include protk I should get it automatically right?). After doing this though I was getting an error on installing protk about "List index out of range". >>>>>> >>>>>> Just to summarize, I can't see how to setup my dependencies properly so that the executables installed by protk get run with the correct environment. >>>>>> >>>>>> The repository I'm trying to install is >>>>>> >>>>>> http://testtoolshed.g2.bx.psu.edu/view/iracooke/omssa >>>>>> >>>>>> I'm sure I've just got muddled somewhere ... but it feels like I'm just messing around and getting it wrong. I'd rather do it in the way you envisioned. >>>>>> >>>>>> If you could give me some pointers on what I've done wrong it would help alot >>>>>> >>>>>> Thanks >>>>>> Ira >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 29/10/2013, at 8:14 PM, Bjoern Gruening <bjoern.gruening(a)gmail.com> wrote: >>>>>> >>>>>>> >>>>>>>> Ahhh ... Thanks .. I can't believe I missed that. Thanks for spotting it. >>>>>>> >>>>>>> No problem! >>>>>>> >>>>>>>> I'm not sure I understand enough about how this works to answer your ruby_home question. Ill play with this more tomorrow and see how things work with my tools and maybe have more feedback then >>>>>>> >>>>>>> Sure, thanks for working on that! >>>>>>> >>>>>>> Here is the promised trello card: https://trello.com/c/l9ZlGa8R >>>>>>> Ciao, >>>>>>> Bjoern >>>>>>> >>>>>>>> Cheers >>>>>>>> Ira >>>>>>>> >>>>>>>> >>>>>>>>> On 29 Oct 2013, at 7:17 pm, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: >>>>>>>>> >>>>>>>>> Hi Ira, >>>>>>>>> >>>>>>>>> as far as I can see you missed the <actions> tag. That results in no log >>>>>>>>> message at all, since no action type is executed. Galaxy should probably >>>>>>>>> give an error here. Or better validate that during tool upload. I will >>>>>>>>> fill a Trello Card for it. >>>>>>>>> >>>>>>>>> The following code snipped worked for me: >>>>>>>>> >>>>>>>>> <?xml version="1.0"?> >>>>>>>>> <tool_dependency> >>>>>>>>> <package name="protk" version="1.2.5"> >>>>>>>>> <install version="1.0"> >>>>>>>>> <actions> >>>>>>>>> <action type="setup_ruby_environment"> >>>>>>>>> <repository changeset_revision="a0494c6e1c63" >>>>>>>>> name="package_ruby_2_0" owner="bgruening" >>>>>>>>> toolshed="http://testtoolshed.g2.bx.psu.edu"> >>>>>>>>> <package name="ruby" version="2.0" /> >>>>>>>>> </repository> >>>>>>>>> <package>protk=1.2.6.pre1</package> >>>>>>>>> </action> >>>>>>>>> </actions> >>>>>>>>> </install> >>>>>>>>> <readme> >>>>>>>>> Installs version 1.2.5 of the protk rubygem from rubygems.org >>>>>>>>> </readme> >>>>>>>>> </package> >>>>>>>>> </tool_dependency> >>>>>>>>> >>>>>>>>> Keep in mind that the repository tag should be filled. If you have a >>>>>>>>> actions and action tag it will be filled automatically by the TS during >>>>>>>>> upload. But better to check :) >>>>>>>>> >>>>>>>>> To populate the RUBY_HOME env var ... do you think its better to set it >>>>>>>>> via the installation automatically? My idea was to just "require" ruby >>>>>>>>> in the tool? If you want to set it in your protk definition you need to >>>>>>>>> use the $ENV[] construct. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Bjoern >>>>>>>>> >>>>>>>>>> Hi Bjoern, >>>>>>>>>> >>>>>>>>>> Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. >>>>>>>>>> >>>>>>>>>> I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. >>>>>>>>>> >>>>>>>>>> Cheers >>>>>>>>>> Ira >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 29/10/2013, at 10:06 AM, Björn Grüning <bjoern.gruening(a)pharmazie.uni-freiburg.de> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ira, >>>>>>>>>>> >>>>>>>>>>> I can reproduce that error here, will try to study it tomorrow more >>>>>>>>>>> deeply. For the meantime can you try to remove the following lines? >>>>>>>>>>> >>>>>>>>>>> <action type="set_environment_for_install"> >>>>>>>>>>> <repository name="package_ruby_2_0" owner="bgruening"> >>>>>>>>>>> <package name="ruby" version="2.0" /> >>>>>>>>>>> </repository> >>>>>>>>>>> </action> >>>>>>>>>>> >>>>>>>>>>> It's not necessary. >>>>>>>>>>> Will get back to you tomorrow. >>>>>>>>>>> Bjoern >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the "Installing" phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but since the tool install never completes I can never see this. What I usually do is try to recreate the environment that galaxy sees during tool install in a normal shell .. and then run it manually .. which sometimes tells me the problem ... but not always (see below). >>>>>>>>>>>> >>>>>>>>>>>> The longer version of my problem is below. >>>>>>>>>>>> >>>>>>>>>>>> In my case I am struggling with the following tool >>>>>>>>>>>> >>>>>>>>>>>> http://testtoolshed.g2.bx.psu.edu/view/iracooke/package_protk_1_2_5 >>>>>>>>>>>> >>>>>>>>>>>> Which uses the new "setup_ruby_environment" tag ... so the tool should do the following >>>>>>>>>>>> >>>>>>>>>>>> (1) Install the dependency .. Ruby 2.0 ... which it does successfully >>>>>>>>>>>> (2) Install my protk rubygem from rubygems.org .. which it gets stuck doing. >>>>>>>>>>>> >>>>>>>>>>>> The last few lines from my galaxy server log during the failed install are; >>>>>>>>>>>> >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:26 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:30 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' >> /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/env.sh >>>>>>>>>>>> [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdQaLI7O >>>>>>>>>>>> ruby version 2.0 installed in /home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63 >>>>>>>>>>>> [localhost] local: touch /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'RUBYLIB=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/lib/:$RUBYLIB; export RUBYLIB' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'PATH=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin:$PATH; export PATH' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: echo 'GALAXY_RUBY_HOME=/home/vagrant/tool_dependencies/ruby/2.0/bgruening/package_ruby_2_0/a0494c6e1c63/bin; export GALAXY_RUBY_HOME' >> /home/vagrant/tool_dependencies/ruby/2.0/iracooke/package_protk_1_2_5/876e44dd4609/env.sh >>>>>>>>>>>> [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdJYyhO0 >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:34 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:47:22 +0000] "POST /admin_toolshed/manage_repositories HTTP/1.1" 302 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> 10.0.2.2 - - [28/Oct/2013:21:54:37 +0000] "GET /admin_toolshed/monitor_repository_installation?tool_shed_repository_ids=f597429621d6eb2b&tool_shed_repository_ids=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8088/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> With a normal tool install I would see the last line of this output repeat lots of times as installation is monitored ... but for some reason in this case monitoring just shuts down and I don't see any further updates. >>>>>>>>>>>> >>>>>>>>>>>> If I recreate the situation on the command-line (eg by manually setting the RUBYLIB and GEM_HOME environment variables appropriately I can install my gem without any errors. >>>>>>>>>>>> >>>>>>>>>>>> More details on my setup: >>>>>>>>>>>> >>>>>>>>>>>> I'm using a vagrant VM for testing which is based on Ubuntu precise64 ... with a minimal number of packages installed ... >>>>>>>>>>>> (My VM is documented in full here: https://bitbucket.org/iracooke/protk_vagrant/src/0ef8d86cc3dcec4ee3a9a83edf… ) >>>>>>>>>>>> >>>>>>>>>>>> I'm testing based on the current default branch of galaxy-central .... which I regularly tear down and re-clone from scratch ... always with the same result. >>>>>>>>>>>> >>>>>>>>>>>> Any help or pointers to how to resolve this much appreciated. >>>>>>>>>>>> >>>>>>>>>>>> Cheers >>>>>>>>>>>> Ira >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> <add_more_env_vars_to_ruby_install_environment.patch> >>> >> >> >> >

2 1