September 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy VM
by kauerbach＠comcast.net 14 Sep '12

14 Sep '12

Hello, Can anyone tell me if there is a Galaxy VM available for download that definitely contains the NCBI-BLAST tools? I have seen several download links for Galaxy VMs that include some tools, but it is not clear what tools precisely they contain, and if BLAST is one of them. Thank you.

1 0

Re: [galaxy-dev] Galaxy Deployers/Users: Web conference Wednesday, September 19th 10 am
by Ann Black-Ziegelbein 14 Sep '12

14 Sep '12

Just a quick followup. I was recently alerted that Blackboard Collaborate client does NOT support Java v7 and has some know issues when running with Java v7. Please launch the web start application with Java v6 to avoid complications. If you don't know what version of java you have, open up a command prompt or terminal on your client machine and type: java --fullversion This will show you the java version on your path. Thanks so much! Ann Ann Black-Ziegelbein Senior Application Developer University of Iowa Center for Bioinformatics and Computational Biology Iowa Institute for Human Genetics From: Ann Black-Ziegelbein <annblack(a)eng.uiowa.edu> Date: Wednesday, September 12, 2012 10:53 PM To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Cc: Oleksandr Moskalenko <om(a)hpc.ufl.edu>, "Maddhi, Srinivas" <srinivas-maddhi(a)uiowa.edu> Subject: Galaxy Deployers/Users: Web conference Wednesday, September 19th 10 am Thank you to everyone who helped provide doodle poll input for the second Galaxy Deployers/Users web conference! We have set the date: Date: Wednesday, September 19th Time: 10:00 AM Central Time (USA Chicago time zone) Web Conference (this will be recorded and posted for playback): We will be using a Blackboard Collaborate web conference. See the section below for more tips on how to use Blackboard Collaborate (Elluminate Live) if you have not used it before. https://globalcampus.uiowa.edu/join_meeting.html?meetingId=1262330108904 Agenda: * Name change! GalaxyCzars may not fit us right please bring your suggestions or email them ahead of time. We will provide a virtual toast to the winning suggestion! * Presentation: Alex Moskalenko will present on the University of Florida's Galaxy and experiences with migrating to the latest Galaxy release. * Open Mic: Bring your problems, your successes, and questions for discussion. Blackboard Collaborate (Ellumiate Live!) Tips: For our web conferences we will try out Blackboard Collaborate web conferencing software. You do not need any special client, just a web browser and java installed (the client will run as a java web start app). You don't need to use a headset, but if you have one it does improve your audio. I will try to open up the session 15 minutes early for people to join and try out the technology. A couple of things to mention, when you launch your session - you can tune your connection speed: Talking works like a walkie-talkie ... you have to select "Talk" (see capture below) to open up your audio line. There can only be 6 simultaneous talkers. Please only select the talk button when you need to actually talk to reduce background noise and to free up lines for others to participate. Thanks! Ann Black-Ziegelbein University of Iowa

1 0

Automatic installation of third party dependancies
by Lance Parsons 14 Sep '12

14 Sep '12

I've put together a tool wrapper for the htseq-count script that is part of the HTSeq python package and uploaded that to the tool shed. However, I have discovered that the tool dependancies do not install properly. There are a couple of issues that I've run into. 1) The biggest issue is the inability to set both the PYTHONPATH and PATH environment variables as part of installation. If I put two separate <action type="set_environment"> tags, then the second overwrites the first in the env.sh file. If instead, I put two <environment_variable> tags within the action, only one (the second) gets executed during installation. 2) Another issue is the inability to install Numpy as a dependancy to HTSeq. I can get Numpy to install, but it won't be in the PYTHONPATH when HTSeq is installed (or executed). Any thoughts on how to address these issues would be appreciated. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

2 4

Workflow Queue
by Edward Hills 13 Sep '12

13 Sep '12

Hi Galaxy team, I am wanting to know where to find the queue that stores all tool's that are going to be executed in the pipeline. For example, I am running a workflow of 5 tools all linked together (the details are not important). Programatically I wish to know the names of the tools that are going to be run. This is because I wish to perform some pre-run analysis on all the tools in the workflow before they get executed. Thanks, Edward Hills Department of Biochemistry, University of Otago, Dunedin, New Zealand.

2 1

Galaxy Deployers/Users: Web conference Wednesday, September 19th 10 am
by Ann Black-Ziegelbein 13 Sep '12

13 Sep '12

Thank you to everyone who helped provide doodle poll input for the second Galaxy Deployers/Users web conference! We have set the date: Date: Wednesday, September 19th Time: 10:00 AM Central Time (USA Chicago time zone) Web Conference (this will be recorded and posted for playback): We will be using a Blackboard Collaborate web conference. See the section below for more tips on how to use Blackboard Collaborate (Elluminate Live) if you have not used it before. https://globalcampus.uiowa.edu/join_meeting.html?meetingId=1262330108904 Agenda: * Name change! GalaxyCzars may not fit us right please bring your suggestions or email them ahead of time. We will provide a virtual toast to the winning suggestion! * Presentation: Alex Moskalenko will present on the University of Florida's Galaxy and experiences with migrating to the latest Galaxy release. * Open Mic: Bring your problems, your successes, and questions for discussion. Blackboard Collaborate (Ellumiate Live!) Tips: For our web conferences we will try out Blackboard Collaborate web conferencing software. You do not need any special client, just a web browser and java installed (the client will run as a java web start app). You don't need to use a headset, but if you have one it does improve your audio. I will try to open up the session 15 minutes early for people to join and try out the technology. A couple of things to mention, when you launch your session - you can tune your connection speed: Talking works like a walkie-talkie ... you have to select "Talk" (see capture below) to open up your audio line. There can only be 6 simultaneous talkers. Please only select the talk button when you need to actually talk to reduce background noise and to free up lines for others to participate. Thanks! Ann Black-Ziegelbein University of Iowa

1 0

Re: [galaxy-dev] environment variables and paths for toolshed tools
by Greg Von Kuster 13 Sep '12

13 Sep '12

Hi David, With the exception of requiring versions strings in <requirement> tags in the tool config in order to enable versioned tool dependency installation (which is documented in the tool shed wiki at http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_third-party_tool_dependency…) this is the first change to the way Galaxy tool configs should be written. I'll get a wiki put together for this new enhancement as soon as possible. I'm hoping to have it as part of the next Galaxy dist release when the feature is released. Greg Von Kuster On Sep 12, 2012, at 1:47 PM, David Hoover wrote: > On the topic of toolshed tools, is there any documented guidance about how to create a tool for the toolshed? There is some syntactical help on the wiki, but no comprehensive guide. > > On Sep 12, 2012, at 1:41 PM, Greg Von Kuster wrote: > >> It'll be available in Galaxy central sooner than Galaxy dist. I'll send you an email when I commit it. >> >> On Sep 12, 2012, at 1:31 PM, David Hoover wrote: >> >>> Will it be available through galaxy-dist or galaxy-central? I have started using the galaxy-central repository just to be on the bleeding edge. >>> >>> On Sep 12, 2012, at 1:15 PM, Greg Von Kuster wrote: >>> >>>> Hello David, >>>> >>>> This is not currently possible, but will be available in the next Galaxy distribution release, currently scheduled for about 10 days from now. You will include a tool_dependencies.xml file in your tool shed repository that looks something like the following. >>>> >>>> <?xml version="1.0"?> >>>> <tool_dependency> >>>> <set_environment version="1.0"> >>>> <environment_variable name="JAVA_JAR_PATH" action="set_to">$INSTALL_DIR</environment_variable> >>>> </set_environment> >>>> </tool_dependency> >>>> >>>> Your tool will find the required files via the defined JAVA_JAR_PATH by using a <requirements> tag set in the tool config, something like this: >>>> >>>> <requirements> >>>> <requirement type="set_environment">JAVA_JAR_PATH </requirement> >>>> </requirements> >>>> >>>> The <command> tag in the tool config would be something like this: >>>> >>>> <command interpreter="python"> >>>> python_wrapper.py $JAVA_JAR_PATH/some_file.jar $param1 $param2... >>>> </command> >>>> >>>> I'm close to having this working, so if you are interested in testing before the next Galaxy dist release, let me know and I'll tell you when the Galaxy central repository has the new feature. >>>> >>>> Thanks! >>>> >>>> Greg Von Kuster >>>> >>>> >>>> On Sep 12, 2012, at 1:00 PM, David Hoover wrote: >>>> >>>>> Are there any environment variables that are honored by toolshed installed tools? I tried creating a tool that uses ${GALAXY_DATA_INDEX_DIR} or $GALAXY_HOME, then uploaded it the test toolshed, then installed it automatically. Neither of these resolved to what I expected. I don't want to hard-code the path in the xml tool file, but rather have a default location for other executables and jar files. How should this best be done? >>>>> >>>>> David Hoover >>>>> Helix Systems Staff >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> >>> >>> > >

1 0

FreeBayes Installation Bug
by Lance Parsons 13 Sep '12

13 Sep '12

Today I ran into a new issues with installing a specific version of Freebayes. It turns out to be an issue with how git deals with submodules, something I'm only just learning about myself. In order to ensure that FreeBayes is built with the correct versions of all of the various submodules for the desired revision, it seems that two commands are needed: git checkout 9608597d12e127c847ae03aa03440ab63992fedf git submodule update --recursive In the current version of tool_dependencies.xml, there is a git reset --hard 9608597d12e127c847ae03aa03440ab63992fedf which seems to work similarly to the checkout command above, however, it does not put the submodules back at the correct revisions. That is only now an issue since they were updated only recently. I've attached a patch to the tool_dependencies.xml file that should once again allow FreeBayes to be built and installed automatically. It would be great if you could update the Tool Shed repo. Thanks. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

2 1

cufflinks and cuffdiff FPKM values
by suzan katie 13 Sep '12

13 Sep '12

Hello everyone, I am comparing two samples (control and treated) paired end RNA Seq data. I am getting different FPKM values on my cufflinks output of 2 samples when compared to the 2 values that cuffdiff.. Below is the snap shot of a gene with different FPKM values. (I am not displaying the gene name). Cufflinks output: Gene length coverage FPKM FPKM_conf_lo FPKM_conf_hi FPKM_status Sample xyz - - 39.9094 38.9553 40.8635 OK Control xyz - - 19.4664 18.7786 20.1542 OK Treated cuffdiff output: Gene sample_1 sample_2 status value_1/Control value_2/ treated log2(fold_change) test_stat p_value q_value significant xyz q1 q2 OK 59.6706 27.1551 -1.1358 5.06151 4.16E-07 4.00E-05 yes I am assuming that FPKM values from cufflinks and cuffdiff output should be matching. Any kind of suggestions are appreciated. Thanks

1 0

Re: [galaxy-dev] Automatic installation of third party dependancies
by Lance Parsons 13 Sep '12

13 Sep '12

I've updated my development system now, and when I try to get updates for that particular tool (htseq_count) I run into the following error. Any ideas on how I can/should fix this? Thanks. URL: http://galaxy-dev.princeton.edu/admin_toolshed/update_to_changeset_revision… File '/data/galaxy-dev/galaxy-dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/data/galaxy-dev/galaxy-dev/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/web/controllers/admin_toolshed.py', line 1469 in update_to_changeset_revision update_repository( repo, latest_ctx_rev ) File '/data/galaxy-dev/galaxy-dev/lib/galaxy/util/shed_util.py', line 1655 in update_repository rev=ctx_rev ) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/commands.py', line 5661 in update rev = scmutil.revsingle(repo, rev, rev).rev() File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/scmutil.py', line 515 in revsingle l = revrange(repo, [revspec]) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/scmutil.py', line 588 in revrange for r in m(repo, range(len(repo))): File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 1414 in mfunc return getset(repo, subset, tree) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 156 in getset return methods[x[0]](repo, subset, *x[1:]) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 171 in symbolset return stringset(repo, subset, x) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/revset.py', line 161 in stringset x = repo[x].rev() File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/localrepo.py', line 235 in __getitem__ return context.changectx(self, changeid) File '/data/galaxy-dev/galaxy-dev/eggs/mercurial-2.2.3-py2.6-linux-x86_64-ucs2.egg/mercurial/context.py', line 103 in __init__ _("unknown revision '%s'") % changeid) RepoLookupError: unknown revision '4' Lance Parsons wrote: > > Thanks Greg. I used you're updated version and added numpy as a > separate dependency. It seems to work in my development system. I > haven't updated my qa or production versions yet, so I can't check > there. Perhaps you could test it and let me know if there are issues, etc. > > Glad I can be of help. > > Lance > > Greg Von Kuster wrote: >> >> Hello Lance, >> >> See my inline comments. >> >> >> On Sep 4, 2012, at 3:15 PM, Lance Parsons wrote: >> >>> >>> I've put together a tool wrapper for the htseq-count script that is >>> part of the HTSeq python package and uploaded that to the tool >>> shed. However, I have discovered that the tool dependancies do not >>> install properly. There are a couple of issues that I've run into. >>> >>> 1) The biggest issue is the inability to set both the PYTHONPATH and >>> PATH environment variables as part of installation. If I put two >>> separate<action type="set_environment"> tags, then the second >>> overwrites the first in the env.sh file. If instead, I put >>> two<environment_variable> tags within the action, only one (the >>> second) gets executed during installation. >> >> >> The above issues have been corrected in changeset 7621:108cda898646, >> which is currently available only in the Galaxy central repository. >> There is a new Galaxy distribution scheduled to go out today, but >> unfortunately this changeset will not be included. From here on, >> Galaxy releases are tentatively scheduled for every 2 weeks, so if >> you don't want to pull from the Galaxy central repository, the fix >> should be available in the next release 2 weeks from now. >> >> I've slightly altered your tool_dependencies.xml file (mostly because >> I've implemented support for a new "make_directory" tag. It is >> attached below - you should updated your tool_dependencies.xml file >> in your htseq_count repository with this updated version. >> >> >>> >>> 2) Another issue is the inability to install Numpy as a dependancy >>> to HTSeq. I can get Numpy to install, but it won't be in the >>> PYTHONPATH when HTSeq is installed (or executed). >> >> >> Were you attempting to install numpy as another package defined in >> your tool-dependencies.xml file? If so, the fixes in the above >> changeset should allow you to do that now since you can install it >> before htseq and set PYTHONPATH to point to numpy as part of the >> installation. >> >> Let me know if you need additional help or bump into further >> problems. The tool dependency installation components are fairly >> immature, so adding enhancements for new tools like yours is very >> valuable. >> >> Thanks for your help on this, >> >> Greg Von Kuster >> >> >>> >>> Any thoughts on how to address these issues would be appreciated. >>> >>> -- >>> Lance Parsons - Scientific Programmer >>> 134 Carl C. Icahn Laboratory >>> Lewis-Sigler Institute for Integrative Genomics >>> Princeton University >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> > > > -- > Lance Parsons - Scientific Programmer > 134 Carl C. Icahn Laboratory > Lewis-Sigler Institute for Integrative Genomics > Princeton University > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/

1 0

Re: [galaxy-dev] Recovery of broken toolshed installs
by Greg Von Kuster 12 Sep '12

12 Sep '12

Hi Alex, Assuming the installed repository is stuck in a "Cloning" state (you can see this by clicking the Admin menu link "Manage installed tool shed repositories"), you can fix it by manually running the following sql command in your db environment (string case is required). update tool_shed_repository set status = 'New' where status = 'Cloning'; If the status is not 'Cloning', you may need to remove the directory where the repository was in the process of being cloned as well, and perhaps manually delete the any tool entries from you shed_tool_conf.xml file. This process is currently manual, but the next release of Galaxy (coming in about 10 days) will provide the ability to do this via a Galaxy Admin UI feature. Greg Von Kuster On Sep 11, 2012, at 11:48 PM, Oleksandr Moskalenko wrote: > Hi Greg, > > What's the best way to completely clean up a borked repository on disk and in the Galaxy database that failed to install for any odd reason, but hangs around causing Apache server errors [1] on any action afterwards? > > Thanks, > > Alex > > 1. "Server Error. An error occurred. See the error logs for more information. (Turn debug on to display exception reports here)"

1 0