May 2013 - galaxy-dev - lists.galaxyproject.org

Next GalaxyAdmins Meetup: May 15; Galaxy @ Pathogen Portal
by Dave Clements 14 May '13

14 May '13

Hello all, The next meeting<http://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2013_05_15> of the GalaxyAdmins Group<http://wiki.galaxyproject.org/Community/GalaxyAdmins> will be held on May 15, 2013, at 10 AM Central US time<http://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2013_05_15> . Andrew Warren of the Cyberinfrastructure Division<http://www.vbi.vt.edu/faculty/group_overview/Cyberinfrastructure_Division> of the Virginia Bioinformatics Institute <https://www.vbi.vt.edu/> at Virginia Tech will talk about their Galaxy deployment<http://rnaseq.pathogenportal.org/> at Pathogen Portal <http://pathogenportal.org/>, a highly customized Galaxy installation, and also about the group's objectives and future plans. Dannon Baker <http://wiki.galaxyproject.org/DannonBaker> will bring the group up to speed on what's happening in the Galaxy project. Date May 15, 2013 Time 10 am Central US Time (-5 GMT) Presentations *Galaxy <http://rnaseq.pathogenportal.org/> at Pathogen Portal<http://pathogenportal.org/> * Andrew Warren, Virginia Bioinformatics Institute <https://www.vbi.vt.edu/>, Virginia Tech *Galaxy Project Update* Dannon Baker <http://wiki.galaxyproject.org/DannonBaker> Links Meetup Link<https://globalcampus.uiowa.edu/join_meeting.html?meetingId=1262346908659> Add to calendar<https://globalcampus.uiowa.edu/build_calendar.event?meetingId=1262346908659> We use the Blackboard Collaborate Web Conferencing system<http://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/WebinarTech> for the meetup. Downloading the required applets in advance and using a headphone with microphone to prevent audio feedback during the call is recommended. GalaxyAdmins <http://wiki.galaxyproject.org/Community/GalaxyAdmins> is a discussion group for Galaxy community members who are responsible for large Galaxy installations. Thanks, Dave Clements -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 1

tool shed repository update bug
by Björn Grüning 14 May '13

14 May '13

Hi Greg, during the addition of the next dependency (libeigen) I found an other bug related to repository updates. Please see revision 6c8f72ee4a51 of confab in the testtoolshed. I installed that and want to update to revision aac0c82ac354 with new dependency. I saw that a new metadata is associated, but I'm not able to update: 'The installed repository named 'confab' is current, there are no updates available.' Cheers, Bjoern

2 3

lists.bx.psu.edu is down
by Peter Cock 14 May '13

14 May '13

Dear all, Something seems to have happened to the lists.bx.psu.edu server recently - which is unfortunate as I've made a habit of using this for linking to past email threads via the nice mailman archive listing which was here: http://lists.bx.psu.edu/pipermail/galaxy-dev/ Was this deliberate or does someone need to kick a server? ;) Thanks, Peter

2 1

Suggestion for improved feedback on failing tests
by Ira Cooke 14 May '13

14 May '13

Hi All, My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem One potential improvement that I think could make it easier to debug this situation would be to show test status for repositories even when they contain no tools (the test would simply attempt an install and show the installation log under the test details). This would be particularly useful for repositories that exist purely to install a dependency eg http://testtoolshed.g2.bx.psu.edu/view/iracooke/galaxy_protk Naturally a tool with proper functional tests would fail if its dependency installations fail .. but it would be a huge help in narrowing down the issue to be able to see where those failures occurred. Another point is that this is pretty much the only way of getting feedback on why a tool might fail installation on the build-bot without having a perfect clone of the build-bot locally. Cheers Ira

3 4

Track Job Runtime
by Geert Vandeweyer 14 May '13

14 May '13

Hi, Are there options available to track the actual runtime of jobs on a cluster and store them in the database? Or are there fields in the database that approximate the job execution duration? This might be useful for fine-grained wall-time estimation in a crowded cluster environment. What I'd like to do is fetch an average runtime / mb of input data for a specific tool from the database, and than use this for wall-time estimation of new jobs in a dynamic job runner script. Has this been done before? Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726

4 5

history page not opening
by sridhar srinivasan 13 May '13

13 May '13

Dear Developers, After configuring the galaxy.. i could'nt see the history column in the web page.. it shows the error as "Content Encoding Error The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression." Best Sridhar

2 2

Re: [galaxy-dev] [galaxy-iuc] Problems running functional tests
by Ira Cooke 13 May '13

13 May '13

Yes I agree that gnuplot should be removed from the list ... to tell the truth I think my dependency should build without it. I'm also wondering if the other packages have been added to the build-bot yet? My tool comes up as failing tests ... but no test result is shown, and last I saw it was failing due to a missing package dependency. Ira On 14/05/2013, at 12:12 AM, Nate Coraor <nate(a)bx.psu.edu> wrote: > On May 10, 2013, at 10:16 AM, Dave Bouvier wrote: > >> Ira, >> >> I've created a trello card (https://trello.com/c/7VTlX9rD) with the current list of recommended packages for tool dependency installation, based on this email and the galaxy-iuc thread. If there are further packages that are determined to be required, let me know and I'll update the list. >> >> --Dave B. >> >> On 5/10/13 07:02:50.000, Ira Cooke wrote: >>> Hi Dave, >>> >>> Thanks for the info and workaround. Actually now when I look at my test results on the test toolshed I can see that the error is most likely due to one of my dependencies failing installation. My guess is that one of the system packages required by my dependency is not installed on the build-bot. >>> >>> I'm not sure how best to approach this ... perhaps I need a setup similar to the build-bot myself for testing. I'm not sure how far you've got towards deciding on the package lists you would have on the build-bot ... but FYI the packages I need on a fresh install of ubuntu are below. >>> >>> build-essential mercurial libreadline6-dev zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev autoconf libgdbm-dev libncurses5-dev automake libtool bison pkg-config libffi-dev subversion libbz2-dev swig expat libpng12-dev gawk gnuplot libperl-dev libfuse-dev libcurl4-openssl-dev libxml2-dev libgd2-xpm-dev liblocal-lib-perl cpanminus unzip > > Just a thought, but some of these may be best to place in the toolshed. The version control systems and build tools are fine, as are (probably) the -dev versions of core system libraries like png, ssl, zlib, etc. But gnuplot should be a repository. > > --nate > >>> >>> Cheers >>> Ira >>> >>> >>> On 10/05/2013, at 12:51 AM, Dave Bouvier <dave(a)bx.psu.edu> wrote: >>> >>>> This only affects tests that are run manually, the automated testing framework correctly resolves tool dependencies. >>>> >>>> --Dave B. >>>> >>>> On 5/9/13 10:41:22.000, Peter Cock wrote: >>>>>> On 5/8/13 20:08:08.000, Ira Cooke wrote: >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I'm starting to add functional tests to my tools but I've found that I >>>>>>> can't get the test framework to work well with tool dependencies. >>>>>>> >>>>>>> The tool I'm considering is xtandem >>>>>>> >>>>>>> http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem >>>>>>> >>>>>>> When I install this tool on my local galaxy instance (on a clean install) >>>>>>> the tool runs fine and if I look in the handler logs I see >>>>>>> >>>>>>> executing: >>>>>>> PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; >>>>>>> export PACKAGE_BASE; . >>>>>>> /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; >>>>>>> PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; >>>>>>> export PACKAGE_BASE; . >>>>>>> /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; >>>>>>> rvm 1.9.3(a)protk-1.2.0 do tandem_search.rb -d >>>>>>> /home/iracooke/galaxy-central/database/files/000/dataset_1.dat >>>>>>> --var-mods=' 57.021464__at__C ' --fix-mods=' >>>>>>> 15.994915__at__M ' >>>>>>> /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o >>>>>>> /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r >>>>>>> --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 >>>>>>> --strict-monoisotopic-mass --keep-params-files >>>>>>> galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination >>>>>>> (destination id: local:///) >>>>>>> >>>>>>> >>>>>>> Note that in this case the tool environments from my tool dependencies are >>>>>>> loaded ... and the tool runs fine. >>>>>>> >>>>>>> When running functional tests however the tests are executed without first >>>>>>> loading the environments from tool dependencies ... and naturally the tool >>>>>>> fails. On my local machine I get the following when running the tests (Note >>>>>>> that no loading of dependency shel environments occurs). (This seems to be >>>>>>> the same issue on the testtoolshed.g2.bx.psu.edu ). >>>>>>> >>>>>>> Job dispatched >>>>>>> galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm >>>>>>> 1.9.3(a)protk-1.2.0 do tandem_search.rb -d >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' >>>>>>> 15.994915__at__M , ' --fix-mods=' >>>>>>> 57.021464__at__C ' >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin >>>>>>> --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 >>>>>>> --strict-monoisotopic-mass --keep-params-files >>>>>>> galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination >>>>>>> (destination id: local:///) >>>>>>> galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution >>>>>>> finished: rvm 1.9.3(a)protk-1.2.0 do tandem_search.rb -d >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' >>>>>>> 15.994915__at__M ,' --fix-mods=' >>>>>>> 57.021464__at__C ' >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o >>>>>>> /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin >>>>>>> --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 >>>>>>> --strict-monoisotopic-mass --keep-params-files >>>>>>> ... >>>>>>> galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR >>>>>>> >>>>>>> >>>>>>> Is there something wrong with the way I've defined my tool dependencies >>>>>>> for this tool? Or perhaps it's a bug in the testing framework. >>>>>>> >>>>>>> Cheers >>>>>>> Ira >>>>> >>>>> On Thu, May 9, 2013 at 3:10 PM, Dave Bouvier <dave(a)bx.psu.edu> wrote: >>>>>> Ira, >>>>>> >>>>>> There is a known issue with resolving tool dependencies when running >>>>>> functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR >>>>>> environment variable to the path configured in your universe_wsgi.ini. >>>>>> >>>>>> This issue is documented in https://trello.com/c/eVT3QWJF >>>>>> >>>>>> --Dave B. >>>>> >>>>> Does this issue affect the Tool Shed tests as well, or just when run >>>>> via run_functional_test.sh? (I'm working though a couple of test >>>>> failures related to the dependency installation breaking somehow). >>>>> >>>>> Thanks, >>>>> >>>>> Peter >>>>> >>> >>> >> _______________________________________________ >> galaxy-iuc mailing list >> galaxy-iuc(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-iuc >

1 0

Script to help maintain toolshed repos across toolsheds
by Ira Cooke 13 May '13

13 May '13

Hi all, I've written a script to help deal with the problem of maintaining toolshed tools across multiple toolsheds (eg test and release) The problem I encountered was that switching between test and production versions of a suite of tools can be quite painful because every repository definition like this <repository toolshed="http://toolshed.g2.bx.psu.edu" name="proteomics_datatypes" owner="iracooke" changeset_revision="463328a6967f"/> needs to be updated to a different toolshed url and (by extension) a different changeset revision. The idea with this script is that you should be able to point it at a directory containing a toolshed repository and it will create a copy of that repository in which the toolshed urls (and changeset revisions) have been updated to correct values for a different toolshed. I'm not sure how others are dealing with this issue (perhaps there is another easier way) .. but I've found this helped me alot so I thought I'd share https://bitbucket.org/iracooke/galaxy_repo_bundler/ Cheers Ira

4 16

Function for deleting specific datasets from the history
by Aristos Aristodimou 13 May '13

13 May '13

Dear dev, I have two questions 1) Is there a function that I can use to delete a dataset from the history? I have a tool that uses as input a hidden dataset and I want to delete the hidden dataset once the tool is executed. I have the filename of the dataset and I was wondering if there is a function that I can use for deleting it. 2) When a user deletes a dataset is there a way to get the filename of the deleted dataset? Thanks, Aris

2 1

How is the paster.py running to start Galaxy Framework ?
by 王聃Don 13 May '13

13 May '13

Hello, Galaxy developers: I'm a galaxy user and have customized a Galaxy instance locally. I'm very interested in Galaxy framework and the way it's running. I've search over the doc from readthedocs and look through some source code files. But I'm confused at first: How to run paster.py to start all process to build a Galaxy server? I've no idea why the last line of code "command.run()" can start the whole Galaxy server and thanks for someone to point it out. Thanks for your help! wangdan

2 1