- galaxy-dev - lists.galaxyproject.org

job not starting on cluster
by Langhorst, Brad 06 Aug '14

06 Aug '14

I’m not aware of any recent changes to the configuration of galaxy… It seems that jobs are not being distributed at the moment. I have 2 job handlers... It seems that when I restart them only a single job is queued, it has an error and then sticks on "Cleaning up external metadata files” forever. any ideas? I have 700+ jobs in the queue and would like to get them running asap... one hanlder just stays at Starting server in PID 4868. serving on http://127.0.0.1:8090 here’s the log from the other. Starting server in PID 4873. serving on http://127.0.0.1:8091 galaxy.jobs DEBUG 2014-08-05 17:34:08,388 (125206) Working directory for job is: /mnt/galaxy/data/galaxy/job_working_directory/000/125/125206 galaxy.jobs.handler DEBUG 2014-08-05 17:34:08,410 (125206) Dispatching to gridengine runner galaxy.jobs DEBUG 2014-08-05 17:34:08,479 (125206) Persisting job destination (destination id: gridengine) galaxy.jobs.handler INFO 2014-08-05 17:34:08,498 (125206) Job dispatched galaxy.tools.deps DEBUG 2014-08-05 17:34:08,862 Building dependency shell command for dependency 'bedtools2' galaxy.tools.deps WARNING 2014-08-05 17:34:08,865 Failed to resolve dependency on 'bedtools2', ignoring galaxy.jobs.runners DEBUG 2014-08-05 17:34:08,966 (125206) command is: bedtools --version > /mnt/galaxy/data/galaxy/tmp/GALAXY_VERSION_STRING_125206 2>&1; bedtools intersect -v -a /mnt/galaxy/data/galaxy/user-data/000/170/dataset_170995.dat -b /mnt/galaxy/data/galaxy/user-data/000/169/dataset_169046.dat > "/mnt/galaxy/data/galaxy/user-data/000/170/dataset_170999.dat"; return_code=$?; cd /mnt/galaxy/data/galaxy/galaxy-dist; /mnt/galaxy/data/galaxy/galaxy-dist/set_metadata.sh /mnt/galaxy/data/galaxy/user-data /mnt/galaxy/data/galaxy/job_working_directory/000/125/125206 . /mnt/galaxy/data/galaxy/galaxy-dist/universe_wsgi.ini /mnt/galaxy/data/galaxy/tmp/tmpLCBPvF /mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/galaxy.json /mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/metadata_in_HistoryDatasetAssociation_194826_qIy33T,/mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/metadata_kwds_HistoryDatasetAssociation_194826_SJhToq,/mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/metadata_out_HistoryDatasetAssociation_194826_4Lf956,/mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/metadata_results_HistoryDatasetAssociation_194826_YzEZK8,,/mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/metadata_override_HistoryDatasetAssociation_194826_19nZAm; sh -c "exit $return_code" galaxy.jobs.runners.drmaa DEBUG 2014-08-05 17:34:08,994 (125206) submitting file /mnt/galaxy/data/galaxy/job_working_directory/000/125/125206/galaxy_125206.sh galaxy.jobs.runners.drmaa INFO 2014-08-05 17:34:09,023 (125206) queued as 1539118 galaxy.jobs DEBUG 2014-08-05 17:34:09,048 (125206) Persisting job destination (destination id: gridengine) galaxy.jobs.runners.drmaa DEBUG 2014-08-05 17:34:09,451 (125206/1539118) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2014-08-05 17:34:11,510 (125206/1539118) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2014-08-05 17:34:31,208 (125206/1539118) state change: job finished normally galaxy.jobs DEBUG 2014-08-05 17:34:31,440 setting dataset state to ERROR galaxy.jobs INFO 2014-08-05 17:34:31,652 Collecting job metrics for <galaxy.model.Job object at 0x7f42e030f850> galaxy.jobs DEBUG 2014-08-05 17:34:31,664 job 125206 ended galaxy.datatypes.metadata DEBUG 2014-08-05 17:34:31,665 Cleaning up external metadata files -- Bradley W. Langhorst, Ph.D. Applications and Product Development Scientist

2 1

PHYLIP
by Philippe Moncuquet 05 Aug '14

05 Aug '14

Hi, I haven't been able to find any implementation of the PHYLIP tools into Galaxy. I know some datatype are part of the emboss datatypes but what about the tools ? I reckon I am not the only one looking for this so I fear I might have missed something, if this is the case could you please point in the right direction to look ? Regards, Philip

1 0

cufflinks suite 2.2.1
by Geert Vandeweyer 04 Aug '14

04 Aug '14

Hi all, I've updated the tool wrappers for the cufflinks tools to options supported by cufflinks 2.2.1. I've also added the cuffquant and cuffnorm tools, together with the cxb datatype. As discussed with Jeremy, these updates were welcome :-) Everything is available on the test toolshed, tested and installed on at least one instance (http://biomina.be/app/galaxy) although mixing main and test toolshed installations doesn't work well. The tools are present twice. Therefore, feel free to integrate the updated wrappers & tools into the main cufflinks package (by devteam) after further testing. Best, geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

start and end times?
by Evan Bollig 04 Aug '14

04 Aug '14

Jobs executed in the workflow appear to have a creation time, but no execution start and end time listed. How does one determine when something executed? I'd like to know exactly where the time is spent computing... -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan(a)msi.umn.edu boll0107(a)umn.edu

2 3

Re: [galaxy-dev] "HTTPerror : HTTP error 307 : temporary redirect"
by Perley, Danielle 04 Aug '14

04 Aug '14

Solved! After finding documentation indicating that changing the URL of the test toolshed from http to https in the tool_sheds_config.xml file solved some redirection issues, I tried the same thing for the URL of the main toolshed. Message: 7 Date: Fri, 1 Aug 2014 20:30:14 +0000 From: "Perley, Danielle" <danielle.perley(a)med.und.edu> To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: [galaxy-dev] Error while installing tools from main toolshed: "HTTPerror : HTTP error 307 : temporary redirect" Message-ID: <1406925112700.23132(a)med.und.edu> Content-Type: text/plain; charset="iso-8859-1" Hi, I'm trying to install tools from the main toolshed, but I keep getting the same error regardless of the tool that I'm installing HTTPError: HTTP Error 307: Temporary Redirect URL: http://localhost:8080/admin_toolshed/prepare_for_install?tool_shed_url=http… Module weberror.evalexception.middleware:364 in respond view >> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view >> body = method( trans, **kwargs ) Module galaxy.web.framework:377 in decorator view >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1116 in prepare_for_install view >> updating=updating ) Module tool_shed.util.common_install_util:110 in get_dependencies_for_repository view >> all_repo_info_dict = get_required_repo_info_dicts( trans, tool_shed_url, util.listify( repo_info_dict ) ) Module tool_shed.util.common_install_util:451 in get_required_repo_info_dicts view >> response = urllib2.urlopen( request ).read() Module urllib2:127 in urlopen view >> return _opener.open(url, data, timeout) Module urllib2:410 in open view >> response = meth(req, response) Module urllib2:523 in http_response view >> 'http', request, response, code, msg, hdrs) Module urllib2:442 in error view >> result = self._call_chain(*args) Module urllib2:382 in _call_chain view >> result = func(*args) Module urllib2:608 in http_error_302 view >> new = self.redirect_request(req, fp, code, msg, headers, newurl) Module urllib2:569 in redirect_request view >> raise HTTPError(req.get_full_url(), code, msg, headers, fp) HTTPError: HTTP Error 307: Temporary Redirect I've just installed Galaxy locally for the first time on a Mac OS 10.9.3. The only changes I've made to the universe_wsg.ini file are: setting host=0.0.0.0 (The error also occurred with host=127.0.0.1?) specifying a full path for Tool_depenendency_dir giving myself administrator privileges with admin_users=myemail any ideas? Danielle

1 0

Adding Reference Genomes, builds.txt vs manual_builds.txt
by Lance Parsons 04 Aug '14

04 Aug '14

I noticed that the instructions at https://wiki.galaxyproject.org/Admin/DataIntegration direct admins to update the builds.txt file when adding a new reference genome to local Galaxy instances. Up to this point I had been actually modifying the manual_builds.txt file and then rebuilding the builds.txt file using the updateucsc.sh script. This allows me to continue to use the updateuscs.sh script without overwriting my changes and also generates a BUILD.len file in the chrom/ directory. It does require some scripting since one needs to add the chromosome lengths to manual_builds.txt, but this is relatively straight forward using an indexed fasta file (.fai) or similar. Perhaps a larger issue is that the manual_builds.txt file is currently under source control in mercurial (instead of being generated by a .sample version). So I have two questions: 1) What would be "best practice" for adding a new genome? Updating builds.txt or manual_builds.txt? 2) Can the manual_builds.txt file be generated from a sample and moved out of source control? (I'll add a Trello card if this seems reasonable). -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

1 0

XLS TO CSV
by Mert Mehnur KIRKALI 04 Aug '14

04 Aug '14

Hello, How can i convert xls file to csv file on galaxy ? Is that possible ? Best Regards,Mert.

4 4

creating datatype repository : no module named binary
by Geert Vandeweyer 04 Aug '14

04 Aug '14

Hi all, I'm trying to create a new binary datatype in galaxy, but I'm running into troubles. The current installer is available at https://testtoolshed.g2.bx.psu.edu/view/geert-vandeweyer/cuffquant_datatype When I install the repository to a local server, I get the errors below (no module binary). Since I'm new to defining datatypes, any insights on what I'm doing wrong is welcome :-) Best, Geert tool_shed.util.repository_dependency_util DEBUG 2014-08-04 14:39:07,313 Building repository dependency relationships... 127.0.0.1 - - [04/Aug/2014:14:39:03 +0200] "POST /admin_toolshed/prepare_for_install HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=http…" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" 127.0.0.1 - - [04/Aug/2014:14:39:10 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,362 Loading datatypes from /home/gvandeweyer/galaxy-dist/galaxy-dist/database/tmp/tmp-toolshed-acalpdGx1tEP galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,363 Exception importing proprietary code file /home/gvandeweyer/galaxy-dist/shed_tools/testtoolshed.g2.bx.psu.edu/repos/geert-vandeweyer/cuffquant_datatype/f7801f8191e0/cuffquant_datatype/binary: No module named binary galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,363 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes datatype_class = getattr( module, datatype_class_name ) AttributeError: 'module' object has no attribute 'Cuffquant' galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,364 Error calling method Cuffquant from class <module 'galaxy.datatypes.binary' from '/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.pyc'>: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 349, in load_datatype_sniffers aclass = getattr( module, datatype_class_name )() AttributeError: 'module' object has no attribute 'Cuffquant' -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

Patch for libxslt and libxml packages
by Jan Kanis 04 Aug '14

04 Aug '14

Hi all, I was told that updates to packages owned by devteam should just be sent here, since there is no public repo. I have patches for the libxml2 and libxslt tool_dependency.xml's, that set some more environment variables that my tool needs (specifically, the python lxml package needs some of these). Can they be applied to the official packages? Jan

1 0

August 2014 Galaxy Newsletter
by Dave Clements 02 Aug '14

02 Aug '14

Hello all, The August 2014 Galaxy Update Newsletter <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08> is out. Here's a preview of what's inside: - GCC2015 will be held in Norwich, United Kingdom, 6-8 July 2015 <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#GCC2015:_Norwich.2C_Un…> - GCC2014 was a tremendous success <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#GCC2014_Report>. Watch the talks, review the slides, posters, and BoF writeups. - ISMB and BOSC 2014 slides, posters and videos <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#Galaxy_.40_ISMB_and_BO…> too. - 53 new papers <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#New_Papers> - One new public server (IM-PET) <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#New_Public_Servers> - Who's hiring? <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#Who.27s_Hiring> It turns out that lots of places are. Happy August, Dave Clements and the Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0