June 2012 - galaxy-dev - lists.galaxyproject.org

human genome variation tool error
by Shantanu Pavgi 08 Jun '12

08 Jun '12

I am getting following errors in the galaxy log: {{{ Error reading tool from path: human_genome_variation/sift.xml Traceback (most recent call last): File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/tools/__init__.py", line 315, in load_tool_tag_set tool = self.load_tool( os.path.join( tool_path, path ), guid=guid ) File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/tools/__init__.py", line 420, in load_tool tree = util.parse_xml( config_file ) File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/util/__init__.py", line 105, in parse_xml tree = ElementTree.parse(fname) File "/home/shantanu/galaxy/galaxy-uab/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/home/shantanu/galaxy/galaxy-uab/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") IOError: [Errno 2] No such file or directory: './tools/human_genome_variation/sift.xml' galaxy.tools ERROR 2012-06-04 09:57:18,028 Error reading tool from path: human_genome_variation/linkToGProfile.xml Traceback (most recent call last): File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/tools/__init__.py", line 315, in load_tool_tag_set tool = self.load_tool( os.path.join( tool_path, path ), guid=guid ) File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/tools/__init__.py", line 420, in load_tool tree = util.parse_xml( config_file ) File "/home/shantanu/galaxy/galaxy-uab/lib/galaxy/util/__init__.py", line 105, in parse_xml tree = ElementTree.parse(fname) File "/home/shantanu/galaxy/galaxy-uab/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/home/shantanu/galaxy/galaxy-uab/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") }}} This tool was renamed or moved in the latest galaxy-dist update. So I think the error should get resolved after removing corresponding 'hgv' entry from tool_conf.xml file. Is that correct way to resolve it or do I need to perform any other/additional steps? -- Thanks, Shantanu

2 1

Implement scaling and load balancing galaxy on a 8-core host with cluster backend
by Derrick Lin 08 Jun '12

08 Jun '12

Hi guys, Today my institute ran a galaxy training with about 20 people. We didn't implement multiple instances for galaxy python. When everyone started submitting jobs, the thread pool filled up very quickly, soon after the galaxy threw worker queue full error then became unresponsive. It had to be restarted. So we decide to implement the multiple instances overnight in prepare for 2nd day of the training. Our setup is a 8-core host runs the galaxy server itself, and a 500-core cluster handles all the jobs. So I am wondering how I should distribute 8 cores for different roles (web, manager, job handler). In the wiki, the author said he runs six web server processes on 8 CPUs server, I assume he only runs one manager and one job handler. Is one job handler is more than enough? Even for the public galaxy? Second question is, it makes sense to have all web roles listen on a public IP (with different ports). For manager, job handler, can I just set them to listen 127.0.0.1? Or they have to be listening the same IP as the web roles? Regards, Derrick

2 1

Galaxy update / -- reload option
by Sarah Maman 08 Jun '12

08 Jun '12

Hello Nate, As agreed at the conference in Lyon, here's a screen printed my instance of Galaxy (please look at the end of this mail), but the layout is not exactly the same as that seen in the instance of Galaxy on the public server. To further explain my approach, here are the steps of my local installation: 1 In January, I've installed a local instance of Galaxy (connected to our cluster) from a source repository: tarball availbale here : http://dist.g2.bx.psu.edu/galaxy-dist.b258de1e6cea.tar.gz 2 In April / May, I've installed Mercurial, recovered sources with hg clone (Date: Wed March 07 2012, changeset: 6799:40 f1816d6857) , then updated galaxy sources with all changes and settings made between January and March. Therefore, the configuration file universe.ini of March was more complete than universe file of January. Can I just make a tkdiff of these two configuration files (January to March) and copy / paste the code added? Moreover, on my instance, if I use the -- reload option with run.sh script, port 80 is no longer available. So, I must kill the python process underway before running the script run.sh. Therefore, the service is interrupted for users.... Do you have an idea to avoid this interruption ? Concerning bug reports, is it possible to configure my local instance of Galaxy so that the mail 'report bug' is sent to the administrator (defines in galaxy configuration file universe) instead of being sent to the Galaxy team? Thank you in advance, Sarah Maman

2 1

tool_dependency_dir not being picked up?
by Ann Black-Ziegelbein 08 Jun '12

08 Jun '12

Hi - I have been trying to run a quick test using the tool_dependency_dir configuration option to prove out how it works ... but unfortunately I can't get it to function right. I was hoping someone could point out my error. I have written a simple tool in galaxy, version_test, that will just echo out a version number to a text file so I can test out tool dependencies and install locations. But my required package is not getting resolved by galaxy.... Snippet from my configuration file: # Directory which contains dependent tool binaries or a env.sh to set env vars in order to find specific versions. tool_dependency_dir = /opt My simple tool: <tool id="version_test" name="version_test" version="1.0.0"> <requirements> <requirement type="package">galaxy_test</requirement> </requirements> <description>provides simple stats on BAM files</description> <command>test.sh "$output1" </command> <inputs> <param name="input1" type="data" format="tabular,binary" label="Holder for input file - not used." /> </inputs> <outputs> <data name="output1" format="txt" /> </outputs> </tool> My filesystem: [galaxy@galaxy-0-4:galaxy-dist]$ ls -lat /opt/galaxy_test/ total 20 drwxr-xr-x 3 root root 4096 May 16 14:10 2.0 drwxr-xr-x 3 root root 4096 May 16 14:09 1.0 drwxr-xr-x 4 root root 4096 May 16 14:05 . -rw-r--r-- 1 root root 50 May 16 14:05 env.sh drwxr-xr-x 72 root root 4096 May 16 14:04 .. My env file: #!/bin/bash PATH=/opt/galaxy_test/1.0/bin:$PATH export PATH Galaxy log messages when invoking my test tool: galaxy.tools DEBUG 2012-05-31 21:59:17,349 Dependency galaxy_test galaxy.tools WARNING 2012-05-31 21:59:17,349 Failed to resolve dependency on 'galaxy_test', ignoring If I manually source the env.sh file , my test.sh is found and executes as appropriately. Where am I going wrong? Thanks much! Ann

2 1

Error starting Galaxy after upgrade.
by Luobin Yang 08 Jun '12

08 Jun '12

Hi, I upgraded Galaxy to the latest version using the command: "hg pull -u", then when I run "sh run.sh" I got the following error messages: Initializing openid_conf.xml from openid_conf.xml.sample Initializing tool-data/bowtie2_indices.loc from bowtie2_indices.loc.sample Some eggs are out of date, attempting to fetch... Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 30, in <module> c.resolve() # Only fetch eggs required by the config File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 546, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: mercurial==2.1.2 Fetch failed. What happened here? Thanks, Luobin

2 1

Toolshed error with proprietary datatypes
by Björn Grüning 08 Jun '12

08 Jun '12

Hi, has anyone encountered that problem or has an idea how to solve it? Traceback (most recent call last): File "/home/ctb/galaxy-dist/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/home/ctb/galaxy-dist/lib/galaxy/app.py", line 64, in __init__ self.installed_repository_manager.load_proprietary_datatypes() File "/home/ctb/galaxy-dist/lib/galaxy/tool_shed/__init__.py", line 26, in load_proprietary_datatypes load_datatype_items( self.app, tool_shed_repository, relative_install_dir ) File "/home/ctb/galaxy-dist/lib/galaxy/util/shed_util.py", line 1036, in load_datatype_items app.datatypes_registry.load_datatype_converters( app.toolbox, installed_repository_dict=repository_dict, deactivate=deactivate ) AttributeError: 'UniverseApplication' object has no attribute 'toolbox' All files are in one repository ... xml, the python-class und the converters. I was able to successfully upload it to our test-toolshed but after installing it (also successfully) galaxy is not able to start. Also the converters are recognised as tools in the toolshed is that intended? Thanks! Bjoern

2 7

input and output extensions
by Nicholas Robinson 08 Jun '12

08 Jun '12

Hi, 1. INPUT I am trying to make a simple tool that sends a command to run crimap without a wrapper ie. ............. <command> crimap $param_file $option >$output1 </command> .............. However, crimap expects a .par file as input (ie. $param_file should be a .par file). Because of this I get the error: ERROR: Parameter file '/home/galaxy/galaxy-dist/database/job_working_directory/001/1434/tmpe5hErw' does not have '.par' suffix Is there a simple way around this (using more lines in the command)? If I need to create a wrapper script, does somebody have an example that could help me to understand how it should be created? I am not familiar with python or perl and have written my other tools with R scripts embedded in the xml file ie. <command interpreter="sh">r_wrapper.sh $script_file</command> <configfiles> <configfile name="script_file"> etc etc </configfile> </configfiles> 2. OUTPUT This is a similar issue. This time I am trying to run another program called "crigen" without a wrapper as follows: ................ <command> crigen -g $myData_gen -o $crimap_number -size $num_individ -gen $num_gen >$output1 </command> <inputs> <param name="label" type="text" value="name" size="30" label="short title for run"> </param> <param format="tabular" name="myData_gen" type="data" label="Choose .gen file" help="" > </param> <param name="crimap_number" type="integer" value="1" min="1" max="100000" label="number to identify output"> </param> <param name="num_individ" type="integer" value="50" min="1" max="10000" label="Number of individuals per family" help="limit of 1 to 10000"> </param> <param name="num_gen" type="integer" value="3" min="1" max="20" label="Number of generations" help="limit of 1 to 20"> </param> </inputs> <outputs> <data format="tabular" name="output1" label="crigen output ${label}" /> </outputs> In this case you need to use a number to identify the output in crigen. Running the tool is successful, but I get an empty output. Appreciate any wrapper examples that deal with this sort of requirement too. Thanks and cheers, Nick

2 1

PBS error 15023: Bad user - no password entry
by Marc Bras 08 Jun '12

08 Jun '12

Hi, When I try to use pbs job runners, I get this message in the red box: Unable to run this job due to a cluster error, please retry it later And I've got this message in my nohup: galaxy.jobs.runners.pbs DEBUG 2012-06-08 10:12:12,030 (453) pbs_submit failed, PBS error 15023: Bad user - no password entry I use a galaxy user to launch my Galaxy server. I already done all steps described in "Unified Method" like this command: sudo usermod -s /bin/bash galaxy Please, I need help !! Thanks a lot, Marc BRAS Fondation Imagine 156, rue Vaugirard 75015 PARIS

1 0

populating history with input files automatically from remote site?
by Nikhil Joshi 08 Jun '12

08 Jun '12

Hi all, I have a galaxy instance running in the Amazon Cloud that I have customized and I want to be able to transfer files (via scp) to the instance (or its attached storage) and then have those files show up automatically in the history. Is this possible? Alternatively, is there some way to automatically populate the history of a registered user, so that when that user logs in, the files are automatically in a saved history? A third possibility would be to transfer the files and restart galaxy and then have the files show up automatically? Just to be clear, I do not want to use the Upload Files tool at all. Is there any way to implement any of these options? Any help would be highly appreciated. Thanks! - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 1

Galaxy Test Not Producing Output
by Kipper Fletez-Brant 08 Jun '12

08 Jun '12

Hi List, I'm trying to write a test for Galaxy and it fails on the result of the test differing from the expected output. As the output of diff (below) lists nothing but local_file data, I'm assuming that the issue is that no output is being generated by the test. The test config is a series of passed values, with the exception of one parameter, which is actually an entry pulled from a loc file directing my tool to the location of the file on disk (i.e. it's a identifier value). I'm not really sure what's happening, but I'd appreciate any help or pointers. Thanks, Kipper --- local_file +++ history_data @@ -1,100 +1,100 @@ -chr10 4522044 4522438 -chr10 5650896 5651310 -chr10 5772258 5772663 -chr10 9135867 9136467 -chr10 9946900 9947273 -chr10 10793689 10794313 -chr10 15790023 15790478 -chr10 16048435 16049028 -chr10 17620155 17620367 -chr10 21718404 21719020 -chr10 22967169 22967533 -chr10 22999383 22999802 -chr10 24511845 24512748 -chr10 24773636 24774006 -chr10 25975943 25976389 -chr10 26620032 26620396 -chr10 26907992 26908408 -chr10 28309833 28310034 -chr10 30908407 30909220 -chr10 31184776 31185151 -chr10 31191717 31192170 -chr10 31651098 31651551 -chr10 32422502 32422880 -chr10 33316453 33316817 -chr10 34353700 34354168 -chr10 34570612 34571026 -chr10 34853178 34853542 -chr10 34857471 34857818 -chr10 34994111 34994269 -chr10 38557788 38558691 -chr10 43296854 43297454 -chr10 44159102 44159480 -chr10 45208753 45209123 -chr10 45246414 45246794 -chr10 46861707 46862080 -chr10 47021051 47021457 -chr10 47716908 47717259

1 0