March 2012 - galaxy-dev - lists.galaxyproject.org

tool_conf.xml.main vs tool_conf.xml.sample
by Anne Pajon 27 Mar '12

27 Mar '12

Hi, I would like to know which tool_conf.xml to use, and would like to know the difference between the main and the sample one. I know that the functional tests are running against the tool_conf.xml.sample but what tool_conf.xml.main is used for? Thanks. Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

3 5

an error in geneTrack
by Liusong Yang 27 Mar '12

27 Mar '12

Hello, I got this error when I am using the link of 'View in geneTrack' for a bed file in current history. I am wondering if anyone know it? Thanks. URL: http://128.146.132.147:1234/display_application/9005c5112febe774/genetrack_… File '/home/galaxy/galaxy-dist/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/controllers/dataset.py', line 829 in display_application display_link = display_app.get_link( link_name, data, dataset_hash, user_hash, trans ) File '/home/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/application.py', line 185 in get_link return PopulatedDisplayApplicationLink( self.links[ link_name ], data, dataset_hash, user_hash, trans ) File '/home/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/application.py', line 125 in __init__ self.ready, self.parameters = self.link.build_parameter_dict( self.data, self.dataset_hash, self.user_hash, trans ) File '/home/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/application.py', line 66 in build_parameter_dict if param.ready( other_values ): File '/home/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/parameters.py', line 119 in ready raise Exception( 'A data display parameter is in the error state: %s' % ( self.name ) ) Exception: A data display parameter is in the error state: genetrack_file Lisa

2 1

Galaxy crashing when downloading large files
by Brian Luerman 27 Mar '12

27 Mar '12

I have a custom module that gets BAM files via HTTP web services. It appears to work perfectly for smaller BAM files, but when I jump up to the 10's of gigs, Galaxy displays some aggravating behavior. I am able to get the data just fine and wholly intact. However, when the transfer is done, Galaxy does not appear to recognize it. It just continues to run the download job, not realizing the file is done. If I try and do anything with the job still in progress (View in browser or view details), Galaxy seems to crash and I see the following errors: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /galaxy/datasets/4ff6f47412c3e65e/display/. Reason: Error reading from remote server -And- python26: ./Modules/cStringIO.c:419: O_cwrite: Assertion `oself->pos + l < 2147483647' failed. run.sh: line 49: 8801 Aborted python26 ./scripts/paster.py serve universe_wsgi.ini $@ And then finally when I try and refresh Galaxy: Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. I have to reboot it in order for it to work again. One time (and I have not been able to replicate this), the file, after causing Galaxy to crash, showed up in my Datasets history, intact. I can see each file I've attempted to download in the file system, but only the one time have I had access to it in Galaxy. Anyone have any ideas what's going on and how to fix this? Thanks! Brian Luerman Intrepid Bioinformatics (502) 212-2699 (ext 3) http://intrepidbio.com/

1 0

Re: [galaxy-dev] functional test issue - PDF output file comparison
by Anne Pajon 27 Mar '12

27 Mar '12

Hi Greg, I did not get any answer unfortunately. I still do have errors when running the functional tests and I do not know how the tool shed will solve the problem as Nate replied to a related email regarding functional test failures. As far as I am aware, the most used tool for comparing the test outputs is 'diff'. It is perfectly fine when comparing text file but there are plenty of tools that produce PDF files, and as you mention they differ a lot even when generated under identical circumstances. However, to temporarily solve these differences, I've generated manually these files using the export GALAXY_TEST_SAVE=/data_ls/galaxy/test-data ran the test, visually compared the files and copied them back to test-data/ directory. Not very elegant and time consuming. Anyone with better solution? Cheers, Anne. On 26 Mar 2012, at 08:27, Greg Edwards wrote: > Hi Anne, > > Just wondering if you got any response to your question, I'm interested to know the answer myself. > > I'm not a PDF expert, but have done some work over the years with PDFs, PostScript, fonts etc. Even a small thing like the date/time generated in the PDF could make a diff fail, > or the encoding of the system info it was generated on. Font library info gets included that can differ across systems. > Plus diff thinks there is binary data in the PDF, which there can be, so that will cause diff to have hiccups. > > I printed two identical PDF's, just a few chars, on my Mac and they differ a lot, even though created a few seconds apart under identical circumstances. > > I don;t think 'diff' is much use to do a comparison in the functional tests, I'm surprised that they use that. Unless Galaxy has a cut down, simplified PDF generator that doesn;t encode any variable info, which I doubt. > > Cheers, > Greg E > > > > > > On Tue, Mar 6, 2012 at 4:35 AM, Anne Pajon <Anne.Pajon(a)cancer.org.uk> wrote: > Dear All, > > I have noticed that some output files of functional tests are of type PDF. When comparing the one produced by the tool and the one stored in test-data/, they obviously differ even if they visually look identical. > > Is there a better way to compare PDF files within Galaxy than using diff? If not, do you have experience or suggestion at solving these functional test failures? > > Thanks a lot in advance. > Kind regards, > Anne. > -- > Anne Pajon, Ph.D. > Cancer Research UK - Cambridge Research Institute > Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE > anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 > > > > NOTICE AND DISCLAIMER > This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. > > We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. > Cancer Research UK > Registered in England and Wales > Company Registered Number: 4325234. > Registered Charity Number: 1089464 and Scotland SC041666 > Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > -- > Greg Edwards, > Port Jackson Bioinformatics > gedwards2(a)gmail.com > -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

1 0

functional test failures
by Anne Pajon 27 Mar '12

27 Mar '12

Hello, I am trying to have all functional tests on our production server (galaxy-dist 6621:26920e20157f) run successfully but I do still have failures that I do not know how to solve. Theses failures are related to unexpected differences probably due to versions of software that have been upgraded recently. ( Extract genomic DNA 1 ) > Test-1 ( Extract genomic DNA 1 ) > Test-2 ( Extract genomic DNA 1 ) > Test-3 ( Extract genomic DNA 1 ) > Test-4 ( Extract genomic DNA 1 ) > Test-5 ( GeneBed_Maf_Fasta2 ) > Test-1 ( Interval2Maf1 ) > Test-1 ( Interval2Maf1 ) > Test-3 ( Interval_Maf_Merged_Fasta2 ) > Test-1 ( Interval_Maf_Merged_Fasta2 ) > Test-2 ( LinearRegression1 ) > Test-1 ( XY_Plot_1 ) > Test-1 ( cca1 ) > Test-1 ( histogram_rpy ) > Test-1 ( kpca1 ) > Test-1 ( pca1 ) > Test-1 ( pca1 ) > Test-2 ( pca1 ) > Test-3 ( plot_for_lda_output1 ) > Test-1 ( qual_stats_boxplot ) > Test-1 AssertionError: History item 2 different than expected, difference (using diff) ( gops_join_1 ) > Test-4 TwillAssertionError: code is 500 != 200 I do also have failures due to missing data but I am struggling to find out where to download them. ( gd_select_snps ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error 'bighorn' does not appear in location file '/opt/local/home/webapp/galaxy/tool-data/gd.chrlens.loc' ( Annotation_Profiler_0 ) > Test-1 ( Annotation_Profiler_0 ) > Test-2 AssertionError: Configuration error: Table directory is missing (/opt/local/home/webapp/galaxy/tool-data/annotation_profiler/hg18) ( Interval2Maf_pairwise1 ) > Test-1 AssertionError: Attempting to set field 'mafType' to value '['PAIRWISE_hg17_fr1']' in form 'tool_form' threw exception: cannot find value/label "PAIRWISE_hg17_fr1" in list control ( maf_stats1 ) > Test-1 ( maf_stats1 ) > Test-2 AssertionError: Attempting to set field 'maf_source_type|mafType' to value '['8_WAY_MULTIZ_hg17']' in form 'tool_form' threw exception: cannot find value/label "8_WAY_MULTIZ_hg17" in list control ( phastOdds_for_intervals ) > Test-1 AssertionError: Attempting to set field 'score_file' to value '['/galaxy/data/phastOdds_precomputed/encode_SEP-2005_\ tba.v2_phastOdds']' in form 'tool_form' threw exception: cannot find value/label "/galaxy/data/phas..2_phastOdds" in li\ st control ( aggregate_scores_in_intervals2 ) > Test-1 ( aggregate_scores_in_intervals2 ) > Test-2 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error ( gencode_partition1 ) > Test-1 Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error Error loading partitioning dataset. ( lda_analy1 ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error The last failure is related to genome diversity and the fact that it calls python2.5. I've changed it to python for testing but the test still fails. ( gd_extract_flanking_dna ) > Test-1 ( gd_extract_primers ) > Test-1 ( gd_select_restriction_enzymes ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error python2.5: command not found > if changed to python: ERROR: 'NoneType' object has no attribute 'rfind' Is there a way to fix these failures? Any help would be very much appreciated. Thanks, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

2 3

New Galaxy look from usegalaxy.org
by Leandro Hermida 27 Mar '12

27 Mar '12

Dear all, How do I get the new Galaxy look enabled that you see in usegalaxy.org? I have two servers on the latest galaxy-dist. regards, Leandro

3 2

[request] drmaa way too chatty
by Louise-Amélie Schmitt 27 Mar '12

27 Mar '12

Hi everyone, The drmaa runner literally floods the Galaxy output with its own output (dozens of lines every second) which makes the Galaxy log impossible to read. Would there be a way to separate the two of them? I tried to look into the code but I'm not fully sure about what exactly produces this humongous output so I'm walking in the dark. Thanks, L-A

1 0

drmaa module does not load
by Louise-Amélie Schmitt 27 Mar '12

27 Mar '12

Hello everyone, I wanted to start the drmaa job runner and followed the instructions in the wiki, but I have this error message when I start Galaxy: galaxy.jobs ERROR 2012-03-23 15:28:49,845 Job runner is not loadable: galaxy.jobs.runners. drmaa Traceback (most recent call last): File "/g/funcgen/galaxy/lib/galaxy/jobs/__init__.py", line 1195, in _load_plugin module = __import__( module_name ) ImportError: No module named drmaa I checked /g/funcgen/galaxy/lib/galaxy/jobs/runners and it contains the drmaa.py file There was no drmaa egg so I made a copy of it from our other Galaxy install but it didn't solve the problem. I don't really know where to start looking, any idea? Thanks, L-A

2 3

Problem starting toolshed
by Joachim Jacob 27 Mar '12

27 Mar '12

Hi all, In order to ease sharing the developed Galaxy tools in our group, we want to launch a private Toolshed. However, upon running ./run_community.sh, I got following error. Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 218, in run result = self.command() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 224, in loadobj global_conf=global_conf) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 248, in loadcontext global_conf=global_conf) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 278, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 413, in get_context section) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 458, in _context_from_explicit value = import_string(found_expr) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 18, in import_string return pkg_resources.EntryPoint.parse("x="+s).load(False) File "/home/galaxy/galaxy-dist/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/community/buildapp.py", line 16, in <module> import galaxy.webapps.community.model File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/community/model/__init__.py", line 12, in <module> from mercurial import hg, ui ImportError: No module named mercurial Removing PID file community_webapp.pid What I understand, there is a conflict between Python2.6 (manually installed under /opt) and Python2.4 (system-wide version). In version 2.6, I do not have mercurial available apparently. Any hint about a fix? Kind regards, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

2 3

Normal that latest galaxy-dist creates shed_tools directory one level up from the galaxy-dist directory?
by Leandro Hermida 26 Mar '12

26 Mar '12

Dear all, Is it normal that the latest galaxy-dist creates a shed_tools directory one directory up (and outside) from the server galaxy-dist directory? best, Leandro

2 1