September 2011 - galaxy-dev - lists.galaxyproject.org

FW: Galaxy tool error report from khaden@illumina.com
by Haden, Kirt 03 Oct '11

03 Oct '11

Galaxy team, I am running a locally configured Galaxy instance using Python 2.7. I am getting an error on SAM_to_BAM unable to fetch eggs. When I change back to Python 2.6, the problem is removed and I can run the tool. Is there a quick fix for this problem? Thanks, Kirt -----Original Message----- From: khaden(a)illumina.com [mailto:khaden@illumina.com] Sent: Thursday, September 29, 2011 8:07 AM To: Haden, Kirt; Haden, Kirt Subject: Galaxy tool error report from khaden(a)illumina.com GALAXY TOOL ERROR REPORT ------------------------ This error report was sent from the Galaxy instance hosted on the server "ussd-dev-lnx01.illumina.com" ----------------------------------------------------------------------------- This is in reference to dataset id 301 from history id 68 ----------------------------------------------------------------------------- You should be able to view the history containing the related history item 7: SAM-to-BAM on data 6: converted BAM by logging in as a Galaxy admin user to the Galaxy instance referenced above and pointing your browser to the following link. ussd-dev-lnx01.illumina.com/history/view?id=42e99d98e0ee2844 ----------------------------------------------------------------------------- The user 'khaden(a)illumina.com' provided the following information: ----------------------------------------------------------------------------- job id: 200 tool id: sam_to_bam ----------------------------------------------------------------------------- job command line: python /home/galaxy/eval/sept09/galaxy-central/tools/samtools/sam_to_bam.py --input1=/home/galaxy/production/Sept06/galaxy-central/database/files/000/dataset_300.dat --dbkey=PhiX --output1=/home/galaxy/production/Sept06/galaxy-central/database/files/000/dataset_301.dat --index_dir=/home/galaxy/eval/sept09/galaxy-central/tool-data ----------------------------------------------------------------------------- job stderr: Traceback (most recent call last): File "/home/galaxy/eval/sept09/galaxy-central/tools/samtools/sam_to_bam.py", line 14, in ? import pkg_resources; pkg_resources.require( "bx-python" ) File "/home/galaxy/eval/sept09/galaxy-central/lib/galaxy/eggs/__init__.py", line 415, in require raise EggNotFetchable( str( [ egg.name for egg in e.eggs ] ) ) galaxy.eggs.EggNotFetchable: ['bx_python'] ----------------------------------------------------------------------------- job stdout: ----------------------------------------------------------------------------- job info: None ----------------------------------------------------------------------------- job traceback: None ----------------------------------------------------------------------------- (This is an automated message).

3 2

Re: [galaxy-dev] [galaxy-user] problem with displaying tracks from Galaxy
by Nate Coraor 03 Oct '11

03 Oct '11

Sergei Ryazansky wrote: > Hello all, > > we have the UCSC genome browser mirror as well as Galaxy mirror. The > Galaxy has a feature enabling a user to display the data at UCSC genome > browser as custom tracks. I have configured the galaxy to display the data > to our UCSC browser mirror but it doesn't work properly: after the > redirecting to genome browser page the "redirected to non-http(s): /root" > error message is appeared. At the same time displaying Galaxy data at > official UCSC works excellent. What are the possible reasons of it? > Thank you in advance! Hi Sergei, If your Galaxy server is behind a proxy server serving via https, have you set the following header: RequestHeader set X-URL-SCHEME https Please see the "SSL" section of the ApacheProxy page for more information: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy --nate > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

4 7

Python2.7 SAM to BAM Egg Not Fetchable
by Haden, Kirt 30 Sep '11

30 Sep '11

Hi, When I run Galaxy under Python 2.7, I am unable to run SAM to BAM tools without getting the following error (below). The same configuration with Python 2.6 does not have an issue. The error seems to indicate bx_python eggs are not fetchable. What am I doing wrong? Error report below: GALAXY TOOL ERROR REPORT ------------------------ This error report was sent from the Galaxy instance hosted on the server "ussd-dev-lnx01.illumina.com" ----------------------------------------------------------------------------- This is in reference to dataset id 301 from history id 68 ----------------------------------------------------------------------------- You should be able to view the history containing the related history item 7: SAM-to-BAM on data 6: converted BAM by logging in as a Galaxy admin user to the Galaxy instance referenced above and pointing your browser to the following link. ussd-dev-lnx01.illumina.com/history/view?id=42e99d98e0ee2844 ----------------------------------------------------------------------------- The user 'khaden(a)illumina.com' provided the following information: ----------------------------------------------------------------------------- job id: 200 tool id: sam_to_bam ----------------------------------------------------------------------------- job command line: python /home/galaxy/eval/sept09/galaxy-central/tools/samtools/sam_to_bam.py --input1=/home/galaxy/production/Sept06/galaxy-central/database/files/000/dataset_300.dat --dbkey=PhiX --output1=/home/galaxy/production/Sept06/galaxy-central/database/files/000/dataset_301.dat --index_dir=/home/galaxy/eval/sept09/galaxy-central/tool-data ----------------------------------------------------------------------------- job stderr: Traceback (most recent call last): File "/home/galaxy/eval/sept09/galaxy-central/tools/samtools/sam_to_bam.py", line 14, in ? import pkg_resources; pkg_resources.require( "bx-python" ) File "/home/galaxy/eval/sept09/galaxy-central/lib/galaxy/eggs/__init__.py", line 415, in require raise EggNotFetchable( str( [ egg.name for egg in e.eggs ] ) ) galaxy.eggs.EggNotFetchable: ['bx_python'] ----------------------------------------------------------------------------- job stdout: ----------------------------------------------------------------------------- job info: None ----------------------------------------------------------------------------- job traceback: None ----------------------------------------------------------------------------- (This is an automated message). Thanks, Kirt Haden Sr. Mgr. Engineering Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858.202.4655 Fax: 858.202.4545 Email: khaden(a)illumina.com<mailto:khaden@illumina.com> Website: www.illumina.com<http://www.illumina.com/>

4 7

Setting http_proxy for Python
by Matloob Khushi 30 Sep '11

30 Sep '11

Hi Guys I am trying to install Galaxy on Red Hat Enterprise Linux Server release 5.7 (Tikanga). I have installed Python 2.6. Our server is behind a proxy that needs to be authenticated. I have set up the proxy for Mercurial as described in http://www.selenic.com/mercurial/hgrc.5#http-proxy Now Mercurial is working and I have cloned the repository, however, when I issue sh run.sh command I get the following error. Though I have set the environment variable http_proxy by issuing export command on command line: export HTTP_PROXY=UserName:MyUniPass@servername.edu.au:8080 I have also tried various export syntax such as: export http_proxy=http://UserName:MyUniPass@ servername.edu.au:8080 I suspect Python is not reading the proxy setting to access internet. Help please. Some eggs are out of date, attempting to fetch... Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 30, in ? c.resolve() # Only fetch eggs required by the config File "/opt/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/opt/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/opt/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conf lict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg. fetch ) File "/opt/galaxy-dist/lib/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: simplejson==2.1.1 Fetch failed. Regards Matloob

2 2

Re: [galaxy-dev] [galaxy-user] Add library to dataset performance metric: developer vs production instances
by Jennifer Jackson 29 Sep '11

29 Sep '11

Hi Roman, This is a good question for the development community to provide feedback on, so I'll cross-post your question over to that list. Best, Jen Galaxy team On 9/19/11 2:30 PM, Roman Valls wrote: > Hello, > > Today I was routinely adding a 27GB Illumina lane on my galaxy instance > running on a cluster node. Just the regular cloned-from-hg type of > instance with set_metadata_externally, no more tuning. > > It took more than 10 minutes to have the dataset imported into a data > library via the filesystem path upload method... not copying it into > galaxy, just "linking". > > galaxy.jobs INFO 2011-09-19 18:05:08,641 job 120 dispatched > (...) > galaxy.jobs DEBUG 2011-09-19 18:16:52,822 job 120 ended > galaxy.datatypes.metadata DEBUG 2011-09-19 18:16:52,824 Cleaning up > external metadata files > > Since I cannot add datasets to libraries in usegalaxy.org and compare, I > was wondering if someone can state an approximated average time *for a > production* galaxy installation to do that operation. > > I would like to have some empirical number to show on how a production > deployment[1] could speed things up, as opposed to having individual > galaxy instances per user in a cluster (as per IT policies): > > http://blogs.nopcode.org/brainstorm/2011/08/22/galaxy-on-uppmax-simplified/ > > Thanks in advance ! > Roman > > [1] http://usegalaxy.org/production > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support

2 1

Additional eggs
by Chorny, Ilya 29 Sep '11

29 Sep '11

Could you add PyX and Matplotlib(http://sourceforge.net/projects/matplotlib/files/matplotlib/matp… to the eggs? They would be very useful for creating reports. Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

1 0

2 questions about Galaxy's functional testing support
by Paniagua, Eric 29 Sep '11

29 Sep '11

Hi all, I've read the Wiki page on Writing Functional Tests (http://wiki.g2.bx.psu.edu/Admin/Tools/Writing%20Tests) and I've been looking through test/base and test/functional and I am left with two questions: * Is it possible to write a test to validate metadata directly on an (optionally composite) output dataset? Everything described on the above page is file oriented. I see that there is TwillTestCase.check_metadata_for_string, but as far as I can tell this is a bit nonspecific since it appears to just do a text search on the Edit page. I don't yet fully understand the context in which tests run, but is there some way to access a "live" dataset's metadata directly, either as a dictionary or just as attributes? Or even to get the actual dataset object? * Does the test harness support retaining output files only for failed tests? Ideally with a cap on how much output data to save. If not, would this be difficult to configure? Thanks, Eric

1 0

Re: [galaxy-dev] GATK unified genotyper
by Chorny, Ilya 29 Sep '11

29 Sep '11

To prevent user error in mixing genomes, would you be interested in a feature in which the unified genotyper choses the genome in a fashion similar to how samtools know which which genome to use? <when value="cached"> <param name="input1" type="data" format="sam" metadata_name="dbkey" label="SAM File to Convert"> <validator type="unspecified_build" /> <validator type="dataset_metadata_in_file" filename="sam_fa_indices.loc" metadata_name="dbkey" metadata_column="1" message="Sequences are not currently available for the specified build." line_startswith="index" /> </param> </when> Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 1

Re: [galaxy-dev] GTF files in Cufflinks and Tophat
by Chorny, Ilya 28 Sep '11

28 Sep '11

Jeremy, 1.) Would you be interested in a feature in which the gtf files could be "locally cached" instead of only "from history"? 2.) Would you be interested in a feature in which cufflinks choses the genome in a fashion similar to how samtools know which which genome to use? i.e.: <when value="cached"> <param name="input1" type="data" format="sam" metadata_name="dbkey" label="SAM File to Convert"> <validator type="unspecified_build" /> <validator type="dataset_metadata_in_file" filename="sam_fa_indices.loc" metadata_name="dbkey" metadata_column="1" message="Sequences are not currently available for the specified build." line_startswith="index" /> </param> </when> I am happy on working on these changes if they will be put into galaxy-central. Best, Ilya Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

1 1

Attribute Error when uploading data on local
by Colleen Doherty 28 Sep '11

28 Sep '11

Hi, I get the following error when trying to upload data to Galaxy AttributeError: 'NoneType' object has no attribute 'lower' Same error for many different types of files (fastq, sam, bam) some of which uploaded last week. What I've tried: Deleting browser cache Deleting cookie Deleting database/compiled_templates Deleting eggs/ Please let me know what else I should try / what I am doing wrong. More info below. Thank you! More information: TRACEBACK: URL: http://127.0.0.1:8080/datasets/59c76a119581e190/display/?preview=True File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/home/colleen/Work/workspace/ChIPSeq/tools/galaxy-dist/lib/galaxy/web/controllers/dataset.py', line 361 in display mime = trans.app.datatypes_registry.get_mimetype_by_extension( data.extension.lower() ) AttributeError: 'NoneType' object has no attribute 'lower' Extra Data CGI Variables CONTENT_LENGTH '0' HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.7' HTTP_ACCEPT_ENCODING 'gzip, deflate' HTTP_ACCEPT_LANGUAGE 'en-us,en;q=0.5' HTTP_CONNECTION 'keep-alive' HTTP_COOKIE 'galaxysession=c6ca0ddb55be603a2c2796e41769bc2412b1c87332f54f840b1876eab065f3113e1ca20ff6a4ab5c' HTTP_HOST '127.0.0.1:8080' HTTP_REFERER 'http://127.0.0.1:8080/history' HTTP_USER_AGENT 'Mozilla/5.0 (X11; Linux x86_64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2' PATH_INFO '/datasets/59c76a119581e190/display/' QUERY_STRING 'preview=True' REMOTE_ADDR '127.0.0.1' REQUEST_METHOD 'GET' SERVER_NAME '127.0.0.1' SERVER_PORT '8080' SERVER_PROTOCOL 'HTTP/1.1' Extra Data WSGI Variables application <paste.debug.prints.PrintDebugMiddleware object at 0x7834350> paste.cookies (<SimpleCookie: galaxysession='c6ca0ddb55be603a2c2796e41769bc2412b1c87332f54f840b1876eab065f3113e1ca20ff6a4ab5c'>, 'galaxysession=c6ca0ddb55be603a2c2796e41769bc2412b1c87332f54f840b1876eab065f3113e1ca20ff6a4ab5c') paste.evalexception <weberror.evalexception.middleware.EvalException object at 0x6dd11d0> paste.evalexception.debug_count 1316648566 paste.expected_exceptions [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions <paste.httpexceptions.HTTPExceptionHandler object at 0x7823090> paste.httpserver.thread_pool <paste.httpserver.ThreadPool object at 0x2b66f90> paste.printdebug_listeners [<cStringIO.StringO object at 0x6e24fb8>, <open file '<stderr>', mode 'w' at 0x7fb208c901e0>] paste.recursive.forward <paste.recursive.Forwarder from /> paste.recursive.include <paste.recursive.Includer from /> paste.recursive.include_app_iter <paste.recursive.IncluderAppIter from /> paste.recursive.script_name '' paste.remove_printdebug <function remove_printdebug at 0x6e23a28> paste.throw_errors True weberror.evalexception <weberror.evalexception.middleware.EvalException object at 0x6dd11d0> weberror.evalexception.debug_count 1316648566 webob._parsed_query_vars (MultiDict([('preview', 'True')]), 'preview=True') wsgi process 'Multithreaded'

2 2