September 2010 - galaxy-dev - lists.galaxyproject.org

Sharing tool definitions between XML files
by Peter 02 Jun '11

02 Jun '11

Hi all, I'm wondering if there is any established way to share parameter definitions between tool wrapper XML files? For example, I am currently working on NCBI BLAST+ wrappers, and these tools have a lot in common. For example, the output format option will be a select parameter, and it will be the same for blastn, blastp, tblastn, etc. I can just cut and paste the definition but this seems inelegant and will be a long term maintenance burden if it ever needs updating (lots of places would need to be updated the same way). I'd like to have a shared XML snippet defining this parameter which I could then reference/include from each tool wrapper than needs it. I'm thinking of something like XML's <!ENTITY ...> might work. Is this possible? Peter

2 2

swap issue
by Isabelle Phan 22 Mar '11

22 Mar '11

Hello, I am running galaxy, using a postgres backend. The galaxy server systematically gets killed whenever I try uploading a bam file. The samtools process is still running. This is the end of the log: 2010-08-26 18:08:11,371 INFO sqlalchemy.engine.threadlocal.TLEngine.0x...4610 {'param_1': 8} sqlalchemy.engine.threadlocal.TLEngine.0x...4610 INFO 2010-08-26 18:08:11,371 {'param_1': 8} 2010-08-26 18:08:11,378 INFO sqlalchemy.pool.QueuePool.0x...4ed0 Connection <connection object at 0x567bf30; dsn: 'dbname=galaxy host=/var closed: 0> being returned to pool sqlalchemy.pool.QueuePool.0x...4ed0 INFO 2010-08-26 18:08:11,378 Connection <connection object at 0x567bf30; dsn: 'dbname=galaxy host=/var closed: 0> being returned to pool 10.1.20.44 - - [26/Aug/2010:18:08:06 -0700] "POST /root/history_item_updates HTTP/1.1" 200 - "http://galaxy:8080/history" "Mozilla/5.0 (X1 (x86_64); en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8" Killed I have the following options set in universe_wsgi.ini: database_engine_option_server_side_cursors = True database_engine_option_strategy = threadlocal It seems to be running out of swap. I monitored memory usage with 'top' and galaxy died when available swap hit 0k free. I am not superuser on the machine where galaxy is running and I have no other server available. Is there something I could do to prevent galaxy from dying? Thanks, Isabelle -- Isabelle Phan, DPhil Seattle Biomedical Research Institute +1(206)256 7113

3 4

Galaxy wrappers for TMHMM and SignalP
by Peter 15 Nov '10

15 Nov '10

Hello, Are there any existing Galaxy wrappers for the command line tools (or the online services) TMHMM v2.0 (Trans-Membrane Hidden Markov Model) and SignalP v3.0 (Signal Peptide)? http://www.cbs.dtu.dk/cgi-bin/nph-sw_request?tmhmm http://www.cbs.dtu.dk/cgi-bin/nph-sw_request?signalp Both take protein FASTA files for input, but there are a few extra considerations here like limiting each call to TMHMM to 4000 proteins (IIRC), the possibility to split the job in order to take advantage of multiple cores, and parsing the output into a Galaxy friendly tabular form. Thanks, Peter

2 7

How to implement the button "send query to galaxy" in UCSC Main tool
by Zhe Chen 09 Nov '10

09 Nov '10

Hi, How to implement the button "send query to galaxy" in (Get Data)->(UCSC Main tool) on the second page? Do you have a code example of it? Thanks

2 1

Fastq_filter fails on large files
by Isabelle Phan 18 Oct '10

18 Oct '10

Hello, The tool fastq_filter worked on 1M reads, but fails (hangs) on 15M reads. I had to kill the job after the user let it run for a whole day. The debug.txt file containing a python function "fastq_read_pass_filter" is created in the files/000/dataset_xxx_files directory. I am getting no error from the galaxy server. I wonder what could cause fastq_filter to fail? The fastx equivalent tool works, but it misses all the options of fastq_filter. I'd be grateful for any hints to help me get fastq_filter to work on large fastq files. Thanks Isabelle

2 2

NCBI BLAST+ wrappers in Galaxy?
by Peter 13 Oct '10

13 Oct '10

Hi all, Something I expect to find useful in several analysis pipelines is a Galaxy wrapper for the NCBI BLAST+ tools (or even the old NCBI "legacy" BLAST tools if such a wrapper exists). I've been looking over the tools in galaxy-dist and galaxy-central and the only NCBI BLAST wrapper I can see is for MEGABLAST, under tools/metag_tools. Is there some more general NCBI BLAST+ wrappers that I have missed? Or is anyone already working on this? Thanks, Peter

3 17

Using rsyslog for logging from Galaxy
by Peter 05 Oct '10

05 Oct '10

Hi all, For our production server we are using rsyslog for centralised logging, http://www.rsyslog.com/ Galaxy is using "import logging" which can output to rsyslog, using SysLogHandler rather than StreamHandler or FileHandler. Based on grep'ing the source code Galaxy doesn't support this yet. I could probably add it but it isn't clear to me where this would be done... functions configure_logging() in lib/galaxy/config.py and in lib/galaxy/webapps/community/config.py look like strong possibilities but both allow for some global_conf_parser override (and I'm not sure what that does). Has anyone looked at modifying this area before, or could offer some tips? Thanks, Peter

2 5

Update .hgignore for typical Unix editor backup files
by Peter 01 Oct '10

01 Oct '10

Hi all, Would it make sense to add the following to the official .hgignore file to ignore backup files from common Unix editors, *~ *.swp *.bak Thanks, Peter

2 1

target in links
by Belinda M. Giardine 01 Oct '10

01 Oct '10

Am I missing something or is it not possible to set the target in the links from reStructuredText? It would be nice to be able to open help links in a separate window. Belinda

2 1

Error running workflow
by Branden Timm 01 Oct '10

01 Oct '10

Hello, I seem to be having database-related issues when running my workflow. galaxy.jobs DEBUG 2010-09-09 21:14:48,967 dispatching job 180 to local runner galaxy.jobs INFO 2010-09-09 21:14:49,058 job 180 dispatched galaxy.jobs.runners.local DEBUG 2010-09-09 21:14:49,697 executing: soap -a /home/galaxy/galaxy_dist/database/files/000/dataset_400.dat -D /home/galaxy/galaxy_dist/soap_indices/ecoli_U00096.fasta.index -o /home/galaxy/galaxy_dist/database/files/000/dataset_431.dat -u None -r 1 -n 5 -v 2 -M 4 2>&1 galaxy.jobs ERROR 2010-09-09 21:14:50,209 failure running job 181 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 185, in __monitor_step self.sa_session.refresh( job ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 926, in refresh self._validate_persistent(state) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1234, in _validate_persistent raise sa_exc.InvalidRequestError( InvalidRequestError: Instance '<Job at 0x1a4074d0>' is not persistent within this Session 146.151.211.150 - - [09/Sep/2010:21:14:48 -0500] "POST /workflow/run?id=f597429621d6eb2b HTTP/1.1" 200 - "http://galaxy.glbrc.wisc.edu/workflow/run?id=f597429621d6eb2b" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8" galaxy.jobs ERROR 2010-09-09 21:14:51,219 failure running job 182 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 185, in __monitor_step self.sa_session.refresh( job ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 926, in refresh self._validate_persistent(state) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1234, in _validate_persistent raise sa_exc.InvalidRequestError( InvalidRequestError: Instance '<Job at 0x2aaab001acd0>' is not persistent within this Session galaxy.jobs.runners.local DEBUG 2010-09-09 21:14:52,460 execution finished: soap -a /home/galaxy/galaxy_dist/database/files/000/dataset_400.dat -D /home/galaxy/galaxy_dist/soap_indices/ecoli_U00096.fasta.index -o /home/galaxy/galaxy_dist/database/files/000/dataset_431.dat -u None -r 1 -n 5 -v 2 -M 4 2>&1 In Galaxy itself, the SOAP tool returns its results and the rest of the jobs just wait to run. Why am I having these errors? Cheers -- Branden Timm btimm(a)glbrc.wisc.edu

2 1