February 2012 - galaxy-dev - lists.galaxyproject.org

loading tool values from other databases
by Matloob Khushi 04 Feb '12

04 Feb '12

Dear Galaxy Developers I wish to write set of tools for Galaxy Community; tools will use Postgresql/MySQL database (other than the default galaxy database). Tool's dropdown boxes will load values from the database on the fly. I had a look at tool configuration at Galaxy WiKi and I can't seems to track the right configuration. What's the best way to approach that? Other solution I was thinking to open a new web app in galaxy_main iframe, however, that seems to involve creating new controller. My aim is achieve this by minimal modification to core code. Reference to any existing similar tool will certainly help. Any other guidance will be appreciated (keep in mind I am a newbie). Thanks. Mat

2 3

Multiple input file bug
by Fields, Christopher J 03 Feb '12

03 Feb '12

Just filed as Issue #714 on bitbucket. I noticed that multiple input seems to be broken on workflows when selecting more that one file (a single file works). This is happening on our local instance, using Jan 27 galaxy-dist release, as well as both main/test.g2.bx.psu.edu Galaxy servers (latter sites just report a server error). Debugging output: URL: http://galaxy.igb.uiuc.edu/workflow/run?id=a799d38679e985db File '/home/a-m/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/home/a-m/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/a-m/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/a-m/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', line 111 in __call__ return self.app( environ, start_response ) File '/home/a-m/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/web/controllers/workflow.py', line 1458 in run update_only=True, old_errors=old_errors ) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py', line 1460 in update_state value, error = check_param( trans, input, incoming_value, context ) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/tools/parameters/__init__.py', line 60 in check_param param.validate( filtered_value, trans.history ) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/tools/parameters/basic.py', line 1547 in validate validator.validate( value, history ) File '/home/a-m/galaxy/galaxy-dist/lib/galaxy/tools/parameters/validation.py', line 185 in validate if value and value.missing_meta( check = self.check, skip = self.skip ): AttributeError: 'list' object has no attribute 'missing_meta' chris

2 4

Re: [galaxy-dev] Some eggs are out of date...Fetch failed (Mac OS X 10.6)
by Nate Coraor 03 Feb '12

03 Feb '12

On Feb 1, 2012, at 9:45 AM, Lucilla Luzi wrote: > Thank you Nate, > > maybe there is something that I'm still missing... > > When I type what you suggest, I get again an error message that is > > $ python scripts/fetch_eggs.py > Traceback (most recent call last): > File "scripts/fetch_eggs.py", line 30, in <module> > c.resolve() # Only fetch eggs required by the config > File "/Users/lluzi/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve > egg.resolve() > File "/Users/lluzi/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve > return self.version_conflict( e.args[0], e.args[1] ) > File "/Users/lluzi/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict > r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) > File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 518, in resolve > raise DistributionNotFound(req) # XXX put more info here > pkg_resources.DistributionNotFound: numpy==1.6.0 > > Do I correctly understand your suggestion? > > Should I manually download all the eggs from your site? If yes, do I need of all of them? > Any other idea to overcome the problem? It looks like you have a conflicting version of numpy already installed on the system and Galaxy's conflict resolution code isn't handling this situation. You may be able to fetch all the eggs with: % python -ES ./scripts/fetch_eggs.py If this works and running Galaxy still fails, you may need to set up a virtualenv as described here: http://usegalaxy.org/production Sorry for the inconvenience. --nate > > Thanks again! > Lu > > > > > On 1/30/12 20:37 , Nate Coraor wrote: >> On Jan 30, 2012, at 1:46 PM, Lucilla Luzi wrote: >> >> >>> Thank you Nate for your reply!! :) >>> >>> This is what I got. >>> >>> $ python ./scripts/get_platforms.py >>> macosx-10.6-universal-ucs2 >>> >>> (just FYI, I run this command from the galaxy-dist folder that has a path like this one: >>> /Users/me/My_programs/Galaxy/galaxy-dist/ ) >>> >> Okay, you should definitely be able to download prebuilt eggs for this platform. Have you tried running python ./scripts/fetch_eggs.py multiple times? >> >> >>> BTW; have you also noticed my message of 25th of January?) >>> >> Yes, thanks for the extra info. >> >> --nate >> >> >>> Thank you a lot!! >>> Lu >>> >>> >>> On 1/30/12 18:40 , Nate Coraor wrote: >>> >>>> On Jan 24, 2012, at 4:16 PM, Lucilla Luzi wrote: >>>> >>>> >>>> >>>>> ERRATA CORRIGE: sorry :( >>>>> >>>>> $ echo $PATH >>>>> /System/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin >>>>> >>>>> -------- Original Message -------- >>>>> Subject: [galaxy-dev] Some eggs are out of date...Fetch failed (Mac OS X 10.6) >>>>> Date: Tue, 24 Jan 2012 22:06:42 +0100 >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I'm pretty new to Galaxy, at least to this side. >>>>> I'm sorry to bother you with my issue, that, I read, was quite extensively discussed previously, but even if I think to have followed >>>>> correctly the instruction and further suggestions, I'm still not able to overcome the problem. >>>>> >>>>> I'm trying to install a local instance of Galaxy on my MAC (OS X10.6.8) >>>>> My Python version is 2.6.1 and I just installed Mercurial 2.0.2 for MacOS X 10.6. >>>>> >>>>> FYI >>>>> >>>>> $ python --version >>>>> Python 2.6.1 >>>>> >>>>> --->> wrong <<---see above >>>>> $ echo $PATH >>>>> /Users/myName/System/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin >>>>> >>>>> When I try to run the run.sh I get the error attached below. >>>>> Can you envisage a possible reason why I'm getting this error? >>>>> Sorry for the naive admission, but from the error message >>>>> I'm not even capable to understand which are the "missing eggs"... >>>>> >>>>> If I'd have ignored a previous post with the solution, I apologize in advance. >>>>> Thank you a lot for your attention. >>>>> >>>>> >>>> Hi Lu, >>>> >>>> Could you provide the output of the following?: >>>> >>>> % python ./scripts/get_platforms.py >>>> >>>> Thanks, >>>> --nate >>>> >>>> >>>> >>>> >>>>> Lu >>>>> >>>>> ------------------ >>>>> $ sh run.sh >>>>> Some eggs are out of date, attempting to fetch... >>>>> Traceback (most recent call last): >>>>> File "./scripts/fetch_eggs.py", line 30, in <module> >>>>> c.resolve() # Only fetch eggs required by the config >>>>> File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve >>>>> egg.resolve() >>>>> File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve >>>>> return self.version_conflict( e.args[0], e.args[1] ) >>>>> File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict >>>>> r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) >>>>> File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 518, in resolve >>>>> raise DistributionNotFound(req) # XXX put more info here >>>>> pkg_resources.DistributionNotFound: numpy==1.6.0 >>>>> Fetch failed. >>>>> >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> >>>>> >>>>> http://lists.bx.psu.edu/ >> >

1 0

Missing stdout and stderr files from SGE cluster
by Peter Cock 03 Feb '12

03 Feb '12

Hi all, I'm trying to configure Galaxy to talk to our SGE cluster via DRMAA. I've got jobs submitting, and can verify they are in the queue and then run via qstat (from the cluster). However, something isn't right. I'm testing with blastp using the following in universe_wsgi.ini [galaxy:tool_runners] ncbi_blastp_wrapper = drmaa:/// ... Job output from run.sh, galaxy.jobs DEBUG 2012-02-02 11:49:16,707 (25) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/25 galaxy.jobs DEBUG 2012-02-02 11:49:16,710 dispatching job 25 to drmaa runner galaxy.jobs INFO 2012-02-02 11:49:16,818 job 25 dispatched galaxy.jobs.runners.drmaa DEBUG 2012-02-02 11:49:17,174 (25) submitting file /mnt/galaxy/galaxy-central/database/pbs/galaxy_25.sh galaxy.jobs.runners.drmaa DEBUG 2012-02-02 11:49:17,174 (25) command is: blastp -version &> /mnt/galaxy/galaxy-central/database/tmp/GALAXY_VERSION_STRING_25; python /mnt/galaxy/galaxy-central/tools/ncbi_blast_plus/hide_stderr.py blastp -query "/mnt/galaxy/galaxy-central/database/files/000/dataset_5.dat" -subject "/mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat" -task blastp -evalue 0.001 -out /mnt/galaxy/galaxy-central/database/files/000/dataset_25.dat -outfmt 6 -num_threads 8 galaxy.jobs.runners.drmaa INFO 2012-02-02 11:49:17,178 (25) queued as 223 galaxy.jobs.runners.drmaa DEBUG 2012-02-02 11:49:18,175 (25/223) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2012-02-02 11:49:32,176 (25/223) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2012-02-02 11:49:43,367 (25/223) state change: job finished normally galaxy.jobs DEBUG 2012-02-02 11:49:43,679 job 25 ended galaxy.jobs.runners.drmaa WARNING 2012-02-02 11:49:43,699 Unable to cleanup: [Errno 2] No such file or directory: '/mnt/galaxy/galaxy-central/database/job_working_directory/000/25/25.drmout' galaxy.jobs.runners.drmaa WARNING 2012-02-02 11:49:43,699 Unable to cleanup: [Errno 2] No such file or directory: '/mnt/galaxy/galaxy-central/database/job_working_directory/000/25/25.drmerr' Interestingly, via the Galaxy interface it does seem to have captured stdout (emtpy) and stderr anyway: <quote> Error invoking command: blastp -query /mnt/galaxy/galaxy-central/database/files/000/dataset_5.dat -subject /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat -task blastp -evalue 0.001 -out /mnt/galaxy/galaxy-central/database/files/000/dataset_25.dat -outfmt 6 -num_threads 8 [Errno 2] No such file or directory </quote> The error in this case is some kind of path problem, which I could 'fix' by adding 'source ~/.bashrc' to the drm_template defined in lib/galaxy/jobs/runners/drmaa.py (although there should be a more elegant solution to this!). Anyway, why is the DRMAA code giving me these warnings about missing the stdout and stderr files - even when the run runs fine? Thanks, Peter

4 6

Re: [galaxy-dev] Issue 710 in galaxy-central: An error occurred running this job- No alignments are available for sacCer3
by ambarish biswas 02 Feb '12

02 Feb '12

Hi, when I try to fetch sequence for SacCer3 using UCSC Main table Browser, I received the following error: "An error occurred running this job: No sequences are available for 'sacCer3', request them by reporting this error." I noticed that, It's possible to extract sequence for SacCer2, but again none of the SacCer2 or SacCer3 has allowed "Fetch Alignments->Extract MAF Blocks" Is it a bug or compatibility issue? Hope this can be fixed or else will be a problem for me. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866 On Thu, Feb 2, 2012 at 3:01 PM, Bitbucket <issues-reply(a)bitbucket.org>wrote: > --- you can reply above this line --- > > Issue 710: An error occurred running this job- No sequences are available > for sacCer3 > > https://bitbucket.org/galaxy/galaxy-central/issue/710/an-error-occurred-run… > > ambarishbiswas on Thu, 2 Feb 2012 03:01:34 +0100: > > Changes: > component: nothing -> tools > > -- > > This is an issue notification from bitbucket.org. You are receiving > this either because you are the owner of the issue, or you are > following the issue. >

2 2

galaxy dependency
by bmoumen＠jouy.inra.fr 02 Feb '12

02 Feb '12

Dear galaxy team thank you a lot for providing a complete solution for biologists to analyse their data, i have one question about galaxy : i see that there are a lot of tools for example for ngs, and i could not understand or find the tool in galaxy directories, is that means that all the tool are rewritten in python ? eg bowtie, samtools, fastx and ... Or the binaries are located in some folder that i can not see . thanks

3 2

GALAXY_LIB path in job scripts
by Shantanu Pavgi 02 Feb '12

02 Feb '12

The job scripts generated by galaxy contain following code. I was wondering what's the purpose of if-block after setting 'GALAXY_LIB="None"' assignment. {{{ GALAXY_LIB="None" if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi }}} -- Shantanu

2 1

naming of history steps
by Langhorst, Brad 02 Feb '12

02 Feb '12

I'm having a tough time keeping track of which data is which after analysis... I can do a bunch of work customizing each tool and each workflow, renaming results etc. But I think it might be a lot easier to manage if step names were based on the titles of the history items instead of "data 2" or whatever. Has this been tried and rejected for some reason? Would a pull request implementing this change be welcomed? Am I just "doing it wrong"? Any suggestions are welcome. Brad -- Brad Langhorst New England Biolabs langhorst(a)neb.com

3 5

"changeset" inconsistency
by Hans-Rudolf Hotz 02 Feb '12

02 Feb '12

Hi I am in the process of upgrading all our Galaxy servers to the current changeset ("b258de1e6cea", Nov. 18) and I have noticed an inconsistency: If I upgrade one of our old servers (which was on "720455407d1c", June 23) with 'hg pull -u -r b258de1e6cea' I get the following: haruhotz@silicon: hg heads changeset: 6297:b258de1e6cea tag: tip user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Thu Nov 17 15:08:49 2011 -0500 summary: Pack js scripts. haruhotz@silicon: If I create a new installation with 'hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist' I get the following: haruhotz@silicon: hg heads changeset: 6298:b258de1e6cea tag: tip user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Thu Nov 17 15:08:49 2011 -0500 summary: Pack js scripts. changeset: 5832:70b8039a2eef branch: feature/ws parent: 5826:6206a27bd2ae user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Jul 27 10:48:03 2011 -0400 summary: Closed the feature/ws branch, changes from this branch were merged in 5827:f3a1086fac91. haruhotz@silicon: I am not worry about the merged branched, I am just a little bit surprised about the differences: "6298:b258de1e6cea" versus "6297:b258de1e6cea" Can anybody with more mercurial experience enlighten me what went wrong? and even better tell me that there is nothing to worry about... Regards, Hans

2 5

How to import data from a non-local source using SCP?
by Ryan 01 Feb '12

01 Feb '12

I have a cluster (cluster A) set up with Galaxy. Our sequencing data gets mapped to hg19, then the resulting BAM files are placed on a SAN connected to a different cluster (cluster B) that cluster A does not have NFS access to. We cannot install an FTP server on cluster B either. The only way to get data from cluster B to cluster A is to use scp. Is there a way to set up a Data Library in Galaxy on cluster A that refers to non-local data and transfers the data from cluster B when needed? Or is it possible to have a Galaxy instance on cluster B share data with a Galaxy instance on cluster A?

3 2