August 2011 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] faToTwoBit (and other Kent tools like liftOver)
by Peter Cock 12 Aug '11

12 Aug '11

On Fri, Aug 12, 2011 at 11:38 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > Hi all, > > I recently started trying "Extract Genomic DNA" which requires > faToTwoBit, which wasn't listed on the dependencies page > (fixed now), http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies > > See also: > http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-August/006366.html > > In updating the wiki I've assumed faToTwoBit comes from the same > place as twoBitToFa, where the link given is simply: > http://hgdownload.cse.ucsc.edu/admin/exe/ > > That just contains pre-compiled binaries - but there is no sign > of the source code or a licence file. Are you just using these > binaries? Would I be right to guess the Penn State cluster is all 64bit Linux? Some of our cluster nodes are just 32bit, so I wanted to grab the 32bit binaries for faToTwoBit, twoBitToFa and liftOver - but this folder just contains liftOver at the moment: http://hgdownload.cse.ucsc.edu/admin/exe/linux.i386/ I guess I'll have to compile faToTwoBit and twoBitToFa from source... > .. > The licensing is murky, apparently free to use for personal, > academic, and non-profit use - but no mention about potential > redistribution or modification rights: > http://hgwdev.cse.ucsc.edu/~kent/src/unzipped/README > >> ... All files are copyrighted, but license is hereby granted >> for personal, academic, and non-profit use. license is >> also granted for the contents of the top level lib, inc and >> utils directories for commercial users. ... > > Have the Galaxy team made special arrangements to license > faToTwoBit for use on the Penn State public server, or is this > fine because faToTwoBit is under the top level utils directory? That's fine for faToTwoBit and twoBitToFa, but... What about the liftOver tool? This is under the hg top level folder, and thus is NOT licensed for commercial users - only for personal, academic, and non-profit use. How is Galaxy able to use this tool on the public Penn State server? Regards, Peter

1 0

Fixed: Error uploading files through filesystem paths: MacOS, Postgres and galaxy-central
by Camille Stephan 11 Aug '11

11 Aug '11

Sorry guys, didn't change file permissions for the new 'galaxy' user. Thanks anyway, Camille. -- *** Camille Stephan-Otto Attolini, PhD Senior Research Officer, Bioinformatics and Biostatistics unit IRB Barcelona Tel (+34) 93 402 0553

1 0

What happened to 'normal' priority on the issue tracker?
by Peter Cock 11 Aug '11

11 Aug '11

Hi all, I'm just filing a bug on bitbucket, and noticed the default priority is now major (!) We can choose: Trivial, Minor, Major, Critical, Blocker Wasn't there previously a 'Normal' which was the default? Peter

3 2

using <validator> tags with regular expressions or mathematical expressions...
by Nikhil Joshi 09 Aug '11

09 Aug '11

Hi all, I would like to use the validator tag to validate data using either regexes or math expressions. How can I do this? I tried using the "regex" type, but I'm not sure where to put the regex itself. I tried putting it in a file that was in the filename attribute, but that didn't seem to work. Essentially, I want to check if an integer input is even or not. - Nik. -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center University of California, Davis Davis, CA http://bioinformatics.ucdavis.edu

1 1

syntax and #if and #for statements within the <command> tag
by Nikhil Joshi 09 Aug '11

09 Aug '11

Hi all, So I've gotten some simple custom tools to work on our local Galaxy, however, I am having trouble figuring out exactly how the syntax works for the commands and lines within the <command> tag. E.g., I've seen #if and #for statements and also lines using functions like "str", but I can't really find any documentation about exactly how the syntax works. It kinda seems like PHP syntax, but not quite? Anyways, any help would be highly appreciated. Thanks! - Nik. On Wed, Aug 3, 2011 at 6:20 AM, Greg Von Kuster <greg(a)bx.psu.edu> wrote: > Hello Nikhil, > > I am responsible for the tool shed ( both implementation as well as > documentation ), and finally got a chance to get the wiki started yesterday > (that's why it is currently lacking in information). I will be working on > the wiki as the top priority now, so it should be fairly useful within the > next day or so. In the meantime, please feel free to direct any questions > to the mail list, and I'll answer them in short order. > > Getting started with the tool shed is fairly straight-forward. You can > browse existing repositories to see how others have designed their file > hierarchies. Many very good examples of repositories have been created by > Peter Cock ( his public username on the tool shed is peterjc ), so browse > some of his repositories to get an idea. Here's a good example of a repo > that includes a single tool that uses a .loc file: > > Contents: > effectivet3<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > tool-data/<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > effectiveT3.loc.sample<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > tools/<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > protein_analysis/<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > effectiveT3.py<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > effectiveT3.txt<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > effectiveT3.xml<http://toolshed.g2.bx.psu.edu/repository/browse_repository?id=92c548528a9a6…> > > > You can browse repos without authenticating, but in order to create a repo, > you'll need to authenticate, so create an account on the tool shed if you > don't yet have one. > > Regarding your questions about the "XML format for the XML file", the > repository is simply a container for files of any type ( tool configs, data, > binaries, python scripts, exported Galaxy workflows, etc ), so the > repository itself has no requirement for XML formats, etc. Generally, XML > files will simply be the Galaxy tool config, the syntax of which is defined > on our wiki at > http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax > > Thanks very much for you interest in the tool shed, > > Greg Von Kuster > > > On Aug 2, 2011, at 10:36 PM, Nikhil Joshi wrote: > > Hi all, > > I have been using Galaxy for a few years now and I am about to start > creating my own repositories for the new Tool Shed. However, I can't seem > to find any good documentation on how to do that exactly. Can someone point > me in the right direction? I need to figure out the best way to organize a > repository, the XML format for the XML file, and any other important things > that I might need to know? Is there a good resource for all that? The wiki > has a section entitled "Create a new repository", but there is nothing in > it. Any help would highly appreciated. Thanks! > > - Nik. > > -- > Nikhil Joshi > Bioinformatics Programmer > UC Davis Genome Center > University of California, Davis > Davis, CA > http://bioinformatics.ucdavis.edu > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > Greg Von Kuster > Galaxy Development Team > greg(a)bx.psu.edu > > > > -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center University of California, Davis Davis, CA http://bioinformatics.ucdavis.edu

2 1

Failure to prepare job error
by Faiaz Khan 09 Aug '11

09 Aug '11

Hello, I'm running a Python/R tool from my Amazon Galaxy instance. To use R, I import a package I've installed called PypeR. My tool works fine from the command line, but when I try to run it on my Amazon instance, it gives me this error: Traceback (most recent call last): File "/nfs/software/galaxy/lib/galaxy/jobs/runners/local.py", line 61, in run_job job_wrapper.prepare() File "/nfs/software/galaxy/lib/galaxy/jobs/__init__.py", line 371, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/nfs/software/galaxy/lib/galaxy/tools/__init__.py", line 1543, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/nfs/software/galaxy/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/nfs/software/galaxy/eggs/Cheetah-2.2.2-py2.6-linux-i686-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1311891781_86_41894.py", line 83, in respond EOFError: EOF when reading a line With the additional caption "failure to prepare job". I've been struggling with this for days, and I'm pretty sure I don't have a typo in my .xml file. How can I solve this? Thanks, Rafi

2 1

Catch problems preparing cluster job scripts
by Peter Cock 09 Aug '11

09 Aug '11

Hi all, I've run into some file permissions problems as part of using the same mapped directory on both the Galaxy server and our cluster. In the process I wrote the following patch which fixes the following bug - where Galaxy seems to leave the job in the pending state: galaxy.jobs INFO 2011-07-29 14:06:46,170 job 30 dispatched galaxy.jobs.runners.drmaa ERROR 2011-07-29 14:06:46,582 Uncaught exception queueing job Traceback (most recent call last): File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 114, in run_next self.queue_job( obj ) File "/data/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 164, in queue_job os.chmod( jt.remoteCommand, 0750 ) OSError: [Errno 1] Operation not permitted: '/data/galaxy/galaxy-central/database/pbs/galaxy_30.sh' The jobs was left stuck in the grey pending state. It looks like this exception should have been caught and the job put into an error state as in this patch: https://bitbucket.org/peterjc/galaxy-central/changeset/c5fa48633c0b This is currently the one and only change on this branch: https://bitbucket.org/peterjc/galaxy-central/src/job_scripts Please could this be reviewed and applied to the trunk. Thanks, Peter

2 4

[API] About workflow parameters :)
by Louise-Amélie Schmitt 09 Aug '11

09 Aug '11

Hello, Just a quick message to ask if you had the time to work on passing parameters to workflows through the API. Just to know, since I'm using workflows with the API. Best, L-A

2 3

Re: [galaxy-dev] [galaxy-user] configuring apache proxy server issue
by Jennifer Jackson 08 Aug '11

08 Aug '11

Hello, I am going to move this over to the galaxy-dev list so that the larger development community can more easily access your question and provide some feedback. Thanks! Jen Galaxy team On 8/4/11 12:55 PM, Sridhar A Malkaram wrote: > Hi, > > I have been using galaxy succesfully for my work using its internal > web-server. Recently I wanted to use it over the internet and so, I > configured it with apache at /galaxy of apache's root. I followed all > the instructions in setting up the proxy redirection. But I think the > re-direction is not happening. I highly appreciate any suggestions to > correct the issue. > > I checked for the mod_proxy, mod_proxy_http etc modules and they are > present and switched on. > > Apache gives the following error in its error_log file: > > [Thu Aug 04 14:15:56 2011] [error] [client 127.0.0.1] File does not > exist: /usr/local/apache/htdocs/galaxy > > Galaxy was set up in the directory /home/galaxy/galaxy-dist > The lines specific for proxy setting in universe_wsgi.ini are as follows: > use = egg:Paste#http > port = 8080 > host = localhost > [filter:proxy-prefix] > use = egg:PasteDeploy#prefix > prefix = /galaxy > [app:main] > filter-with = proxy-prefix > cookie_path = /galaxy > use_remote_user = True > remote_user_maildomain = unl.edu <http://unl.edu> > > httpd.conf settings: > > RewriteEngine on > <Proxy http://localhost:8080> > Order deny,allow > Allow from all > </Proxy> > <Location "/galaxy"> > Order deny,allow > Allow from all > # Take the $REMOTE_USER environment variable and set it as a header > in the proxy request. > RewriteCond %{IS_SUBREQ} ^false$ > RewriteCond %{LA-U:REMOTE_USER} (.+) > RewriteRule . - [E=RU:%1] > RequestHeader set REMOTE_USER %{RU}e > RequestHeader set X-URL-SCHEME https > RewriteRule ^/galaxy$ /galaxy/ [R] > RewriteRule ^/galaxy/(.*) http://localhost:8080/$1 [P] > # Compress all uncompressed content. > SetOutputFilter DEFLATE > SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary > SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary > </Location> > > > > > Thanks, > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support

3 2

Filtering Illumina CASAVA 1.8 FASTQ files
by Assaf Gordon 07 Aug '11

07 Aug '11

Hello, With the recent upgrade to CASAVA 1.8, Illumina's FASTQ files now contain both filtered and non-filtered reads (i.e. high and low quality reads, what used to be called "reads that passed filtering"). Here's a program (+galaxy wrapper) that can filter those FASTQ files, preparing them for the down-stream genome mapping programs (Probably a temporary fix until mapping programs add a feature to ignore those reads). http://cancan.cshl.edu/labmembers/gordon/fastq_illumina_filter/ Comments are welcomed, -gordon

1 0