March 2012 - galaxy-dev - lists.galaxyproject.org

BOSC 2012 Call for Abstracts
by Brad Chapman 05 Mar '12

05 Mar '12

Call for Abstracts for the 13th Annual Bioinformatics Open Source Conference (BOSC 2012) A Special Interest Group (SIG) of ISMB 2012 Dates: July 13-14, 2012 Location: Long Beach, California Web site: http://www.open-bio.org/wiki/BOSC_2012 Email: bosc(a)open-bio.org BOSC announcements mailing list: http://lists.open-bio.org/mailman/listinfo/bosc-announce Important Dates: April 13, 2012: Deadline for submitting abstracts May 7, 2012: Notification of accepted talk abstracts emailed to authors July 11-12, 2012: Codefest 2012 programming session July 13-14, 2012: BOSC 2012 July 15-17, 2012: ISMB 2012 The Bioinformatics Open Source Conference (BOSC) is sponsored by the Open Bioinformatics Foundation (O|B|F), a non-profit group dedicated to promoting the practice and philosophy of Open Source software development within the biological research community. To be considered for acceptance, software systems representing the central topic in a presentation submitted to BOSC must be licensed with a recognized Open Source License, and be freely available for download in source code form. We invite you to submit one-page abstracts for talks and posters. This year's session topics are: - Cloud and Parallel Computing - Linked Data - Genome-scale Data Management - Data Visualization and Imaging - Translational Bioinformatics - Software Interoperability (possibly a joint session with BSI-SIG, the Bioinformatics Software Interoperability SIG) - Bioinformatics Open Source Project Updates - Interfacing with Industry (panel) Thanks to generous sponsorship from Eagle Genomics and an anonymous donor, we are pleased to announce a competition for three Student Travel Awards. Each winner will be awarded $250 to defray the costs of travel to BOSC 2012. For instructions on submitting your abstract, please visit http://www.open-bio.org/wiki/BOSC_2012#Submitting_Abstracts BOSC 2012 Organizing Committee: Nomi Harris (chair), Jan Aerts, Brad Chapman, Peter Cock, Chris Fields, Erwin Frise, Peter Rice

1 0

Defining Job Runners Dynamically
by John Chilton 05 Mar '12

05 Mar '12

Hello All, I just issued a pull request that augments Galaxy to allow defining job runners dynamically at runtime (https://bitbucket.org/galaxy/galaxy-central/pull-request/12/dynamic-job-run…) Whether it makes the cut or not, I thought I would describe enhancements here in case anyone else would find it useful. There a couple use cases we hope this will help us address for our institution - one is dynamically switching queues based on user (we have a very nice shared memory resource that can only be used by researchers with NIH funding) and the other is inspecting input sizes to give more accurate max walltimes to pbs (a small number of cufflinks jobs for instance take over three days on our cluster but defining max walltimes in excess of that for all jobs could result in our queue sitting idle around our monthly downtimes). You might also imagine using this to dynamically switch queues entirely based on input sizes or parameters, or alter queue priorities based on the submitting user or input sizes/parameters. There are two steps to use this - you must add a line in universe.ini and define a function to compute the true job runner string in the new file lib/galaxy/jobs/rules.py. This first step is similar to what you would do to statically assign a tool to a particular job runner. If you would like to dynamically assign a job runner for cufflinks you would start by adding a line like one of the following to universe.ini cufflinks = dynamic:///python -or- cufflinks = dynamic:///python/compute_runner If you use the first form, a function called cufflinks must be defined in rules.py, adding the extra argument after python/ lets you specify a particular function by name (compute_runner in this example). This second option could let you assign job runners with the same function for multiple tools. The only other step is to define a python function in rules.py that produces a string corresponding to a valid job runner such as "local:///" or "pbs:///queue/-l walltime=48:00:00/". If the functions defined in this file take in arguments, these arguments should have names from the follow list: job_wrapper, user_email, app, job, tool, tool_id, job_id, user. The plumbing will map these arguments to the implied galaxy object. For instance, job_wrapper is the JobWrapper instance for the job that gets passed to the job runner, user_email is the user's email address or None, app is the main application configuration object used throughout the code base that can be used for instance to get values defined in universe.ini, job, tool, and user are model objects, and job_id and tool_id the relevant ids. If you are writing a function that routes a certain list of users to a particular queue or increases their priority, you will probably only need to take in one argument - user_email. However, if you are going to look at input file sizes you may want to take in an argument called job and use the following piece of code to find the input size for input named "input1" in the tool xml. inp_data = dict( [ ( da.name, da.dataset ) for da in job.input_datasets ] ) inp_data.update( [ ( da.name, da.dataset ) for da in job.input_library_datasets ] ) input1_file = inp_data[ "input1" ].file_name input1_size = os.path.getsize( input1_file ) This whole concept works for a couple of small tests on my local machine, but there are certain aspects of the job runner code that makes me feel there may be corner cases I am not seeing where this approach may not work - so your millage may vary. -John ------------------------------------------------ John Chilton Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223 E-Mail: chilton(a)msi.umn.edu

3 2

Production environment and Apache's sendfile
by Jennifer Jackson 05 Mar '12

05 Mar '12

Hello, I am going to forward you question to the galaxy-dev(a)bx.psu.edu mailing list so that the development community can offer advice. http://wiki.g2.bx.psu.edu/Support#Mailing_Lists http://wiki.g2.bx.psu.edu/Mailing%20Lists#The_lists Thanks, Jen Galaxy team On 3/2/12 8:00 AM, Yuri D'Elia wrote: > Hi everyone. I've set up a galaxy production environment and it seems to be working. > I've identified a couple of issues though: > > If I enable apache_xsendfile, the little "eye" icon ("Display data in browser") on each dataset stops working. I get the following instead: > > The requested URL /datasets/7e93e3cf926fb2b3/display/ was not found on this server > > My document root is not the same as the galaxy root (as suggested). Everything else seems to work. Is this a bug? > > Second question: > > I want to rotate the galaxy log file (as specified with paster.py serve --log-file. I can't find a way to reopen the logfile in galaxy without taking down the whole service, or a way to redirect the log to a pipe together while using --daemon. > > I cannot let the file grow forever. > Any suggestions? > > Thanks. > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

1 0

snpEff: html report is not displaying after update
by Jennifer Jackson 05 Mar '12

05 Mar '12

Hello Raj, This tool is from the Tool Shed and is maintained by the tool author. To contact them about functionality or problems, go to the Tool Shed at http://toolshed.g2.bx.psu.edu/, locate the repository (search by 'snpEff'), click on the name, in the the top far right corner locate the pull down menu "Repository Actions". Select "Contact repository owner" to send them a message. There have been many updates to Galaxy recently and it is possible that one of these had an impact on this tool. I am going to send your email to galaxy-dev(a)bx.psu.edu, to see if anyone else has had problems with this tool since the last distribution update. You may be asked to provide more information (be as specific as you can): the hg pull #, any customizations, python version, etc. Regards, Jen Galaxy team On 2/14/12 8:28 PM, Praveen Raj Somarajan wrote: > Hi All, > > I updated galaxy recently to the latest version. Everything looks fine, > except snpEff report html view. It was displaying properly (all tables > and summary values) before the update, but the summary values are not > displaying after the update. A sample screen-shot is attached for your > reference. Could you please figure out this issue? > > When I ran the same on command line, the reports were generated > correctly. I assume, something (datatypes or preview) has changed by the > update. Please let me know the work around on this? > > Secondly, as we know, snpEff also generates a gene-wise annotation file > along with other results, but somehow we cannot access this file through > Galaxy. Though we see the link in the html report, but it seems the path > is broken. > > Let me know your suggestions. > > Best, > > Raj > > > ------------------------------------------------------------------------ > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended > solely for the use of the addressee(s). If you are not the intended > recipient, please notify the sender by e-mail and delete the original > message. Further, you are not to copy, disclose, or distribute this > e-mail or its contents to any other person and any such actions that are > unlawful. This e-mail may contain viruses. Ocimum Biosolutions has taken > every reasonable precaution to minimize this risk, but is not liable for > any damage you may sustain as a result of any virus in this e-mail. You > should carry out your own virus checks before opening the e-mail or > attachment. > The information contained in this email and any attachments is > confidential and may be subject to copyright or other intellectual > property protection. If you are not the intended recipient, you are not > authorized to use or disclose this information, and we request that you > notify us by reply mail or telephone and delete the original message > from your mail system. > > OCIMUMBIO SOLUTIONS (P) LTD > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

2 1

Problems getting galaxy to start
by michael burrell (NBI) 05 Mar '12

05 Mar '12

Good evening, I am having difficulty getting galaxy to startup over the last couple of days, I initially suspected that it was an issue with the eggs, but have replaced these and the following errors remain, galaxy@jic55119: $ python --version Python 2.6.6 galaxy@jic55119:galaxy-central$ ./run.sh Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/home/home/galaxy/software/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/home/home/galaxy/software/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/home/home/galaxy/software/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 218, in run result = self.command() File "/home/home/galaxy/software/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/home/home/galaxy/software/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 224, in loadobj global_conf=global_conf) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 248, in loadcontext global_conf=global_conf) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 278, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 413, in get_context section) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 458, in _context_from_explicit value = import_string(found_expr) File "/home/home/galaxy/software/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 18, in import_string return pkg_resources.EntryPoint.parse("x="+s).load(False) File "/home/home/galaxy/software/galaxy-central/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/home/home/galaxy/software/galaxy-central/lib/galaxy/web/buildapp.py", line 19, in <module> from galaxy import config, jobs, util, tools File "/home/home/galaxy/software/galaxy-central/lib/galaxy/jobs/__init__.py", line 4, in <module> from galaxy import util, model EOFError: EOF read where object expected Any suggestions? Thanks in advance Michael.

1 0

large ftp uploads to galaxy cloud
by bermans＠mskcc.org 05 Mar '12

05 Mar '12

I'm having difficulty uploading files > 6GB or so to an EC2 instantiation of galaxy cloud. The size seems to correlate with the amount of space available on the root volume. I can upload a 5.06 GB file but any larger and it never shows up in the galaxy upload history. I'm using a "Large" EC2 instance but the root volume is only 15GB and only has 5.9GB free. I'm using the 861460482541/galaxy-cloudman-2011-03-22 ===================================================================== Please note that this e-mail and any files transmitted from Memorial Sloan-Kettering Cancer Center may be privileged, confidential, and protected from disclosure under applicable law. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this communication or any of its attachments is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting this message, any attachments, and all copies and backups from your computer.

1 0

functional test issue - PDF output file comparison
by Anne Pajon 05 Mar '12

05 Mar '12

Dear All, I have noticed that some output files of functional tests are of type PDF. When comparing the one produced by the tool and the one stored in test-data/, they obviously differ even if they visually look identical. Is there a better way to compare PDF files within Galaxy than using diff? If not, do you have experience or suggestion at solving these functional test failures? Thanks a lot in advance. Kind regards, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

1 0

Problem with python ./scripts/make_egg_packager.py
by Jose William Valero Velasquez 05 Mar '12

05 Mar '12

Hi, I'm writing you because i'm trying to install galaxy in a server running suse behind a proxy, so i have to build the eggs like if the server were offline, in the guide says that i should execute the command ./scripts/make_egg_packager.py but i get the following error, and i don't really understand whats happening: File "scripts/make_egg_packager.py", line 21, in <module> c = Crate() TypeError: __init__() takes at least 2 arguments (1 given) I'd really apreciate your help to figure out how to solve this issue. William Valero Universidad Nacional de Colombia.

2 1

Progress indicators/feedback for long jobs on Galaxy
by Jennifer Jackson 05 Mar '12

05 Mar '12

Hi Greg, I am going to forward your email over to the galaxy-dev(a)bx.psu.edu mailing list where the development community can provide feedback. Best, Jen Galaxy team On 3/4/12 3:26 PM, Greg Edwards wrote: > Hi, > > This has been raised a number of times .. one issue is at ... > > https://bitbucket.org/galaxy/galaxy-central/issue/250/progress-of-running-j… > > but seems to have been dropped from Open to On Hold. There was another > similar request in the issues list but I can't find it just now. > > I'd be gald to know if anything is happening on the idea of simple > progress indicators. The first things that were raised by my "users" > were 1) Don't know how long a data upload is going to take, and b) Don't > know what's happening with my tool execution. (And these were both > operations of only 5-10 minutes or so, they'd go crazy with the multi-GB > uploads and hours-long runs I see reported here). > > > I've implemented my own email-based scheme on a AWS EC2 CloudMan Galay, > seems to be working ok. It uses an Ubuntu package "sendEmail", which is > a self-conatined simple SMTP email sender. It sends progress emails out > of the main Perl tool code, via a bash script, every 1 to N minutes > (selectable), to an email address supplied by the user. The emails have > configurable subject and content with brief display of Start, Progress > (every N minutes) and Success/Failure termination status. > > You have to jump through some hurdles. AWS thinks your'e a spammer and > blocks emails after a while ! but you can submit a request to get around > that. > > If anyone's interested let me know. > > Cheers, > Greg E > > > -- > Greg Edwards, > Port Jackson Bioinformatics > gedwards2(a)gmail.com <mailto:gedwards2@gmail.com> > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

1 0

Dynamic feedback whilst setting tool-parameters in Galaxy web-interface
by Tim Dorscheidt 05 Mar '12

05 Mar '12

Hi, I've been writing several wrapper-files for statistical tools, and I was wondering whether it was possible to enhance the tool-interface currently available in the Galaxy web-interface. Ideally, I am looking for options to already display (some) tool-feedback in the user-interface of a tool dependent on the dataset selected from the "history" and user-defined parameters. An example would be a graph re-drawing itself every time the user selects a different dataset or alters a parameter, but simple text-feedback that dynamically updates itself would be a great start already. The "execute" button is now the only way to run code on user-input, and this makes Galaxy a rather cumbersome environment in the case of tools that require many iterations (or just when playing around with parameters). So far, the only limited example I've seen of such "dynamic feedback" is when the user select a matrix from her "history" and Galaxy is able to read the amount of columns, which strictly speaking is a form of generating immediate feedback on the basis of user-selected input, but of course I'd like to define the code myself that is run on the input and what sort of feedback is derived and displayed from this. I have seen the new "visualizations" section of Galaxy, which indeed allows for some of this, but it is clearly separate from the standard tool-funcitonality of Galaxy, and it does not allow for workflows (whereas the above functionality does not clash with workflows, which simply needs to access the static version of the tool). Any help would be greatly appreciated. Tim Dorscheidt Scientific Programmer Biosystems Data Analysis group, SILS University of Amsterdam

1 0