February 2014 - galaxy-dev - lists.galaxyproject.org

Attempting to bind to AMQP message queue, but kombu dependency unavailable
by Peter Cock 18 Feb '14

18 Feb '14

Hi all, I just updated my development server, and ran into what looks like a missing dependency: $ ./run.sh ... galaxy.jobs.manager DEBUG 2014-02-18 11:58:06,422 Starting job handler galaxy.jobs INFO 2014-02-18 11:58:06,422 Handler 'main' will load all configured runner plugins galaxy.jobs.runners DEBUG 2014-02-18 11:58:06,423 Starting 5 LocalRunner workers galaxy.jobs DEBUG 2014-02-18 11:58:06,425 Loaded job runner 'galaxy.jobs.runners.local:LocalJobRunner' as 'local' galaxy.jobs.runners DEBUG 2014-02-18 11:58:06,460 Starting 3 LWRRunner workers Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/mnt/galaxy/galaxy-central/lib/galaxy/app.py", line 130, in __init__ self.job_manager = manager.JobManager( self ) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/manager.py", line 31, in __init__ self.job_handler = handler.JobHandler( app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 31, in __init__ self.dispatcher = DefaultJobDispatcher( app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 576, in __init__ self.job_runners = self.app.job_config.get_job_runner_plugins( self.app.config.server_name ) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py", line 496, in get_job_runner_plugins rval[id] = runner_class( self.app, runner[ 'workers' ], **runner.get( 'kwds', {} ) ) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/lwr.py", line 45, in __init__ self.client_manager = build_client_manager(**client_manager_kwargs) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/lwr_client/manager.py", line 41, in build_client_manager return MessageQueueClientManager(**kwargs) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/lwr_client/manager.py", line 89, in __init__ self.exchange = LwrExchange(self.url, self.manager_name) File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/lwr_client/amqp_exchange.py", line 37, in __init__ raise Exception(KOMBU_UNAVAILABLE) Exception: Attempting to bind to AMQP message queue, but kombu dependency unavailable $ hg summary parent: 13557:75b13d642f94 tip Allow LWR runner to publish/consume messages via message queue. branch: default commit: 9 modified, 240 unknown update: (current) (Local modifications mostly extra logging lines; nothing which looked relevant here) Regards, Peter

2 3

SnpEff_and_CloudMap_for_other_speices_other_than_worm (galaxy not local)
by Wang, Xiaofei 18 Feb '14

18 Feb '14

Dear, Is it possible to use SnpEff with genomes of other species (e.g. Drosophila) other than C.elegan on galaxy website (not local)? It seems the only option for the genome menu (Caenorhabditis elegants : WS220.64). In fact, I am trying to use CloudMap (the EMS Variant Density Mapping workflow) to analyze data for Drosophila. So, I have to edit the workflow based on my own data. For SnpEff on the workflow, I tried to change "Genome" to other species from Caenorhabditis elegants : WS. But, there is no other options. Then, I changed it "to be set at runtime". When I run the workflow, I still can not change it to other species. Then, I went to see the SnpEff tool on galaxy website and found that there is no other options for "Genome" option. So, could you help me about this? Could I use SnpEff and CloudMap (the EMS Variant Density Mapping workflow) other than worm? Or, I have to install galaxy locally for this objective? Thank you so much! Have a nice day!

3 4

Reference Genome in snpEff Tool
by Clint Christensen 18 Feb '14

18 Feb '14

Howdy! I only see the "Caenorhabditis elegans: WS220.64" reference genome listed in the "Genome" drop down menu in the snpEff tool. If possible, I would like to use the latest human genome hg19. Can it be added as an option? Thanks in advance. Not: I did follow the link to the developer's site, but would prefer to use the tool from within Galaxy. Clint Christensen Research Assistant, Department of Genetics Texas Biomedical Research Institute 7620 NW Loop 410 San Antonio, TX 78245 210-258-9779 clintc(a)txbiomedgenetics.org

3 2

Re: [galaxy-dev] space in text tool results in two arguments
by Ross 18 Feb '14

18 Feb '14

Hi Ketan - great news - all the best with your project... On Tue, Feb 18, 2014 at 3:58 PM, Ketan Maheshwari <ketan(a)mcs.anl.gov> wrote: > Ross, > > That's correct. It seems I tested on a cached copy of tool. Works now! > > Thanks, > Ketan > > > On Mon, Feb 17, 2014 at 10:55 PM, Ross <ross.lazarus(a)gmail.com> wrote: > >> Hi Ketan. >> We routinely recommend quotes as you've shown for this reason and it >> certainly works for me at least ! >> Did you reload the tool xml after editing (via the admin menu or via a >> galaxy restart) before testing? If not, you tested a cached copy of the >> tool! >> >> >> Ross Lazarus MBBS MPH; >> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 >> http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ >> >> >> On Tue, Feb 18, 2014 at 3:40 PM, Ketan Maheshwari < >> ketancmaheshwari(a)gmail.com> wrote: >> >>> Hi Ross, >>> >>> I did try to use double quotes and curly braces: "${outloc}" but it >>> does not seem to address the issue. >>> >>> Thanks, >>> Ketan >>> >>> >>> On Mon, Feb 17, 2014 at 10:37 PM, Ross <ross.lazarus(a)gmail.com> wrote: >>> >>>> Hi Ketan. >>>> Please try quotation marks to enclose any parameter containing spaces >>>> in the tool command template - eg something like: >>>> python myscript.py "$text_with_spaces" $param2 $param3.... >>>> >>>> Please confirm that this solves the problem? >>>> >>>> >>>> >>>> On Tue, Feb 18, 2014 at 12:47 PM, Ketan Maheshwari < >>>> ketancmaheshwari(a)gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> My tool in galaxy accepts text argument which can have zero or more >>>>> spaces depending on user requirements. When user inputs one word it is >>>>> parsed fine but in the case when user inputs more words separated by space >>>>> it results in multiple separate arguments which messes with the way I am >>>>> treating commandline args in my script. Is it possible to tell Galaxy to >>>>> treat textbox as a single argument irrespective of spaces in the values >>>>> provided? >>>>> >>>>> Thanks for any suggestions. >>>>> >>>>> Best, >>>>> -- >>>>> Ketan >>>>> >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> http://lists.bx.psu.edu/ >>>>> >>>>> To search Galaxy mailing lists use the unified search at: >>>>> http://galaxyproject.org/search/mailinglists/ >>>>> >>>> >>>> >>> >>> >>> -- >>> Ketan >>> >>> >> >

1 0

space in text tool results in two arguments
by Ketan Maheshwari 18 Feb '14

18 Feb '14

Hi, My tool in galaxy accepts text argument which can have zero or more spaces depending on user requirements. When user inputs one word it is parsed fine but in the case when user inputs more words separated by space it results in multiple separate arguments which messes with the way I am treating commandline args in my script. Is it possible to tell Galaxy to treat textbox as a single argument irrespective of spaces in the values provided? Thanks for any suggestions. Best, -- Ketan

2 2

Local Galaxy web layout
by shenwiyn＠gmail.com 17 Feb '14

17 Feb '14

Galaxy developers:I want to change my local �Galaxy�web layout,for example,delete the top button of "Visualization" and "Cloud" or change their positions.Could anyone show me some details about this?Thank you very much. shenwiyn(a)gmail.com

3 2

New install and handlers for tools
by Thomas Green 17 Feb '14

17 Feb '14

Galaxy devs, I have just setup a Galaxy instance here in Cardiff University. We have used a VM to host the web frontend and setup DRMAA to connect to our HPC resource. We are still in the process of testing all the functionality it but one aspect which feels a bit strange is how to tell Galaxy to use the web frontend to run the tools which download data. Our HPC resource does not have a connection to the outside world (only the login nodes). I am not sure on the latest version but it seems handlers for tools can only be specified by id not by tool_type or any other "class" descriptor so non-computational tools can be run elsewhere whilst the computational tasks are run on HPC - I am currently using the id for each tool to specify a destination in the job_conf.xml. Am I missing something obvious? Another aspect I found whilst installing this was the delicate nature of permissions when integrating it with the cluster. This is especially true if ACLs are also used on the filesystem. It seems the versions I installed (a version from November) could fail without much information why it failed in the logs. Also had occasions where Galaxy never found out a job failed even though the log file had an error and traceback (usually when the DRMAA submit failed). Any advice (or documentation) what the disk permissions should be? And if anyone else is using NFS then root_squash on NFS really does cause issues. So after a bit of tweaking and getting used to the structure of the code I got something which seemed to work - I have possible a few fixes which others might find useful mainly to give better feedback in the log. Any advice how best to integrate these back to you - is there a ticketing system to track changes? I must really upgrade Galaxy to a more recent version to see whether these have already been fixed. Many thanks for all the work that goes into Galaxy and look forward collaborating with the community. All the best, Tom -- Thomas Green Senior Programmer ARCCA, Redwood Building (Tower), King Edward VII Avenue, Cardiff, Wales, CF10 3NB, UK Tel: +44 (0)29 208 79269 Fax: +44 (0)29 208 70734 Email: greent10(a)cardiff.ac.uk Web: http://www.cardiff.ac.uk

2 1

Parsing input format in galaxy tools
by Nikos Sidiropoulos 14 Feb '14

14 Feb '14

Hi all, I'm writing a tool in Galaxy and I need to parse the input file's format (bam, bed, etc) to pass it as a variable to the command. Something like: command.sh $input ${input.format} $output Is there a way to get it? Bests, Nikos

2 2

Error in Galaxy Reports
by Nikos Sidiropoulos 14 Feb '14

14 Feb '14

Hi all, after the February 2014 galaxy update I'm experiencing errors in Galaxy Reports. It only occurs when I'm trying to see which jobs were ran at a specific date (e.g. I can see how many jobs were ran today, but when I click on the number to see which ones it returns the following error): URL: http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_… Module paste.exceptions.errormiddleware:*144* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> app_iter *=* self*.*application*(*environ*,* sr_checker*)* Module paste.debug.prints:*106* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> environ*,* self*.*app*)* Module paste.wsgilib:*543* in intercept_output >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> app_iter *=* application*(*environ*,* replacement_start_response*)* Module paste.recursive:*84* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* self*.*application*(*environ*,* start_response*)* Module paste.httpexceptions:*633* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* self*.*application*(*environ*,* start_response*)* Module galaxy.web.framework.base:*132* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* self*.*handle_request*(* environ*,* start_response *)* Module galaxy.web.framework.base:*190* in handle_request >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> body *=* method*(* trans*,* ****kwargs *)* Module galaxy.webapps.reports.controllers.jobs:*177* in specified_date_handler >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* self*.*specified_date_list_grid*(* trans*,* ****kwd *)* Module galaxy.web.framework.helpers.grids:*296* in __call__ >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> kwargs*=*kwargs *)* Module galaxy.web.framework:*1184* in fill_template >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* self*.*fill_template_mako*(* filename*,* ****kwargs *)* Module galaxy.web.framework:*1199* in fill_template_mako >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* template*.*render*(* ****data *)* Module mako.template:*296* in render >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* runtime*.*_render*(*self*,* self*.*callable_*,* args*,* data*)* Module mako.runtime:*660* in _render >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> ****_kwargs_for_callable*(*callable_*,* data*)**)* Module mako.runtime:*692* in _render_context >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> _exec_template*(*inherit*,* lclcontext*,* args*=*args*,* kwargs*=*kwargs*)* Module mako.runtime:*718* in _exec_template >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> callable_*(*context*,* ***args*,* ****kwargs*)* Module _base_mako:*45* in render_body >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> __M_writer*(*unicode*(*next*.*body*(**)**)**)* Module _grid_base_mako:*134* in render_body >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> __M_writer*(*unicode*(*self*.*load*(**)**)**)* Module _grid_base_mako:*108* in render_load >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> self*.*grid_javascripts*(**)* Module _grid_base_mako:*517* in render_grid_javascripts >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> __M_writer*(*unicode*(*h*.*to_json_string*(*self*.*grid_config*)**)**)* Module json:*231* in dumps >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* _default_encoder*.*encode*(*obj*)* Module json.encoder:*200* in encode >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> chunks *=* self*.*iterencode*(*o*,* _one_shot*=*True*)* Module json.encoder:*263* in iterencode >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *return* _iterencode*(*o*,* *0**)* Module json.encoder:*177* in default >> <http://localhost:9001/jobs/specified_date_handler?webapp=reports&specified_…> *raise* TypeError*(*repr*(*o*)* *+* " is not JSON serializable"*)* *TypeError: datetime.datetime(2014, 2, 13, 12, 12, 24, 966618) is not JSON serializable* Bests, Nikos

3 4

WARNING:galaxy.datatypes.registry:Overriding conflicting datatype
by Jim Johnson 14 Feb '14

14 Feb '14

I now have 2 versions of snpeff installed from the toolshed on my galaxy server. Each snpeff version includes an identical datatypes_conf.xml The galaxy server is setting metadata externally. When any job runs, (in may case I was running a picard tool), the following message is written to the job stderr: WARNING:galaxy.datatypes.registry:Overriding conflicting datatype with extension 'snpeffdb', using datatype from /galaxy/database/tmp/tmpdKGVnQ. Without stdio tags or otherwise catching the stderr, the job state is set to error. I'm now assuming that best practice would be to put the datatypes in a separate toolshed repository that would be a dependency for the tools repository. Thus the datatypes would not loaded multiple times and avoid overrride conflicts. In the meantime, I have just commented out the warning message. Any other advice on this issue? Thanks, JJ -- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota

3 4