January 2016 - galaxy-dev - lists.galaxyproject.org

Galaxy sending jobs to multiple clusters
by Nikolay Aleksandrov Vazov 04 Feb '16

04 Feb '16

Hi, John, Dan, Carrie and all others, I am considering a task of setting up a Galaxy instance which shall send jobs to more than on cluster at a time. In my case I am using drmaa-python and I was wondering if it was possible to configure multiple drmaa runners each "pointing" at a different (slurm) control host, e.g. local drmaa1 drmaa2 Thanks a lot for your advice Nikolay =============== Nikolay Vazov, PhD Department for Research Computing, University of Oslo

4 9

REST API - Tool input information
by Koehorst, Jasper 01 Feb '16

01 Feb '16

Is it possible to get tool input information through the REST api? For example, I would like to know for a specific tool to retrieve the inputs needed and the types of inputs being either integer, dataset, string, etc… Kind regards, Jasper Koehorst

2 1

Running Galaxy jobs remotely via ssh / rsh
by D K 01 Feb '16

01 Feb '16

Hi, I would like to run the Galaxy framework on server A, while performing all of the jobs on server B using ssh. Looking at the documentation here: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CLI, this seems like it should be possible. However, the documentation states that the cli runner requires, at a minimum, two parameters, one for the shell (which I'm selecting SecureShell) and a Job plugin. I'm not sure what this should be since the ones available are Torque, Slurm, and SlurmTorque and I'm not running any of these. Can anyone give me any hints? My current job_conf.xml looks like this: <?xml version="1.0"?> <job_conf> <plugins>  <plugin id="cli" type="runner" load="galaxy.jobs.runners.cli:ShellJobRunner"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="cli_default"> <destination id="cli_default" runner="cli"> <param id="shell_plugin">SecureShell</param> <param id="job_plugin">cli</param> <param id="shell_hostname">computehost</param> <param id="shell_username">galaxy</param> </destination> </destinations> </job_conf> As an alternative, in a tool xml I tried directly sshing between the <command> tags e.g.: <command>ssh galaxy@serverB 'sh $__tool_directory__/runTool.sh --verbose > $output 2>&1' </command> This appears to work. Is it possible to "inject" this ssh command into tool execution in Galaxy? I was looking to do this to avoid having to install a scheduler like SGE or using Pulsar. Any suggestions would be greatly appreciated. Thanks!

2 3

RedHat versus CentOs
by Hans-Rudolf Hotz 01 Feb '16

01 Feb '16

Dear All Our production Galaxy server has been running smoothly on a single multicore box with Red Hat 6.7. As this box gets older and older, it is in now time to replace it. At the same time, we are also considering switching the operating system to CentOS 7. Has anyone switched a production installation from Red Hat to CentOS ? 'Galaxy Main' is running on CentOS, isn't-it? I am also interested in plain numbers: how many of you are using Red Hat, how many CentOS and which version for your production server. I will probably need such numbers to convince our IT department. There is no need to reply to the mailing list, just reply to me (hrh(a)fmi.ch) and if I get enough I will then collate and anonymize all the data and post back. Maybe we can even add such information to the wiki (https://wiki.galaxyproject.org/Community/Deployments)? Thank you very much for your help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland

3 3

Current state of cloudman etc. within private AWS VPCs?
by Chris Dagdigian 29 Jan '16

29 Jan '16

Apologies if this is asked often, I did a bit of searching and google-foo but could not find very new news... I've long been a fan/user of cloudman-managed Galaxy clusters but for the first time I need galaxy in an AWS environment where we absolutely cannot stand up any sort of public facing service or any sort of host with a visible EIC IP address. The hosts can access the internet through NAT etc. but we can't run them in a public VPC or otherwise expose them to the Internet at large. My experiment last night with cloudman launching inside a private VPC (with NAT'ed internet access and working forward/reverse DNS resolution for all hosts in the subnet) seemed to have failed - the key error in the cloudman logs was an endless iteration of "attempting to learn public hostname" log entries. I blew away the install when I saw that error message endlessly repeating because there is zero chance our hosts in this VPC subnet will ever have a public FDQN hostname. They all have private IPs and private (but DNS resolvable) hostnames. Can anyone give a current state update on cloudman inside private VPCs? Is this something I should have debugged further or am I better off just rolling my own AMI and installing galaxy and perhaps the cloudman elements by hand? Thanks! -Chris

4 3

'bp_genbank2gff3' installation error and running error
by Wang, Xiaofei 28 Jan '16

28 Jan '16

Dear there, I'd like to convert genebank flat file to GFF. So, I am trying to install this tool shed [cid:2d047a8e-cfc4-4e3e-97b4-5225f8dde128] to my local galaxy. But I got an error as below. Does it associated with dependencies? I see it needs these ? [cid:51e76e29-daa7-4f92-a9a8-3687d969f3e6] . How can I check if I have all these? ?[cid:53e1eb6a-f27d-471c-8122-0d575a63c4f2] Also, I see there is the same tool in this website https://galaxy.cbio.mskcc.org/root?tool_id=fml_gbk2gff (see below). It should be also galaxy but I need to register an account for it. Do you know what is the relationship between this website and the galaxy? BTW, I received an error when I used the tool on this website. I think if they are same tool, the galaxy team should know the reason for it. Here is the error as below. GFF Toolkit (v0.1) GROOMING PROGRAMS Examine_GFF<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=gff_examine> file Parent-Child relations FeatureScan<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=gff_available_limits> gives an overview about GFF feature content CONVERTER PROGRAMS GTF_to_GFF3<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=fml_gtf2gff3> converter BED_to_GFF3<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=fml_bed2gff> converter GFF3_to_BED<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=fml_gff2bed> converter GBK-to-GFF<https://galaxy.cbio.mskcc.org/tool_runner?tool_id=fml_gbk2gff> converter ?[cid:56985bc0-77c2-4719-b0f9-d4b36aeb03c0] Thanks! Best, Xiaofei

2 2

Re: [galaxy-dev] Loading or Upload Large datasets into Galaxy History
by Martin Čech 28 Jan '16

28 Jan '16

Amos, I have no experience with the tool you mentioned and I am not surprised it does not work correctly as it has been updated 4years ago and Galaxy has changed in the meantime. If you want to get your local data to your local Galaxy the best option is to import it into Data Library and use 'link to files' option which will prevent creating duplicates of the dataset. Some details are available here: https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles I hope this helps. thanks Martin On Thu, Jan 28, 2016 at 3:14 PM Raphenya, Amogelang <raphenar(a)mcmaster.ca> wrote: > Hi Martin, > > I have local instance. > > I uploaded 1GB fastq compressed with *.gz* using *Upload_local_file *tool > (installed from galaxy tool shed). > > After the upload when I run *fastqc* it failed BUT when I upload the same > file using *Upload_file, **fastqc *works fine. > > Cheers. > > > *Amos Raphenya * > *Bioinformatics Software Developer* > *Department of Biochemistry & Biomedical Sciences* > *McMaster University, MDCL **2317* > *p: (905) 525-9140 **ext: **22787* > *a: 1280 Main St W.,Hamilton, Ontario, Canada L8S 4K1* > *e: raphenar(a)mcmaster.ca <raphenar(a)mcmaster.ca> * > *skype: amos_raphenya* > > On Jan 28, 2016, at 3:08 PM, Martin Čech <marten(a)bx.psu.edu> wrote: > > Hi Amos, > > what Galaxy are you using and where are you seeing these messages? > > thanks > > Martin > > On Thu, Jan 28, 2016 at 2:56 PM Raphenya, Amogelang <raphenar(a)mcmaster.ca> > wrote: > >> Hi All, >> >> Is there a faster way to upload or load large datasets into galaxy >> history without using FTP? >> >> I tried *Upload_local_file* (*to current history*) and it doesn’t upload >> *.gz* files correctly as compared to when using* Upload_file *(*from >> your computer*). >> >> Cheers. >> >> >> *Amos Raphenya * >> *Bioinformatics Software Developer* >> *Department of Biochemistry & Biomedical Sciences* >> *McMaster University, MDCL **2317* >> *p: (905) 525-9140 **ext: **22787* >> *a: 1280 Main St W.,Hamilton, Ontario, Canada L8S 4K1* >> *e: raphenar(a)mcmaster.ca <raphenar(a)mcmaster.ca> * >> *skype: amos_raphenya* >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > >

1 0

Loading or Upload Large datasets into Galaxy History
by Raphenya, Amogelang 28 Jan '16

28 Jan '16

3 2

Trimmomatic Tool dependency installation issue
by Aevermann, Brian 28 Jan '16

28 Jan '16

Hello, I am having troubles installing Trimmomatic on my local galaxy on my mac 10.10.03. After following the instructions on the public tool shed and I end up to an tool dependency installation error for the Trimmomatic 0.32 package. The error is below. It appears that the installer is looking for the folder listed; however, the a folder does exist with only one Trimmomatic-0.32 subfolder, not two. Please advise. Thanks! Brian Aevermann [Errno 2] No such file or directory: './database/tmp/tmp-toolshed-mtdHb41Oo/Trimmomatic-0.32/Trimmomatic-0.32/'

2 1

about the queue in the galaxy
by 杜云飞 28 Jan '16

28 Jan '16

Hi: We found that there have two types of queue in the galaxy. One is the queue imported from kombu, the other is the Python queue. What is the function difference between them? Why not using only the queue from combo. Thanks!

1 0