June 2015 - galaxy-dev - lists.galaxyproject.org

LDAP and new history creation
by Erwan DELAGE 29 Jun '15

29 Jun '15

Dear all, We have a galaxy instance that uses LDAP for authentication and it works fine. The only problem is that a new history is created each time a user log in. When we disable LDAP to use the classical authentication method, the problem goes away. Any idea of what could happen here ? Thanks Erwan

2 2

Download of hg19 genome by data manager fails
by Tobias Zimmermann 29 Jun '15

29 Jun '15

Dear All, I have some problems with the download of genome files by the data manager. I tried to download some genome files from the UCSC server with the data manager of galaxy ( name: data_manager_fetch_genome_all_fasta, revision: cca219f2b212), but it fails because of a timeout. Our galaxy server is in an internal network without connection to the Internet. To access the Internet we are required to go trough a http-proxy before any web-services can be accessed. The file /shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_all_fasta/cca219f2b212/data_manager_fetch_genome_all_fasta/data_manager/data_manager_fetch_genome_all_fasta.py manages the download of the file. Line 183 shows the problem. Here the FTP item from the python ftplib module tries to establishes the connection to the UCSC ftp server. This connection fails. So here is my question: How can i download data by the data manager with a http-proxy? Thanks for help! Tobias

1 0

Set build when discovering output in collection
by Alexander Vowinkel 26 Jun '15

26 Jun '15

Hi, as many of you probably know by now, I have a tool that creates an output collection by discovering the datasets in this collection like propsed there: [1] Specifically I have a fastq input with set build information (a ref genome). The output is a collection of fastq files. After I managed to give them the same format like the input (hacky version), I now want them to adopt the build ID. How can I do that? Thanks, Alexander [1] https://bitbucket.org/galaxy/galaxy-central/pull-request/634/allow-tools-to…

1 2

input dataset(s) and collections
by Ryan G 25 Jun '15

25 Jun '15

Hi all - I'm constructing a wrapper for a tool I have and the input to the tool can be: 1) a single fastq file (single end sample) 2) multiple singled-end fastq files 3) a single paired-end sample 4) multiple paired-end samples. I have cases #1 and #2 handled, however case #3 is presenting a problem. If the user select "Paired-End" sample, I want to restrict them to selecting only a single fastq1 and fastq2 file. If they want to submit multiple paired-end samples I only want to allow them to submit them as a list of dataset pairs. I can get this part to work. I just need to restrict users to selecting only a single fastq1 and fastq2 file when the input_type is Paired-End. My XML is as follows. Any help would be appreciated  <conditional name="input_type"> <param name="input_type_selector" type="select" label="Select input type" help="Select between single and paired fastq data"> <option value="single">Single-End</option> <option value="paired">Paired-End</option> <option value="paired_collection">Paired Collection</option> </param> <when value="single"> <param name="fastq_input1" type="data" format="fastq" label="Select fastq dataset" multiple="true" help="Specify dataset with single end reads" /> </when> <when value="paired"> <param name="fastq_input1" type="data" format="fastq" label="Select fastq dataset" help="Specify dataset with 1st of paired-end reads" /> <param name="fastq_input2" type="data" format="fastq" label="Select fastq dataset" help="Specify dataset with 2nd of paired-end reads" /> </when> <when value="paired_collection"> <param name="fastq_collection" type="data_collection" collection_type="list:paired" label="Select a paired collection" help="See help section for an explanation of dataset collections"/> </when> </conditional>

2 2

Data Collections
by Alexander Vowinkel 25 Jun '15

25 Jun '15

Hi Folks, thank you so far for the previous help. I got much further. Now I'm stuck with data collections. Because this is quite a list, I appreciate also answers to parts of my questions ;) I have two issues: A) manual definition of data collections (any type) by user and/or admin B) definition of data collections as input/output of a tool and inside a workflow A) manual Basically I would like to create i) a list of fastq files (unpaired) ii) a paired set of two fastq files iii) a list of each two paired fastq files How can I do that? By using the web app? As user? As admin? By working via ssh on the server? B) in tool/workflow Here I also have different approaches I would like to realize: i) use a collection as input for a tool ii) create a collection as output of a tool ii.1) from known # of output parameters ii.2) from unknown # of output parameters For these things I was trying to find some tools in toolshed to see how they do it, but I couldn't quite adopt it. i) use a collection as input for a tool this is good documented - realizable by type="data_collection" and the collection_type. Unfortunately I can't test this because I can't create a collection so far ;) - see A ii) create a collection as output of a tool Here it gets blurry for me. ii.1) from known # of output parameters Here I didn't find a tool. I just thought, it might be a simpler case than ii.2 and good to understand the concept. I would be glad if someone could explain the way(s) to do this. ii.2) from unknown # of output parameters For this I found barcode splitter tools (also from devteam) that have different approaches. But. Their output (defined in xml) is only some report file. The output files seem to be fed into the history. And here I don't know how to get hands on these files when I want to use them to feed them into the next step during a workflow. Help highly appreciated! Thanks! Alexander

2 3

Using API to identify all datasets that were part of a workflow?
by Ben Bimber 25 Jun '15

25 Jun '15

Hello, I'm still relatively new to galaxy. I'm trying to use the API to identify the string of jobs/datasets that were created as part of executing a workflow. So far as I can tell, the API gives me the ID of the job, which corresponds to one step in the workflow. Each of these has inputs/outputs. I can walk outwards and try to connect any other jobs that happen to use one of these files as an input or output; however, I am not seeing any key that provides a more direct indication that a set of steps was executed as part of a given workflow. Am I missing something? Thanks in advance, Ben

2 2

bowtie dataset pair input
by Ryan G 25 Jun '15

25 Jun '15

Hi all - It looks like bowtie's wrapper is working incorrectly for a list of dataset pairs. Its expecting all the forward reads in one dataset list and the reverse reads in a separate dataset list. Instead, I have a list of dataset pairs (for paired-end data). This cannot be provided to bowtie as input. If I correct this, should I create a pull request for it? Alternatively, does someone already have a corrected version of this?

3 2

Using 'display_servers' even if not REMOTE_USER a good idea?
by David Trudgian 23 Jun '15

23 Jun '15

Hi, Spent a little while today trying to hook our Galaxy up to our local UCSC Browser. There was various stuff on the web but is a bit confusing since the 'display_servers' setting in galaxy.ini which comes up only applies when REMOTE_USER is in use. There's a separate UCSC_SERVERS list in lib/galaxy/web/framework/webapp.py and a sites Bunch in galaxy/security/__init__.py which define the hgwx.cse... URLs. Would it be reasonable if I came up with a little PR which makes it possible to override these from display_servers too? The one question I have is when are the security/__init.py__ checks used, and when is it framework/webapp.py? I couldn't work that out by looking. Thanks, DT -- David Trudgian Ph.D. Computational Scientist, BioHPC UT Southwestern Medical Center Dallas, TX 75390-9039 Tel: (214) 648-4833 ________________________________ UT Southwestern Medical Center The future of medicine, today.

1 0

cloudman error somewhere
by Alexander Vowinkel 23 Jun '15

23 Jun '15

Hi, I have the following problem: First, the job scheduling didn't work. I tried to upload a file, but the job was waiting. Then I checked "squeue", which timed out. The slurm service was running. Then I restarted the cluster. Now I even have problems access the cloudman admin page. Meaning, the page is loading endless. The machine is not busy (CPU and mem @ htop) In the paster.log it tries this every 4 seconds: 2015-06-16 18:35:56,547 DEBUG master:2579 Trying to setup AMQP > connection; conn = '<cm.util.comm.CMMasterComm object at 0x7f0aa26d72d0>' > 2015-06-16 18:35:56,547 DEBUG comm:30 Setting up a new AMQP > connection > 2015-06-16 18:35:56,548 DEBUG comm:49 AMQP Connection Failure: > [Errno 111] Connection refused Any idea what this might be? Something seems to be really weird. I am using the next version of CloudMan (dev). I created an own instance with the Cloud Man playbook. Best, Alexander

2 4

Cloudman cluster not starting workers
by Nicholas Dickens 23 Jun '15

23 Jun '15

Hi All, First time post, as a quick intro, I’m have some reasonable experience with EC2 & AWS for developing our own pipelines, I’m comfortable in python and *nix flavours and I am developing a completely custom galaxy in AWS for members of the WTCMP, Glasgow. In the meantime, I have thrown up a quick cloudman galaxy using the cloudstart and the cloudman 2.3 ami (ami-a7dbf6ce) in us-east-1. Auto–scaling didn’t seem to work so I’ve switched it off and added nodes manually, I tried various sizes including ‘same as master’ instances but they just don’t start – in the EC2 console I can see them and see them running. But they’re constantly pending in the /cloud interface and in the log they reboot 4 times and then terminate – apparently not responding "10:16:56 - Instance i-xxxxxx not responding after 4 reboots. Terminating instance". It’s out of the box, I editted the universe_wsgi.ini… file to disallow user registration and allow me to impersonate users but didn’t change anything else. The only other configuration I’ve done is associate an elastic IP with the master instance so I can have a more static url for a couple of test users (if I need to destroy it and start again, etc). I’m new to the system so don’t know which logs are best to check…and am I missing something obvious? It there a known bug when using elastic IPs? I’ve googled but with no joy. Thanks for your help and best wishes, Nick -- Nick Dickens DPhil BSc ARCS Bioinformatics Team Leader Wellcome Trust Centre for Molecular Parasitology B6-21 SGDB 120 University Place Glasgow G12 8TA Tel: +44 141 330 8282 http://fb.me/WTCMPbix @WTCMPbix http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/ http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparative…

3 6