- galaxy-dev - lists.galaxyproject.org

Set build when discovering output in collection
by Alexander Vowinkel 26 Jun '15

26 Jun '15

Hi, as many of you probably know by now, I have a tool that creates an output collection by discovering the datasets in this collection like propsed there: [1] Specifically I have a fastq input with set build information (a ref genome). The output is a collection of fastq files. After I managed to give them the same format like the input (hacky version), I now want them to adopt the build ID. How can I do that? Thanks, Alexander [1] https://bitbucket.org/galaxy/galaxy-central/pull-request/634/allow-tools-to…

1 2

input dataset(s) and collections
by Ryan G 25 Jun '15

25 Jun '15

Hi all - I'm constructing a wrapper for a tool I have and the input to the tool can be: 1) a single fastq file (single end sample) 2) multiple singled-end fastq files 3) a single paired-end sample 4) multiple paired-end samples. I have cases #1 and #2 handled, however case #3 is presenting a problem. If the user select "Paired-End" sample, I want to restrict them to selecting only a single fastq1 and fastq2 file. If they want to submit multiple paired-end samples I only want to allow them to submit them as a list of dataset pairs. I can get this part to work. I just need to restrict users to selecting only a single fastq1 and fastq2 file when the input_type is Paired-End. My XML is as follows. Any help would be appreciated  <conditional name="input_type"> <param name="input_type_selector" type="select" label="Select input type" help="Select between single and paired fastq data"> <option value="single">Single-End</option> <option value="paired">Paired-End</option> <option value="paired_collection">Paired Collection</option> </param> <when value="single"> <param name="fastq_input1" type="data" format="fastq" label="Select fastq dataset" multiple="true" help="Specify dataset with single end reads" /> </when> <when value="paired"> <param name="fastq_input1" type="data" format="fastq" label="Select fastq dataset" help="Specify dataset with 1st of paired-end reads" /> <param name="fastq_input2" type="data" format="fastq" label="Select fastq dataset" help="Specify dataset with 2nd of paired-end reads" /> </when> <when value="paired_collection"> <param name="fastq_collection" type="data_collection" collection_type="list:paired" label="Select a paired collection" help="See help section for an explanation of dataset collections"/> </when> </conditional>

2 2

Data Collections
by Alexander Vowinkel 25 Jun '15

25 Jun '15

Hi Folks, thank you so far for the previous help. I got much further. Now I'm stuck with data collections. Because this is quite a list, I appreciate also answers to parts of my questions ;) I have two issues: A) manual definition of data collections (any type) by user and/or admin B) definition of data collections as input/output of a tool and inside a workflow A) manual Basically I would like to create i) a list of fastq files (unpaired) ii) a paired set of two fastq files iii) a list of each two paired fastq files How can I do that? By using the web app? As user? As admin? By working via ssh on the server? B) in tool/workflow Here I also have different approaches I would like to realize: i) use a collection as input for a tool ii) create a collection as output of a tool ii.1) from known # of output parameters ii.2) from unknown # of output parameters For these things I was trying to find some tools in toolshed to see how they do it, but I couldn't quite adopt it. i) use a collection as input for a tool this is good documented - realizable by type="data_collection" and the collection_type. Unfortunately I can't test this because I can't create a collection so far ;) - see A ii) create a collection as output of a tool Here it gets blurry for me. ii.1) from known # of output parameters Here I didn't find a tool. I just thought, it might be a simpler case than ii.2 and good to understand the concept. I would be glad if someone could explain the way(s) to do this. ii.2) from unknown # of output parameters For this I found barcode splitter tools (also from devteam) that have different approaches. But. Their output (defined in xml) is only some report file. The output files seem to be fed into the history. And here I don't know how to get hands on these files when I want to use them to feed them into the next step during a workflow. Help highly appreciated! Thanks! Alexander

2 3

Using API to identify all datasets that were part of a workflow?
by Ben Bimber 25 Jun '15

25 Jun '15

Hello, I'm still relatively new to galaxy. I'm trying to use the API to identify the string of jobs/datasets that were created as part of executing a workflow. So far as I can tell, the API gives me the ID of the job, which corresponds to one step in the workflow. Each of these has inputs/outputs. I can walk outwards and try to connect any other jobs that happen to use one of these files as an input or output; however, I am not seeing any key that provides a more direct indication that a set of steps was executed as part of a given workflow. Am I missing something? Thanks in advance, Ben

2 2

bowtie dataset pair input
by Ryan G 25 Jun '15

25 Jun '15

Hi all - It looks like bowtie's wrapper is working incorrectly for a list of dataset pairs. Its expecting all the forward reads in one dataset list and the reverse reads in a separate dataset list. Instead, I have a list of dataset pairs (for paired-end data). This cannot be provided to bowtie as input. If I correct this, should I create a pull request for it? Alternatively, does someone already have a corrected version of this?

3 2

Using 'display_servers' even if not REMOTE_USER a good idea?
by David Trudgian 23 Jun '15

23 Jun '15

Hi, Spent a little while today trying to hook our Galaxy up to our local UCSC Browser. There was various stuff on the web but is a bit confusing since the 'display_servers' setting in galaxy.ini which comes up only applies when REMOTE_USER is in use. There's a separate UCSC_SERVERS list in lib/galaxy/web/framework/webapp.py and a sites Bunch in galaxy/security/__init__.py which define the hgwx.cse... URLs. Would it be reasonable if I came up with a little PR which makes it possible to override these from display_servers too? The one question I have is when are the security/__init.py__ checks used, and when is it framework/webapp.py? I couldn't work that out by looking. Thanks, DT -- David Trudgian Ph.D. Computational Scientist, BioHPC UT Southwestern Medical Center Dallas, TX 75390-9039 Tel: (214) 648-4833 ________________________________ UT Southwestern Medical Center The future of medicine, today.

1 0

cloudman error somewhere
by Alexander Vowinkel 23 Jun '15

23 Jun '15

Hi, I have the following problem: First, the job scheduling didn't work. I tried to upload a file, but the job was waiting. Then I checked "squeue", which timed out. The slurm service was running. Then I restarted the cluster. Now I even have problems access the cloudman admin page. Meaning, the page is loading endless. The machine is not busy (CPU and mem @ htop) In the paster.log it tries this every 4 seconds: 2015-06-16 18:35:56,547 DEBUG master:2579 Trying to setup AMQP > connection; conn = '<cm.util.comm.CMMasterComm object at 0x7f0aa26d72d0>' > 2015-06-16 18:35:56,547 DEBUG comm:30 Setting up a new AMQP > connection > 2015-06-16 18:35:56,548 DEBUG comm:49 AMQP Connection Failure: > [Errno 111] Connection refused Any idea what this might be? Something seems to be really weird. I am using the next version of CloudMan (dev). I created an own instance with the Cloud Man playbook. Best, Alexander

2 4

Cloudman cluster not starting workers
by Nicholas Dickens 23 Jun '15

23 Jun '15

Hi All, First time post, as a quick intro, I’m have some reasonable experience with EC2 & AWS for developing our own pipelines, I’m comfortable in python and *nix flavours and I am developing a completely custom galaxy in AWS for members of the WTCMP, Glasgow. In the meantime, I have thrown up a quick cloudman galaxy using the cloudstart and the cloudman 2.3 ami (ami-a7dbf6ce) in us-east-1. Auto–scaling didn’t seem to work so I’ve switched it off and added nodes manually, I tried various sizes including ‘same as master’ instances but they just don’t start – in the EC2 console I can see them and see them running. But they’re constantly pending in the /cloud interface and in the log they reboot 4 times and then terminate – apparently not responding "10:16:56 - Instance i-xxxxxx not responding after 4 reboots. Terminating instance". It’s out of the box, I editted the universe_wsgi.ini… file to disallow user registration and allow me to impersonate users but didn’t change anything else. The only other configuration I’ve done is associate an elastic IP with the master instance so I can have a more static url for a couple of test users (if I need to destroy it and start again, etc). I’m new to the system so don’t know which logs are best to check…and am I missing something obvious? It there a known bug when using elastic IPs? I’ve googled but with no joy. Thanks for your help and best wishes, Nick -- Nick Dickens DPhil BSc ARCS Bioinformatics Team Leader Wellcome Trust Centre for Molecular Parasitology B6-21 SGDB 120 University Place Glasgow G12 8TA Tel: +44 141 330 8282 http://fb.me/WTCMPbix @WTCMPbix http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/ http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparative…

3 6

Re: [galaxy-dev] Galaxy admin access
by Dannon Baker 23 Jun '15

23 Jun '15

In your terminal, you'll need to navigate to the galaxy config directory (you'll see the .sample file there) first. Once there, look at a directory list and make sure that an existing galaxy.ini isn't already there. Also, please keep the galaxy-dev list cc'd instead of replying directly so that others might be able to help you as well. On Wed, Jun 17, 2015 at 12:18 PM Davenport, Aneisha <ADAVENP3(a)gru.edu> wrote: > I'm typing the command in terminal > > > -------- Original message -------- > From: Dannon Baker <dannon.baker(a)gmail.com> > Date: 06/17/2015 12:05 PM (GMT-05:00) > To: "Davenport, Aneisha" <ADAVENP3(a)gru.edu>, > galaxy-dev(a)lists.galaxyproject.org > Subject: Re: [galaxy-dev] Galaxy admin access > > Where are you executing this? You'll need to be in the galaxy/config > directory, where galaxy.ini.sample exists. > > I'm looping the mailing list back in on this so other people might also be > able to help. > > On Wed, Jun 17, 2015 at 8:28 AM Davenport, Aneisha <ADAVENP3(a)gru.edu> > wrote: > >> When I type in cp galaxy.ini.sample galaxy.ini in terminal it outputs no >> such file or directory >> >> >> -------- Original message -------- >> From: Dannon Baker <dannon.baker(a)gmail.com> >> Date: 06/15/2015 3:14 PM (GMT-05:00) >> To: "Davenport, Aneisha" <ADAVENP3(a)gru.edu> >> Subject: Re: [galaxy-dev] Galaxy admin access >> >> I'm not sure what you mean. You can edit the file using any text >> editor, try 'nano', or emacs or vim if you're more comfortable with those. >> >> Does the file exist or not? If not, simply "cp galaxy.ini.sample >> galaxy.ini" will do it. >> >> On Mon, Jun 15, 2015 at 7:27 AM Davenport, Aneisha <ADAVENP3(a)gru.edu> >> wrote: >> >>> How do I create it and edit it on terminal? >>> >>> Aneisha Davenport >>> ________________________________________ >>> From: Dannon Baker [dannon.baker(a)gmail.com] >>> Sent: Friday, June 12, 2015 9:44 PM >>> To: Davenport, Aneisha; galaxy-dev(a)lists.galaxyproject.org >>> Subject: Re: [galaxy-dev] Galaxy admin access >>> >>> Hi Aneisha, >>> >>> When you say you cannot alter config/galaxy.ini, what do you mean? You >>> should be able to create (if not there) and edit this file in your regular >>> terminal on that machine. >>> >>> Does a different user own the file, perhaps a 'galaxy' one? If so, you >>> could change to that user and then edit. >>> >>> On Fri, Jun 12, 2015 at 5:13 PM Davenport, Aneisha <ADAVENP3(a)gru.edu >>> <mailto:ADAVENP3@gru.edu>> wrote: >>> Hi, I work at the It service desk at the GRU Health and Science campus >>> Ed Commons GB building and I have a customer trying to get admin access for >>> the local installation of galaxy. We tried to follow the online >>> instructions "In order to control your new Galaxy through the UI >>> (installing tools, managing users, creating groups etc.) you have to become >>> an administrator. First register as a new user and then give the user admin >>> privileges like this: You add the Galaxy login ( email ) to the Galaxy >>> configuration file (config/galaxy.ini). If the file does not exist you can >>> copy it from the provided sample (config/galaxy.ini.sample). Note that you >>> have to restart Galaxy after modifying the configuration for changes to >>> take effect. >>> >>> >>> # this should be a comma-separated list of valid Galaxy users >>> admin_users = user1(a)example.com<mailto:user1@example.com>, >>> user2(a)example.com<mailto:user2@example.com>" but we cannot alter the >>> config/galaxy.ini file. Can you please give us assistance? Do you run in >>> console or Terminal (this is a Macbook Air Laptop)? >>> >>> >>> Aneisha Davenport >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >>

2 7

Strange repository dependency behavior with toolshed-installed naive_variant_caller
by Brian Claywell 19 Jun '15

19 Jun '15

First, I'm at a bit of a loss to even describe what's going on, so I apologize in advance for a rambling bug report. Please bear with me! The naive_variant_caller tool was failing on our instance with "Failed to resolve dependency" errors on numpy, pyBamParser, and pyBamTools, despite the tool dependencies and their respective dependency packages being installed. I attempted to use the "repair repository" action on naive_variant_caller with no success. I then uninstalled naive_variant_caller, package_numpy_1_7, package_pybamparser_0_0_1, and package_pybamtools_0_0_1 and attempted to reinstall naive_variant_caller from scratch, again allowing Galaxy to manage the dependencies. This time, the dependencies resolved (at least as far as the "Building dependency shell command" messages were concerned), but naive_variant_caller still failed, this time with an import error: ImportError: No module named pyBamTools.genotyping.naive Digging around a little I found that the env.sh that was sourced for pyBamTools was looking for an env.sh in another pyBamTools revision directory, but the revision it was looking for didn't exist. Then I looked in the admin interface and saw that package_pybamtools_0_0_1 and package_pybamparser_0_0_1 were each installed twice; each had both the most recent revision and an older revision with the "updates are available" icon. At this point I uninstalled naive_variant_caller and all of its dependency packages again. I then installed naive_variant_caller from the toolshed *without* dependency management, and manually installed the required revisions of numpy, pybamparser, and pybamtools. Finally naive_variant_caller started working. However, naive_variant_caller is now shown in the admin interface as "Installed, missing repository dependencies", and the installed repository page shows that the dependencies are a very strange state (see attached screenshot). That is, the same revision of numpy is listed as installed twice, and the pybamparser and pybamtools dependencies are each listed as simultaneously installed and uninstalled. Can anyone shed some light on what could be happening and what to do about it? Is there some external way to verify the consistency of a tool and its dependencies? Thanks! Cheers, Brian -- Brian Claywell | programmer/analyst Matsen Group | http://matsen.fredhutch.org <http://matsen.fhcrc.org/> Fred Hutchinson Cancer Research Center

2 5