June 2011 - galaxy-dev - lists.galaxyproject.org

drmaa://native/ : native options are ignored
by Geert Vandeweyer 20 Jun '11

20 Jun '11

Hi, I'm working on an local installation of galaxy using torque with drmaa (the pbs-torque scramble failed). The torque-drmaa works fine so far, except for one issue. I'd like to specify some tool-dependent requirements from the tool_runners section in universe.wsgi.ini. For now I've been testing it with the setting below to have global native arguments : default_cluster_job_runner = drmaa://-l mem=4gb:nodes=1:ppn=6/ This should request 4gb of memory on a single node with 6 threads, but these requests are ignored. They are not listed on 'qstat -R' and more simultaneous jobs than possible are started if the requirements were taken into account. What am I missing here? Best regards, Geert Vandeweyer

2 1

New "delete and purge" button
by Assaf Gordon 18 Jun '11

18 Jun '11

Hi, Sorry for complaining or sounding ungrateful, but before you push the new "Delete and remove datasets from disk" to galaxy-dist, can you make it optional (with enable/disable in universe_wsgi.ini) ? Or at the very least, add a big warning and a confirmation message box before actually deleting the datasets ? This feature is going to be a support problem - users won't easily tell the difference between "delete" and "delete and remove" (trust me, they won't, even though for you it's the most obvious difference in the world) - then they'll come asking to somehow restore those files (since I guess you don't actually delete them immediately from the disk, right) ? That is, if users actually delete histories, which I still think they rarely do. Thanks, -gordon

2 1

galaxy with SGE cluster
by Shantanu Pavgi 17 Jun '11

17 Jun '11

I need some help in configuring galaxy with SGE scheduler using unified method. The galaxy is running on a system distinct from SGE scheduler install. The cluster nodes can access galaxy install, galaxy-tools and dataset files using NFS. I am not sure how drmaa works and how galaxy submits jobs to the cluster/scheduler. Do we need specify some type of connection string or ssh-config to connect with the cluster/scheduler? Does it need any configuration changes on the SGE scheduler side? Any explanation regarding this will be really helpful. -- Thanks, Shantanu.

3 7

LDAP and API issue?
by Louise-Amélie Schmitt 17 Jun '11

17 Jun '11

Hi everyone I'm currently trying to use the API, but I think that our LDAP logging doesn't really help. When I call the script specifying the port in the url, here's what I get: <urlopen error [Errno 111] Connection refused> Then when I don't specify it Here's what I get: HTTP Error 401: Authorization Required <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>401 Authorization Required</title> </head><body> <h1>Authorization Required</h1> <p>This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.</p> <hr> <address>Apache/2.2.3 (CentOS) Server at localhost Port 80</address> </body></html> Did I miss anything? Is there a way to avoid that? Thanks LA

2 3

Galaxy Workshops @ USC June 23-24
by Dave Clements 17 Jun '11

17 Jun '11

Hello all, There will be two Galaxy workshops at the University of Southern California (USC) next week. Both are presented by Jeremy Goecks of Emory University and the Galaxy team. *Both workshops are open to the public:* *Galaxy: A web‐based workbench for interactive and reproducible analysis of high‐throughput sequencing data, Thursday, June 23, 2011* At Aresty Auditorium, Harylene J Norris Cancer Research Tower, USC Health Science Campus, 1450 Biggy Street Registration is free, but required. Register at https://uschsl.qualtrics.com/SE/?SID=SV_bDaxnZwEfrfk1BG 9:15am-Noon Introduction to Galaxy including 30 min. Q&A Session 2:00pm-5:00pm High‐throughput sequencing data analysis including 45 min. Q&A Session *Progress and Challenges in Developing a Web-based Platform for Computational Biomedical Research, Friday, June 24, 2011 * See USC ISI AI Seminar Page ( http://ai.isi.edu/index.php?module=seminars/index) for details on time and location. The recent reliance on computation in biology has created an informatics crisis for biomedical researchers: computational resources are often difficult to use, communicating techniques and experiments is challenging, and reproducibility is very limited. Galaxy is one approach for addressing these problems. Galaxy is a popular Web-based platform for performing accessible, reproducible, and transparent genomics research. Galaxy provides a collaborative environment for performing complex analyses with automatic and unobtrusive provenance tracking; these features allow transparent sharing of both the precise computational details underlying an analysis and also intent, context, and narrative. Based on experiences with Galaxy, Jeremy will discuss some open problems that might be addressed using artificial intelligence methods and techniques. Thanks, Dave C. -- http://getgalaxy.org http://usegalaxy.org/

1 0

Re: [galaxy-dev] [galaxy-user] SRMA on a local galaxy instance?
by Kelly Vincent 16 Jun '11

16 Jun '11

Henry, I'm moving this over to galaxy-dev since it has to do with a local instance. It definitely sounds like you've got things set up right. You don't say what the result of your run is--is it "successful" (green) but an empty file? Or is there an error? If it's empty, it's possible your data just didn't yield any results. If there's an error, please share that with us. There are a couple of other things you could do to start figure out where the problem is. Have you tried running the functional tests for SRMA? There are two and you could comment out the first one in the xml since it relies on hg18chr21 being in the loc file, but if the second one passes then we'd know SRMA is installed correctly but there is something with your data (the command is: sh run_functional_tests.sh - id srma_wrapper). You also could try running your data through on our test server's (test.g2.bx.psu.edu) version of SRMA to see if if produces any results. Let us know if you need further assistance. Thanks, Kelly On Thu Jun 16, at 12:49 PM, Gong, Henry wrote: > I may have just sent an empty reply email; if so, my apologies. > > Yes, I have the srma jar in that folder, without the version number > exactly like that. > > Cheers, > Henry > > -----Original Message----- > From: Nate Coraor [mailto:nate@bx.psu.edu] > Sent: Thu 6/16/2011 7:35 AM > To: Gong, Henry > Cc: galaxy-user(a)lists.bx.psu.edu; Shieh, Joseph > Subject: Re: [galaxy-user] SRMA on a local galaxy instance? > > Gong, Henry wrote: >> >> Does anyone know how to get SRMA to work properly in galaxy? I've >> used the pre-built 0.1.15 jar as well as a 0.1.16 jar I built from >> source, but the result of an SRMA re-alignment job in both >> instances is a python process which only takes a few percent CPU >> (or less, since it only seems to be outputting timestamps to the >> terminal). Here are my system's specs if they're important: 2.8Ghz >> Intel i5, 4 core; 4GB 1333MHz DDR3; 1TB HDD (430GB free). The OS is >> OS X 10.6.7. I've also built the required indices and placed them >> in path/to/galaxy-dist/hg19/srma_path/hg19.dict, hg19.fa.fai, and >> hg19.fa; the hg19 is a concatenated version of the various >> chromosome and contig files, and I've followed the sample files in >> adding these locations to the .loc files (both srma and picard >> tools). >> >> The NGS installation page seems to suggest that SRMA does work, so >> I'm not sure if I'm overreacting, but I'd really appreciate any >> advice on this, since I imagine others have had similar problems. > > Hi Henry, > > Where is your srma JAR file located? It should be > /path/to/galaxy-dist/tool-data/shared/jars/srma.jar > > --nate > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

2 1

Display names accessible from Tool XML?
by Dave Walton 16 Jun '11

16 Jun '11

I'm building a tool that needs the "display name" of a file inside the output of the tool. To clarify, I have a tool that merges gene expression result from cufflinks for many samples. It generates a "matrix" tab-delimited file that provides the results with genes or transcript down one axis and samples across the other with intensity values in the cells of the matrix. I have already built into my galaxy server the ability to carry meaningful sample names from the original input file all the way through to the output of the final output of the cufflinks step of my workflow (this is functionality I've already offered up to the galaxy team to include, and can explain at a later date). Okay, so the problem I have is from the XML, I want to be able to pass my command-line script both the physical file names so the files can be parsed, but I also want to be able to pass the script the "display name" to use as the sample names for the columns in the file. Is there a way using the metadata to grab this from the input? I see that I can do something like input.metadata.dbkey and get the database key, but I can't find a way to do anything like input.metadata.name or input.metadata.display_name to give me what I want. My example XML file is below. Thanks, Dave <tool id="cuff_exp_2_matrix" name="Merge Cufflinks Exp Files to Matrix(files as sample names)" version="1.1.1"> <description>merges multiple cufflinks expression output files (gene or transcript) into a single matrix formatted file (id by sample, with intensity values)</description> <command interpreter="python"> cuff_exp_2_matrix_new.py $input1.metadata.display_name $input1 $output1 #for $i in $inputs ${i.input.metadata.display_name} ${i.input} #end for </command> <inputs> <param name="input1" label="First file" type="data" format="tabular" help="Need to add more files? Use controls below."/> <repeat name="inputs" title="Input Files"> <param name="input" label="Add file" type="data" format="tabular" /> </repeat> </inputs> <outputs> <data format="tabular" name="output1" label="${tool.name} on ${on_string}: Expression matrix" /> </outputs> <help> **What it does** This is a Jackson Laboratory custom tool for taking the output for several runs of cufflinks, where each run represents a sample, and merging these results into a file that contains a matrix of gene/transcript id by sample, with the cells of the matrix filled with the intensity values. The end result is a file that is easily imported into R or other tools for expression analysis. </help> </tool>

1 1

Fwd: a Trackster Question
by Jeremy Goecks 16 Jun '11

16 Jun '11

Matt, I'm moving your Trackster question onto the galaxy-dev list: > one issue I keep running into is that it's not super clear how to go about configuring the dynamic tracks you showed off in your talk. > > Is there some guidance somewhere that I am missing on this? I confess my own motive is to be able to dynamically interact with a sliding window-based algorithm for measuring DNA replication status from sequencing data. This would help us parameterize the algorithm before running it on the entire data set. This (very new) galaxy-central changeset: https://bitbucket.org/galaxy/galaxy-central/changeset/a7e9af183746 formalizes tool integration into Trackster. To enable the "Show Tool" option in Trackster for a particular tool, two things need to happen: (a) add the <trackster_conf/> tag to the tool XML file; (b) ensure that the tool has parameters supported by Trackster; only integer, float, and static select parameters are currently supported in a tool's interface, but it is straightforward to add support for other parameter types as well, and we'll do so as needed. We'll create a wiki page soon so that this information is easier to find. Let us know if you have more questions. Best, J.

1 0

Updating tools on new hg based Galaxy Tool Shed
by Peter Cock 16 Jun '11

16 Jun '11

Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear "update" action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-up… I tried using the "upload files to repository" option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-f… Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-… Are you expecting tool authors to work primarily at the hg level? Regards, Peter

3 7

Cluster Usage
by Robert Jackson 16 Jun '11

16 Jun '11

Does anyone have practical information for installation and set up for the galaxy distribution on a cluster?? Robert C. Jackson Software Systems Specialist III The University of Texas Pan-American Computer Support Services Division of Information Technology Phone: 956-665-2455 Fax: 956-665-8777

2 1