August 2011 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] suggestion for multithreading
by Andrew Warren 15 Aug '11

15 Aug '11

So would the current correct method for setting up multi-threaded jobs on a cluster be to specify custom runners in the [galaxy:tool_runners] section of the universe config file for EVERY tool that uses a multiple threads (assuming the default is set to one)? For example, for the bowtie program and a queue named "galaxy": *bowtie = pbs:///galaxy/-l ppn=4,mem=16gb/* * * Is this currently the only way for galaxy to inform the queuing system how many threads a program will use? And does this mean that without custom runners in the config file any muti-threaded program that has multiple instances in an asychronous workflow has the opportunity to overload a cluster node since the queuing system doesn't "know" how many threads the program will be using? Just want to make sure I'm not missing out on the latest and greatest method for process management. :) Thanks, Andrew * * Louise-Amélie Schmitt wrote: > > > > default_cluster_job_runner will remain for backwards compatibility, but > > we'll ship a sample job_conf.xml that runs everything locally by > > default. > > > > --nate > > Haha, and I did that before realizing I could do just what I needed by > writing tool-specific *pbs*:// URLs at the end of the config file... I'm such > an idiot. Haha, okay, I don't think i even noticed since I was distracted by your implementation being a step in the way we want to go with it. > But I really like what you did of it and I have a couple of questions. > > Concerning the single-threaded tools, what would happen if the number of > threads set in the xml file was >1 ? It'd consume extra slots, but the tool itself would just run as usual. > Could it be possible to forbid a tool to run on a given node? Hrm. In *PBS* you could do it using node properties/neednodes or resource requirements. I'd have to think a bit about how to do this in a more general way in the XML. --nate > > Thanks, > L-A > > > > > >> > >> Peter > >>

3 2

Re: [galaxy-dev] Adding Link to Sequencing Requests Page
by Greg Von Kuster 15 Aug '11

15 Aug '11

Hello Joe, Galaxy uses a combination of Python and mako templates for rendering - see http://www.makotemplates.org/ for information about mako templates. Assuming the "Sequencing Requests" page to which you refer looks something like the following, you'll need to modify the Python code in the RequestsGrid class in ~/lib/galaxy/web/controllers/requests_common.py. Good luck, and thanks for using Galaxy, Greg Von Kuster Sequencing Requests Create new request Advanced Search Name Description Samples Type Last Updated ↑ State No Items For 0 selected items: > Submitted by jcruz7(a)gmail.com > Hello Everyone, > > I would like to add a link to the Sequencing Requests page and modify it > further in the future but I'm unsure of how to go about doing that, since > there is no static html file for the webpage (at least none that I can find) > and I'm a newbie when it comes to web application development. > > I appreciate your help and patience. > > joe

1 0

global configuration of email "from" possible?
by Holger Klein 15 Aug '11

15 Aug '11

Hi all, for the email functionality of our local galaxy instance I'm forced to use SMTP authentication and to use a specific "from", which is different to what galaxy generates automatically. Now galaxy generates a "from" like galaxy(a)computeserver.imb.uni-mainz.de but I want to set this to galaxy(a)imb-mainz.de. A quick check shows that the send_mail function defined in lib/galaxy/util/__init__.py is called in various places. Is there a way to set the "from" to a specific value? If not, it send_mail the only galaxy-function used to send mail? Cheers, Holger -- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

3 4

Whole genome simulation with dwgsim
by Nils Homer 14 Aug '11

14 Aug '11

I wanted to plug two tools that we have wrapped up for Galaxy: whole genome simulation of NGS reads, as well as the evaluation of the resulting mappings of those reads. This could be very useful for those that want to evaluate mapping algorithms, or experiment with simulated NGS data. Check out the "dwgsim" and "dwgsim_eval" tools at http://toolshed.g2.bx.psu.edu/ The tools and wrapper code can also be found at http://dnaa.sf.net, and described at https://sourceforge.net/apps/mediawiki/dnaa/index.php?title=Whole_Genome_Si… Feedback and comments are very welcome, Nils Homer

1 0

Re: [galaxy-dev] trackster reference genome
by Jeremy Goecks 14 Aug '11

14 Aug '11

Colin, > How is it possible to preinstall a reference genome for trackster (hg19) within galaxy. > > i tried to find a dedicated help page but could not find any. Could you please link me to relevant information? See the section titled 'Setup for local instances' on this page: http://wiki.g2.bx.psu.edu/Learn/Visualization Finally, please send all emails to our mailing lists--in this case, galaxy-dev, which I've cc'd--rather than individual Galaxy developers or other list members. Sending email to the mailing lists ensures that everyone can see and contribute to questions and issues; in addition, emails to mailing lists are tracked and archived so that they are available in the future. Best, J.

1 0

Picard installation problem
by colin molter 14 Aug '11

14 Aug '11

Hello all, I get troubles installing Picard. In short is there a dedicated link with precise instructions on how to install it on Galaxy? I spent hours trying to install it and if I made a subtle step forward in the installation, I am still stuck with exceptions when trying to run any picard tool. My first problem was to have a link to a built-in reference genome. I finally fixed it. I explain my solution here for interested persons. (1) create the 'dict' file from your reference fasta file by using the following picard tool: java -jar CreateSequenceDictionary.jar -R=<ref-genome.fa> -O=<ref-genome.dict> (2) modify the picard.loc index file to point to the correct location (3) -- the step that took me hours to find -- modify the all-fasta.loc file to point to the correct location. Second -and still unsolved- problem: When i try to run a picard tool, I get the following exception: File "/home/hg/Galaxy/galaxy-dist/tools/picard/picard_wrapper.py", line 40 class PicardBase(): ^ SyntaxError: invalid syntax I downloaded the last release of picard (picard-tools-1.48) and put all the jars file in tool-data/shared/jars/ permission of the files are : -rw-r--r-- 1 apache adm I suspect that galaxy can't find the jars file (i get the same error if I remove them from that location). Am I missing something? Should I precise something about the classpath in galaxY? Thank you for your help, Colin

3 4

Extracting regions of sequences given table of co-ords
by Peter Cock 12 Aug '11

12 Aug '11

Hi all, I'm about to start working on a general tool for extracting sequences (from FASTA etc) based on a tabular file describing desired regions (this could be a BED or GFF3 file), giving a new sequence file with one entry for each region in the tabular file. Essentially I want as input columns describing: * FASTA sequence ID * Start * End * Strand (optional) * Region ID for this sequence in the output file (optional) In the case of a GFF3 input, those would be columns 1, 4, 5, 7 (with the ID being hidden inside column 9). But this isn't really driven by GFF3 where you'd probably not want to treat each line separately anyway (compound features). I'm thinking of very general examples like extracting general search matches (BLAST, HMMER, motifs, etc), but specifically things like cleaving proteins to remove a predicted signal peptide. This could even be used for trimming of sequencing reads (for small datasets, e.g. capillary reads). Is there anything like this in Galaxy already? If so I can't find it ;) I've also skimmed the tool shed. Regards, Peter

3 8

Toronto GMOD meeting registration now open
by Dave Clements 12 Aug '11

12 Aug '11

Hello all, The next GMOD meeting has been schedule for this October in Toronto. See below for details. Dave C ---------- Forwarded message ---------- From: Scott Cain <scott(a)scottcain.net> Date: Fri, Aug 12, 2011 at 9:23 AM Subject: [Gmod-announce] Toronto GMOD meeting registration now open Hello, I am pleased to announce that registration is now open for the October 12-13 GMOD meeting at OICR in Toronto, Canada. Please visit the registration site at: http://gmod.eventbrite.com/ The registration fee for the meeting is $40 if registered before September 13. After September 13, the registration fee goes up to $50. For more information about the meeting, including information on our excellent keynote speakers, Gary Bader and Michael Brudno, please visit the meeting page: http://gmod.org/wiki/October_2011_GMOD_Meeting While at the meeting page, please make sure to suggest topics, talks and satellites for the meeting. I look forward to seeing you in Toronto! Scott -- ------------------------------------------------------------------------ Scott Cain, Ph. D. scott at scottcain dot net GMOD Coordinator (http://gmod.org/) 216-392-3087 Ontario Institute for Cancer Research -------------------------------------------------------------------------------- http://getgalaxy.org http://usegalaxy.org/ http://galaxyproject.org/wiki

1 0

Quota
by Steve Taylor 12 Aug '11

12 Aug '11

Hi, Are there any plans to have a disk space quota for Galaxy users that can be set by admins? Thanks, Steve

4 3

faToTwoBit
by Peter Cock 12 Aug '11

12 Aug '11

Hi all, I recently started trying "Extract Genomic DNA" which requires faToTwoBit, which wasn't listed on the dependencies page (fixed now), http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies See also: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-August/006366.html In updating the wiki I've assumed faToTwoBit comes from the same place as twoBitToFa, where the link given is simply: http://hgdownload.cse.ucsc.edu/admin/exe/ That just contains pre-compiled binaries - but there is no sign of the source code or a licence file. Are you just using these binaries? I feel like I've been getting lost in the UCSC website... I eventually found Jim Kent's pages, which appears to host the twoBitToFa source code along with other tools like BLAT, http://hgwdev.cse.ucsc.edu/~kent/src/unzipped/utils/twoBitToFa/ The log files hinted at there being a git repository, which I found here, http://genome-source.cse.ucsc.edu/ http://genome.ucsc.edu/admin/git.html The licensing is murky, apparently free to use for personal, academic, and non-profit use - but no mention about potential redistribution or modification rights: http://hgwdev.cse.ucsc.edu/~kent/src/unzipped/README > ... All files are copyrighted, but license is hereby granted > for personal, academic, and non-profit use. license is > also granted for the contents of the top level lib, inc and > utils directories for commercial users. ... Have the Galaxy team made special arrangements to license faToTwoBit for use on the Penn State public server, or is this fine because faToTwoBit is under the top level utils directory? Have I overlooked some general page introducing the Kent tools like faToTwoBit? If such a page exists, could it be linked to from the Galaxy Tool Dependencies page? Thanks, Peter

2 3