July 2011 - galaxy-dev - lists.galaxyproject.org

Metadata in Galaxy Tool Shed
by Lance Parsons 22 Jul '11

22 Jul '11

I recently created a new repository in the galaxy tool shed for the cutadapt tool and "imported" my old hg repository into it. However, it doesn't seem to be picking up the metadata correctly. Does this take time, or is there something I need to do to get the metadata generated correctly? Also, any suggestions on how to organize the files in the repository are welcome. Thanks. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

2 1

new data type.
by Luobin Yang 22 Jul '11

22 Jul '11

Hi, I created an XML file for a tool. This tool has an Nexus input. When I click the input field of the web interface for this tool, it shows all kinds of data that I can select. How do I restrict the list to show only data in nexus format? I added a line in the registration part and the sniffer part of the datatypes_conf.xml and I also created a class called Nexus which inherits the class galaxy.datatypes.Sequence, but this doesn't solve the problem. Thanks, Luobin

2 2

Scroll bars not present using i.e.
by Chorny, Ilya 22 Jul '11

22 Jul '11

http://test.g2.bx.psu.edu/ Cheers, Ilya Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

error reading tool from path (epigraph.xml, epigraph_test.xml, access_libraries.xml)
by Leon Mei 22 Jul '11

22 Jul '11

Dear all, I installed a local galaxy using the latest codebase yesterday. During the startup, the loading of 3 tool wrappers are failed: epigraph.xml, epigraph_test.xml, access_libraries.xml. Similar errors were reported in http://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg01839.html Have other people experienced the same problem? Any suggested solutions? Thanks, Leon ==== paster.log below === ... galaxy.tools DEBUG 2011-07-22 16:02:47,084 Loaded tool: hbvar 1.0.0 galaxy.tools DEBUG 2011-07-22 16:02:47,088 Loaded tool: fix_errors 1.0.0 galaxy.tools DEBUG 2011-07-22 16:02:47,089 Loading section: Send Data galaxy.tools ERROR 2011-07-22 16:02:47,089 error reading tool from path: data_destination/epigraph.xml Traceback (most recent call last): File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( os.path.join( self.tool_root_dir, path ) ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 169, in load_tool return ToolClass( config_file, root, self.app ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 340, in __init__ self.parse( root ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 385, in parse self.interpreter = command.get("interpreter", None) AttributeError: 'NoneType' object has no attribute 'get' galaxy.tools ERROR 2011-07-22 16:02:47,090 error reading tool from path: data_destination/epigraph_test.xml Traceback (most recent call last): File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( os.path.join( self.tool_root_dir, path ) ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 169, in load_tool return ToolClass( config_file, root, self.app ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 340, in __init__ self.parse( root ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 385, in parse self.interpreter = command.get("interpreter", None) AttributeError: 'NoneType' object has no attribute 'get' ... galaxy.tools DEBUG 2011-07-22 16:02:50,556 Loading section: SNP/WGA: Data; Filters galaxy.tools DEBUG 2011-07-22 16:02:50,587 Loaded tool: upload1 1.1.3 galaxy.tools ERROR 2011-07-22 16:02:50,587 error reading tool from path: data_source/access_libraries.xml Traceback (most recent call last): File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( os.path.join( self.tool_root_dir, path ) ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 169, in load_tool return ToolClass( config_file, root, self.app ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 340, in __init__ self.parse( root ) File "/opt/galaxy/prog/galaxy-2011-7-21/lib/galaxy/tools/__init__.py", line 385, in parse self.interpreter = command.get("interpreter", None) AttributeError: 'NoneType' object has no attribute 'get' ... -- Hailiang (Leon) Mei Netherlands Bioinformatics Center (http://www.nbic.nl/) Skype: leon_mei Mobile: +31 6 41709231

2 1

Where to find the toolconfig parser code?
by Samuel Lampa 22 Jul '11

22 Jul '11

Quick question to you devs, I need to find the code where the tool configs are parsed ... especially the "free-text" if-else clauses in the command-tag. Could someone point me to the file? TIA // Samuel -- System Expert / Bioinformatician SNIC-UPPMAX / SciLifeLab Uppsala Uppsala University, Sweden -------------------------------------- E-mail: samuel.lampa(a)scilifelab.uu.se Phone: +46 (0)18 - 471 1060 WWW: http://www.uppmax.uu.se Uppnex: https://www.uppnex.uu.se

3 5

Re: [galaxy-dev] Error moving tool-data dir ... bowite indexes missing
by Bossers, Alex 22 Jul '11

22 Jul '11

Ross, I think I don't understand what you mean. Do you mean the actual reference to the files contained in the loc file? I have all non-relative paths to the bowtie indexes (snippet below). The last column is an additional column for some plotting tools to have a sensible short name and works fine in the original location. Separators are tabs. Alex {still confused} == Part of the bowtie_indices.loc -- Bos_taurus_ensembl Bos_taurus_ensembl Bos taurus ensembl 4.0.61 /share/bowtie/indexes/Bos_taurus_ensembl BosTaurus Homo_sapiens.GRCh37 Homo_sapiens.GRCh37 Homo sapiens ensembl GRch37 /share/bowtie/indexes/Homo_sapiens.GRCh37 Human Macaca_mulatta_NCBIdraft20110405 Macaca_mulatta_NCBIdraft20110405 Macaca mulatta (rhesus monkey draft NCBI) /share/bowtie/indexes/Macaca_mulatta_NCBIdraft20110405 Macaca Mus_musculus.NCBIM37 Mus_musculus.NCBIM37 Mus musculus NCBIM37 /share/bowtie/indexes/Mus_musculus.NCBIM37 Mouse -- -----Oorspronkelijk bericht----- Van: Ross [mailto:ross.lazarus@gmail.com] Verzonden: vrijdag 22 juli 2011 15:15 Aan: Bossers, Alex Onderwerp: Re: [galaxy-dev] Error moving tool-data dir ... bowite indexes missing If those changes invalidate your loc files, you may need to adjust them? On Fri, Jul 22, 2011 at 11:02 PM, Bossers, Alex <Alex.Bossers(a)wur.nl<mailto:Alex.Bossers@wur.nl>> wrote: > On our up-to-date galaxy_central version I have successfully moved the > database/files directory to mounted storage. After adapting the > universe_wsgi.ini it works fine after galaxy restart. > > > > Now the tool-data dir becomes bigger and bigger and I wanted to move > that to mounted storage as well. > > I moved the tool-data dir to the other location, made sure file > permissions were ok and adapted the universe_wsgi.ini tool_data_path > to match the new tool-data location. > > > > Galaxy restarts and the loc files are read (since blast recognises its > databases). However, the bowtie indexes are not read since they do not > show up in the NGS bowtie mapper for illumina..... what could be wrong? > > > > When I restore the universe_wsgi.ini it works all fine. > > I am puzzled. > > > > > > Alex > > > > > > > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and other > Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Director of Bioinformatics, Channing Lab; Tel: +1 617 505 4850; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;

1 0

Per-user tools permissions
by Assaf Gordon 22 Jul '11

22 Jul '11

Hi, I've noticed a ticket on the wiki for per-user tool permissions: https://bitbucket.org/galaxy/galaxy-central/issue/269/use-galaxy-security-t… If anyone is interested, the following patch implements such thing (but requires manual code changes whenever the access needs to be changed). The method "tool_allowed_for_user()" can of course be extended/rewritten to read access permissions from file or from a database. Comments are welcomed, -gordon

2 1

Re: [galaxy-dev] Add Genome
by Jennifer Jackson 21 Jul '11

21 Jul '11

Hello Somayyeh, These data are from the same authors/release build. At Ensembl, the same 5 core chromosomes plus an additional 78 unplaced scaffolds are available. The data authors decided not to include these scaffolds in their publication at NCBI. For a primary genome, I suggest that we use the core 5 chromosomes as this is what was published externally to NCBI by the authors. Would this also be your preference? Both options are possible. Many genomes have unmapped scaffolds included in the Galaxy genome builds, however they are usually published at NCBI if/when the genome is submitted there. Thank you! Jennifer On 7/21/11 5:41 AM, Somayyeh Fahiminiya wrote: > Hi Jen, > > > Thanks for your email and sorry to be late . > > Please look at : > wget > ftp://ftp.ensemblgenomes.org/pub/plants/current/fasta/brachypodium_distachy… > > I dont know if it is more recent or the same version as yours > > > > Thanks again > > somayyeh > > > > > On Mon, Jul 11, 2011 at 9:05 PM, Jennifer Jackson <jen(a)bx.psu.edu > <mailto:jen@bx.psu.edu>> wrote: > > Hello, > > To confirm: the correct source would be this project, only has > chromosomes 1-5, at NCBI? Or is there a preferred (more > complete/recent) source? > > http://www.ncbi.nlm.nih.gov/__bioproject/32607 > <http://www.ncbi.nlm.nih.gov/bioproject/32607> > > Once we have the source confirmed, we can add this genome to our > to-add list, > > Best, > > Jen > Galaxy team > > > On 7/7/11 6:25 AM, Somayyeh Fahiminiya wrote: > > Hello everyone, > > Could you plz add the genome of "*Brachypodium distachyon*" to > the Galaxy. > > Thanks in advance > > > > _____________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > -- > Jennifer Jackson > http://usegalaxy.org/ > http://galaxyproject.org/ > > -- Jennifer Jackson http://usegalaxy.org/ http://galaxyproject.org/

1 0

workflows are broken in current galaxy-central version
by Edward Kirton 21 Jul '11

21 Jul '11

URL: https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8 Module paste.exceptions.errormiddleware:*143* in __call__ Module paste.debug.prints:*98* in __call__ Module paste.wsgilib:*539* in intercept_output Module paste.recursive:*80* in __call__ Module paste.httpexceptions:*632* in __call__ Module galaxy.web.framework.base:*145* in __call__ >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>body * =* method*(* trans*,* ****kwargs *)* Module galaxy.web.framework:*84* in decorator >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>* return* func*(* self*,* trans*,* ***args*,* ****kwargs *)* Module galaxy.web.controllers.workflow:*686* in editor >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>* return* trans*.*fill_template*(* "workflow/editor.mako"*,* stored*=*stored*, * annotation*=*self*.*get_item_annotation_str*(* trans*.*sa_session*,* trans *.*user*,* stored *)* *)* Module galaxy.web.framework:*661* in fill_template >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>* return* self*.*fill_template_mako*(* filename*,* ****kwargs *)* Module galaxy.web.framework:*672* in fill_template_mako >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>* return* template*.*render*(* ****data *)* Module mako.template:*296* in render >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>* return* runtime*.*_render*(*self*,* self*.*callable_*,* args*,* data*)* Module mako.runtime:*660* in _render >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#>**** _kwargs_for_callable*(*callable_*,* data*)**)* Module mako.runtime:*692* in _render_context >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#> _exec_template*(*inherit*,* lclcontext*,* args*=*args*,* kwargs*=*kwargs*)* Module mako.runtime:*718* in _exec_template >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#> callable_*(*context*,* ***args*,* ****kwargs*)* Module _base_panels_mako:*87* in render_body >> <https://galaxy.jgi-psf.org/workflow/editor?id=3918559ba2a747d8#> __M_writer*(*unicode*(*self*.*overlay*(*visible*=*self*.*overlay_visible*)** )**)* *TypeError: render_overlay() got an unexpected keyword argument 'visible'*

2 2

Detailed SGE timing information about galaxy jobs
by Assaf Gordon 21 Jul '11

21 Jul '11

Hi all, I got around to writing the report I always needed: how much time each job actually runs and how much time it spends waiting in the SGE queue. The attached shell script produces the report, combining information from SGE's QACCT with the galaxy job/dataset information. The output contains: job, user, tool name, dbkey, total input size in bytes, waiting time in SGE queue, actual SGE executiong running time, and some other tidbits. This allows finding how much running time each user had on the cluster, how much time each tool/user spend idly waiting, and some possible correlations between tools, dbkeys, input size and running time. The script is tightly coupled with SGE and PostgreSQL, but can probably be adapted to PBS/MySQL. Hope this helps someone, -gordon

3 3