- galaxy-dev - lists.galaxyproject.org

(no subject)
by Kristen Eisenberg 06 Sep '11

06 Sep '11

[galaxy-dev] Failure to prepare job error Hello, I'm running a Python/R tool from my Amazon Galaxy instance. To use R, I import a package I've installed called PypeR. My tool works fine from the command line, but when I try to run it on my Amazon instance, it gives me this error: Traceback (most recent call last): File "/nfs/software/galaxy/lib/galaxy/jobs/runners/local.py", line 61, in run_job job_wrapper.prepare() File "/nfs/software/galaxy/lib/galaxy/jobs/__init__.py", line 371, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/nfs/software/galaxy/lib/galaxy/tools/__init__.py", line 1543, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/nfs/software/galaxy/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/nfs/software/galaxy/eggs/Cheetah-2.2.2-py2.6-linux-i686-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1311891781_86_41894.py", line 83, in respond EOFError: EOF when reading a line With the additional caption "failure to prepare job". I've been struggling with this for days, and I'm pretty sure I don't have a typo in my .xml file. How can I solve this? Kristen Eisenberg Billige Flüge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de - Billige Flüge vergleichen

2 1

downloading pre-indexed genomes to our galaxy server
by Nikhil Joshi 06 Sep '11

06 Sep '11

Hi all, Is there a repository of pre-indexed genomes for base-space/color-space, bwa, bam, sam, etc...? It would be much easier than having to index them all ourselves.... :) - Nik.

2 1

handling galaxy updates
by Shantanu Pavgi 06 Sep '11

06 Sep '11

Hi, I am curious to know which revision of galaxy code gets deployed in the main ( http://main.g2.bx.psu.edu/ ) and test ( http://test.g2.bx.psu.edu/ ) instances of galaxy. I was thinking active development repository galaxy-central code gets deployed in the test galaxy instance and stable galaxy-dist code gets deployed in the main galaxy instance. However, it seems like main galaxy instance is updated more frequently than galaxy-dist repository. Should local galaxy instances keep up with PSU's main galaxy instance revisions or wait for stable code to be released in galaxy-dist repository? We have been keeping up with galaxy-dist repository and not the galaxy-central repository updates. Are other sites following similar update model? -- Thanks, Shantanu.

3 3

link file bug in the new version
by remy d1 06 Sep '11

06 Sep '11

Hi, We found a little problem in the new galaxy release. When we upload a dataset library from filesystem : Admin > Manage data libraries > create new data library > Add dataset > Upload files from filesystem path > Link to files without copying If galaxy user is the owner of this file or if he has write permission on it (on the filesystem), the file is deleted !! I do not think it is the normal behaviour... Regards.

2 3

rpy - No module named rpy CentoOs install
by Joseph Hargitai 06 Sep '11

06 Sep '11

Hi, On our Ubuntu install stat packages and all that require rpy work fine. On our CentOs install seeing this stubborn error that I did see from previous post to be difficult to fix. At first suspected the SGE issue - environment not transferring to compute nodes. After changing the app to run local had the same issue. CentOs: 2.6.18-92.1.13.el5 rpy module is in: /apps1/python/2.6.6/intel/lib/python2.6/site-packages _rpy2122.so _rpy2131.so version: [galaxy@compute-0-65 galaxy-dist]$ python -c "import rpy; print rpy.__version__" 1.5.1 path: python -c 'import sys; print "\n".join( sys.path )' /apps1/python/2.6.6/intel/lib/python2.6/site-packages/simplejson-2.0.9-py2.6-linux-x86_64.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/Sphinx-1.0.7-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/docutils-0.7-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/Jinja2-2.5.5-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/Pygments-1.4-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/Traits-3.5.0-py2.6-linux-x86_64.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/nibabel-1.0.0-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/nipype-0.0.0-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/setuptools-0.6c12dev_r88846-py2.6.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/birdsuite-1.0-py2.5.egg /apps1/python/2.6.6/intel/lib/python2.6/site-packages/mpgutils-0.7-py2.5.egg /apps1/python/2.6.6/intel/lib/python26.zip /apps1/python/2.6.6/intel/lib/python2.6 /apps1/python/2.6.6/intel/lib/python2.6/plat-linux2 /apps1/python/2.6.6/intel/lib/python2.6/lib-tk /apps1/python/2.6.6/intel/lib/python2.6/lib-old /apps1/python/2.6.6/intel/lib/python2.6/lib-dynload /apps1/python/2.6.6/intel/lib/python2.6/site-packages /apps1/python/2.6.6/intel/lib/python2.6/site-packages/PIL compiled against /R/2.13.1 env: export PATH=.:\ /apps1/R/2.13.1/intel/bin:\ /apps1/python/2.6.6/intel/bin:\ /apps1/pipe/bowtie/0.12.7/intel:\ /apps1/pipe/bwa/0.5.9/intel:\ /apps1/samtools/0.1.13/intel/bin:\ /apps1/fastx_toolkit/0.0.13/intel/bin:\ /apps1/maq/maq-0.7.1:\ /apps1/maq/maq-0.7.1/scripts:\ /apps1/bfast/bfast-0.6.5a/butil:\ /apps1/bfast/bfast-0.6.5a/scripts:\ /apps1/abyss/1.2.7/intel/bin:\ /apps1/velvet/velvet_1.0.12:\ /apps1/pipe/tophat/1.3.0/intel/bin:\ /apps1/pipe/cufflinks/1.0.3/intel/bin:\ /apps1/blast/2.2.25/gnu/bin:\ /apps1/blast+/2.2.5/gnu/bin:\ /apps1/sputnik/intel/bin:\ /apps1/taxonomy/intel/bin:\ /apps1/add_scores/add_scores:\ /apps1/emboss/6.4.0/intel/bin:\ /apps1/hyphy/hyphy/HYPHY:\ /apps1/lastz/1.02.00:\ /apps1/perm/0.3.6/intel/bin:\ /apps1/beam2/intel/bin:\ /apps1/pass2/intel/bin:\ /apps1/plink/1.07/intel/bin:\ /apps1/fbat/2.0.3/bin:\ /apps1/eigensoft/3.0/intel/bin:\ /apps1/mosaik/Mosaik-1.1.0021-Linux-x64/bin:\ /apps1/freebayes/freebayes.git/bin:\ $PATH export LD_LIBRARY_PATH=.:\ /apps1/python/2.6.6/intel/lib:\ /apps1/libgtextutils/0.6/intel/lib:\ /apps1/emboss/6.4.0/intel/lib:\ /apps1/intel/lib/intel64:\ /apps1/intel/mkl/lib/em64t:\ /apps1/tcltk/8.5.9/intel/lib:\ /apps1/zlib/1.2.5/intel/lib:\ /apps1/graphviz/2.26.3/intel/lib:\ /apps1/python/2.6.6/intel/lib/python2.6/site-packages/simtk/chem/openmm/OpenMM:\ /apps1/python/2.6.6/intel/lib/python2.6/site-packages:\ /apps1/libpng/1.5.0/intel/lib:\ /apps1/R/2.13.1/intel/lib64/R/lib:\ $LD_LIBRARY_PATH export PKG_CONFIG_PATH=.:\ /apps1/R/2.13.1/intel/lib64/pkgconfig:\ /apps1/libgtextutils/0.6/intel/lib/pkgconfig:\ /apps1/sparsehash/1.11/intel/lib/pkgconfig:\ $PKG_CONFIG_PATH export CLASSPATH=.:\ /apps1/gatk/gatk-git/dist:\ /apps1/gatk/gatk-git/lib:\ /apps1/srma/srma-0.1.13:\ /apps1/haploview/4.2:\ /apps1/picard/picard-tools-1.50:\ /apps1/fastqc/fastqc-0.9.5:\ $CLASSPATH best, joe

2 4

Regarding display application
by shashi shekhar 06 Sep '11

06 Sep '11

Hi All., I need help in understanding the xml tag for display_application. i don't know how all these xml tags works . can u provide me links for understanding those tags like target_frame type="template" viewable="False <display id="genetrack_interval" version="1.0.0" name="view in"> <link id="genetrack" name="GeneTrack"> <url target_frame="galaxy_main">http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&h…</url> <param type="data" name="bed_file" viewable="False" format="bed6,genetrack"/>  <param type="data" dataset="bed_file" name="genetrack_file" format="genetrack" viewable="False" /> <param type="template" name="galaxy_url" strip="True" > ${BASE_URL}/tool_runner?tool_id=predict2genetrack </param> <param type="template" name="hash_key" strip="True" > #from galaxy.util.hash_util import hmac_new ${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )} </param> <param type="template" name="encoded_filename" strip="True" > #import binascii ${binascii.hexlify( $genetrack_file.file_name )} </param> </link> </display> Regards shashi shekhar

2 3

Feedback appreciated for new unified tabular format: GTrack
by Sveinung Gundersen 05 Sep '11

05 Sep '11

Dear Galaxy-dev guys! Our research group have the last years been developing the Genomic HyperBrowser (http://hyperbrowser.uio.no) which is a system for statistical analysis of genomic data, built on top of Galaxy. The system currently includes 17 hypothesis tests and 53 descriptional statistics. In the process of developing the HyperBrowser, we have experienced several shortcomings of the usual tabular formats for genomic datasets: BED, WIG, GFF, BedGraph, etc. This has lead us to define (yet another) format for genomic data: the GTrack format (short for Genomic Track). The format will hopefully be published in an article, currently under review (together with an extension of the XML format BioXSD, which supports many of the same properties as the GTrack format). In the process of article review, we would be very interested in the feedback from you guys, in order for the format to be as good as possible. The basic issues handled are the following: 1. The first issue is the very existence of so many formats. We have, in the article, defined 15 different track formats and believe that these track formats are the main reason for the proliferation of tabular formats. The track formats are the usual Segments (as in BED) or Valued Segments (as in BedGraph), but also include other types as Points or Step Function. In addition, we introduce linked track types usable for analysis of three-dimensional data set. The GTrack format handles all 15 track types. 2. Simple to create. Allthough the GTrack format specification document is quite large, the format is still quite simple to handle. It is based on fixed columns (not attributes like GFF), but allows custom columns (unlike BED). If you allready have scripts creating output in a common tabular format, they should require little change to support GTrack. 3. Customizability. GTrack allows any number of custom columns to be added, in any order. Also, GTrack supports a scheme for creating GTrack subtypes. A GTrack subtype is a particular configuration of GTrack files explicitly created for specific uses/tools. All GTrack subtypes can still be handled by generic GTrack parsers. 4. Simple to parse. We have tried to make GTrack as simple as possible to parse. This includes the use of header lines for defining properties of a file. This eases parsing by telling the parses what is coming, plus it allows quick and dirty parsers to explicitly assert what they are able to handle (so as to abort with a clearly stated reason instead of failing silently). Also, the GTrack subtyping scheme allows parsers to limit their support to a subset of the GTrack specification, e.g. files with a fixed number and order of columns. 5. Advanced functionality. GTrack supports more advanced functionality such as networks of track elements and the option of defining the domain of a track (e.g. the genomic regions for which the track is defined). 6. Syntax, not semantics. As with BED or WIG, the GTrack format focuses on the structural elements of the data, e.g. how to represent data mathematically/informatically. We leave the specifics of interpretation to others (who can, for instance, use their definitons to create GTrack subtypes). We have also been in contact with the Galaxy team and have received positive signals regarding future support for GTrack in Galaxy. Note that tools for converting between GTrack and other formats, in addition to a tool to help create GTrack headers, will be available soon. We hope you find the format interesting and welcome all kinds of feedback/suggestions. As the paper is in a review process, we would appreciatee feedback within the next two weeks. Version 1.0b2 of the GTrack specification and an illustration of the 15 track types are available here: http://hyperbrowser.uio.no/hb/static/hyperbrowser/files/gtrack/GTrack_speci… http://hyperbrowser.uio.no/hb/static/hyperbrowser/files/gtrack/track_types.… For the HyperBrowser team, Sveinung Gundersen -- Sveinung Gundersen PhD Student, Bioinformatics, Dept. of Tumor Biology, Inst. for Cancer Research, The Norwegian Radium Hospital, Montebello, 0310 Oslo, Norway E-mail: sveinung.gundersen(a)medisin.uio.no, Phone: +47 93 00 94 54

1 0

installing software for Galaxy
by Mattias de Hollander 05 Sep '11

05 Sep '11

Hello Galaxy-developers and community, I would like to ask your advice on installing software. I used the cloudman scripts to install galaxy on a multi-core server. I use the fabric scripts from cloudman/mi-diployment to install some software but I am also looking at repositories like biolinux and the NBIC RPM repository. There has been a thread about that over here: http://gmod.827538.n3.nabble.com/RPM-repository-for-NGS-tools-in-Galaxy-tp2… As James says in that thread I also prefer to install software in isolated directories to keep track of different version. Did someone use the biolinux repository to install software in a galaxy accepted path: $GALAXY_APPS/package/version/? How does the Galaxy Cloud team does that? (Enis?) I was thinking to use 'sudo apt-get --download-only' to first get the debs and then install them using 'dpkg --instdir' and then specifying the directory. Or is there a smarter way to do this? Any thoughts on this issue are appreciated. Thanks. -- Bioinformatician Netherlands Institute of Ecology (NIOO-KNAW) Wageningen, the Netherlands

3 3

adding display application in local instance of galaxy
by shashi shekhar 05 Sep '11

05 Sep '11

Hi All , i am working in galaxy tool . i want to add one visualization tools in galaxy . i don't understand how the display applications works . tag ox xml file is not clear for display application . can u just explain the meaning of type="template" tag , target_frame tag , &amp tag ,BASE URL , strip tag and viewable tag in this xml? <display id="genetrack_interval" version="1.0.0" name="view in"> <link id="genetrack" name="GeneTrack"> <url target_frame="galaxy_main">http://genetrack.g2.bx.psu.edu/galaxy?filename=${encoded_filename.qp}&h…</url> <param type="data" name="bed_file" viewable="False" format="bed6,genetrack"/>  <param type="data" dataset="bed_file" name="genetrack_file" format="genetrack" viewable="False" /> <param type="template" name="galaxy_url" strip="True" > ${BASE_URL}/tool_runner?tool_id=predict2genetrack </param> <param type="template" name="hash_key" strip="True" > #from galaxy.util.hash_util import hmac_new ${hmac_new( $APP.config.tool_secret, $genetrack_file.file_name )} </param> <param type="template" name="encoded_filename" strip="True" > #import binascii ${binascii.hexlify( $genetrack_file.file_name )} </param> </link> </display> Regards shashi shekhar

1 0

Suggestion / Request for Comments on Galaxy Best Practices - Gradual migration to standard indentation
by Trevor Wennblom 05 Sep '11

05 Sep '11

i've been developing within Galaxy for awhile now and have been enjoying it quite a bit. i've seen portions of similar ideas in the past, but never cohesively assembled and so well realized. i'd like to make some suggestions for further code collaboration / community contributions. my default development environment seems to occasionally be causing havoc within my local repository. upon saving, i normally automatically strip right-sided whitespace to stay compatible with other developers i collaborate with. this also fixes the auto-indentation left by my editor when it occasionally leaves a line consisting of only spaces. given that python has syntactically significant whitespace, i also try to maintain the convention of indentation with four-spaces. i've noticed this isn't consistent within the codebase, but does seem to be the preferred style such as in `lib/galaxy/datatypes/`. python comes packaged with the script `reindent.py`: Change Python (.py) files to use 4-space indents and no hard tab characters. Also trim excess spaces and tabs from ends of lines, and remove empty lines at the end of files. Also ensure the last line ends with a newline. this is recommended practice per PEP 8: http://www.python.org/dev/peps/pep-0008/ Tabs or Spaces? Never mix tabs and spaces. The most popular way of indenting Python is with spaces only. The second-most popular way is with tabs only. Code indented with a mixture of tabs and spaces should be converted to using spaces exclusively. When invoking the Python command line interpreter with the -t option, it issues warnings about code that illegally mixes tabs and spaces. When using -tt these warnings become errors. These options are highly recommended! For new projects, spaces-only are strongly recommended over tabs. Most editors have features that make this easy to do. other comments in regards to collaboration: http://rails-bestpractices.com/posts/60-remove-trailing-whitespace http://blogobaggins.com/2009/03/31/waging-war-on-whitespace.html hooks can be added to the repository for consistency, as i'm sure with the many contributions received this would be easy to miss: http://mercurial.selenic.com/wiki/CheckFilesExtension would anyone be opposed to me fixing up the current codebase to adhere to this? running `reindent.py` on the files is easy enough, i'm willing to step through the files (`opendiff` / `FileMerge.app`) and verify no unlikely syntactic changes have occurred. i can also deliver changes in gradual "chunked" pull requests to ease current developers getting possibly bit by merge issues. would anyone be willing to add the appropriate hooks to the central repository as well? trevor trevor(a)well.com https://github.com/trevor

6 6