- galaxy-dev - lists.galaxyproject.org

How can I populate a data_collection output ?
by julie dubois 30 Jul '14

30 Jul '14

Hi all, I'm working with data_collection as type of input in xml files. but I use all the datasets of a collection to compute some results that I want to push in a "data_collection" output. I know how take each dataset of a data_collection input with my script, like this : <configfile name="shscript"> ... for $f in $input ... </configfile> where $input is defined by <param name=$input1 type=data_collection collection_type=list... in the top of my xml file. But if my script produce some outputs (number of outputs not defined in advance), how can I populate an output as a data_collection with my shscript and how should I define this output in the xml ? Thanks Julie

3 4

Uploads with embedded citations causing red error on Tool Shed
by Peter Cock 30 Jul '14

30 Jul '14

Hi John, Following the work at the BOSC 2014 CodeFest to support embedded citations within Galaxy Tool XML files [1], and your work adding this to the BLAST tools as an example [2], I tried uploading a minor tool using this to the Tool Shed. The upload seems to have worked, but there was a scary red error message about metadata... see below (both main and test toolsheds affected). Regards, Peter [1] https://bitbucket.org/galaxy/galaxy-central/pull-request/440/ [2] https://github.com/peterjc/galaxy_blast/commit/9d2e3906915895765ecc3f48421b… -- Uploading on the Test Tool Shed, red error: Metadata may have been defined for some items in revision '796dc2ff8e8e'. Correct the following problems if necessary and reset metadata. blastxml_to_top_descr.xml - 'UniverseApplication' object has no attribute 'citations_manager' https://testtoolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr -- Uploading on the Tool Shed, red error: Metadata may have been defined for some items in revision 'fe1ed74793c9'. Correct the following problems if necessary and reset metadata. blastxml_to_top_descr.xml - 'UniverseApplication' object has no attribute 'citations_manager' https://toolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr

3 6

change in file of Galaxy version
by yassine 30 Jul '14

30 Jul '14

Hello, In our old version of Galaxy (July,2012), we have this file GalaxyDIR/templates/webapps/galaxy/base_panels.mako that has changed a lot of to actual version (latest_2014.06.02). Do you have some documentation about this? Because when we used an iframe, all upper menu disappear, Any clue? Thank in advance for your answer -- With Regards Yassine

1 0

Main Galaxy Tool Shed is running the next-stable branch
by Greg Von Kuster 29 Jul '14

29 Jul '14

Hello all, The main Galaxy Tool Shed is now running the next-stable branch ( e6813f244a2a (next-stable) tip ) in preparation for next upcoming Galaxy release. Please let us know if you encounter anything strange or broken and we'll get things fixed asap. The main Tool Shed will be updated to the stable branch when it is created for the upcoming Galaxy release. Thanks very much, Greg Von Kuster

1 0

Re: [galaxy-dev] Error while installing DESeq-hts
by Geert Vandeweyer 29 Jul '14

29 Jul '14

I know this is a very old entry to the mailing list, but I thought this might be useful for some people. Recently, I set up DESeq-hts and its siblings on our galaxy instance. To succeed, I performed the steps below. @vipin: would you consider revising the tool_dependencies.xml for these tools to take these steps into account? If you're interested, I might compose a preliminary version. I believe that most, if not all of these settings can be automated using the tool_dependencies files. I did not manage to get DEXSeq working, as it throws errors on the 'estimateDispersions' step in R. I'm not sure how to solve this, as it gives me the same errors using the demonstration data from the pasilla library, following the manual steps. (R 3.0.2, bioconductor 2.13, DEXSeq 1.8). Any hints on that are welcome. Best, Geert ## GALAXY PACKAGES IN TOOLSHED ####################### - install the DESeq-hts tools from the main toolshed (cec4b4fb30be from vipints) - install R into galaxy (I used package_R_3_0_2 from iuc (50f7e1e71271)) - install package_scipi and package_numpi (from iuc) - install package_samtools-0.1.18 (I used devteam/package_samtools_0_1_18/c0f72bdba484) - install htseq_count package (I used d5edaf8dc974 from lparsons) ## INSTALL DESeq bioconductor package ######################### - open R from the commandline (using the R version installed earlier !), install bioconductor package 'DESeq' : - source("http://bioconductor.org/biocLite.R") - biocLite('DESeq') - biocLite('DESeq2') - biocLite('DEXSeq') (needed extra ubuntu package libcurl4-gnutls-dev for this) ## complie samtools from source (temporary) ############################# - download samtools 0.1.18 source code into a temporary location and unpack - compile with : "make CXXFLAGS=-fPIC CFLAGS=-fPIC CPPFLAGS=-fPIC" ## install octave ########### - OS-dependent, ubuntu: apt-get install octave ## set up DESeq-hts-1.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0 - run : "setup_deseq-hts.sh" and provide paths as requested. - NOTE: set samtools path to the source compiled version for now (shed version fails to find sam.h)!! - NOTE: also add NumPy path to the SCIPY_PATH variable - rest: example (available in bin/deseq_config.sh): export LD_LIBRARY_PATH= export ENVIRONMENT=galaxy export DESEQ_VERSION=1.12.1 export DESEQ_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_ht… export DESEQ_SRC_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/dese… export DESEQ_BIN_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/dese… export INTERPRETER=octave export MATLAB_BIN_PATH= export MATLAB_MEX_PATH= export MATLAB_INCLUDE_DIR= export OCTAVE_BIN_PATH=/opt/software/octave-3.8.0/bin/octave export OCTAVE_MKOCT=/opt/software/octave-3.8.0/bin/mkoctfile export SAMTOOLS_DIR=/galaxy/packages_sources/samtools-0.1.18_fPIC #export SAMTOOLS_DIR=/galaxy/galaxy_tool_binaries/samtools/0.1.18/devteam/package_samtools_0_1_18/c0f72bdba484/bin/ export PYTHON_PATH=/galaxy/galaxy_env/bin/python export SCIPY_PATH=/galaxy/galaxy_tool_binaries/scipy/0.12.0/iuc/package_scipy_0_12/984d208b0808/lib/python/:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python/ ## USE THE TOOLSHED VERSION, The one you installed bioclite packages with. export R_PATH=/galaxy/galaxy_tool_binaries/R_3_0_2/3.0.2/iuc/package_r_3_0_2/50f7e1e71271/R - enter mex directory, run "make octave" - swap SAMTOOLS_DIR in bin/deseq_config.sh to point to the toolshed version of samtools 0.1.18 ## RUN DESeq-hts-1.0 TEST ################## enter src directory, run test from the toolconf xml file: "./deseq-hts.sh ../test_data/deseq_c_elegans_WS200-I-regions.gff3 ../test_data/deseq_c_elegans_WS200-I-regions_deseq.txt ../test_data/genes.mat ../test_data/deseq_c_elegans_WS200-I-regions-SRX001872.bam ../test_data/deseq_c_elegans_WS200-I-regions-SRX001875.bam" ## set up DESeq-hts-2.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_2.0 - similar to 1.0: setup config file, - point to src compiled version of samtools - add scipy and numpy to scipi_path variable - use the toolshed version of R - in mex, run "make octave" - swap samtools path IMPORTANT: ---------------- - open "src/deseq2-hts.sh" in a text editor and add an extra line after line 38 (echo load the genome annotation....): export PYTHONPATH=$PYTHONPATH:${SCIPY_PATH} - in the same file (src/deseq2-hts.sh), append line 70 (../bin/get_read_counts...) with 2>&1 . This prevents some octave warnings from crashing the job (missing docstring warnings) Test-run in galaxy seems to work. ## set up DEXSeq-hts-1.0 ############### go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/dexseq_1.0 run setup_dexseq-hts.sh - set samtools path - set python_path : I use the galaxy python version (/galaxy/galaxy_env/bin/python), see development docs - set pythonpath: Make sure to include both HTSeq AND numpy here ! : export PYTHONPATH=/galaxy/galaxy_tool_binaries/htseq/0.5.4p5/lparsons/htseq_count/3ffe4e2572a7/lib/python/HTSeq-0.5.4p5-py2.6-linux-x86_64.egg:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python:$PYTHONPATH - set R_path : I use toolshed version package_r_3_0_2 from iuc (with the installed bioclite packages from above) => FAILS : crashes on estimateDispersions / fitDispersionFunction step in run_DEXseq.R -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandeweyer@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

3 2

galaxy won't start
by Sajdak, Doris 29 Jul '14

29 Jul '14

I'm suddenly getting this error today when I tried to restart Galaxy: Starting galaxy... Some eggs are out of date, attempting to fetch... Warning: setuptools (a dependent egg of sqlalchemy-migrate) cannot be fetched Fetched http://eggs.galaxyproject.org/bioblend/bioblend-0.4.2-py2.7.egg One of Galaxy's managed eggs depends on something which is missing, this is almost certainly a bug in the egg distribution. Dependency "bioblend" requires "pyyaml" Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 37, in <module> c.resolve() # Only fetch eggs required by the config File "/ifs/user/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/ifs/user/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 168, in resolve dists = pkg_resources.working_set.resolve( ( self.distribution.as_requirement(), ), env, self.fetch ) File "/ifs/user/galaxy/galaxy-dist/lib/pkg_resources.py", line 569, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (bioblend 0.4.2 (/ifs/user/galaxy/galaxy-dist/eggs/bioblend-0.4.2-py2.7.egg), Requirement.parse('pyyaml')) pyyaml is installed and I haven't updated anything that would cause a conflict. What do I do to fix this? Thanks, Dori ********************************** Dori Sajdak Senior Systems Administrator State University of NY at Buffalo Center for Computational Research 701 Ellicott St Buffalo, NY 14203 Phone: (716) 881-8934 Fax: (716) 849-6656 Web: http://ccr.buffalo.edu **********************************

2 2

is there an issue with displaying txt files in latest galaxy?
by Geert Vandeweyer 28 Jul '14

28 Jul '14

Hi, Is there a known issue in the latest galaxy-dist related to displaying txt files? If I click the eye icon for a 'txt' dataset, nothing is shown. The preview shows the data correctly, and if I check the path on the server, the data and contents are definitely there. main.log doesn't show any errors. I updated today using hg pull && hg update latest_2014.06.02: changeset 13584 Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

1 1

Settings multiple Genome Indices
by Geert Vandeweyer 28 Jul '14

28 Jul '14

Hi, I'm working on tool wrapper that needs three reference genome locations (2bit/lastz, all_fasta and novoalign). Is there a way to fetch the three paths from the loc files using a single select box? I've looked around in a couple of tool wrappers on the toolshed, but could find any leads. Something like the following would be ok: Select options from all_fasta.loc -> upon change, set hidden variables to novoalign.loc + lastz_seqs.loc using a filter tag or something against the selected options from all_fasta. Even more simple : access in the command tag like: --novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the input file (which is a bam file in this case) In this case, error handling should come from the tool, if one of the indices is missing. I'm currently using three selection boxes, but that looks a but redundant to the user, as they all have to select the same thing. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

3 5

GATK problem on Galaxy
by Jingchao Zhang 28 Jul '14

28 Jul '14

Hi All, I am having problems using GATK Indel Realigner. We installed our own galaxy instance on a VM server and configured it to submit jobs through DRMAA to the clusters. The first error I got is: " ERROR MESSAGE: Running with the -et NO_ET or -et STDOUT option requires a GATK Key file. Please see http://gatkforums.broadinstitute.org/discussion/1250/what-is-phone-home-and… for more information and instructions on how to obtain a key. " I solved this by modifying the "indel_realigner.xml" file and changed the -rt "NO_ET" to "-et STANDARD". Then comes the second the error as below: " ERROR MESSAGE: Invalid command line: Failed to load reference dictionary " Stdout has: " [bam_header_read] EOF marker is absent. The input is probably truncated. [bam_header_read] invalid BAM binary header (this is not a BAM file). [bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM file. " The "dataset.dat" file has the following: " INFO 09:17:58,332 RodBindingArgumentTypeDescriptor - Dynamically determined type of /tmp/tmp-gatk-2gGVA2/gatk_target_intervals.bed to be BED INFO 09:17:58,452 GenomeAnalysisEngine - Strictness is SILENT WARN 09:17:58,502 FSLockWithShared - WARNING: Unable to lock file /work/galaxy/galaxy/genomes/mm9/picard/mm9.dict: Function not implemented. INFO 09:17:58,504 ReferenceDataSource - Unable to create a lock on dictionary file: Function not implemented INFO 09:17:58,505 ReferenceDataSource - Treating existing dictionary file as complete. WARN 09:17:58,507 FSLockWithShared - WARNING: Unable to lock file /work/galaxy/galaxy/genomes/mm9/picard/mm9.fa.fai: Function not implemented. INFO 09:17:58,509 ReferenceDataSource - Unable to create a lock on index file: Function not implemented INFO 09:17:58,510 ReferenceDataSource - Treating existing index file as complete. " I don't have any bio background and lost track of the error messages. Is this error caused by the Galaxy GATK wrapper or the input data files? I got those input files from my colleague who used to run them successfully on another older Galaxy server. That makes me wonder if there is something with our GATK configuration. My apologies if asking this question in the wrong place. Any suggestion is appreciated. Thanks, Jingchao

1 0

Jobs stay queued
by julie dubois 28 Jul '14

28 Jul '14

Hi all, I've seen several posts for jobs which stay queued and never start with an installation of galaxy on a cluster. But our installation is just a local instance of galaxy in 1 PC and I have more than 100 jobs which stay queued and never start. How can I debug this problem (my version of galaxy is the latest 2june2014). And is it possible to run those jobs without restart galaxy because it would be very long to rerun each job manually after reloading galaxy. Thanks. Julie

3 7