- galaxy-dev - lists.galaxyproject.org

Samtools 1.1 Dependency
by Hans Vasquez-Gross 13 Jul '15

13 Jul '15

Hi All, I have a galaxy installation on an OSX system that has been running fine with our custom tools. However, now we want to start using for some simple BAC mapping. When I go to the main toolshed to install bwa, it lists the following dependencies. Repository *package_bwa_0_7_10_039ea20639* revision *5b9aca1e1c07* owned by *devteam*Repository *package_samtools_1_1* revision *43f2fbec5d52* owned by *iuc* However, when running through this process, it fails at the package_samtools_1_1 stage with an Error. Looking at the INSTALLATION.LOG i see the following. ############################################# sed -i 's/-lcurses/-lncurses/' Makefile STDOUT ############################################# ############################################# sed -i 's/-lcurses/-lncurses/' Makefile STDERR sed: 1: "Makefile": invalid command code M ############################################# However, looking at the tool_dependencies.xml for this particular package, I see the following. I've highlighted in yellow a key configuration. <?xml version="1.0"?> <tool_dependency> <package name="samtools" version="1.1"> <install version="1.0"> <actions_group> <actions architecture="x86_64" os="linux"> <action type="download_by_url"> http://depot.galaxyproject.org/package/linux/x86_64/samtools/samtools-1.1-L… </action> <action type="move_directory_files"> <source_directory>.</source_directory> <destination_directory>$INSTALL_DIR</destination_directory> </action> </actions> <actions> <package name="ncurses" version="5.9"> <repository changeset_revision="5e1760c773ba" name="package_ncurses_5_9" owner="iuc" prior_installation_required="True" toolshed="https://toolshed.g2.bx.psu.edu" /> </package> <package name="zlib" version="1.2.8"> <repository changeset_revision="63a4a902cda2" name="package_zlib_1_2_8" owner="iuc" prior_installation_required="True" toolshed="https://toolshed.g2.bx.psu.edu" /> </package> <action type="download_by_url"> http://downloads.sourceforge.net/project/samtools/samtools/1.1/samtools-1.1… </action> <action type="set_environment_for_install"> <repository changeset_revision="5e1760c773ba" name="package_ncurses_5_9" owner="iuc" toolshed=" https://toolshed.g2.bx.psu.edu"> <package name="ncurses" version="5.9" /> </repository> <repository changeset_revision="63a4a902cda2" name="package_zlib_1_2_8" owner="iuc" toolshed=" https://toolshed.g2.bx.psu.edu"> <package name="zlib" version="1.2.8" /> </repository> </action> <action type="shell_command">sed -i 's/-lcurses/-lncurses/' Makefile</action> <action type="shell_command">sed -i -e "s|CFLAGS=\s*-g\s*-Wall\s*-O2\s*|CFLAGS= -g -Wall -O2 -I$NCURSES_INCLUDE_PATH/ncurses/ -I$NCURSES_INCLUDE_PATH -L$NCURSES_LIB_PATH|g" Makefile< /action> <action type="shell_command">make</action> <action type="move_file"> <source>samtools</source> <destination>$INSTALL_DIR/bin</destination> </action> </actions> <action type="set_environment"> <environment_variable action="prepend_to" name="PATH">$INSTALL_DIR/bin</environment_variable> <environment_variable action="set_to" name="SAMTOOLS_ROOT_PATH">$INSTALL_DIR</environment_variable> </action> </actions_group> </install> So, to me, it sounds like this is trying to compile/install a distribution meant for linux, but I'm on OSX. I was able to successfully install samtools 1.2 with no errors. Is there anyway to update, or modify the code for the BWA installation recipe to use as a dependency samtools 1.2 rather then 1.1? Or does anyone have any suggestions on how to get samtools 1.1 to work? I tried making a bin/ directory and moving a precompiled samtools binary for 1.1 in the directory. But galaxy didn't seem to recognize that the dependency was satisfied manually. Any advice would be greatly appreciated. Thank you, -Hans

2 1

problem with npm install for galaxy ipython
by Christian Brenninkmeijer 09 Jul '15

09 Jul '15

Thank you for the excellent training session on using ipython in galaxy. http://gcc2015.tsl.ac.uk/training-day/#Galaxy_Interactive_Environments However when I tried to repeat it on my desktop I had an error doing the step Change directory into: lib/galaxy/web/proxy/js/ $ cd lib/galaxy/web/proxy/js/ Then install the node proxy with “npm”: $ npm install . I get: npm http GET https://registry.npmjs.org/sqlite3/3.0.2 npm http 304 https://registry.npmjs.org/sqlite3/3.0.2 npm http GET https://registry.npmjs.org/nan/1.3.0 npm http 304 https://registry.npmjs.org/nan/1.3.0 > sqlite3(a)3.0.2 install /home/christian/galaxy/lib/galaxy/web/proxy/js/node_modules/sqlite3 > node-pre-gyp install --fallback-to-build /usr/bin/env: node: No such file or directory npm WARN This failure might be due to the use of legacy binary "node" npm WARN For further explanations, please read /usr/share/doc/nodejs/README.Debian npm ERR! weird error 127 npm ERR! not ok code 0 System Information: Galaxy master revision:6f53676f482634212bac512ec91dbf24e4afacff (also tried against today's dev) OS: Distributor ID: Ubuntu Description: Ubuntu 14.04.2 LTS Release: 14.04 Codename: trusty npm version: 1.3.10 Which is rather strange as these are all exactly the same as in the Virtual machine used. Any one have any ideas? Thanks in advance Christian PS. Below is the content of: /usr/share/doc/nodejs/README.Debian nodejs for Debian ================= packaged modules ---------------- The global search path for modules is /usr/lib/nodejs Future packages of node modules will use that directory, so it should be used wisely. user modules ------------ Node looks for modules in ./node_modules directory first; please read node#modules documentation carefully for more information. Node does not look for modules in /usr/local/lib/node_modules, where npm put them. Please read npm-link(1) of npm package, to understand how to properly use npm-installed modules in a project. Note that require.paths is not supported in future node versions. See also node(1) for more information about NODE_PATH. nodejs command -------------- The upstream name for the Node.js interpreter command is "node". In Debian the interpreter command has been changed to "nodejs". This was done to prevent a namespace collision: other commands use the same name in their upstreams, such as ax25-node from the "node" package. Scripts calling Node.js as a shell command must be changed to instead use the "nodejs" command.

2 1

problem with view of list collection in history view
by Alexander Vowinkel 09 Jul '15

09 Jul '15

Hi Team, my tool creates dynamically 96 datasets bundled into a list. In the history I can see the number 96 in the top as hidden datasets (6 shown, 96 hidden) When I open the list, I just can see 64 items. Now I run the job again and I have 96 more hidden items. I open the new list and can see 66 items in that new list. What is going on here? Is that "just" a visual bug? Or are my datasets affected? Thanks, Alexander PS: I use postgres

2 2

Change Password
by Mariel Voutounou 08 Jul '15

08 Jul '15

Hello, I am trying to re-set my password but I am not receiving the required email. Please let me know on how to proceed Kind Regards, Mariel

1 0

Error in Reset Password User Notification
by Matthias Enders 08 Jul '15

08 Jul '15

Dear Galaxy Community, as we tried to establish the mail functionalities for our Galaxy instance, we managed to send email upon the "I forgot my password" user request. In the Email a Link is provided. Here two errors arise: 1. In the File "galaxy/lib/galaxy/webapps/galaxy/controllers/user.pyc" HTML Tags for this Email are generated, but the Email is send as plain/text, therefore the HTML Tags, as well as the (doubled) Link are visible for the user. 2. In our case, the provided link takes us to the normal user login page, where the user has no option of resetting his/her password. Can you provide me some assistance to tackle the problems? Kind Regards, Matthias Enders

1 0

Cosmetic Makeblastdb error promulgated through wrapper
by Dooley, Damion 08 Jul '15

08 Jul '15

Makeblastdb can spit out warning notices when a higher count of N ambiguous nucleotides occurs in the first row of fasta data, but goes ahead and completes the blast database. "Error: (1431.1) FASTA-Reader: Warning: FASTA-Reader: First data line in seq is about 45% ambiguous nucleotides (shouldn't be over 40%)" At moment the NCBI Blast+ wrappers (toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/0.1.01) are saying "yup those are errors", but i think they should be downgraded to just Notices? If the tool is part of a workflow it fails - on db's like RDP RNA this is quite a nuisance. Anyone else run into the same thing? d. Also see: http://www.acgt.me/blog/2014/5/15/fun-with-an-error-message-from-ncbi-blast Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

2 5

Problem retrieving password
by Jonathan Baker 07 Jul '15

07 Jul '15

Hello, I used Galaxy for several months when I had to use the reset password function. I tried to reset my password, using the googlemail.com address that I used in the original registration, several times but no "Galaxy Password Reset" email are coming though. Out of the 4 or-so requests, only one reset email came through hours later in the spam folder. The reset password that I got didn't work. The Galaxy website recommends to "contact an administrator" but this but I can't find an email address or web form for that. Can you offer any help with this issue? Kind regards, Jon

1 0

Parallelism using metadata
by Marco Albuquerque 06 Jul '15

06 Jul '15

Hello Galaxy Dev, I have a question regarding parallelism on a BAM file. I have currently implemented 3 split options for the BAM datatype 1) by_rname -> splits the bam into files based on the chromosome 2) by_interval -> splits the bam into files based on a defined bp length, and does so across the entire genome present in the BAM file 3) by_read -> splits the bam into files based on the number of reads encountered (if multiple files, all other files match the interval as the first) Now, as you can imagine, reading and writing large BAM files is a pain, and I personally think this is not the best solution for Galaxy. What I was hoping to implement (but don't know how) is to create a new metadata option in bam (bam.metadata.bam_interval) which would generate the interval without creating a new file (essentially, I would create a symbolic link to the old large file, and then update the metadata.bam_interval, this would contain some string of the form chrom:start-end which could then be used in a variety of tools which accept an interval as an option (for example samtools view)) This would be far more efficient then my first implementation, but the thing I don't know how to do is specify some kind of metadata at the split level. I was hoping maybe you could direct me to an example that does this? I have added the following to my metadata.py file: class IntervalParameter( MetadataParamter ) def __init__( self, spec ): MetadataParamter.__init__( self, spec ): self.rname = self.spec.get( "rname" ) self.start = self.spec.get( "start" ) self.end = self.spec.get( "end" ) def to_string(self): if self.rname = 'all': return '' else: return ''.join([self.rname, ':', self.start, '-', self.end]) And the following to my binary.py file: ### UNDER THE BAM CLASS MetadataElement( name="bam_interval", desc="BAM Interval", param=metadata.IntervalParameter, rname="all", start="", end="", visible=False, optional=True) I somehow want rname="all" to be the default, but upon parallelism, I want to be able to adjust this parameter in the split functions. So, <parallelism method="multi" split_inputs="normal,tumour" split_mode="by_interval" split_size="50000000" merge_outputs="output"/> Would actually change the metadata of each file, and not create sub-bams. PLEASE HELP!!! Marco

3 4

Fix for missing included files from history export and import
by Vimalkumar Velayudhan 03 Jul '15

03 Jul '15

Hi all, For some of the recent tools we have developed on our Galaxy instance ( http://ribogalaxy.ucc.ie) we were using the approach mentioned in the wiki for writing a single HTML output file with links to additional output files (images, csv etc.,). This is much nicer than generating multiple output files in the history panel. However, the exported histories did not contain these included files. I have now fixed this bug on our instance and have also created a pull request on Github (#417 <https://github.com/galaxyproject/galaxy/pull/417> ). Implementation details: 1. Include the extra_files_path attribute on the dataset which is being exported. 2. Check if there are any files in the extra_files_path for each dataset that will be included in the archive. 3. If there are files present, create a new directory and save files to this directory. This directory will then be included in exported archive. The dataset's history ID is appended to both the directory and the dataset's name to prevent conflict with a dataset of the same name. Testing: A simple tool to test html export with included files is available at: https://testtoolshed.g2.bx.psu.edu/repos/vimalkumarvelayudhan/html_export Regards, Vimalkumar Velayudhan

1 0

Problem finding depency for none toolshed tools
by Christian Brenninkmeijer 03 Jul '15

03 Jul '15

Hi, I have been testing tools not yet ready for a tool-shed but had trouble finding the dependencies as there was no database entries to load these. The code in lib/galaxy/tools/deps/resolvers/galaxy_packages.py assumes that the env.sh or bin directory will be in dependency_dir/package/version when in fact it is often in dependency_dir/ package/version/owner/package_name/revision. I have created a pull request with some code to find the dependency in the sub directory https://github.com/galaxyproject/galaxy/pull/413 As I am relatively new to galaxy I have probably broken lots of rules/agreements on how I did the pull request. If so please let me know where I went wrong so I can redo it properly. Christian PS. This is my proposed change to lib/galaxy/tools/deps/resolvers/galaxy_packages.py: def _find_dep_versioned( self, name, version, type='package', **kwds ): #First try the way without owner/name/revision path = join( self.base_path, name, version ) package = self._galaxy_package_dep(path, version) if package != INDETERMINATE_DEPENDENCY: return package #now try with an owner/name/revision for owner in listdir(path): owner_path = path + "/" + owner for package_name in listdir(owner_path): if package_name.startswith("package_"+name): package_path = owner_path + "/" + package_name for revision in listdir(package_path): revision_path = package_path + "/" + revision package = self._galaxy_package_dep(revision_path, version) if package != INDETERMINATE_DEPENDENCY: return package return INDETERMINATE_DEPENDENCY

1 1