May 2018 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Tool to download galaxy datasets locally
by Nicola Soranzo 09 May '18

09 May '18

Dear Evan, you need to setup an environment variable for the tool to specify the directory where users are expected to save their files. One way to do that is: 1) create a directory called EXPORT_DIR_PREFIX in your tool dependencies directory (which by default is `database/dependencies/` ) 2) in this directory create a file called env.sh with a content similar to the following template: EXPORT_DIR_PREFIX=/PATH/WHERE/TO/SAVE/FILES; export EXPORT_DIR_PREFIX 3) create a symlink called `default` inside the EXPORT_DIR_PREFIX directory pointing to the same directory, e.g.: $ cd TOOL_DEPENDENCIES_DIR/EXPORT_DIR_PREFIX/ $ ln -s . default Alternatively, you could define the EXPORT_DIR_PREFIX environment variable in your config/job_conf.xml for the destination where the export_to_cluster tool is going to run (for an example, see `_JAVA_OPTIONS` in config/job_conf.xml.sample_advanced ), but this would define the environment variable also for all the other tools that use that destination (normally not a problem, but worth to mention). Cheers, Nicola On 05/05/18 15:04, Evan Clark wrote: > Have you ever seen this error with export to cluster? > > export_to_cluster.py: error: Directory prefix cannot be empty > > Nicola Soranzo wrote: >> >> Hi Evan, >> you can have a look at >> https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/ >> <https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/b97775e9f…> >> >> Hope that helps! >> >> Cheers, >> Nicola >> >> On 27/03/18 16:03, evan clark wrote: >>> >>> Has anyone attempted to write a tool that allows copying of galaxy >>> datasets locally, i.e. without downloading via http. We have scenario >>> where there are some additional tools available only via command line >>> and it would be very useful to be able to directly copy galaxy >>> datasets from the datasets folder. >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/ >>

2 1

ELIXIR Workshop for Galaxy training material and skills improvement
by Nicola Soranzo 08 May '18

08 May '18

Dear all, we'd like to invite you to the "ELIXIR Workshop for Galaxy training material and skills improvement", which will be held 21-23 May at the Earlham Institute in Norwich, UK: http://www.earlham.ac.uk/elixir-workshop-galaxy-training-material-and-skill… The Galaxy Training Network (GTN) has been working for several years with GOBLET and the ELIXIR Training Platform to create material and deliver training for scientists, developers and system administrators. A selection of 72 tutorials, developed by more than 60 contributors, is already available at http://training.galaxyproject.org/ In this workshop, we intend to improve the participants’ understanding of learning principles, training techniques and best practices for materials preparation. We will then work together to expand the existing collection of Galaxy training materials by covering more topics relevant to the ELIXIR use cases, add missing annotation and metadata and make the materials easily accessible. The workshop will start with a 1-day "Train the Trainer" course, similar to the Carpentries' Instructor Training and the ELIXIR-EXCELERATE Train the Trainer. Here Galaxy trainers will develop an insight into different learning styles, understand what makes a good trainer, and learn new approaches to training from experienced trainers. This course will help them to better contribute and shape the materials during the hackathon. This workshop is kindly sponsored by ELIXIR-UK and the ELIXIR Training Platform. Registration closes soon, hurry up! https://www.eiseverywhere.com/galaxy We look forward to seeing many of you in Norwich! Nicola -- Nicola Soranzo, Ph.D. Data Infrastructure & Algorithms group Earlham Institute Norwich Research Park, Norwich, NR4 7UZ, UK http://www.earlham.ac.uk/nicola-soranzo

1 0

GCCBOSC 2018 Early Registration Ends this Friday (May 11)
by Dave Clements 07 May '18

07 May '18

Hi All, Just a reminder that early registration for the Galaxy Community's biggest annual gathering ends this Friday. You can also register for housing and childcare too. https://galaxyproject.org/events/gccbosc2018/register/ https://galaxyproject.org/events/gccbosc2018/venue/#housing https://galaxyproject.org/events/gccbosc2018/childcare/ Hope to see you there, Dave C on behalf of the GCCBOSC 2018 Organizing Committee -- https://gccbosc2018.sched.com/ https://galaxyproject.org/ https://usegalaxy.org/

1 0

Best practices for utilising Unique Molecular Indexing (UMI's)
by Brooks, Tony 04 May '18

04 May '18

Hi We are currently seeing a number of methods that are utilising the power of unique molecular indexing. Unfortunately, there is no consensus on how libraries should be configured, and therefore no consensus for how to deal with them within Galaxy. Often libraries that have the UMI placed directly downstream of the first (i7) index, such as ones using the IDT xGen adapter set (https://www.idtdna.com/pages/products/next-generation-sequencing/adapters/x…) Sometimes UMI's exist in place of the second (i5) index (https://www.neb.com/nebnext-direct/nebnext-direct-for-target-enrichment) In both cases, the recommended workflows are convoluted and all the necessary tools do not currently exist in the toolshed (so that the datasets need to be taken out of galaxy, processed and reloaded). It is possible to use bcl2fastq to output the UMI as an additional fastq file, but this would then require me to create a dataset triplicate (not pair) which afaik we can't do (yet). A quick Google/toolshed search had me find UMI-Tools & Je-Suite which both exist in Galaxy. Both these tools assume the UMI is "in-line" (i.e. at the beginning of the read 1 or read2 - not its own read), extract/remove the UMI and place it in the read header, where it is then used further down the line to dedup the bam file. Does anyone know of any tools that would take the UMI from a separate fastq and use it to tag the headers of actual read data. Or alternatively, a tool that will paste the UMI tag onto the 5' end of the read fastq? And whether these steps can be done within Galaxy, or maybe prior to fastq upload? Anyone have a method/workflow for UMI's? Thanks in advance Tony

2 6

May Galaxy News: GCCBOSC, Training Material Workshop, New pubs, jobs, ...
by Dave Clements 02 May '18

02 May '18

Hello all, The May 2018 Galactic News <https://galaxyproject.org/galaxy-updates/2018-05/> highlights what's going on in the Galaxy this month, including: - Lots of GCCBOSC 2018 news <https://galaxyproject.org/galaxy-updates/2018-04/#gccbosc-2018>: - *Early registration ends May 11 <https://galaxyproject.org/galaxy-updates/2018-05/#early-registration-ends-m…>* - Galaxy talks have been posted to the schedule <https://galaxyproject.org/galaxy-updates/2018-05/#gcc2018-schedule-posted> . - BOSC deadline for late-breaking lightning talks, demos and posters *is May 17 <https://galaxyproject.org/galaxy-updates/2018-05/#bosc-deadline-for-late-br…>* - Please welcome *Google Cloud <https://galaxyproject.org/galaxy-updates/2018-05/#gigascience-and-oxford-un…>* and *Advanced HPC <https://galaxyproject.org/galaxy-updates/2018-05/#advanced-hpc>* as GCCBOSC sponsors. *Both are offering goodies for conference participants.* - Other event news: - ELIXIR Workshop for Galaxy training material and skills improvement <https://galaxyproject.org/galaxy-updates/2018-05/#elixir-workshop-for-galax…> is 21-23 May. Register right now. - Galaxy Africa Report <https://galaxyproject.org/galaxy-updates/2018-05/#galaxy-africa-report> - Other Upcoming events <https://galaxyproject.org/galaxy-updates/2018-05/#upcoming-events> (on three continents in 6 countries) - New: - 132 publications <https://galaxyproject.org/galaxy-updates/2018-05/#publications> - Openings in 10 groups on 2 continents <https://galaxyproject.org/galaxy-updates/2018-05/#whos-hiring> - Public Galaxy Server News <https://galaxyproject.org/galaxy-updates/2018-05/#public-galaxy-server-news> : IRProfiler <https://galaxyproject.org/galaxy-updates/2018-05/#irprofiler> - ToolShed contributions <https://galaxyproject.org/galaxy-updates/2018-05/#toolshed-contributions> - Releases <https://galaxyproject.org/galaxy-updates/2018-05/#releases> of BioBlend and Starforge See the newsletter <https://galaxyproject.org/galaxy-updates/2018-05/> for the full story, and we hope to see you in Portland! Dave C on behalf of, well, everyone -- https://gccbosc2018.sched.com/ https://galaxyproject.org/ https://usegalaxy.org/

1 0

extract extra info (logs) from history
by Jochen Bick 02 May '18

02 May '18

Hi, I would like to use the info (log) HISAT is showing, for another tool. e.g.: 26781724 reads; of these: 26781724 (100.00%) were unpaired; of these: 2445411 (9.13%) aligned 0 times 20399099 (76.17%) aligned exactly 1 time 3937214 (14.70%) aligned >1 times 90.87% overall alignment rate [bam_sort_core] merging from 21 f Is it possible to access these logs? Cheers Jochen

4 7