Re: [galaxy-dev] Tool to download galaxy datasets locally
by Nicola Soranzo
Dear Evan,
you need to setup an environment variable for the tool to specify the
directory where users are expected to save their files. One way to do
that is:
1) create a directory called EXPORT_DIR_PREFIX in your tool dependencies
directory (which by default is `database/dependencies/` )
2) in this directory create a file called env.sh with a content similar
to the following template:
EXPORT_DIR_PREFIX=/PATH/WHERE/TO/SAVE/FILES; export EXPORT_DIR_PREFIX
3) create a symlink called `default` inside the EXPORT_DIR_PREFIX
directory pointing to the same directory, e.g.:
$ cd TOOL_DEPENDENCIES_DIR/EXPORT_DIR_PREFIX/
$ ln -s . default
Alternatively, you could define the EXPORT_DIR_PREFIX environment
variable in your config/job_conf.xml for the destination where the
export_to_cluster tool is going to run (for an example, see
`_JAVA_OPTIONS` in config/job_conf.xml.sample_advanced ), but this would
define the environment variable also for all the other tools that use
that destination (normally not a problem, but worth to mention).
Cheers,
Nicola
On 05/05/18 15:04, Evan Clark wrote:
> Have you ever seen this error with export to cluster?
>
> export_to_cluster.py: error: Directory prefix cannot be empty
>
> Nicola Soranzo wrote:
>>
>> Hi Evan,
>> you can have a look at
>> https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/
>> <https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/b97775e...>
>>
>> Hope that helps!
>>
>> Cheers,
>> Nicola
>>
>> On 27/03/18 16:03, evan clark wrote:
>>>
>>> Has anyone attempted to write a tool that allows copying of galaxy
>>> datasets locally, i.e. without downloading via http. We have scenario
>>> where there are some additional tools available only via command line
>>> and it would be very useful to be able to directly copy galaxy
>>> datasets from the datasets folder.
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client. To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>> http://galaxyproject.org/search/
>>
2 years, 8 months
ELIXIR Workshop for Galaxy training material and skills improvement
by Nicola Soranzo
Dear all,
we'd like to invite you to the "ELIXIR Workshop for Galaxy training
material and skills improvement", which will be held 21-23 May at the
Earlham Institute in Norwich, UK:
http://www.earlham.ac.uk/elixir-workshop-galaxy-training-material-and-ski...
The Galaxy Training Network (GTN) has been working for several years
with GOBLET and the ELIXIR Training Platform to create material and
deliver training for scientists, developers and system administrators. A
selection of 72 tutorials, developed by more than 60 contributors, is
already available at http://training.galaxyproject.org/
In this workshop, we intend to improve the participants’ understanding
of learning principles, training techniques and best practices for
materials preparation. We will then work together to expand the existing
collection of Galaxy training materials by covering more topics relevant
to the ELIXIR use cases, add missing annotation and metadata and make
the materials easily accessible.
The workshop will start with a 1-day "Train the Trainer" course, similar
to the Carpentries' Instructor Training and the ELIXIR-EXCELERATE Train
the Trainer. Here Galaxy trainers will develop an insight into different
learning styles, understand what makes a good trainer, and learn new
approaches to training from experienced trainers. This course will help
them to better contribute and shape the materials during the hackathon.
This workshop is kindly sponsored by ELIXIR-UK and the ELIXIR Training
Platform.
Registration closes soon, hurry up! https://www.eiseverywhere.com/galaxy
We look forward to seeing many of you in Norwich!
Nicola
--
Nicola Soranzo, Ph.D.
Data Infrastructure & Algorithms group
Earlham Institute
Norwich Research Park, Norwich, NR4 7UZ, UK
http://www.earlham.ac.uk/nicola-soranzo
2 years, 8 months
Best practices for utilising Unique Molecular Indexing (UMI's)
by Brooks, Tony
Hi
We are currently seeing a number of methods that are utilising the power of unique molecular indexing. Unfortunately, there is no consensus on how libraries should be configured, and therefore no consensus for how to deal with them within Galaxy.
Often libraries that have the UMI placed directly downstream of the first (i7) index, such as ones using the IDT xGen adapter set
(https://www.idtdna.com/pages/products/next-generation-sequencing/adapters...). Sometimes UMI's exist in place of the second (i5) index (https://www.neb.com/nebnext-direct/nebnext-direct-for-target-enrichment).
In both cases, the recommended workflows are convoluted and all the necessary tools do not currently exist in the toolshed (so that the datasets need to be taken out of galaxy, processed and reloaded).
It is possible to use bcl2fastq to output the UMI as an additional fastq file, but this would then require me to create a dataset triplicate (not pair) which afaik we can't do (yet).
A quick Google/toolshed search had me find UMI-Tools & Je-Suite which both exist in Galaxy.
Both these tools assume the UMI is "in-line" (i.e. at the beginning of the read 1 or read2 - not its own read), extract/remove the UMI and place it in the read header, where it is then used further down the line to dedup the bam file.
Does anyone know of any tools that would take the UMI from a separate fastq and use it to tag the headers of actual read data. Or alternatively, a tool that will paste the UMI tag onto the 5' end of the read fastq? And whether these steps can be done within Galaxy, or maybe prior to fastq upload?
Anyone have a method/workflow for UMI's?
Thanks in advance
Tony
2 years, 8 months
May Galaxy News: GCCBOSC, Training Material Workshop, New pubs, jobs, ...
by Dave Clements
Hello all,
The May 2018 Galactic News
<https://galaxyproject.org/galaxy-updates/2018-05/> highlights what's going
on in the Galaxy this month, including:
- Lots of GCCBOSC 2018 news
<https://galaxyproject.org/galaxy-updates/2018-04/#gccbosc-2018>:
- *Early registration ends May 11
<https://galaxyproject.org/galaxy-updates/2018-05/#early-registration-ends...>*
- Galaxy talks have been posted to the schedule
<https://galaxyproject.org/galaxy-updates/2018-05/#gcc2018-schedule-posted>
.
- BOSC deadline for late-breaking lightning talks, demos and posters *is
May 17
<https://galaxyproject.org/galaxy-updates/2018-05/#bosc-deadline-for-late-...>*
- Please welcome *Google Cloud
<https://galaxyproject.org/galaxy-updates/2018-05/#gigascience-and-oxford-...>*
and *Advanced HPC
<https://galaxyproject.org/galaxy-updates/2018-05/#advanced-hpc>* as
GCCBOSC sponsors. *Both are offering goodies for conference
participants.*
- Other event news:
- ELIXIR Workshop for Galaxy training material and skills improvement
<https://galaxyproject.org/galaxy-updates/2018-05/#elixir-workshop-for-gal...>
is 21-23 May. Register right now.
- Galaxy Africa Report
<https://galaxyproject.org/galaxy-updates/2018-05/#galaxy-africa-report>
- Other Upcoming events
<https://galaxyproject.org/galaxy-updates/2018-05/#upcoming-events> (on
three continents in 6 countries)
- New:
- 132 publications
<https://galaxyproject.org/galaxy-updates/2018-05/#publications>
- Openings in 10 groups on 2 continents
<https://galaxyproject.org/galaxy-updates/2018-05/#whos-hiring>
- Public Galaxy Server News
<https://galaxyproject.org/galaxy-updates/2018-05/#public-galaxy-server-news>
: IRProfiler
<https://galaxyproject.org/galaxy-updates/2018-05/#irprofiler>
- ToolShed contributions
<https://galaxyproject.org/galaxy-updates/2018-05/#toolshed-contributions>
- Releases
<https://galaxyproject.org/galaxy-updates/2018-05/#releases> of
BioBlend and Starforge
See the newsletter <https://galaxyproject.org/galaxy-updates/2018-05/> for
the full story, and we hope to see you in Portland!
Dave C on behalf of, well, everyone
--
https://gccbosc2018.sched.com/
https://galaxyproject.org/
https://usegalaxy.org/
2 years, 8 months
extract extra info (logs) from history
by Jochen Bick
Hi,
I would like to use the info (log) HISAT is showing, for another tool.
e.g.:
26781724 reads; of these:
26781724 (100.00%) were unpaired; of these:
2445411 (9.13%) aligned 0 times
20399099 (76.17%) aligned exactly 1 time
3937214 (14.70%) aligned >1 times
90.87% overall alignment rate
[bam_sort_core] merging from 21 f
Is it possible to access these logs?
Cheers Jochen
2 years, 8 months
