April 2018 - galaxy-dev - lists.galaxyproject.org

Monitoring Dashboard for Galaxy
by evan clark 03 Sep '18

03 Sep '18

I remember there previously being an additional tool that ran on a different port from galaxy that allowed for monitoring of performance and jobs running. Is this tool still packaged with galaxy and if so how can it be activated.

3 3

Job Script Integrity
by John Letaw 04 Jun '18

04 Jun '18

Hi all, I’m installing via GalaxyKickStart… I’m getting the following error: galaxy.jobs.runners ERROR 2017-11-06 19:14:05,263 (19) Failure preparing job Traceback (most recent call last): File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", line 175, in prepare_job modify_command_for_container=modify_command_for_container File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", line 209, in build_command_line container=container File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", line 84, in build_command externalized_commands = __externalize_commands(job_wrapper, external_command_shell, commands_builder, remote_command_params) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", line 143, in __externalize_commands write_script(local_container_script, script_contents, config) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/util/job_script/__init__.py", line 112, in write_script _handle_script_integrity(path, config) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/util/job_script/__init__.py", line 147, in _handle_script_integrity raise Exception("Failed to write job script, could not verify job script integrity.") Exception: Failed to write job script, could not verify job script integrity. galaxy.model.metadata DEBUG 2017-11-06 19:14:05,541 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-11-06 19:14:05,576 Failed to cleanup MetadataTempFile temp files from /home/exacloud/lustre1/galaxydev/galaxyuser/database/jobs/000/19/metadata_out_HistoryDatasetAssociation_16_I8bhLX: No JSON object could be decoded I would like to further understand what it means to not verify integrity of a job script. Does this just mean there is a permissions error? Ownership doesn’t match up? Thanks, John

2 1

Text file busy and conda
by John Letaw 16 May '18

16 May '18

Hi all, I’m receiving the following error trying to access tools autoinstalled via conda: Fatal error: Exit code 126 () ~/galaxy/database/jobs_directory/000/38/tool_script.sh: line 41: ~/galaxy/database/dependencies/_conda/envs/mulled-v1-bb83f93efd111e042823e53ddfa74c32d81ba74cceca9445dfddfc9e940ff738/bin/samtools: Text file busy So, something is happening too fast I guess, meaning a process is attempting access while this is being created. Any ideas on how I can diagnose this? Not seeing any other errors floating around the logs. Maybe we are dealing with a lustre file locking issue? Thanks, John

2 2

Best practices for utilising Unique Molecular Indexing (UMI's)
by Brooks, Tony 04 May '18

04 May '18

Hi We are currently seeing a number of methods that are utilising the power of unique molecular indexing. Unfortunately, there is no consensus on how libraries should be configured, and therefore no consensus for how to deal with them within Galaxy. Often libraries that have the UMI placed directly downstream of the first (i7) index, such as ones using the IDT xGen adapter set (https://www.idtdna.com/pages/products/next-generation-sequencing/adapters/x…) Sometimes UMI's exist in place of the second (i5) index (https://www.neb.com/nebnext-direct/nebnext-direct-for-target-enrichment) In both cases, the recommended workflows are convoluted and all the necessary tools do not currently exist in the toolshed (so that the datasets need to be taken out of galaxy, processed and reloaded). It is possible to use bcl2fastq to output the UMI as an additional fastq file, but this would then require me to create a dataset triplicate (not pair) which afaik we can't do (yet). A quick Google/toolshed search had me find UMI-Tools & Je-Suite which both exist in Galaxy. Both these tools assume the UMI is "in-line" (i.e. at the beginning of the read 1 or read2 - not its own read), extract/remove the UMI and place it in the read header, where it is then used further down the line to dedup the bam file. Does anyone know of any tools that would take the UMI from a separate fastq and use it to tag the headers of actual read data. Or alternatively, a tool that will paste the UMI tag onto the 5' end of the read fastq? And whether these steps can be done within Galaxy, or maybe prior to fastq upload? Anyone have a method/workflow for UMI's? Thanks in advance Tony

2 6

extract extra info (logs) from history
by Jochen Bick 02 May '18

02 May '18

Hi, I would like to use the info (log) HISAT is showing, for another tool. e.g.: 26781724 reads; of these: 26781724 (100.00%) were unpaired; of these: 2445411 (9.13%) aligned 0 times 20399099 (76.17%) aligned exactly 1 time 3937214 (14.70%) aligned >1 times 90.87% overall alignment rate [bam_sort_core] merging from 21 f Is it possible to access these logs? Cheers Jochen

4 7

Gateways 2018 Call for Participation: Extended to May 16
by Katherine Lawrence 30 Apr '18

30 Apr '18

The deadline for the Gateways 2018 Call for Participation has been extended to Wednesday, May 16, 2018. Gateways 2018 (September 25-27, at the University of Texas at Austin) is accepting submissions of papers, demos, tutorials, and panels (2-4 pages) on the topic of science or engineering gateways. Topics may include their design, use, impact, development processes, sustainability, best practices, or any other aspect that you think fellow gateway creators or users will find interesting to learn. We also welcome educational topics directed toward the next generation of gateway creators. Submissions of papers, demos, tutorials, and panels are due May 16, 2018. There will be no further extension of this deadline. The poster session deadline (open to all) is August 1, 2018. Full submission details can be found here: https://sciencegateways.org/gateways2018/call

1 0

tool_conf.xml
by Nancy Ide 30 Apr '18

30 Apr '18

Hello I’m trying to include some of the tools available from usegalaxy.org in the tool menu of our instance of Galaxy. When I look at the list of tools in the Galaxy tool_conf.xml.main file, it does not seem to line up with what you see in the menu at usegalaxy.org. I’ve searched through various versions etc. on GitHub, but no luck—they all seem to be identical and do not seem to correspond with the menu at usegalaxy.org (I am looking specifically at the Text Manipulation part of the menu). Any clues on where I should look? Thanks, Nancy Ide ----------------------------------------------------- Nancy Ide Professor of Computer Science Department of Computer Science Vassar College Poughkeepsie, New York 12604-0520 USA tel: (+1 845) 437 5988 fax: (+1 845) 437 7498 email: ide(a)cs.vassar.edu http://www.cs.vassar.edu/~ide -----------------------------------------------------

2 2

Galaxy Cleanup
by Previti 27 Apr '18

27 Apr '18

Hi everybody, I have a hopefully basic questions regarding temporary output files. A user was generating >4TB output files via the program bedtools intersectbed. The jobs were cancelled, but I still see the (temporary) output files that were generated. Does Galaxy clean these up? Does it do this regularly? Would a quota prevent something like this from happening again or would the jobs have to finish before being "counted"? Thanks and best regards, Christopher -- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

3 3

fonts on headless compute nodes
by Matthias Bernt 25 Apr '18

25 Apr '18

Dear list, I found that some tools (eg FastQC and MetaProSIP) have problems with missing fonts (the former is java and the latter caused by R) on our compute nodes which do not have xorg and no fonts installed. On the cluster head nodes several font related packages are installed: dejavu-fonts-common-2.33-1.el6.noarch xorg-x11-fonts-Type1-7.2-11.el6.noarch liberation-fonts-common-1.05.1.20090721-5.el6.noarch liberation-sans-fonts-1.05.1.20090721-5.el6.noarch dejavu-sans-fonts-2.33-1.el6.noarch urw-fonts-2.4-11.el6.noarch ghostscript-fonts-5.50-23.2.el6.noarch xorg-x11-fonts-misc-7.2-11.el6.noarch It seems that conda no fonts are available. Is there somebody with similar problems. Any hint which of the packages is/are actually needed. Maybe our admins could install them... I also found for MetaProSIP that several xorg-... packages are needed. Should such dependencies be handled by the conda packages or by the Galaxy tool. Best, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

3 4

Samtools running on submit-node and as a slurm job
by Previti 24 Apr '18

24 Apr '18

Dear all, We found something strange with our Galaxy instance... Our Galaxy server is configured as a submit node using slurm to submit to our cluster. This works fine for the most part, but today we saw a samtools index job running "locally" on the submit node instead of a cluster node. Is there any way of stopping this behavior? The submit nodes are not configured for data-intensive computation and everything runs significantly slower. I was running bamtools (splitting a bam-file into mapped into unmapped) at the time and the output bam-files were being indexed. Thanks and best regards, Christopher -- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

2 1

ng-chm path error
by Aswathymary Paul 24 Apr '18

24 Apr '18

Hello, I tried to install ng-chm. I have done all the steps described in that manual. But when I generate the heat map, I am not getting a menu for visualization in galaxy and still the path is shown as /shed_tool/.... in my home directory not /galaxy/config. I have moved the zipped heat map into the visualization folder inside plugin. And unzipped there itself. But the path is not taking. Thank you Regards ------------------------------------------------------- Aswathy Mary Paul Ph.D. Student Computational biology and Bioinformatics Cancer Research Program [CRP-9] Rajiv Gandhi Center for Biotechnology Thycaud(P.O), Poojappura Thiruvananthapuram-695014, Kerala, lab extn : 247

1 0

Save data from visualization plugin
by Keith Suderman 23 Apr '18

23 Apr '18

My Google-Fu is failing me... We have a visualization plugin that can also serve as an editor to modify the data it is visualizing. What is the best way to save the data from the visualizer back to Galaxy? I seem to recall reading (somewhere, sometime) that the REST API could not do this and the "best" solution is to invoke the upload tool to upload data into the current history. I have used the upload tool from the Python and Blend4J APIs, but I can not find a JavaScript example for invoking the upload tool. So two questions: 1. Is the upload tool (still) the recommended way to put data into the current history via API calls? 2. How do I invoke the upload tool from JavaScript? Thanks, Keith ---------------------- Keith Suderman Research Associate Department of Computer Science Vassar College, Poughkeepsie NY suderman(a)cs.vassar.edu

2 2

Remove entry from Tool Data Table using via Data Manager
by Dan Fornika 23 Apr '18

23 Apr '18

Hello, I'm working on galaxy integration for an MLST tool called MentaLiST: https://github.com/WGS-TB/MentaLiST/tree/master/galaxy I've created data managers that create kmer databases and store their locations in a Tool Data Table. I'd like to allow users to remove entries from the Tool Data Table. Has anyone seen a Data Manager that provides that functionality? Should I create separate data managers for add and remove functionality, or can it be integrated into one? Thanks, Dan

2 3

FTP service for galaxy with external authentication via SAML using shibboleth
by Martin Demko 17 Apr '18

17 Apr '18

Hi, I'm currently solving a problem with authentication to FTP service for users of our galaxy server. We've successfully established an authentication via shibboleth behind Nginx (not very easy, but doable :) but that also means that ProFTPD is not working anymore, as it doesn't support SAML authentication. So my question is obvious, I'm looking for an easy and free FTP server with SAML support, can anybody help me with an advice? I've already found CompleteFTP and CrushFTP but both are paid and one is Windows-only allegedly. So how do you - people using external authentication via SAML - do this? Thanks in advance for any kind of useful advice. Best wishes, Martin Demko

3 3

planemo/toolshed error, assistance request
by Dennis, H. E. Cicada Brokaw 13 Apr '18

13 Apr '18

Hello, I am having a problem using planemo to upload a suite of tools into the testtoolshed. The tools at this point are all in the toolshed from using the shed_create and shed_update commands in the suite directory. However, along the way there were some errors in uploading some of the tools that I had to fix. Those are all fixed and all the tools show as up to date, except for the suite, which is returning the following error message, about which I have no idea what to do (and I have tried a lot of different things): The repository_dependencies.xml file contains an invalid <repository> tag. Invalid latest installable changeset_revision 000000000000 retrieved for repository ctat_metagenomics owned by trinity_ctat. Does anyone have an idea of what to do to fix this? Thanks, Cicada Dennis

2 1

Doubts about FTP upload.
by Álvaro Rios 11 Apr '18

11 Apr '18

Dear I am trying to upload files via FTP. However, the following message appears. "Authentication required ftp://usegalaxy.org Your connection to this site is not private." I do not have the experience to use galaxy, What I need to do? Thank you. Álvaro Rios.

2 1

April Galactic News: UseGalaxy.org gets some siblings, GCCBOSC and other event news; new pubs, servers, jobs, ...
by Dave Clements 11 Apr '18

11 Apr '18

Hello all, The April 2018 Galactic News <https://galaxyproject.org/galaxy-updates/2018-04/> highlights what's going on in the Galaxy (and it's actually quite cool). Take a look: - *UseGalaxy.eu and UseGalaxy.org.au Launched <https://galaxyproject.org/galaxy-updates/2018-04/#usegalaxyeu-and-usegalaxy…>* - Lots of GCCBOSC 2018 news <https://galaxyproject.org/galaxy-updates/2018-04/#gccbosc-2018>: - We are pleased to announce that Lucia Peixoto <https://galaxyproject.org/galaxy-updates/2018-04/#keynote-speakers> of Washington State University will be our third keynote speaker. - Please welcome *Google Cloud <https://galaxyproject.org/galaxy-updates/2018-04/#google-cloud>* as a GCCBOSC Platinum sponsor. - *GCC Lightning talk, poster and demo submissions are due by April 16 <https://galaxyproject.org/galaxy-updates/2018-04/#submit-gcc-lightning-talk…>* - Childcare for GCCBOSC <https://galaxyproject.org/galaxy-updates/2018-04/#childcare-at-gccbosc>! Space is limited, so register *now*. - Upcoming Events: - 2018 Big Genomics Data Skills Training course <https://galaxyproject.org/galaxy-updates/2018-04/#2018-big-genomics-data-sk…> - how to integrate genomic data analysis into undergraduate coursework - Gateways 2018 Call for Participation <https://galaxyproject.org/galaxy-updates/2018-04/#gateways-2018-call-for-pa…> - Upcoming events <https://galaxyproject.org/galaxy-updates/2018-04/#upcoming-events> (on four continents) - New: - 119 publications <https://galaxyproject.org/galaxy-updates/2018-04/#publications> (including 9 highlighted ones) - Openings in 9 groups on 2 continents <https://galaxyproject.org/galaxy-updates/2018-04/#whos-hiring> - Public Galaxy Server News <https://galaxyproject.org/galaxy-updates/2018-04/#public-galaxy-server-news> (a new server and an updated one too) - ToolShed contributions <https://galaxyproject.org/galaxy-updates/2018-04/#toolshed-contributions> - Releases <https://galaxyproject.org/galaxy-updates/2018-04/#releases> of Galaxy, Galaxy Docker, blend4j, and galaxy-lib - And other news <https://galaxyproject.org/galaxy-updates/2018-04/#other-news> too See the newsletter <https://galaxyproject.org/galaxy-updates/2018-04/> for the full story, and we hope to see you in Portland! Dave C, on behalf of the Galaxy -- https://gccbosc2018.sched.com/ https://galaxyproject.org/ https://usegalaxy.org/

1 0

question regarding GALXY software
by דויד קורנספן [David Kornspan] 10 Apr '18

10 Apr '18

Hi, I would appreciate your advice regarding the recommended portable computer specifications which will enable running the GALXY software for analysis of DNA and RNA seq of bacterial genomes. Thanks, David Kornspan

2 1

Ballgown does not work
by Jochen Bick 10 Apr '18

10 Apr '18

Hi, I'm trying to install Ballgown but it throws some error message after running it: cannot find 'dataset' while searching for 'param_name.dataset.dataset' the script does not have test-data file and it also does not install the requirements. Cheers Jochen

3 6