- galaxy-dev - lists.galaxyproject.org

Upload bigger 2GB
by Jochen Bick 13 Mar '19

13 Mar '19

Hi, since some months we have the problem that users are allowed to upload files bigger 2GB. Is there some kind of options in the galaxy config that can stop this? Our data storage is not able to handle such big data duplication, we used to use Shared Data->Data libraries to reduce data duplication. Thanks in advance. Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

2 2

Gateways 2019 Call for Participation (Abstracts due April 29; Full submissions due May 6)
by Katherine Lawrence 08 Mar '19

08 Mar '19

* Call for Participation: Gateways 2019(September 23–25, San Diego, California) is now accepting submissions of papers, demos, tutorials, and panels (2–4 pages) on the topic of gateways for science, engineering, or other disciplines. Gateways are user-friendly interfaces to scientific computing, data, and other domain-specific resources to support research and education. Topics may include their design, use, impact, development processes, sustainability, best practices, or any other aspect that you think fellow gateway creators or users will find interesting to learn. We also welcome educational topics directed toward the next generation of gateway creators. In addition, this year Gateways 2019 is co-located with the eScience <https://escience2019.sdsc.edu/>conference. The two conferences will offer shared sessions and connected registration.Abstracts are due April 29, and full submissions are due May 6, 2019.A poster session deadline (open to all attendees) will be August 15. Read more details in the Call for Participation:http://sciencegateways.org/gateways2019/call*

1 0

Bismark - Bug in Bismark Pretty Report ?
by Previti 08 Mar '19

08 Mar '19

Dear Galaxy Community, I just recently started testing the Bismark methylation tools, everything works fine until I try to make a "pretty report". Some information is not included in the report, instead you see placeholders I attached the output I get and shown below are the job parameters. Does anybody know where to fix this? Best regards, Christopher Bismark Pretty Report Dataset Information Number: 198 Name: Bismark Pretty Report on data 190, data 189, and others: Download pretty html report Created: Thu 07 Mar 2019 03:24:40 PM (UTC) Filesize: 2.7 MB Dbkey: ? Format: html Job Information Galaxy Tool ID: toolshed.g2.bx.psu.edu/repos/bgruening/bismark/bismark_pretty_report/0.20.0 Galaxy Tool Version: 0.20.0 Tool Version: Tool Standard Output: stdout <http://dkfzgalaxy/datasets/971c2f882eb14ac3/stdout> Tool Standard Error: stderr <http://dkfzgalaxy/datasets/971c2f882eb14ac3/stderr> Tool Exit Code: 0 History Content API ID: 971c2f882eb14ac3 (41433) Job API ID: 58816697357e736f (22051) History API ID: 5fccce9c1b3793da (807) UUID: 45bedd12-6ce5-4d63-a1e2-e05883b0c356 Full Path: /opt/galaxy/galaxy/database/files/038/dataset_38737.dat Tool Parameters Input Parameter Value Note for rerun Submit a Bismark mapping report 191: Bismark Meth. Extractor on data 60 and data 133: Genome-wide methylation report. <http://dkfzgalaxy/datasets/c53067deacfc4f28/show_params> additional_reports Submit the corresponding Bismark deduplication report 197: Bismark Deduplicate on data 133: deduplication report <http://dkfzgalaxy/datasets/622377f5e70addd2/show_params> Submit the corresponding Bismark splitting report 189: Bismark Meth. Extractor on data 60 and data 133: Splitting Report <http://dkfzgalaxy/datasets/aa40ffd063a4645f/show_params> Submit the corresponding Bismark M-bias report 190: Bismark Meth. Extractor on data 60 and data 133: Mbias Report <http://dkfzgalaxy/datasets/47db6b20871008b9/show_params> Submit the corresponding Bismark nucleotide report Create a separate logfile, otherwise logs are added to the dataset info. False Inheritance Chain Bismark Pretty Report on data 190, data 189, and others: Download pretty html report Command Line python '/opt/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/bgruening/bismark/7bffcb6fc81d/bismark/bismark2report_wrapper.py' --alignment_report '/opt/galaxy/galaxy/database/files/038/dataset_38715.dat' --dedup_report '/opt/galaxy/galaxy/database/files/038/dataset_38736.dat' --splitting_report '/opt/galaxy/galaxy/database/files/038/dataset_38713.dat' --mbias_report '/opt/galaxy/galaxy/database/files/038/dataset_38714.dat' --output_html_report '/opt/galaxy/galaxy/database/files/038/dataset_38737.dat' Job Metrics core Cores Allocated 8 Job End Time 2019-03-07 16:49:58 Job Runtime (Wall Clock) 25 minutes Job Start Time 2019-03-07 16:24:43 Memory Allocated (MB) 510000 Job Dependencies Dependency Dependency Type Version bismark conda 0.20.0 -- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

1 0

March 2019 Galaxy News: GCC2019 childcare sponsors, and more!
by Mo Heydarian 04 Mar '19

04 Mar '19

Hello All, The *March 2019 Galactic News <https://galaxyproject.org/news/2019-03-galaxy-update/>* is here! - Event News - eLife and F1000Research sponsor childcare at GCC2019 <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> - The GalaxyAdmins <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> meetup is back March 21! - 184 new publications <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…>, great resources lead to great insight. - Some most excellent Galaxy Platform News <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…>, including metagenomics, predictive model building, epigenome-wide association studies, immunogenomics, DNA methylation analysis, and more! - An interview on The Galactic Blog <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> with Nate Coraor <https://github.com/galaxyproject/galaxy-hub/blob/master/src/people/nate/ind…> about CVMFS & Galaxy. - At least 15 Open positions <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> in five countries on two continents. - A burst of Training Updates <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> . - ToolShed contributions <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> . - Galaxy 19.01 released <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> ! - And a bunch of other news <https://github.com/galaxyproject/galaxy-hub/blob/master/src/news/2019-03-ga…> too. If you have anything to add to next month's newsletter, then please send it to outreach(a)galaxyproject.org. Cheers, Mo Heydarian, on behalf of everyone

1 0

Galaxy Release 19.01
by Dave Clements 28 Feb '19

28 Feb '19

We are pleased to announce the *release of Galaxy 19.01 <https://docs.galaxyproject.org/en/release_19.01/releases/19.01_announce.html>*. A few release highlights are: <https://galaxyproject.org/news/2019-1-galaxy-release/#revised-ui-style> Revised UI Style The Galaxy stylesheet has been substantially reworked resulting in the largest visual refresh to Galaxy in years. <https://galaxyproject.org/news/2019-1-galaxy-release/#more-colorful-tags> More Colorful Tags Use consistently hashed randomized colors for history dataset name tags to increase visual distinction between tags, making it easier to track data flow in complex histories. Thanks to @erasche <https://github.com/erasche>. <https://galaxyproject.org/news/2019-1-galaxy-release/#extensive-workflow-en…> Extensive Workflow Enhancements (continued) Galaxy 18.09 featured a bunch of important and diverse fixes and enhancements to workflows on both the frontend and backend. That progress continues in release 19.01. The workflow editor now features explicit step parameter nodes for non-file data (integers, strings, booleans, etc) and the ability to connect these to all tool inputs - not just files. The YAML-based Galaxy Format 2 workflow language is now included in Galaxy as a beta option and allows import and export of human readable workflows as well as extensions to allow Galaxy to serve as a file editor for such workflows. The workflow editor features more helpful labels on inputs and outputs and important fixes including logic for connecting collections and dealing with missing tools. The workflow run form now deals with default values more correctly. <https://galaxyproject.org/news/2019-1-galaxy-release/#enhanced-support-for-…> Enhanced Support for Singularity Singularity <https://singularity.lbl.gov/> container support in Galaxy has been brought to parity with Docker container support. Specifically, Galaxy can now be configured to fetch Biocontainers <https://biocontainers.pro/> automatically for enabled job destinations. Moreover, explicitly annotated Docker containers (either in tools or job destinations) can be used with Singularity. Thanks to @mvdbeek <https://github.com/mvdbeek> Please see the full release notes <https://docs.galaxyproject.org/en/release_19.01/releases/19.01_announce.html> for more information, including how to upgrade today! *Thanks for using Galaxy!* -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

A question about Galaxy
by Abdullah El Kurdi 21 Feb '19

21 Feb '19

Dear all, I would like to ask if is there any way that I could copy my galaxy docker instance from a server to another including the "export" directory, without having to re-install all the tools again. Thanks in advance.

2 1

[GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019 Participation
by Robin Haw 11 Feb '19

11 Feb '19

Dear All, The Open Genome Informatics team serves as an “umbrella" organization to support the efforts of many open access open-source bioinformatics projects for Google Summer of Code (GSoC)<https://summerofcode.withgoogle.com/>. Among this list of projects are GMOD and its software projects -- GBrowse, JBrowse; Galaxy; Reactome; WormBase; DockStore; Bioconda; and others. Call for 2019 Project Ideas and Mentors: We are seeking project ideas to post and attract talented students to this year’s Summer of Code competition. If you have a project idea for which you would like to mentor a student, please contact Robin Haw, Marc Gillespie, Dave Clements, Dannon Baker, and Scott Cain (emails above). You can also submit your ideas here<http://gmod.org/wiki/GSOC_Project_Ideas_2019> or this online form<https://docs.google.com/forms/d/1uQlpsbPCnhNwO5OFKAB-IiY9DLWlmaHZATnaeP6KHhw>. For more information please refer to the Open Genome Informatics page<http://gmod.org/wiki/GSoC>. The mentoring organization application deadline with GSoC is February 6th, 2019 at 3 pm EST. So, if you are interested in taking part with the team please let us know as soon as possible. Please forward this to others who might be interested in taking part. If you have any questions please let us know. Thanks, Robin, Marc, Dave, Dannon, and Scott.

1 1

Uncaught exception in exposed API method:
by Jochen Bick 07 Feb '19

07 Feb '19

Hi, I'm having this error since some minutes. The error message is coming in the moment I click to any tool (e.g.: featurecounts) that provides data collections. I think it has to do with a collection in my history. Here some detail of my log file: galaxy.web.framework.decorators ERROR 2019-02-06 15:50:09,026 Uncaught exception in exposed API method: Traceback (most recent call last): File "/opt/galaxy/production/galaxy-dist/lib/galaxy/web/framework/decorators.py", line 283, in decorator rval = func(self, trans, *args, **kwargs) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 113, in build return tool.to_json(trans, kwd.get('inputs', kwd)) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/__init__.py", line 1903, in to_json self.populate_model(request_context, self.inputs, state_inputs, tool_model['inputs']) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/__init__.py", line 1951, in populate_model tool_dict = input.to_dict(request_context) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 701, in to_dict cond_dict["cases"] = list(map(nested_to_dict, self.cases)) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 699, in nested_to_dict return input.to_dict(trans) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 719, in to_dict when_dict["inputs"] = list(map(input_to_dict, self.inputs.values())) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 717, in input_to_dict return input.to_dict(trans) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1966, in to_dict match = dataset_collection_matcher.hdca_match(hdca) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/dataset_matcher.py", line 226, in hdca_match converted_ext, _ = datatypes_registry.find_conversion_destination_for_dataset_by_extensions(extension, formats) File "/opt/galaxy/production/galaxy-dist/lib/galaxy/datatypes/registry.py", line 892, in find_conversion_destination_for_dataset_by_extensions self.log.warning("Datatype class not found for extension '%s', which is used as target for conversion from datatype '%s'" % (convert_ext, dataset.ext)) AttributeError: 'NoneType' object has no attribute 'ext' 82.130.95.243 - - [06/Feb/2019:15:50:08 +0200] "GET /api/tools/toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/1.6.3%2Bgalaxy2/build?version=1.6.3%2Bgalaxy2&__identifer=ku6ydfn1c8&tool_version=1.6.3%252Bgalaxy2 HTTP/1.1" 500 - "http://green-l-bio.ethz.ch:8090/?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2Fi…" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.81 Safari/537.36" Any ideas? Errors only occur if I'm in that history. I can also provide more information. This is running on a Galaxy 18.09. The data collections contains 8 fastq files. Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

1 2

Question using sftp to upload file to galaxy
by Rui Wang 06 Feb '19

06 Feb '19

Hey Folks, I'm looking at the instructions of using ftp with proftpd. There is a section talking about extending it to use sftp. However, the sample config isn't comprehensive. I'm wondering if anyone has a working config for reference? What's the setting of user and group? It says it should match the one in the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but start galaxy as bioinfoadmin(normal user with sudo). Just fyi, my proftpd config module and config file are pasted below. I'm working it out on a trial and error fashion, please feel free to point out if anything is wrong! Cheers, Rui modules: $ sbin/proftpd -l Compiled-in modules: mod_core.c mod_xfer.c mod_rlimit.c mod_auth_unix.c mod_auth.c mod_ls.c mod_log.c mod_site.c mod_delay.c mod_facts.c mod_sql.c mod_sql_postgres.c mod_sql_passwd.c mod_sftp.c mod_cap.c etc/proftpd.conf ServerType standalone # You must put this in a virtual host if you want it to listen on its own port. VHost != Apache Vhost. <VirtualHost 10.3.17.42> Port 2222 SFTPEngine on AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will get weird disconnects SFTPHostKey /etc/ssh/ssh_host_rsa_key RequireValidShell no MaxLoginAttempts 6 ServerName "Galaxy SFTP" DefaultServer on Umask 077 User bioinfoadmin Group bioinfoadmin UseFtpUsers off DefaultRoot ~ AllowOverwrite on AllowStoreRestart on SQLEngine on SQLGroupInfo sftp_groups name id members # Do not authenticate against real (system) users <IfModule mod_auth_pam.c> AuthPAM off </IfModule> # Common SQL authentication options SQLPasswordEngine on SQLBackend postgres SQLConnectInfo galaxy@galaxy.my.org:5432 bioinfoadmin dbpwd SQLAuthenticate users # Configuration that handles PBKDF2 encryption # Set up mod_sql to authenticate against the Galaxy database SQLAuthTypes PBKDF2 SQLPasswordPBKDF2 SHA256 10000 24 SQLPasswordEncoding base64 SQLPasswordUserSalt sql:/GetUserSalt # Define a custom query for lookup that returns a passwd-like entry. Replace 512s with the UID and GID of the user running the Galaxy server SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38 for 69) ELSE password END) AS password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" # Define custom query to fetch the password salt SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt FROM galaxy_user WHERE email='%U'" </VirtualHost> # Don't use IPv6 support by default. UseIPv6 off MaxInstances 30 # To cause every FTP user to be "jailed" (chrooted) into their home # directory, uncomment this line. # Bar use of SITE CHMOD by default <Limit SITE_CHMOD> DenyAll </Limit> # Bar use of RETR (download) since this is not a public file drop <Limit RETR> DenyAll </Limit> ~

2 4

featureCounts with GFF
by Jochen Bick 05 Feb '19

05 Feb '19

Hi, our group would like to switch to featureCounts <http://green-l-bio.ethz.ch:8090/tool_runner?tool_id=toolshed.g2.bx.psu.edu%…> but we are using only GFF from NCBI. It looks like featureCounts <http://green-l-bio.ethz.ch:8090/tool_runner?tool_id=toolshed.g2.bx.psu.edu%…> provides GFF but out of the box it does not provide a proper count table. The output looks like this: Status HISAT2 on data 33 Assigned 5186527 Unassigned_Unmapped 518985 Unassigned_MappingQuality 1353602 Unassigned_Chimera 0 Unassigned_FragmentLength 0 Unassigned_Duplicate 0 Unassigned_MultiMapping 0 Unassigned_Secondary 0 Unassigned_Nonjunction 0 Unassigned_NoFeatures 7515020 Unassigned_Overlapping_Length 0 Unassigned_Ambiguity 0 Geneid HISAT2 on data 33 5186527 I guess I need some advanced options? Which are different to the default run settings? Does anyone have a tutorial or example on how to configure the tool correctly with a standard GFF file, or does anyone also use GFF files? Or is there a better tool to use? Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

1 0