November 2017 - galaxy-dev - lists.galaxyproject.org

European Galaxy Administrator Workshop, 8-12 January 2018, Oslo. FREE
by Dave Clements 16 Nov '17

16 Nov '17

Hello all, *We are pleased to announce that registration <https://skjema.uio.no/galaxy-admin-workshop-2018> is now open for the European Galaxy Administrator Workshop <https://www.elixir-europe.org/events/european-galaxy-administrator-workshop>* . - Mon 8 January 2018 to Fri 12 January 2018 CET - Python room, Blindern, UiO campus, Ole Johan Dahls hus <https://www.uio.no/english/about/getting-around/areas/gaustad/ga06/>, Gaustadalléen 23B, Oslo, 0373, Norway Participants will learn how to install, configure, customize, and extend their own Galaxy servers. Topics include tool configuration, authentication and user management, using heterogeneous storage and compute services, and many other topics that will enable you to get your own Galaxy server up and running, performing well, and used by your community. Have a look at the programme <https://github.com/elixir-no-nels/dagobah-training/blob/2018-oslo/README.md> to see more details on the planned content of the workshop. *Maximum number of participants is limited to 30, and we will manage registrations to allow for geographical distribution and proritize participants with responsibility of administrating galaxy servers.* After registering you will get a second confirmation if/when a slot has been allocated to you. Although located in Europe, and named to reflect this, the workshop is definitely *open to a world-wide audience.* The workshop is a result of the collaboration between the Elixir Galaxy WG <https://www.elixir-europe.org/about/groups/galaxy-wg> and the Galaxy Project, and so far known, this is the only Galaxy Admin workshop planned for 2018. *Participants are to cover their own travel and accomodation, while the workshop participation is free of charge.* Instructors - Björn Grüning <https://github.com/bgruening> (Elixir Galaxy WG <https://www.elixir-europe.org/about/groups/galaxy-wg>, Elixir Germany <https://www.elixir-europe.org/about-us/who-we-are/nodes/germany>, University of Freiburg <https://www.uni-freiburg.de/?set_language=en>) - Marius van den Beek <https://github.com/mvdbeek> (Elixir Galaxy WG <https://www.elixir-europe.org/about/groups/galaxy-wg>, Institute Curie <https://science.institut-curie.org/>, France) - Nicola Soranzo <http://www.earlham.ac.uk/nicola-soranzo> (Elixir Galaxy WG <https://www.elixir-europe.org/about/groups/galaxy-wg>, Earlham Institute <http://www.earlham.ac.uk/>, UK) - Enis Afgan <https://galaxyproject.org/people/enis-afgan/> (Galaxy Project <https://galaxyproject.org/>, Johns Hopkins University <https://jhu.edu/>, USA) - Abdulrahman Azab <http://www.usit.uio.no/om/organisasjon/itf/ft/ansatte/azab/index.html> (Elixir Norway <https://www.elixir-europe.org/about-us/who-we-are/nodes/norway>, USIT <http://www.usit.uio.no/>, University of Oslo <http://www.uio.no/>, Norway) -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

Active Directory Authentication question
by Hakeem Almabrazi 13 Nov '17

13 Nov '17

Hello, I have installed the latest galaxy and I would like to use the MS Active Directory 2012 for authentication. I tried to follow the instructions outlined here https://galaxyproject.org/admin/config/external-user-auth/ without touching the lib/galaxy/auth/providers/ldap_ad_py since I thought this is related to ldap not AD. Here is the exact auth_config.xml file +++++++++++++++++++++++++ <?xml version="1.0"?> <auth> <authenticator> <type>ldap</type> <options> <allow-register>True</allow-register> <auto-register>True</auto-register> <allow-password-change>False</allow-password-change> <login-use-username>True</login-use-username>  <search-fields>sAMAccountName,mail</search-fields> <search-base>dc=dc1,dc=example,dc=com</search-base>  <search-filter>(&(objectClass=user)(sAMAccountName={username}))</search-filter> </options> </authenticator> </auth> +++++++++++++++++ When I try to login using a real user email and pwd, it says "No user of valid password" in the Galaxy page. And here is the error log I keep getting. galaxy.webapps.galaxy.controllers.user DEBUG 2017-11-09 13:55:37,940 trans.app.config.auth_config_file: ./config/auth_conf.xml galaxy.auth.providers.ldap_ad DEBUG 2017-11-09 13:55:37,940 LDAP authenticate: email is user(a)example.com galaxy.auth.providers.ldap_ad DEBUG 2017-11-09 13:55:37,940 LDAP authenticate: username is None galaxy.auth.providers.ldap_ad DEBUG 2017-11-09 13:55:37,940 LDAP authenticate: options are {'search-fields': 'sAMAccountName,mail', 'login-use-username': 'True', 'allow-register': 'True', 'auto-register': 'True', 'search-base': 'dc=dc1,dc=example,dc=com', 'search-filter': '(&(objectClass=user)(sAMAccountName={username}))', 'allow-password-change': 'False'} galaxy.auth.providers.ldap_ad DEBUG 2017-11-09 13:55:37,940 LDAP authenticate: username must be used to login, cannot be None galaxy.auth DEBUG 2017-11-09 13:55:37,941 Email: , Username , stopping due to failed non-continue Is there anything missing from my auth_config.xml file? I appreciate any kind of help figuring this out. Best regards, Hak Disclaimer: This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Sidra Medical and Research Center.

2 4

Fwd: Job Script Integrity with GalaxyKickStart (galaxy-dev Digest, Vol 137, Issue 5)
by Christophe Antoniewski 09 Nov '17

09 Nov '17

Hi John, Can you also raise an issue in https://github.com/ARTbio/ GalaxyKickStart/issues ? In order to help, I will need to know the configuration of your GalaxyKickStart (the variables you modified in the playbook, group_vars and inventory_files). Did you use the cloud_setup role ? In that case Enis Afgan https://github.com/afgane may help. Best regards Chris Christophe Antoniewski Institut de Biologie Paris Seine <http://www.ibps.upmc.fr/en> 9, Quai St Bernard, Boîte courrier 24 75252 Paris Cedex 05 ARTbio <http://artbio.fr/> Bâtiment B, 7e étage, porte 725 Tel +33 1 44 2 *7 70 05*Mobile +33 6 68 60 51 50 <06%2068%2060%2051%2050> Pour accéder à la Plateforme Bâtiment B, 7e étage, Porte 725 <https://www.google.com/maps/d/u/0/edit?mid=zmZz-3Vin5D0.kjRSV6vitXE8> https://twitter.com/ARTbio_IBPS 2017-11-07 18:00 GMT+01:00 <galaxy-dev-request(a)lists.galaxyproject.org>: > Send galaxy-dev mailing list submissions to > galaxy-dev(a)lists.galaxyproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.galaxyproject.org/listinfo/galaxy-dev > or, via email, send a message with subject or body 'help' to > galaxy-dev-request(a)lists.galaxyproject.org > > You can reach the person managing the list at > galaxy-dev-owner(a)lists.galaxyproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of galaxy-dev digest..." > > > HEY! This is important! If you reply to a thread in a digest, please > 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to > the original subject for the thread. > 2. Strip out everything else in the digest that is not part of the thread > you are responding to. > > Why? > 1. This will keep the subject meaningful. People will have some idea from > the subject line if they should read it or not. > 2. Not doing this greatly increases the number of emails that match search > queries, but that aren't actually informative. > > Today's Topics: > > 1. Job Script Integrity (John Letaw) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 7 Nov 2017 03:20:49 +0000 > From: "John Letaw" <letaw(a)ohsu.edu> > To: "galaxy-dev(a)lists.galaxyproject.org" > <galaxy-dev(a)lists.galaxyproject.org> > Subject: [galaxy-dev] Job Script Integrity > Message-ID: <FBF795C3-8F01-47EF-8033-F14DD8694328(a)ohsu.edu> > Content-Type: text/plain; charset="utf-8" > > Hi all, > > I’m installing via GalaxyKickStart… > > I’m getting the following error: > > galaxy.jobs.runners ERROR 2017-11-06 19:14:05,263 (19) Failure preparing > job > Traceback (most recent call last): > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", > line 175, in prepare_job > modify_command_for_container=modify_command_for_container > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", > line 209, in build_command_line > container=container > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", > line 84, in build_command > externalized_commands = __externalize_commands(job_wrapper, > external_command_shell, commands_builder, remote_command_params) > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", > line 143, in __externalize_commands > write_script(local_container_script, script_contents, config) > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/ > jobs/runners/util/job_script/__init__.py", line 112, in write_script > _handle_script_integrity(path, config) > File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/ > jobs/runners/util/job_script/__init__.py", line 147, in > _handle_script_integrity > raise Exception("Failed to write job script, could not verify job > script integrity.") > Exception: Failed to write job script, could not verify job script > integrity. > galaxy.model.metadata DEBUG 2017-11-06 19:14:05,541 Cleaning up external > metadata files > galaxy.model.metadata DEBUG 2017-11-06 19:14:05,576 Failed to cleanup > MetadataTempFile temp files from /home/exacloud/lustre1/galaxyd > ev/galaxyuser/database/jobs/000/19/metadata_out_HistoryDatasetAssociation_16_I8bhLX: > No JSON object could be decoded > > I would like to further understand what it means to not verify integrity > of a job script. Does this just mean there is a permissions error? > Ownership doesn’t match up? > > Thanks, > John > >

3 6

AWS launch error
by David Kovalic 09 Nov '17

09 Nov '17

Enis, I am having issues launching previously existing clusters today, here is some more info: * I go to https://launch.usegalaxy.org/launch * I fill in the required AWS credentials * I click on "Choose a saved cluster" * the page goes into a loop trying to retrieve the 'Saved Cluster' information to populate the drop down list * I can terminate this loop by clicking on the "x" * I can enter the cluster name manually * The cluster bucket entered does exist on my S3 * After entering the cluster name manually I click "Create a cluster" * The next page is shown "Please wait, your scalable cluster is starting on the Amazon - Virginia cloud." * I see no instance stared on the AWS console I have similar issues on the new page "https://beta.launch.usegalaxy.org/", on this page my previous clusters are not shown. Nothing has changed on my end, to my knoledge… Any ideas? Need any more information from my end? lmk and thanks, David David Kovalic Ph.D.President,Analomephone:+314.496.8808web:www.analome.comemail: kovalic(a)analome.com On Fri, Nov 3, 2017 11:00 AM, galaxy-dev-request(a)lists.galaxyproject.org wrote: Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.galaxyproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.galaxyproject.org/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.galaxyproject.org You can reach the person managing the list at galaxy-dev-owner(a)lists.galaxyproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.Today's Topics: 1. Re: AWS launch error (Enis Afgan) 2. Enable viewing of large HTML files (Peter Briggs) 3. Re: Enable viewing of large HTML files (Martin Čech) _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.galaxyproject.org https://lists.galaxyproject.org/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

3 4

Delete a record from a Tool Data Table via the Galaxy API
by Dan Fornika 08 Nov '17

08 Nov '17

Hello, I'm working on adding a feature to blend4j (java interface to the Galaxy API) to access Galaxy Tool Data Tables. My repository is here: https://github.com/dfornika/blend4j/tree/tool-data-client I'm working on a method that deletes entries from Galaxy Tool Data Tables, but I'm having some trouble with the API calls. How can I construct a DELETE api call on an entry in a Tool Data Table? For example, how could I delete hg38 from the "igv_broad_genomes" tool data table (using Postman or curl or whatever HTTP client)? Specifically, how do I pass the extra argument(s) besides the table name (id) that goes in the URL? In bioblend, we pass "a “|” separated list of column contents, there must be a value for all the columns of the data table" https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#bioblend… I can see the "igv_broad_genomes" tool data table when I send a GET request: GET /api/tool_data/igv_broad_genomes { "model_class": "TabularToolDataTable", "name": "igv_broad_genomes", "columns": [ "name", "url", "value" ], "fields": [ [ "A. baumannii str. ATCC", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/ABaumannii_ATCC_1797… ", "ABaumannii_ATCC_17978" ], [ "A. baumannii str. AYE", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/ABaumannii_AYE_uid61… ", "ABaumannii_AYE_uid61637" ], ...(removed for brevity)... [ "Human hg38", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/hg38.genome", "hg38" ], ...(removed for brevity)... ] } The closest I've got is to send this: DELETE /api/tool_data/igv_broad_genomes Content-Type: application/json { "values": "Human hg38\thttp:// s3.amazonaws.com/igv.broadinstitute.org/genomes/hg38.genome\thg38" } I get the same JSON description of the original Tool Data Table, including hg38: { "model_class": "TabularToolDataTable", "name": "igv_broad_genomes", "columns": [ "name", "url", "value" ], "fields": [ [ "A. baumannii str. ATCC", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/ABaumannii_ATCC_1797… ", "ABaumannii_ATCC_17978" ], [ "A. baumannii str. AYE", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/ABaumannii_AYE_uid61… ", "ABaumannii_AYE_uid61637" ], ...(removed for brevity)... [ "Human hg38", " http://s3.amazonaws.com/igv.broadinstitute.org/genomes/hg38.genome", "hg38" ], ...(removed for brevity)... ] }

1 0

Diffbind error
by Rui Wang 06 Nov '17

06 Nov '17

Hi Folks, I found one problem with the docker galaxy instance we have. While we are trying to run Diffbind tool, it failed to create the env for the tool. The error is Conda dependency seemingly installed but failed to build job environment. I saw the env is defined in envs/__bioconductor-diffbind\(a)2.0.9, so I tried the following: conda list --name __bioconductor-diffbind\(a)2.0.9 --export > file1 conda create -y --unknown --offline --prefix diff_env --file file1 > /dev/null basically I was trying to manually create the env for this tool and see if it works. file1 was generated ok but the second command returns: UnsatisfiableError: The following specifications were found to be in conflict: - bioconductor-annotate 1.50.0 r3.3.2_1 - krb5 1.13.2 0 Use "conda info <package>" to see the dependencies for each package. Have anyone seen this before? Thanks, Rui

1 0

html with styles
by Wolfgang Maier 03 Nov '17

03 Nov '17

Dear all, until recently extra html files linked from html datasets got displayed with style information applied, but this seems to have changed. I did not investigate the change in detail, but is this a consequence of the backported https://docs.galaxyproject.org/en/master/releases/17.09_announce.html#cross… Is downloading the zipped data and opening it locally now the only way to view styled html? Have a nice weekend, Wolfgang

2 5

Upload tool incorrect path
by evan clark 03 Nov '17

03 Nov '17

When using the upload tool on our local galaxy we are experiencing an issue with tool factory pointing to the incorrect directory. Below dhouldn't be /var/web_services but rather /mnt/ceph/..... We are unsure how to change it our how to fix this issue. Traceback (most recent call last): File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 425, in<module> __main__() File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 399, in __main__ registry.load_datatypes(root_dir=sys.argv[1], config=sys.argv[2] ) File "/mnt/ceph/home/galaxy/galaxy/lib/galaxy/datatypes/registry.py", line 119, in load_datatypes raise ConfigurationError( "Directory does not exist: %s" % self.converters_path ) galaxy.datatypes.registry.ConfigurationError: Directory does not exist: /var/web_services/galaxy/lib/galaxy/datatypes/converters

1 0

Enable viewing of large HTML files
by Peter Briggs 03 Nov '17

03 Nov '17

Dear Devs We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files. For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text. As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)? We're currently using 17.05 in our production instance. Thanks for advance for any help or suggestions, Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

AWS launch error
by Sijung Yun 02 Nov '17

02 Nov '17

To whom it may concern, Currently, both of launch.usegalaxy.org and beta.launch.usegalaxy.org has error in launching, so it is not possible to launch at this time. Could you solve this problem as soon as possible? I attach the snapshots. Thanks a lot. Sincerely, Sijung -- Sijung Yun, Ph.D. Founder and CEO Yotta Biomed, LLC. Bioinformatics Unlimited yottabiomed.com Cell: 240-672-3937

2 1