March 2018 - galaxy-dev - lists.galaxyproject.org

Monitoring Dashboard for Galaxy
by evan clark 03 Sep '18

03 Sep '18

I remember there previously being an additional tool that ran on a different port from galaxy that allowed for monitoring of performance and jobs running. Is this tool still packaged with galaxy and if so how can it be activated.

3 3

Job Script Integrity
by John Letaw 04 Jun '18

04 Jun '18

Hi all, I’m installing via GalaxyKickStart… I’m getting the following error: galaxy.jobs.runners ERROR 2017-11-06 19:14:05,263 (19) Failure preparing job Traceback (most recent call last): File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", line 175, in prepare_job modify_command_for_container=modify_command_for_container File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/__init__.py", line 209, in build_command_line container=container File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", line 84, in build_command externalized_commands = __externalize_commands(job_wrapper, external_command_shell, commands_builder, remote_command_params) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/command_factory.py", line 143, in __externalize_commands write_script(local_container_script, script_contents, config) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/util/job_script/__init__.py", line 112, in write_script _handle_script_integrity(path, config) File "/home/exacloud/lustre1/galaxydev/galaxyuser/lib/galaxy/jobs/runners/util/job_script/__init__.py", line 147, in _handle_script_integrity raise Exception("Failed to write job script, could not verify job script integrity.") Exception: Failed to write job script, could not verify job script integrity. galaxy.model.metadata DEBUG 2017-11-06 19:14:05,541 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-11-06 19:14:05,576 Failed to cleanup MetadataTempFile temp files from /home/exacloud/lustre1/galaxydev/galaxyuser/database/jobs/000/19/metadata_out_HistoryDatasetAssociation_16_I8bhLX: No JSON object could be decoded I would like to further understand what it means to not verify integrity of a job script. Does this just mean there is a permissions error? Ownership doesn’t match up? Thanks, John

2 1

Text file busy and conda
by John Letaw 16 May '18

16 May '18

Hi all, I’m receiving the following error trying to access tools autoinstalled via conda: Fatal error: Exit code 126 () ~/galaxy/database/jobs_directory/000/38/tool_script.sh: line 41: ~/galaxy/database/dependencies/_conda/envs/mulled-v1-bb83f93efd111e042823e53ddfa74c32d81ba74cceca9445dfddfc9e940ff738/bin/samtools: Text file busy So, something is happening too fast I guess, meaning a process is attempting access while this is being created. Any ideas on how I can diagnose this? Not seeing any other errors floating around the logs. Maybe we are dealing with a lustre file locking issue? Thanks, John

2 2

Best practices for utilising Unique Molecular Indexing (UMI's)
by Brooks, Tony 04 May '18

04 May '18

Hi We are currently seeing a number of methods that are utilising the power of unique molecular indexing. Unfortunately, there is no consensus on how libraries should be configured, and therefore no consensus for how to deal with them within Galaxy. Often libraries that have the UMI placed directly downstream of the first (i7) index, such as ones using the IDT xGen adapter set (https://www.idtdna.com/pages/products/next-generation-sequencing/adapters/x…) Sometimes UMI's exist in place of the second (i5) index (https://www.neb.com/nebnext-direct/nebnext-direct-for-target-enrichment) In both cases, the recommended workflows are convoluted and all the necessary tools do not currently exist in the toolshed (so that the datasets need to be taken out of galaxy, processed and reloaded). It is possible to use bcl2fastq to output the UMI as an additional fastq file, but this would then require me to create a dataset triplicate (not pair) which afaik we can't do (yet). A quick Google/toolshed search had me find UMI-Tools & Je-Suite which both exist in Galaxy. Both these tools assume the UMI is "in-line" (i.e. at the beginning of the read 1 or read2 - not its own read), extract/remove the UMI and place it in the read header, where it is then used further down the line to dedup the bam file. Does anyone know of any tools that would take the UMI from a separate fastq and use it to tag the headers of actual read data. Or alternatively, a tool that will paste the UMI tag onto the 5' end of the read fastq? And whether these steps can be done within Galaxy, or maybe prior to fastq upload? Anyone have a method/workflow for UMI's? Thanks in advance Tony

2 6

Remove entry from Tool Data Table using via Data Manager
by Dan Fornika 23 Apr '18

23 Apr '18

Hello, I'm working on galaxy integration for an MLST tool called MentaLiST: https://github.com/WGS-TB/MentaLiST/tree/master/galaxy I've created data managers that create kmer databases and store their locations in a Tool Data Table. I'd like to allow users to remove entries from the Tool Data Table. Has anyone seen a Data Manager that provides that functionality? Should I create separate data managers for add and remove functionality, or can it be integrated into one? Thanks, Dan

2 3

Problems running reporting app with reports.yml using --daemon
by Peter Briggs 28 Mar '18

28 Mar '18

Dear devs I'm trying to update my ansible playbooks for deploying Galaxy and I'm having problems when trying to run the reporting app in release_17.09 using a 'reports.yml' file (rather than 'reports.ini'), when specifying the --daemon option. Running without any arguments, the application appears to start okay - but if I do sh ./run_reports.sh --daemon then it exits with the error: /XXXXX/galaxy/.venv/bin/python: unrecognized option '--safe-pidfile' getopt_long() error This seems to be coming from the invocation of uwsgi within the run_reports.sh script, and looks like an issue with Galaxy's version of uWSGI: version 2.0.15 (as specified in the requirements.txt file) doesn't support the '--safe-pidfile' option (that needs uWSGI 2.0.16). Editing requirements.txt to use 2.0.16 seems to address this issue (as in, --daemon no longer fails on startup). Has anyone else encountered this issue? My ultimate goal is to be able to serve the uwsgi-based reports app via an nginx proxy (which used to work with the old i.e. non-uwsgi reports.ini-configured version). Thanks for your help, Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

1 0

Tool to download galaxy datasets locally
by evan clark 27 Mar '18

27 Mar '18

Has anyone attempted to write a tool that allows copying of galaxy datasets locally, i.e. without downloading via http. We have scenario where there are some additional tools available only via command line and it would be very useful to be able to directly copy galaxy datasets from the datasets folder.

2 1

Re: [galaxy-dev] Shiny in Galaxy
by Romain Dallet 27 Mar '18

27 Mar '18

Dear Mohamed, Karim, Hans, I don't know if you are still looking for a Galaxy Interactive Environment with Shiny. But in collaboration with Yvan's team, we have also developed a generic Shiny GIE for the Workflow4Metabolomics project. You can find it with an example on GitHub at the following link : https://github.com/workflow4metabolomics/gie-shiny . If you find mistakes or something require more informations in the tutorial, don't hesitate to make a pull request and if you have some questions, please, ask me. Romain and Gildas >> Début du message réexpédié : >> >> *De: *Yvan Le Bras <yvan.le-bras(a)mnhn.fr <mailto:yvan.le-bras@mnhn.fr>> >> *Objet: **Rép : [galaxy-dev] Shiny in Galaxy* >> *Date: *14 mars 2018 à 10:14:14 UTC+1 >> *À: *Mohamed Kassam <k.mamoud(a)gmail.com <mailto:k.mamoud@gmail.com>>, >> Hans-Rudolf Hotz <hrh(a)fmi.ch <mailto:hrh@fmi.ch>> >> *Cc: *Timothée VIRGOULAY <timothee.virgoulay(a)mnhn.fr >> <mailto:timothee.virgoulay@mnhn.fr>>, Galaxy Dev List >> <galaxy-dev(a)lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>>, Valentin CHAMBON >> <valentin.chambon(a)mnhn.fr <mailto:valentin.chambon@mnhn.fr>> >> >> Dear Mohamed, >> >> Following exchanges with Hans and french colleagues as William from >> the carpem project (university hospital Georges Pompidou), we have >> finalized a first gie shiny template for Galaxy-E project. You can >> refer to the GitHub repo for details, don't hesitate to ask more >> info...https://github.com/65MO/Galaxy-E/tree/master/GIE >> >> Cheers,. >> Yvan >> >> >> >> Envoyé depuis mon smartphone Samsung Galaxy. >> >> -------- Message d'origine -------- >> De : Mohamed Kassam <k.mamoud(a)gmail.com <mailto:k.mamoud@gmail.com>> >> Date : 14/03/2018 09:59 (GMT+01:00) >> À : Hans-Rudolf Hotz <hrh(a)fmi.ch <mailto:hrh@fmi.ch>> >> Cc : Yvan Le Bras <yvan.le-bras(a)mnhn.fr >> <mailto:yvan.le-bras@mnhn.fr>>, "Md. Rezaul Karim" >> <rezaul.karim(a)insight-centre.org >> <mailto:rezaul.karim@insight-centre.org>>, Timothée VIRGOULAY >> <timothee.virgoulay(a)mnhn.fr <mailto:timothee.virgoulay@mnhn.fr>>, >> Valentin CHAMBON <valentin.chambon(a)mnhn.fr >> <mailto:valentin.chambon@mnhn.fr>>, Galaxy Dev List >> <galaxy-dev(a)lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>> >> Objet : Re: [galaxy-dev] Shiny in Galaxy >> >> Dear all >> >> I am contacting about the shiny integration into the Galaxy framewrok. >> >> Now, we are running a shiny app outside the Galaxy framework and we >> are pointing a link in the Galaxy tools that allow the user to open >> another browser for the use of the shiny app. >> >> Howerver we would like to integrate directly the shiny into Galaxy >> framewrok, and I would like to know if someone find a solution for >> doing that. >> >> Best regards >> >> Mohamed >> >> 2017-07-11 13:57 GMT+02:00 Hans-Rudolf Hotz <hrh(a)fmi.ch >> <mailto:hrh@fmi.ch>>: >> >> Hi Yvan >> >> Yes, after an additional week in the South of France, we traveled >> back safely last weekend. >> >> I use the free version of the shiny server. In addition to what >> you already have mentioned, we run into another 'feature' of the >> free version: you cannot set the R path. You need to set the R >> path in the environment of the user, the shiny server is running as. >> >> I am currently not using IE, since I cannot use Docker in our >> production environment. Hence the detour via an html page linking >> to the extra shiny server. Maybe this will change with >> Singularity in the future...... >> >> Using IE is definitively the way forward. >> >> >> Regards, Hans-Rudolf >> >> >> >> >> On 07/06/2017 10:58 PM, Yvan Le Bras wrote: >> >> Hi Hans, Hi everyone, >> >> I hope your return trip from GCC has been safe! >> Unfortunately, I didn't take time during gcc to see you Quick >> and dirty solution... Can we go further on this Shiny / >> Galaxy integration idea ? It seems that the Shiny licence can >> be a problem but it appears to me, if I don't make a mistake, >> that using the open source version of Shiny (without ssl and >> identification) on Galaxy through IE can be sufficent... no ? >> >> Cheers, >> >> Yvan >> >> >> ----- Mail d'origine ----- >> De: Hans-Rudolf Hotz <hrh(a)fmi.ch <mailto:hrh@fmi.ch>> >> À: Yvan Le Bras <yvan.le-bras(a)mnhn.fr >> <mailto:yvan.le-bras@mnhn.fr>>, Md. Rezaul Karim >> <rezaul.karim(a)insight-centre.org >> <mailto:rezaul.karim@insight-centre.org>>, Mohamed Kassam >> <k.mamoud(a)gmail.com <mailto:k.mamoud@gmail.com>>, Timothée >> VIRGOULAY <timothee.virgoulay(a)mnhn.fr >> <mailto:timothee.virgoulay@mnhn.fr>>, Valentin CHAMBON >> <valentin.chambon(a)mnhn.fr <mailto:valentin.chambon@mnhn.fr>> >> Cc: Galaxy Dev List <galaxy-dev(a)lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>> >> Envoyé: Mon, 29 May 2017 09:39:10 +0200 (CEST) >> >> Objet: Re: [galaxy-dev] Shiny in Galaxy >> >> Hi Yvan and list >> >> This is great news! >> >> In my limited spare time, I am currently playing with shiny >> as well. My >> current 'quick and dirty' trick goes like this: >> >> The galaxy tool creates a new shiny app and the output of the >> galaxy >> tool is a html page with the URL to the shiny app. I hope, I >> will have >> something ready to show/discuss at GCC >> >> >> Regards, Hans-Rudolf >> >> >> On 05/26/2017 04:13 PM, Yvan Le Bras wrote: >> >> Dear Mohamed, Karim, galaxy-dev list, >> >> A rapid e-mail to inform you that we plan to work on this >> task in the >> upcoming weeks with Thimothée and Valentin, copied. We >> have had some >> exchanges with Eric (Rasche) and Björn (Grüning) and it >> seems that using >> the Galaxy "interactive environment" functionality is a >> good way to >> proceed. Don't hesitate to give us more informations to >> collaborate on >> it... >> >> Wishing you a good week end. >> >> Cheers, >> >> Yvan >> >> >> -------- Message d'origine -------- >> De : "Md. Rezaul Karim" <rezaul.karim(a)insight-centre.org >> <mailto:rezaul.karim@insight-centre.org>> >> Date : 26/05/2017 15:45 (GMT+01:00) >> À : Mohamed Kassam <k.mamoud(a)gmail.com >> <mailto:k.mamoud@gmail.com>> >> Cc : Galaxy Dev List <galaxy-dev(a)lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>> >> Objet : Re: [galaxy-dev] Shiny in Galaxy >> >> +1 >> >> On May 26, 2017 2:44 PM, "Mohamed Kassam" >> <k.mamoud(a)gmail.com <mailto:k.mamoud@gmail.com> >> <mailto:k.mamoud@gmail.com <mailto:k.mamoud@gmail.com>>> >> wrote: >> >> Dear all, >> >> I have a shiny application working in my RStudio, >> but I would like >> to integrate it to Galaxy that the users can call my >> application via >> Galaxy . >> >> Thanks in Advance, >> >> Mohamed >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions >> to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> <https://lists.galaxyproject.org/> >> <https://lists.galaxyproject.org/ >> <https://lists.galaxyproject.org/>> >> >> To search Galaxy mailing lists use the unified >> search at: >> http://galaxyproject.org/search/ >> <http://galaxyproject.org/search/> >> <http://galaxyproject.org/search/ >> <http://galaxyproject.org/search/>> >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> <https://lists.galaxyproject.org/> >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/ >> <http://galaxyproject.org/search/> >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/ >

1 0

GVL/Galaxy ver 18.01 instance error: R_COMM channel basic get exception: [Errno 32] Broken pipe
by Leon Parker 26 Mar '18

26 Mar '18

We just stood up our first ever Genomics Virtual Lab appliance and used it to spool up a Galaxy Cluster for doing bioinformatics work. We’ve worked a bit with the original “Cloudman” application and the ver 17.05 revision of Galaxy and it has worked fine, but we decided it might be a good time to migrate the GVL platform because it uses the latest, greatest version of Galaxy and provides additional tools, and the old “cloud man” approach has been deprecated, giving us further incentive to make the switch. Unfortunately, we’re encountering an error right out of the gate, when we spool up Galaxy through GVL, before we do anything with it. Within two minutes of Galaxy starting, we begin to see the following over and over again in the Cluster Info Log: 20:55:20 - R_COMM channel basic get exception: [Errno 32] Broken pipe In addition, we can’t seem to access any Spot worker node instances we activate. They launch but are never recognized and used. Not sure what this mean, as we are biologists, not informaticians nor python experts. Any help would be most appreciated! --Leon

2 1

Re: [galaxy-dev] firezilla not working
by Atanu Pan 21 Mar '18

21 Mar '18

Hi, This is an intermittent problem, sometime it is connected and sometime not. The error is attached in the file, please help. Regards, Atanu Pan From: "Antara Biswas" <ab3(a)nibmg.ac.in> To: galaxy-dev(a)lists.galaxyproject.org Cc: "Atanu Pan" <ap1(a)nibmg.ac.in> Sent: Tuesday, March 20, 2018 12:19:57 PM Subject: firezilla not working Dear galaxy team, I am having trouble since a week uploading data into filezilla from local computer. Error message says- connection refused by server. Please help. My login id is ab3(a)nibmg.ac.in Thanks Antara Biswas Research Fellow National Institute of Biomedical Genomics India

3 3