- galaxy-dev - lists.galaxyproject.org

Elastic Bean Stalk
by Ryan G 15 Nov '16

15 Nov '16

Hi all - I was wondering, if we deploy Galaxy in AWS, can we take advantage of Elastic BeanStalk to scale our instances based on demand instead of trying to configure Galaxy to perform well for our users? Ryan

2 1

Reporting issues for the wiki? e.g. universe_wgsi.ini still widely used
by Peter Cock 15 Nov '16

15 Nov '16

2 3

purge dataset at a different location
by Mark Xiao 14 Nov '16

14 Nov '16

Dear Galaxy Developers, Because of specific use case, our transfer tool need to keep the datasets in a different location instead of the default location, so we'd like to purge datasets according to its "external_filename" (we found this column in the dataset table in the database). Is there a way or some code to modify so that when we try to purge a dataset or a history in Galaxy, the file located at a dataset's external_filename can also be deleted? Thanks in advance, Mark Xiao

2 1

conda management of tools dependent on python3
by Wolfgang Maier 10 Nov '16

10 Nov '16

Dear all, I'm still pretty new to conda tool dependencies management and there is one thing I cannot find documented anywhere: How can I let conda manage tool dependencies that involve python3? If I let Galaxy install conda for me, it creates an environment with python2, then if there is a tool dependency that has a conda package built for only python3 I'm getting an error because of a conflict between that package and python2 (not surprisingly), but AFAIU conda environments are bound to one particular version of python (or is that incorrect?). Thanks for any help with this and best wishes, Wolfgang

4 9

Compilation of package_bioconductor_limma_3_28_10 crash : source unavailable !
by Emmanuel Quemener 10 Nov '16

10 Nov '16

Hello, The package_bioconductor_limma_3_28_10 refuses to install because the source of software is available in https://depot.galaxyproject.org/software/limma folder. It seems to have been removed because the file https://depot.galaxyproject.org/software/limma/SHA256SUM.txt contain a line with this source. Is it possible to update the package or to add these source in folder ? Best regards. <log message> Tool shed repository 'package_bioconductor_limma_3_28_10' tool dependencies Name Version Type Status Error bioconductor-limma 3.28.10 package Error File "/galaxy2/galaxy/lib/tool_shed/galaxy_install/install_manager.py", line 125, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( install_environment, tool_dependency, actions_dict ) File "/galaxy2/galaxy/lib/tool_shed/galaxy_install/install_manager.py", line 83, in install_and_build_package initial_download=True ) File "/galaxy2/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 31, in execute_step initial_download=initial_download ) File "/galaxy2/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 1362, in execute_step self.url_download( work_dir, filename, url, extract=False, checksums=checksums ) File "/galaxy2/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 184, in url_download raise Exception( err_msg ) Error downloading from URL https://depot.galaxyproject.org/software/limma/limma_3.28.10_src_all.tar.gz: HTTP Error 404: Not Found </log message> -- Dr Emmanuel QUEMENER Tel.: 04 72 72 86 33 Centre Blaise Pascal Ecole Normale Superieure de Lyon 46, allee d'Italie, 69364 Lyon Emmanuel.Quemener(a)ens-lyon.fr -- "La simplicite est la sophistication supreme." Leonard de Vinci

1 0

upload problem for http://deeptools.ie-freiburg.mpg.de
by Robert Hansel-Hertsch 08 Nov '16

08 Nov '16

Dear Galaxy support team, yesterday I was using the Galaxy deeptools server and I encountered no problem by uploading data. Now, I’m not able to upload bigger data volumes, such as bigwig files (100-300 mb). I’m getting the following error message: Could you please advice me here? Best wishes, Robert [cid:191EDBDE-E7F9-47A7-970D-4C7307D18C23@cri.camres.org]

1 0

Errors when building Galaxy RNA-Seq Docker Image
by Stephen Kelly 04 Nov '16

04 Nov '16

I am trying to get this Docker image running: https://github.com/bgruening/galaxy-rna-seq If I let Docker download and start the container using this command: docker run -d -p 8080:80 bgruening/galaxy-rna-seq everything appears to work fine. However, if I clone the repo and try to build it myself like this: docker build -t skelly/galaxy-rna-seq . I get a load of errors. Here are a few: ... ... Step 5 : RUN install-tools $GALAXY_ROOT/tools.yaml ---> Running in 78e96eb73823 Enable installation from the Test Tool Shed. * Starting PostgreSQL 9.3 database server ...done. starting Galaxy ./run.sh: 37: [: --daemon: unexpected operator Activating virtualenv at /galaxy_venv Requirement already satisfied (use --upgrade to upgrade): pip>=8.1 in /galaxy_venv/lib/python2.7/site-packages /galaxy_venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning /galaxy_venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwar…. InsecurePlatformWarning You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command. ... ... But I think the biggest problem is that the tools installation appears to fail: ... ... Activating virtualenv at /galaxy_venv Galaxy is running. (1/10) Installing tool trim_galore from bgruening to section "Quality Control" at revision f1e71aeaa923 (TRT: 0:00:00.610888) Tool trim_galore installed successfully (in 0:01:46.048131) at revision f1e71aeaa923 (2/10) Installing tool suite_samtools_1_2 from devteam to section "SAM Tools" at revision 5b673ccc8747 (TRT: 0:01:46.874497) * Error installing a tool (after 0:00:05.381866 seconds)! Name: suite_samtools_1_2,owner: devteam, revision: 5b673ccc8747, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (3/10) Installing tool deseq2 from iuc to section "RNA-Seq" at revision 89bbd1dbf48d (TRT: 0:01:52.463056) * Error installing a tool (after 0:00:05.147542 seconds)! Name: deseq2,owner: iuc, revision: 89bbd1dbf48d, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (4/10) Installing tool deseq2 from iuc to section "RNA-Seq" at revision 8702e49e68b6 (TRT: 0:01:57.827642) * Error installing a tool (after 0:00:04.991533 seconds)! Name: deseq2,owner: iuc, revision: 8702e49e68b6, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (5/10) Installing tool htseq_count from lparsons to section "RNA-Seq" at revision a6dcb86af112 (TRT: 0:02:03.041254) * Error installing a tool (after 0:00:03.671927 seconds)! Name: htseq_count,owner: lparsons, revision: a6dcb86af112, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (6/10) Installing tool htseq_count from lparsons to section "RNA-Seq" at revision 6f920f33c5eb (TRT: 0:02:06.931447) Tool htseq_count installed successfully (in 0:04:57.743878) at revision 6f920f33c5eb (7/10) Installing tool all_cufflinks_tool_suite from devteam to section "RNA-Seq" at revision d19a1295d99b (TRT: 0:07:04.933382) * Error installing a tool (after 0:00:06.615792 seconds)! Name: all_cufflinks_tool_suite,owner: devteam, revision: d19a1295d99b, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (8/10) Installing tool tophat2 from devteam to section "RNA-Seq" at revision 758594ed0364 (TRT: 0:07:11.768638) Tool tophat2 installed successfully (in 0:01:09.319561) at revision 758594ed0364 (9/10) Installing tool trinity from iuc to section "RNA-Seq" at revision 831abd20e690 (TRT: 0:08:21.314686) * Error installing a tool (after 0:00:08.002569 seconds)! Name: trinity,owner: iuc, revision: 831abd20e690, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} (10/10) Installing tool sailfish from bgruening to section "RNA-Seq" at revision 1b4ed566a41c (TRT: 0:08:29.534125) Tool sailfish installed successfully (in 0:00:02.506058) at revision 1b4ed566a41c Installed tools (4): [('trim_galore', None), ('htseq_count', None), ('tophat2', None), ('sailfish', None)] Skipped tools (0): [] Errored tools (6): [('suite_samtools_1_2', ''), ('deseq2', ''), ('deseq2', ''), ('htseq_count', ''), ('all_cufflinks_tool_suite', ''), ('trinity', '')] All tools have been processed. Total run time: 0:08:32.041138 The command '/bin/sh -c install-tools $GALAXY_ROOT/tools.yaml' returned a non-zero code: 1 Any ideas? I was hoping to use this as a base image and then add my own customizations. Thanks Stephen ------------------------------------------------------------ This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is proprietary, confidential, and exempt from disclosure under applicable law. Any unauthorized review, use, disclosure, or distribution is prohibited. If you have received this email in error please notify the sender by return email and delete the original message. Please note, the recipient should check this email and any attachments for the presence of viruses. The organization accepts no liability for any damage caused by any virus transmitted by this email. =================================

2 1

Last Call for GMOD talks at PAG
by Scott Cain 04 Nov '16

04 Nov '16

Time is short! If you want to attend PAG <http://www.intlpag.org/> and would like to present on a topic that would be of interest to the GMOD community, please send an abstract or at least a descriptive title to help(a)gmod.org. Types of talks typically include updates on GMOD software projects, usage stories for successful sites, proposals for new GMOD projects and descriptions of plugins for existing GMOD software projects like Tripal <http://gmod.org/wiki/Tripal>, JBrowse <http://gmod.org/wiki/JBrowse> and Galaxy <http://gmod.org/wiki/Galaxy>. Please consider giving a talk and sharing your experience and ideas! Scott -- ------------------------------------------------------------------------ Scott Cain, Ph. D. scott at scottcain dot net GMOD Coordinator (http://gmod.org/) 216-392-3087 Ontario Institute for Cancer Research

1 0

Re: [galaxy-dev] galaxy - velvet devteam revision 08256557922f
by Fernandez Edgar 03 Nov '16

03 Nov '16

Hey everyone, I haven’t got a feedback from you. If you have any suggestions, I’m a little bit confused with this problem. Thank you very much! Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-210 • Bur. : 1-514-343-6111 poste 16568 From: Fernandez Edgar Sent: October-31-16 11:40 AM To: galaxy-dev(a)bx.psu.edu Subject: [galaxy-dev] galaxy - velvet devteam revision 08256557922f Hey Guys, I need your help understanding what’s happening with velvet devteam revision 08256557922f. After installing it, velvet was giving me the following error: galaxy.jobs.handler WARNING 2016-10-20 12:15:11,986 (1599) Tool 'toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.0' removed from tool config, unable to run job galaxy.jobs.handler ERROR 2016-10-20 12:15:12,165 (1599) Error checking job readiness galaxy.jobs.handler ERROR 2016-10-20 12:15:12,165 (1599) Error checking job readiness I realized velvet needed that galaxy was restarted. Now, velvet finishes his jobs (becomes green) however, the following message is on galaxy’s Job output not returned from cluster And you can see in the logs of the handler: galaxy.jobs DEBUG 2016-10-31 10:17:08,548 (1669) Working directory for job is: /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669 galaxy.jobs.handler DEBUG 2016-10-31 10:17:08,567 (1669) Dispatching to drmaa runner galaxy.jobs DEBUG 2016-10-31 10:17:08,621 (1669) Persisting job destination (destination id: torque) galaxy.jobs.runners DEBUG 2016-10-31 10:17:08,630 Job [1669] queued (63.397 ms) galaxy.jobs.handler INFO 2016-10-31 10:17:08,652 (1669) Job dispatched galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Building dependency shell command for dependency 'velvet' galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Find dependency velvet version 1.2.10 galaxy.tools.deps DEBUG 2016-10-31 10:17:08,896 Resolver tool_shed_packages returned <galaxy.tools.deps.resolvers.galaxy_packages.GalaxyPackageDependency object at 0x7f80f807bd50> (isnull? False) galaxy.jobs.command_factory INFO 2016-10-31 10:17:08,945 Built script [/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh] for tool command[PACKAGE_BASE=/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f; export PACKAGE_BASE; . /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f/env.sh; velveth 2>&1 | grep "Version" | sed -e 's/Version //' > /home/galaxy/galaxy-prod/database/tmp/GALAXY_VERSION_STRING_1669 2>&1; python /home/galaxy/galaxy-prod/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/velvet/08256557922f/velvet/velveth_wrapper.py '/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat' '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/dataset_4306_files' 21 -fasta -short /home/galaxy/galaxy-prod/database/files/003/dataset_3537.dat] galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Building dependency shell command for dependency 'samtools' galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Find dependency samtools version None galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver tool_shed_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps WARNING 2016-10-31 10:17:09,145 Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners DEBUG 2016-10-31 10:17:09,153 (1669) command is: mkdir -p working; cd working; /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh; return_code=$?; cd '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669'; python "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/set_metadata_KQTExg.py" "/home/galaxy/galaxy-prod/database/tmp/tmpLU0HQy" "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/working/galaxy.json" "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_in_HistoryDatasetAssociation_4437_kIYXWa,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_kwds_HistoryDatasetAssociation_4437_ArCXyb,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_out_HistoryDatasetAssociation_4437_l28qWr,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_results_HistoryDatasetAssociation_4437_oTDjOd,/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_override_HistoryDatasetAssociation_4437_8VzdJw" 5242880; sh -c "exit $return_code" galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,202 (1669) submitting file /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/galaxy_1669.sh galaxy.jobs.runners.drmaa INFO 2016-10-31 10:17:09,210 (1669) queued as 1573.esilbac3a.esi.umontreal.ca galaxy.jobs DEBUG 2016-10-31 10:17:09,214 (1669) Persisting job destination (destination id: torque) galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,642 (1669/1573.esilbac3a.esi.umontreal.ca) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:14,848 (1669/1573.esilbac3a.esi.umontreal.ca) state change: job finished normally galaxy.jobs.runners ERROR 2016-10-31 10:17:24,906 (1669/1573.esilbac3a.esi.umontreal.ca) Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/galaxy_1669.o' galaxy.model.metadata DEBUG 2016-10-31 10:17:25,314 loading metadata from file for: HistoryDatasetAssociation 4437 galaxy.jobs INFO 2016-10-31 10:17:25,541 Collecting metrics for Job 1669 galaxy.jobs DEBUG 2016-10-31 10:17:25,554 job 1669 ended (finish() executed in (636.363 ms)) galaxy.model.metadata DEBUG 2016-10-31 10:17:25,560 Cleaning up external metadata files Do you have an idea why I’m getting this problem? Best regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-210 • Bur. : 1-514-343-6111 poste 16568

1 0

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown
by Peter Cock 03 Nov '16

03 Nov '16

Good news - this is the bug I though it was, the ampersand should be escaped as & in the XML file: https://github.com/galaxyproject/galaxy/issues/3084 More good news: This will likely be fixed in the next release of Galaxy: https://github.com/galaxyproject/galaxy/pull/3122 Right now I suggest you manually edit these XML files to use the word "and" instead of "&". Peter On Thu, Nov 3, 2016 at 2:36 PM, Yip, Miu ki <myip(a)cshl.edu> wrote: > Hi Peter, > > Line 44 is oddly specific. Nevertheless, we have this as the 44th line in the xml file. > > <label id="cshl_rnaseq_proc_qc" text="Post-Mapping Processing & QC" version="" /> > > This tool is not the one trying to be installed. > > Thanks, > Miuki > On Nov 3, 2016, at 10:28 AM, Peter Cock <p.j.a.cock(a)googlemail.com<mailto:p.j.a.cock@googlemail.com>> wrote: > > I think you may have hit the same bug I found recently: > > https://github.com/galaxyproject/galaxy/issues/3084 > > What is line 44 of your integrated_tool_panel.xml file? > > Peter > > On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki <myip(a)cshl.edu> wrote: > Hi all, > > While trying to install tools from the toolshed via the browser interface, an error keeps appearing and nothing is abled to be installed. Currently using the latest version of Galaxy and having tried to install several different tools, the following error keeps appearing in the log file. > > galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 'main' > Exception in thread Thread-2: > Traceback (most recent call last): > File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner > self.run() > File "/usr/lib64/python2.7/threading.py", line 764, in run > self.__target(*self.__args, **self.__kwargs) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 147, in on_any_event > self._handle(event) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 150, in _handle > self.reload_callback() > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", line 78, in <lambda> > self._tool_conf_watcher = get_tool_conf_watcher( lambda: reload_toolbox(app)) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 56, in reload_toolbox > app.toolbox = _get_new_toolbox(app) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 72, in _get_new_toolbox > new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, app.toolbox._tool_conf_watcher) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 110, in __init__ > tool_conf_watcher=tool_conf_watcher > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", line 1037, in __init__ > super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, app, tool_conf_watcher=tool_conf_watcher) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", line 69, in __init__ > self._init_integrated_tool_panel( app.config ) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py", line 36, in _init_integrated_tool_panel > self._load_integrated_tool_panel_keys() > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", line 368, in _load_integrated_tool_panel_keys > tree = parse_xml( self._integrated_tool_panel_config ) > File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 211, in parse_xml > root = tree.parse( fname, parser=ElementTree.XMLParser( target=DoctypeSafeCallbackTarget() ) ) > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse > parser.feed(data) > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed > self._raiseerror(v) > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror > raise err > ParseError: not well-formed (invalid token): line 44, column 71 > > Is there an XML or configuration file which needs to be changed for this to work? Not sure how to proceed. > > Thanks! > Miuki > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >

2 1