- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] MySQL or Postgres size and growth?
by Dannon Baker 23 Feb '15

23 Feb '15

Hey Dwayne, All of Galaxy's datasets are stored in flat files and are not in the database, so the size really doesn't grow that quickly. Our test server's database is on the order of a couple gigabytes with several hundred thousand jobs having been run, if that helps. -Dannon On Mon Feb 23 2015 at 12:24:06 PM McCully, Dwayne (NIH/NIAMS) [C] < dmccully(a)mail.nih.gov> wrote: > When setting up a Galaxy site, what should I expect for MySQL or > Postgres database initial size and growth? > > > > Dwayne > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/

1 0

MySQL or Postgres size and growth?
by McCully, Dwayne (NIH/NIAMS) [C] 23 Feb '15

23 Feb '15

When setting up a Galaxy site, what should I expect for MySQL or Postgres database initial size and growth? Dwayne

1 0

S3/Swift object store cache path and `extra_dir`s
by Brian Claywell 20 Feb '15

20 Feb '15

Our local instance currently uses the traditional directories under `database/` for datasets, job working directories, and temporary files. Ultimately we wish to transition to using our Swift object store for storage. We've been doing some experimentation with Galaxy's Swift backend and have run into a few issues. The first major issue we came across was Swift's 5 GB segment size limit, since the segmentation/multipart upload code is bypassed for instances of SwiftObjectStore [1]. SwiftStack support provided a patch enabling multipart uploads for Swift (PR #648) which has been working well for us so far. (Thanks, Charles!) The next issue is that the path attribute of the cache tag in object_store_conf.xml appears to be ignored. The value does get stored to self.cache_path in _parse_config_xml, but elsewhere in the file self.staging_path is used instead. Finally, adding extra_dir tags to the Swift object store config doesn't appear to do anything. Here's my object_store_conf.xml: <?xml version="1.0"?> <object_store type="hierarchical"> <backends> <object_store type="swift" id="primary" order="0"> <auth access_key="..." secret_key="..."/> <bucket name="galaxy_store"/> <connection host="tin.fhcrc.org" port="443"/> <cache path="database/object_store_cache" size="1000"/> <extra_dir type="temp" path="database/tmp"/> <extra_dir type="job_work" path="database/job_working_directory"/> </object_store> <object_store type="disk" id="secondary" order="1"> <files_dir path="database/files"/> </object_store> </backends> </object_store> The goal with the hierarchical setup above is for new datasets to be created in the primary (Swift) object store, caching to `database/object_store_cache`, while the job and temporary directories remain at `database/job_working_directory` and `database/tmp`, respectively. Existing (pre-Swift) datasets remain in `database/files` and are handled by the secondary disk store. What actually happens (after renaming self.cache_path to self.staging_path in _parse_config_xml to get the cache path working) is this: galaxy.jobs DEBUG 2015-02-06 16:07:26,615 (1) Working directory for job is: /home/bclaywel/workspace/galaxy-central/database/object_store_cache/000/1 That is, the job working directory is created directly under the cache path's hash directories. I assume temp files would probably end up there also. We're quite excited to get Galaxy and Swift working well together, and I'm more than happy to help debug and test! Cheers, Brian [1] https://bitbucket.org/galaxy/galaxy-central/src/54ed3adb6575addba47d627944e… -- Brian Claywell | programmer/analyst Matsen Group | http://matsen.fredhutch.org Fred Hutchinson Cancer Research Center

2 2

(no subject)
by Briand, Sheldon 20 Feb '15

20 Feb '15

Hi, I'm seeing the following error when trying to use upload file with a local file that is larger than 8Kb: galaxy.web.framework.decorators ERROR 2015-02-19 16:51:48,280 Uncaught exception in exposed API method: Traceback (most recent call last): File "/BigData/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py", line 135, in decorator rval = func( self, trans, *args, **kwargs) File "/BigData/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 124, in create tool = trans.app.toolbox.get_tool( payload[ 'tool_id' ] ) if 'tool_id' in payload else None File "/BigData/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py", line 448, in get_tool if tool_id in self.tools_by_id and not get_all_versions: TypeError: unhashable type: 'list' The end of the file in the upload store: GGTGGATCACAAGGTCAGGAGATCGAGACCATCCTGGTTAACATGATGAAACCCCGTGTCTACTAAAAAT ACAAAAAA-----------------------------206772567414653 Content-Disposition: form-data; name="history_id" Null I'm using nginx 1.6.2 with the upload_file module and the latest stable version of galaxy. What am I messing up? Thanks, -Sheldon Sheldon Briand NRC Research Computing Support Analyst / Operations, Science Portfolio Shared Services Canada / Government of Canada sheldon.briand(a)ssc-spc.gc.ca / Tel: 902-426-1677 CNRC Analyst de Support Informatique de Recherche/ Operations, Portefeuil des sciences Services partagés Canada /Gouvernement du Canada sheldon.briand(a)ssc-spc.gc.ca / Tél: 902-426-1677

2 2

biopython installation
by Lapointe, David 20 Feb '15

20 Feb '15

I tried installing biopython and dependancies from the toolshed but this does not seem to be functional even though these tools say INSTALLED. Is there something that I should do that I missed? I sis better to installed them in the regular python fashion? -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu

2 1

History options confirmation dialog boxes
by Jose Juan Almagro Armenteros 19 Feb '15

19 Feb '15

Hi all I have noticed that confirmation dialog boxes that appear in the history options seems to be broken. E.g., when I try to delete permanently a history, if I click cancel in the pop-up window it still deletes the history. Is this some kind of bug or is it related with the browser? Thank you! Jose

1 0

Galaxy CloudMan instance - share history broken
by Anne Pajon 19 Feb '15

19 Feb '15

Dear All, For information, here is how I solved the error I was getting when selecting from the history menu ‘Share and Publish’ on a Galaxy CloudMan instance: Internal Server Error Galaxy was unable to successfully complete your request URL: http://ec2-52-0-174-227.compute-1.amazonaws.com/history/sharing Module galaxy.web.framework.middleware.error:149 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request >> body = method( trans, **kwargs ) Module galaxy.web.framework.decorators:64 in decorator >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.history:735 in sharing >> return trans.fill_template( "/sharing_base.mako", item=history ) Module galaxy.web.framework.webapp:774 in fill_template >> return self.fill_template_mako( filename, **kwargs ) Module galaxy.web.framework.webapp:782 in fill_template_mako >> template = template_lookup.get_template( filename ) Module mako.lookup:218 in get_template >> return self._load(srcfile, uri) Module mako.lookup:283 in _load >> **self.template_args) Module mako.template:218 in __init__ >> module = self._compile_from_file(path, filename) Module mako.template:245 in _compile_from_file >> module = imp.load_source(self.module_id, path, open(path, 'rb')) IOError: [Errno 13] Permission denied: '/mnt/galaxy/galaxy-app/database/compiled_templates/sharing_base.mako.py’ ssh into the instance and then > sudo chgrp -R galaxy /mnt/galaxy/galaxy-app/database/ > sudo chown -R galaxy /mnt/galaxy/galaxy-app/database/ Hope it could be useful to someone else. Please do let me know if there is a better place for reporting these CloudMan issues. Many thanks. Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 769 631

1 0

Re: [galaxy-dev] Galaxy server and ssl
by Dannon Baker 19 Feb '15

19 Feb '15

You may have done this already, but if not -- can you try editing config/tool_sheds_conf.xml and change the two url entries to https? If this file doesn't exist, create it from the .sample and then make the modifications. -Dannon On Thu Feb 19 2015 at 8:17:32 AM Scott Szakonyi <Scott.B.Szakonyi.1(a)nd.edu> wrote: > Hi Dannon, > > There are two things I'm seeing, which seem to be independent problems. > > 1. Https/ssl issue with URLs generated in util.py. It seems the > detection of Https is not working properly, therefore the links underlying > the toolshed functionality are coming up as http rather than https. The > browser then refuses to following them because they are "mixed mode". I've > gone around this for the moment by hacking the code in util.py, but it's > possible (as I'm very new to server administration) that I'm missing some > necessary configuration item that I just don't know about. > 2. Toolshed installation is not working properly when Galaxy is run > with multiple processes. New categories are not created, and the installed > tool does not show up in user menus until the Galaxy server is restarted. > I've worked around this by running the Galaxy server in single process mode > while installing tools. > > Thanks for the link... I'll look into that and see what I can glean from > it. > > Best regards, > > Scott > > On Wed, Feb 18, 2015 at 7:14 PM, Dannon Baker <dannon.baker(a)gmail.com> > wrote: > >> Hey Scott, >> >> What weird behavior are you seeing? Paste itself isn't going to serve >> https, though it's definitely possible to configure an upstream proxy to do >> so. Are you using a setup similar to https://wiki.galaxyproject. >> org/Admin/Config/nginxProxy#SSL ? >> >> -Dannon >> >> On Wed Feb 18 2015 at 4:34:51 PM Scott Szakonyi < >> Scott.B.Szakonyi.1(a)nd.edu> wrote: >> >>> Hello all, >>> >>> Just curious if anyone has successfully implemented Galaxy with ssl and >>> standalone Paste-based processes. We're attempting to do so, and we're >>> getting some weird behavior, particularly with regards to the Toolshed. >>> Just want to know if it can be done successfully. >>> >>> Thanks! >>> >>> -- >>> Scott B. Szakonyi >>> Research Programmer >>> >>> *Center for Research Computing* >>> 107 Information Technology Center >>> Notre Dame, IN 46556 >>> http://crc.nd.edu >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> >> > > > -- > Scott B. Szakonyi > Research Programmer > > *Center for Research Computing* > 107 Information Technology Center > Notre Dame, IN 46556 > http://crc.nd.edu >

2 1

Re: [galaxy-dev] Galaxy server and ssl
by Dannon Baker 19 Feb '15

19 Feb '15

Hey Scott, What weird behavior are you seeing? Paste itself isn't going to serve https, though it's definitely possible to configure an upstream proxy to do so. Are you using a setup similar to https://wiki.galaxyproject.org/Admin/Config/nginxProxy#SSL ? -Dannon On Wed Feb 18 2015 at 4:34:51 PM Scott Szakonyi <Scott.B.Szakonyi.1(a)nd.edu> wrote: > Hello all, > > Just curious if anyone has successfully implemented Galaxy with ssl and > standalone Paste-based processes. We're attempting to do so, and we're > getting some weird behavior, particularly with regards to the Toolshed. > Just want to know if it can be done successfully. > > Thanks! > > -- > Scott B. Szakonyi > Research Programmer > > *Center for Research Computing* > 107 Information Technology Center > Notre Dame, IN 46556 > http://crc.nd.edu > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/

2 1

Galaxy server and ssl
by Scott Szakonyi 18 Feb '15

18 Feb '15

Hello all, Just curious if anyone has successfully implemented Galaxy with ssl and standalone Paste-based processes. We're attempting to do so, and we're getting some weird behavior, particularly with regards to the Toolshed. Just want to know if it can be done successfully. Thanks! -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

1 0