July 2015 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Reasons to choose psql over mysql or not
by Eric Kuyt 27 Jul '15

27 Jul '15

Hi Hans-Rudolf Thanks for the swift reply. So the (Psql > Mysql ) still stands in 2015 that's kind of a pity, installing psql is not a problem but fixing a crashed database (which never happened to us on our mysql -> psql migrated database) would still be me pushing some buttons to fix everything and I know for sure that these sysadmins are better in documenting stuff and following SLA's But a perfect running galaxy server would be concern nr 1. For your concerns about your migration. If "sh manage_db.sh upgrade" works on your new or test psql database after a galaxy upgrade I think you're safe. Eric On 27 July 2015 at 15:35, Hans-Rudolf Hotz <hrh(a)fmi.ch> wrote: > Hi Eric > > Well, there is no official statement besides the one you have quoted > below, but anyone in the Galaxy team I have spoken to recently told me > to move to PostgreSQL. > > And that's why, I've been through the painful process of migrating our > MySQL database to a PostgreSQl database. Actually, the switch for our > production server just happened this morning - and I am still anxious > whether I have done everything correctly. > > Our system admin also prefers MySQl. Hence I had to set up my own > PostgreSQL server. And now, I am running the back-ups myself. > > > For your case: why not moving the Galaxy server to the "new more > powerful environment" but still connected to the old PostgreSQL server? > I don't think the database server (unless it is sqlite) is ever the > bottleneck, is-it? > > > Regards, Hans-Rudolf > > > On 07/27/2015 02:35 PM, Eric Kuyt wrote: > > Hi All, > > > > We are thinking of putting our galaxy instance in a new more powerful > > environment. Galaxy is now on a single server containing a psql database > > which is migrated from mysql a few years ago. > > The new environment already houses a really nice backupped highly > > available mysql server. > > > > as galaxy admin I am quite reluctant of migrating back to mysql, the > > system admin is quite reluctant of installing this psql database with > > the same grade of backup and availability just for galaxy. > > > > I could only come up with the line "/PostgreSQL is much preferred since > > we've found it works better with our DB abstraction layer, SQLAlchemy > > <http://www.sqlalchemy.org/>./" from the galaxy wiki, which is a rather > > slim argument. > > > > My question now is. > > > > Is Postgres still the preferred database or is Mysql now also a safe bet. > > > > For mysql we would have to migrate back, system management would have a > > lot of experience tuning databases and a high quality instance is > > already running. > > > > For Psql a new database instance would have to be installed, less > > knowledge of the database system is available but it is the preferred > > database. > > > > Could someone point out some more pros and cons in this question? > > > > Thanks, > > > > Eric > > > > -- > > Central Veterinary Institute of Wageningen UR (CVI) > > Department of Infection Biology > > PO box 65, 8200 AB Lelystad, NL > > Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad > > > > Tel: +31-(0)320-293391 > > Fax: +31-(0)320-238153 > > E-mail: eric.kuijt(a)wur.nl <mailto:eric.kuijt@wur.nl> > > Web:http://www.cvi.wur.nl <http://www.cvi.wur.nl> > > > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > > -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

1 0

ERROR in Galaxy
by Stephanie Beck Roth 27 Jul '15

27 Jul '15

I have been receiving the following error for over an hour: The server could not complete the request. Please contact the Galaxy Team if this error persists. { "outputs": [], "implicit_collections": [], "jobs": [], "output_collections": [] } ____________________________________________________________________ Stephanie R Beck Roth, PhD Assistant Professor of Mathematics Potomac State College of WVU 101 Fort Ave. Keyser, WV 26726 (304) 788 – 7118 Stephanie.BeckRoth(a)mail.wvu.edu<mailto:Stephanie.BeckRoth@mail.wvu.edu> “If you come to a fork in the road, take it.” ~ Yogi Berra “Nothing will work unless you do.” ~Maya Angelou

2 1

Workflow editor stuck on loading workflow
by Ryan G 27 Jul '15

27 Jul '15

It looks like a recent commit since 6/26 to 15.05 is break the workflow editor. I have a pipeline I'm trying to edit. I updated my instance of Galaxy and now when I try to edit a workflow, the editor is stuck at "Loading workflow editor".

4 6

Switching history doesn't work with prefix
by Ping Luo 24 Jul '15

24 Jul '15

Hello, Our galaxy server is served under a sub-directory "/myserver". So far everything works except the "Switch to" button in "view all history". When clicking it, there is no response. From apache server access log, I see this message "GET /history/set_as_current?id=9a8f74b3b768f8f1 HTTP/1.1" 404 304 And in the apache error log, I see File does not exist: /www/htdocs/history, referer: https://xxx.xxx.xxx/myserver/history/view_multiple Since the prefix "/myserver" is not properly prepended in the equest, the request is not properly routed. It tries to retrive the history page from the document root /www/htdocs. So far, all other requests seems have a proper prefix, for example "GET /myserver/history/current_history_json HTTP/1.1" 200 1430 "GET /myserver/api/histories/7a11fc800eeb71e9/contents HTTP/1.1" 200 15876 "GET /myserver/history/view_multiple HTTP/1.1" 200 27296 "GET /myserver/static/style/base.css?v=1437576796 HTTP/1.1" 304 - Please help to resolve the issue. Thanks, Ping

3 6

Tool Versions not displaying in a predictable way in the tool menu
by Lance Parsons 22 Jul '15

22 Jul '15

1 0

"Failure when preparing job" on blastp
by Peter van Heusden 22 Jul '15

22 Jul '15

Hi there I'm trying to use the blastp tools from the ncbi_blast_tools, but every time I try and run a job I get the following error: Traceback (most recent call last): File "/net/ datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/lib/galaxy/jobs/runners/__init__.py", line 163, in prepare_job job_wrapper.prepare() File "/net/ datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/lib/galaxy/jobs/__init__.py", line 859, in prepare self.command_line, self.extra_filenames, self.environment_variables = tool_evaluator.build() File "/net/ datasrv3hs.sanbi.ac.za/cip0/software/galaxy/galaxysrv/galaxy/lib/galaxy/tools/evaluation.py", line 422, in build raise e NotFound: cannot find 'files_path' while searching for 'db_opts.histdb.files_path' This is happening when expanding the tool template. The BLAST db in in the history (thus the reference to histdb) but why can it not find this attribute? Peter

2 6

Object-Store, setting filetypes crashes Galaxy
by bjoern.gruening＠googlemail.com 21 Jul '15

21 Jul '15

Hi, I have configured to use the hierarchical object store but as soon as I try to reset the filetpye of a dataset Galaxy is crashing with: galaxy.objectstore DEBUG 2014-06-12 14:39:21,180 Using preferred backend 'files3' for creation of MetadataFile 5963 132.230.153.57 - - [12/Jun/2014:14:39:20 +0200] "POST /datasets/966f24627ef70c12/edit HTTP/1.1" 500 - " http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Fire fox/29.0" Error - <type 'exceptions.OSError'>: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' URL: http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py', line 295 in edit trans.app.datatypes_registry.set_external_metadata_tool.tool_action.execute( trans.app.datatypes_registry.set_external_metadata_tool, trans, incoming = { 'input1':data }, overwrite = False ) #overwrite is False as per existi ng behavior File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 18 in execute overwrite, history, job_params ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 79 in execute_via_app kwds = { 'overwrite' : overwrite } ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 717 in setup_external_metadata shutil.copy( dataset.metadata.get( meta_key, None ).file_name, metadata_temp.file_name ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 575 in file_name self._filename = abspath( tempfile.NamedTemporaryFile( dir = self.tmp_dir, prefix = "metadata_temp_file_" ).name ) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 454 in NamedTemporaryFile (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 235 in _mkstemp_inner fd = _os.open(file, flags, 0600) OSError: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' I have attached my object_store_conf.xml file. Thanks, Bjoern

5 7

Connection reset by peer
by COUNTOURIS Hector 21 Jul '15

21 Jul '15

Hello, i'm actually having a message of "connection reset by peer" when i'm launching a long API request (upload files only links actually, run the workflow other them and retrieve the results into a specific folder). The workflow is working fine when i proceed to an analysis file by file but when it comes to several file processing (like 10 or more), this exception appear. I'm working with Bioblend on a local instance with a postgres SGBD. I've perfom the database performance enhancement you suggest for production mode. But right know i don't have any Apache or Nginx server. Is that mandatory ? Thanks in advance for your help. Best regards. HC.

2 1

Fwd: Re: Change from Bitbucket to git
by Jochen Bick 21 Jul '15

21 Jul '15

Hej Jelle, I also thought of setting it up again from scratch... @Marius I will check again the hg diff. The update with hg update worked fine as far as I can see. So I'm not in a big hurry to change to git. Cheers Jochen On 21.07.2015 13:27, Jelle Scholtalbers wrote: > Hi Jochen, > > I recently took a probably more labour intensive approach. > > # backup! > # mv prev-installation to somewhere else > # git clone to old-location > # git checkout -b master origin/master > # and copy over every config, tool xml, tool dependency directory, > tool-data loc files from the prev-installation to the new git one > # add your own code changes if you have any that you want to keep > # ./run.sh or manage_db.sh upgrade > # check log and probably copy some more missing files > > The upside for me was that I could finally remove all the junk that our > installation has been collecting for the last couple of year. > > Cheers, > Jelle > > > On Tue, Jul 21, 2015 at 12:45 PM, Jochen Bick <jochen.bick(a)usys.ethz.ch> > wrote: > >> Thanks Marius, >> >> it does not look like it's working for me. There are to many changes in >> git diff... >> >> the git pull and cat local_changes.diff |patch -p1 -R gives me an >> endless list of differences. >> >> Cheers Jochen >> >> On 21.07.2015 11:41, Marius van den Beek wrote: >>> Hi Jochen, >>> >>> I recently did this. Instructions are here: >>> https://wiki.galaxyproject.org/Develop/SourceCode >>> >>> This has worked for me: >>> (~/galaxy-dist is my production galaxy dir) >>> cd ~ >>> git clone https://github.com/galaxyproject/galaxy.git >>> cd galaxy >>> git reset --hard 3f1cec451f6dadee52a70a14fc6f6899f7aacdae >>> cd ~/galaxy-dist >>> hg update 17115 >>> mv .hg .hg_backup >>> cp -R ~/galaxy/.git . >>> git pull (at this point I got a merge error!) >>> so I did: >>> git diff > local_changes.diff >>> cat local_changes.diff |patch -p1 -R >>> git pull >>> git checkout release_15.05 >>> >>> Best, >>> Marius >>> >>> On 21 July 2015 at 11:36, Jochen Bick <jochen.bick(a)usys.ethz.ch> wrote: >>> >>>> Hi, >>>> >>>> is it possible to change from bitbucket to git? So I would like >>>> future-update my galaxy from git instead from bitbucket. >>>> >>>> Cheers Jochen >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 Office: Tannenstrasse 1 / TAN D 6.2 8092 Zurich, Switzerland Phone +41 44 632 28 25 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

1 0

Change from Bitbucket to git
by Jochen Bick 21 Jul '15

21 Jul '15

Hi, is it possible to change from bitbucket to git? So I would like future-update my galaxy from git instead from bitbucket. Cheers Jochen

3 3