- galaxy-dev - lists.galaxyproject.org

Cluster Admins - Call for experiences/issues encountered
by Eric Rasche 02 Jun '14

02 Jun '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is specifically to galaxy administrators and developers who have worked with galaxy and cluster deployment. If you have successfully deployed a galaxy instance backed by one (or more) clusters, I'm looking to hear about your experience; - - the configuration - - any pitfalls encountered - - any extra tools that needed to be installed - - any ugly hacks you had to implement The current cluster documentation is somewhat lacking, and I'm willing to compile everyone's experiences deploying to clusters if you're willing to send them in. I'll update the cluster page on the wiki as I receive replies and can allocate time. By way of example, here's the summary of what I would reply to this email with my experiences deploying to an HTCondor cluster - - HTcondor ran fine, accepted jobs and worked for all users - - Connecting galaxy to it would result in some jobs running/some jobs failing. It was determined that those jobs which ran, were only on localhost. - - This was a result of the fact that galaxy's home directory/data was not exported - - Galaxy was made an LDAP user and a folder created in the correct location on all computers - - This still failed, as galaxy does not transfer files by default (as far as I can tell), even though that's available in HTCondor - - Finally galaxy's home directory was NFS exported and things seemed to work. Cheers, Eric - -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr(a)tamu.edu rasche.eric(a)yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAEBAgAGBQJThL4PAAoJEMqDXdrsMcpVJ0wP/jPFrTxRYnWcavbUxhht8tWE VDhid2pst7o+qkkLAUylPyNzJz31movsU1Gd2FJLP4+RHb3lhy2GWm8mR9G4JuMM g7fHAwIRj1qTT5tMXAEN6EoCtrValZH0EkIQPaFUPIiN1wzGTTDP56V+L6f3xQNg zglk2GTO+hM+AunYGuDugDvdKTPyKgR1K2vaCNdc2kZlvQZe7TlOHZEbXaoDGTY4 vwHcNR0+8fvf6Hs1liCuCuh8nlTBABZLxMl5LgxOaZqbaAVQWsqXNdh1tF/rlERL +PlNv5HRBWz26iGE04jbAD1pKmxg33Knl3l4MXrUnB5ZJ2dIDJL0YHOhXM3mjI4T Hf8mv/iVr21xx3zrrAbAlRIt3Npw4DtaGSqBFdKvnxQZygjKEF7ZKTGfA4xLNRHh 4dPElUpv8b3jUe2GqhXyrmHYuBdWqBC5Y/ii0k4c/bHjoXW6yg0mPs7po7i5+5JS wnO3mfoFhnyEALd45EBprkQY+kx05zjSg4ntpeywl8fIM2uGT0GyANA0/prKhcmB 5PiYUejsaDTY4GigiiZqz0tnC5z7VDyjByfyl7ufNaaCWNRXlTNIzH0ePy0ZjLrU Sj8XyI1XoOSdtyxxvNNe8OPDxxffnJxi0rres4Q3CC/EbZK4vBF7chsisHd28Q29 M1e+lNdO8FBIoy5+MR3H =6Jbk -----END PGP SIGNATURE-----

2 3

stdout in history
by Neil.Burdett＠csiro.au 02 Jun '14

02 Jun '14

Hi, It seems that the first 4/5 lines that are printed in the source code tools appear in the history (when expended) and in stdout link. Is there anyway to stop "print" statements appearing in the history panel? Thanks Neil

4 3

tool dependencies fail
by Isabelle Phan 31 May '14

31 May '14

Hello, I'm following "Managed Tool Dependencies" to the letter and still can't get Galaxy to find the executable. Is this page https://wiki.galaxyproject.org/Admin/Config/ToolDependencies up to date? If yes, what am I doing wrong? Any help greatly appreciated! # universe_wsgi.ini # I also tried the absolute path, it made no difference tool_dependency_dir = tool_dependencies # tool_dependencies is set like instructed: $ ls tool_dependencies/hhsuite/ 2.0.16/ default@ $ ls tool_dependencies/hhsuite/2.0.16/bin ffindex_build* ffindex_get* hhalign* hhblits* hhconsensus* hhfilter* hhmake* hhsearch* My tool: <tool id="hhpred_hhblits_01" name="Create MSA from single fasta for hhsearch" hidden="false" version="1.0.0" force_history_refresh="true"> <requirement type="package">hhsuite</requirement> <description>Searches single fasta iteratively against hmm db to build a MSA</description> <command>hhblits -i $input_file -d $database -oa3m $outfile</command> etc... Galaxy throws Error: hhblits not found. I've tried setting the PATH in the tool_dependencies/hhsuite/2.0.16/env.sh: #!/bin/bash # configure PATH to hhsuite binaries PATH="/opt/galaxy-dist/tool_dependencies/hhsuite/default/bin:$PATH" export PATH still getting hhblits not found :-( What am I doing wrong? Isabelle

3 2

stdout in history
by Neil.Burdett＠csiro.au 31 May '14

31 May '14

Yes that would work. Can you supply an example if possible please? Thanks Neil Message: 3 Date: Fri, 30 May 2014 15:04:47 +0200 From: Wolfgang Maier <wolfgang.maier(a)biologie.uni-freiburg.de> To: galaxy-dev(a)lists.bx.psu.edu Subject: Re: [galaxy-dev] stdout in history Message-ID: <lm9vle$70e$2(a)ger.gmane.org> Content-Type: text/plain; charset=UTF-8; format=flowed you could just redirect stdout to /dev/null in the tool's xml <command> line ? Is that an option for you ? Cheers, Wolfgang On 30.05.2014 14:35, Neil.Burdett(a)csiro.au wrote: > Hi, > It seems that the first 4/5 lines that are printed in the source > code tools appear in the history (when expended) and in stdout link. Is > there anyway to stop "print" statements appearing in the history panel? > > Thanks > Neil > > ------------------------------ _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ End of galaxy-dev Digest, Vol 95, Issue 29 ******************************************

2 1

June 2014 Galaxy Update Newsletter: GCC2014 registration closes in 1 week!
by Dave Clements 30 May '14

30 May '14

Hello all, The June 2014 Galaxy Update Newsletter <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06> is out! There's a lot going on in the project and the community right now. The big news for the coming month is - *GCC2014 registration closes June 6! <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#GCC2014:_June_30_-_Jul…>* - (OK, that's a lie, but after June 6, late registration rates kick in, and you really don't want to pay those.) - *GCC2014 starts June 30 <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#GCC2014:_June_30_-_Jul…>!* - *The first ever GCC Hackathon starts June 28 <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#Galaxy_Hackathon_at_GC…>!* - After a very long and productive life, the *Galaxy-User mailing list is retiring on June 6* <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#Galaxy-User_Being_Reti…>. (Don't worry. It's been replaced by Galaxy Biostar <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#Galaxy-User_Being_Reti…> .) There are also at least 14 other Galaxy related events <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#Other_Events> in the next two months in Thailand, Canada, France, the United States, Italy, the Netherlands, Australia, and Brazil. And, in the past month there were: - 49 new papers <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#New_Papers> - One new public server (deepTools) <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#New_Public_Servers> - One new Galaxy deployment description (deepTools) <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#Galaxy_Community_Hubs> - and a stunning number of ToolShed contributions <https://wiki.galaxyproject.org/GalaxyUpdates/2014_06#ToolShed_Contributions> Please let us know if you have anything to include in the July newsletter, and we will see you in Baltimore! Dave Clements and the Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/GCC2014 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Data Libraries are slow
by Kandalaft, Iyad 29 May '14

29 May '14

Hello everyone I have a lot of sequencing data that I wanted to load into one Galaxy data library. I enabled loading the data by filesystem path and successfully "linked to the files" rather than copying them into galaxy. Unfortunately, as I add more folders and data files, the library takes longer and longer to load. At this point, the apache proxy just times out before galaxy returns any data. Should I have more data libraries and fewer files within them? Is there a reason for not implementing paging in the data libraries? The web server hosting galaxy shows only 80% loads 30473 galaxy 20 0 1070m 443m 5160 S 81.6 1.4 9:59.89 python The database server is at 13% load (kind of low) 20890 mysql 20 0 3762m 551m 9104 S 13.4 0.9 796:16.69 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-bin=/var/log/mysql/bin.log --slow-query-log-file=/var/log/mysql/slow-queries.log Regards, Iyad Kandalaft Bioinformatics Application Developer Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada KW Neatby Bldg | éd. KW Neatby 960 Carling Ave| 960, avenue Carling Ottawa, ON | Ottawa (ON) K1A 0C6 E-mail Address / Adresse courriel: Iyad.Kandalaft(a)agr.gc.ca<mailto:Iyad.Kandalaft@agr.gc.ca> Telephone | Téléphone 613- 759-1228 Facsimile | Télécopieur 613-759-1701 Government of Canada | Gouvernement du Canada

2 1

Old Tool Shed URL http://community.g2.bx.psu.edu/ dead
by Peter Cock 28 May '14

28 May '14

Hello all, Either I made a typo in some old README files, or the old URL for the Tool Shed http://community.g2.bx.psu.edu/ has stopped working. Could this redirect to the current Tool Shed address http://toolshed.g2.bx.psu.edu/ please? Thanks, Peter

3 6

Re: [galaxy-dev] Changes in admin menu - Upload files
by Kandalaft, Iyad 28 May '14

28 May '14

Julien, You have a couple of options. Maybe the file you are interested in is the upload tool, which can control if uploads get rejected or not. This is in tools/data_source/upload.py & .xml However, the file you are looking for is ./templates/webapps/galaxy/library/common/common.mako <-- this provides the drop down menu that you want to modify <div class="form-row"> <label>Upload option:</label> <div class="form-row-input"> ${upload_option_select_list.get_html()} </div> <div class="toolParamHelp" style="clear: both;"> Choose upload option (file, directory, filesystem paths, current history). </div> <div style="clear: both"></div> </div> It seems to call upload_option_select_list.get_html() to get the list, which is defined in lib/galaxy/webapps/galaxy/controllers/library_common.py Hope that helps get you started. Regards, Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228 ________________________________ From: Julien Daligault [daligault(a)ijm.univ-paris-diderot.fr] Sent: May 28, 2014 10:29 AM To: Kandalaft, Iyad Subject: Re: [galaxy-dev] Changes in admin menu - Upload files Hi Iyad, We have a SERVER A composed of computing machine with galaxy instances and a storage machine where data are upload. Galaxy instance are proposed on virtual machine, so each team of our institute have is own galaxy instance. To be sure that people upload data on the storage machine and then create a link by filesystem path in galaxy (and not upload directly data on galaxy, so on computing machine), I want that upload option "Upload files from filesystem paths" appears directly when we use Manage data libraries<https://integraseq.mti.univ-paris-diderot.fr/galaxy60/library_admin/browse_…> in the Admin menu. I tried to change "home/galaxy/lib/galaxy/webapps/galaxy/controllers/library_common.py" and "/home/galaxy/lib/galaxy/model/__init__.py" to do that but it is not evident "Upload files" option appears always in first. I just arrived to put "Link to files without copying into Galaxy in first in the Copy data into Galaxy? option by changing "/home/galaxy/templates/library/common/common.mako" file. If you have a solution, it will be wonderful Regards, Julien Julien Daligault Plateforme Génomique-Transcriptomique (583B) INSTITUT JACQUES MONOD CNRS-Université Paris Diderot Bât. Buffon - 15 rue Hélène Brion 75205 Paris Cedex 13 - France tel : +33 1 57 27 81 31 daligault(a)ijm.univ-paris-diderot.fr<mailto:daligault@ijm.univ-paris-diderot.fr> On 05/28/2014 03:54 PM, Kandalaft, Iyad wrote: Hi Julien Do you mean that you want users to only "upload" files to data libraries and use the "link to files" option instead of "copy files into galaxy"? Can you describe your infrastructure setup so that I can provide you with a good response. >From what I understand, you have SERVER A with a galaxy install and SERVER B is your compute machine? Galaxy is submitting jobs to SERVER B for analysis/processing? Or do you only have 1 server with Galaxy on it and it is also doing the processing? Do you have a storage array to house the data? Regards, Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca<mailto:Iyad.Kandalaft@agr.gc.ca> | (613) 759-1228 ________________________________________ From: galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> [galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu>] on behalf of galaxy-dev-request(a)lists.bx.psu.edu<mailto:galaxy-dev-request@lists.bx.psu.edu> [galaxy-dev-request(a)lists.bx.psu.edu<mailto:galaxy-dev-request@lists.bx.psu.edu>] Sent: May 27, 2014 12:00 PM To: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: galaxy-dev Digest, Vol 95, Issue 25 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.bx.psu.edu<mailto:galaxy-dev-request@lists.bx.psu.edu> You can reach the person managing the list at galaxy-dev-owner(a)lists.bx.psu.edu<mailto:galaxy-dev-owner@lists.bx.psu.edu> When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 2. Changes in admin menu - Upload files (Julien Daligault) 3. Re: Changes in admin menu - Upload files (Dannon Baker) 4. Re: New tool on TestToolShed still not tested (Greg Von Kuster) 5. Re: New tool on TestToolShed still not tested (Peter Cock) 6. Re: Main ToolShed wrong report: Repository does not have a test-data directory. (Greg Von Kuster) 7. Re: Installation failure on Test Tool Shed (Greg Von Kuster) 8. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 9. Re: ToolShed: Uploaded archives can only include regular directories and files (Greg Von Kuster) 10. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Peter Cock) 11. Re: ToolShed: Uploaded archives can only include regular directories and files (Peter Cock) 12. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 13. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 14. Uploading files to galaxy from a folder (Kandalaft, Iyad) 15. complex help for conditional param (Jun Fan) 16. Re: complex help for conditional param (Peter Cock) 17. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 18. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 19. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) ---------------------------------------------------------------------- Message: 1 Date: Tue, 27 May 2014 07:40:13 -0400 From: Dannon Baker <dannon.baker(a)gmail.com><mailto:dannon.baker@gmail.com> To: "Paniagua, Eric" <epaniagu(a)cshl.edu><mailto:epaniagu@cshl.edu> Cc: "galaxy-dev(a)lists.bx.psu.edu"<mailto:galaxy-dev@lists.bx.psu.edu> <galaxy-dev(a)lists.bx.psu.edu><mailto:galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host Message-ID: <CAGn_WZn7WkihVYidHLOvL0J4cbU_xEeM02N1fKGGo8bcnYczCA(a)mail.gmail.com><mailto:CAGn_WZn7WkihVYidHLOvL0J4cbU_xEeM02N1fKGGo8bcnYczCA@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hey Eric, It looks like you have connection info for both tcp/ip connections and unix sockets in the connection strings. If you're logging in using "psql -h wigserv5.cshl.edu <snip>", then you only want the tcp/ip connection info. Drop the ?host=tmp off the third option you listed and I think you'll be up and running, so: postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric<mailto:postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric> -Dannon On Sat, May 24, 2014 at 1:49 AM, Paniagua, Eric <epaniagu(a)cshl.edu><mailto:epaniagu@cshl.edu> wrote: Dear Galaxy Developers, I've been banging my head against this one for a few days now. I have two Galaxy instances. One resides on a server called "genomics", which also hosts the corresponding PostgreSQL installation. The second also resides on "genomics", but its database is hosted on "wigserv5". Based on the tests I just ran and code I just read, sqlalchemy (not Galaxy) is ignoring the hostname/port part of the database_connection string. For reference, the connection strings I've tried are: postgresql://glxeric:XXXXX@/glxeric?host=/tmp postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp<mailto:postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp> postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp<mailto:postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp> postgresql://glxeric:XXXXX@adgdgdfdflkhjfdhfkl/glxeric?host=/tmp All of these appear to result in Galaxy connecting to the PostgreSQL installation on genomics, as determined by Galaxy schema version discrepancies and other constraints. With each connection string, Galaxy starts up normally. I force database activity by browsing saved histories. It works every time. By all appearances, the second Galaxy instance is using the PostgreSQL database hosted on "genomics", not on "wigserv5". All databases and roles exist, and the databases are populated. When I comment out the "database_connection" line in universe_wsgi.ini, I get errors arising from the later configuration of PostgreSQL-specific Galaxy options, as expected. I can connect to the database server on "wigserv5" using "psql -h wigserv5.cshl.edu -d glxeric -U glxeric" from the server "genomics". Have you ever observed this behavior from Galaxy or sqlalchemy? Thanks, Eric ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

1 0

Changes in admin menu - Upload files
by Kandalaft, Iyad 28 May '14

28 May '14

Hi Julien Do you mean that you want users to only "upload" files to data libraries and use the "link to files" option instead of "copy files into galaxy"? Can you describe your infrastructure setup so that I can provide you with a good response. >From what I understand, you have SERVER A with a galaxy install and SERVER B is your compute machine? Galaxy is submitting jobs to SERVER B for analysis/processing? Or do you only have 1 server with Galaxy on it and it is also doing the processing? Do you have a storage array to house the data? Regards, Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228 ________________________________________ From: galaxy-dev-bounces(a)lists.bx.psu.edu [galaxy-dev-bounces(a)lists.bx.psu.edu] on behalf of galaxy-dev-request(a)lists.bx.psu.edu [galaxy-dev-request(a)lists.bx.psu.edu] Sent: May 27, 2014 12:00 PM To: galaxy-dev(a)lists.bx.psu.edu Subject: galaxy-dev Digest, Vol 95, Issue 25 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.bx.psu.edu To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.bx.psu.edu You can reach the person managing the list at galaxy-dev-owner(a)lists.bx.psu.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 2. Changes in admin menu - Upload files (Julien Daligault) 3. Re: Changes in admin menu - Upload files (Dannon Baker) 4. Re: New tool on TestToolShed still not tested (Greg Von Kuster) 5. Re: New tool on TestToolShed still not tested (Peter Cock) 6. Re: Main ToolShed wrong report: Repository does not have a test-data directory. (Greg Von Kuster) 7. Re: Installation failure on Test Tool Shed (Greg Von Kuster) 8. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 9. Re: ToolShed: Uploaded archives can only include regular directories and files (Greg Von Kuster) 10. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Peter Cock) 11. Re: ToolShed: Uploaded archives can only include regular directories and files (Peter Cock) 12. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 13. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 14. Uploading files to galaxy from a folder (Kandalaft, Iyad) 15. complex help for conditional param (Jun Fan) 16. Re: complex help for conditional param (Peter Cock) 17. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 18. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 19. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) ---------------------------------------------------------------------- Message: 1 Date: Tue, 27 May 2014 07:40:13 -0400 From: Dannon Baker <dannon.baker(a)gmail.com> To: "Paniagua, Eric" <epaniagu(a)cshl.edu> Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host Message-ID: <CAGn_WZn7WkihVYidHLOvL0J4cbU_xEeM02N1fKGGo8bcnYczCA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hey Eric, It looks like you have connection info for both tcp/ip connections and unix sockets in the connection strings. If you're logging in using "psql -h wigserv5.cshl.edu <snip>", then you only want the tcp/ip connection info. Drop the ?host=tmp off the third option you listed and I think you'll be up and running, so: postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric -Dannon On Sat, May 24, 2014 at 1:49 AM, Paniagua, Eric <epaniagu(a)cshl.edu> wrote: > Dear Galaxy Developers, > > I've been banging my head against this one for a few days now. > > I have two Galaxy instances. One resides on a server called "genomics", > which also hosts the corresponding PostgreSQL installation. The second > also resides on "genomics", but its database is hosted on "wigserv5". > > Based on the tests I just ran and code I just read, sqlalchemy (not > Galaxy) is ignoring the hostname/port part of the database_connection > string. For reference, the connection strings I've tried are: > > postgresql://glxeric:XXXXX@/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp > postgresql://glxeric:XXXXX@adgdgdfdflkhjfdhfkl/glxeric?host=/tmp > > All of these appear to result in Galaxy connecting to the PostgreSQL > installation on genomics, as determined by Galaxy schema version > discrepancies and other constraints. With each connection string, Galaxy > starts up normally. I force database activity by browsing saved histories. > It works every time. By all appearances, the second Galaxy instance is > using the PostgreSQL database hosted on "genomics", not on "wigserv5". > > All databases and roles exist, and the databases are populated. > > When I comment out the "database_connection" line in universe_wsgi.ini, I > get errors arising from the later configuration of PostgreSQL-specific > Galaxy options, as expected. > > I can connect to the database server on "wigserv5" using "psql -h > wigserv5.cshl.edu -d glxeric -U glxeric" from the server "genomics". > > Have you ever observed this behavior from Galaxy or sqlalchemy? > > Thanks, > Eric > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >

1 0

Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker)
by Kandalaft, Iyad 28 May '14

28 May '14

Hey Eric, The connection string dialect is as follows for TCP/IP connections: dialect+driver://username:password@host:port/database So, the host=/tmp is not necessary when are specifying a hostname (resolves to the IP) and a port because that is referring to UNIX sockets. If you were using SQL Alchemy with a unix socket to connect to the postgres server, then it would look something like this. "postgresql+psycopg2://user:password@/dbname?host=/var/lib/postgresql" Note that there's no hostname and port. Just the socket is specified. See http://docs.sqlalchemy.org/en/rel_0_9/core/engines.html for more info. Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228 ________________________________________ From: galaxy-dev-bounces(a)lists.bx.psu.edu [galaxy-dev-bounces(a)lists.bx.psu.edu] on behalf of galaxy-dev-request(a)lists.bx.psu.edu [galaxy-dev-request(a)lists.bx.psu.edu] Sent: May 27, 2014 12:00 PM To: galaxy-dev(a)lists.bx.psu.edu Subject: galaxy-dev Digest, Vol 95, Issue 25 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.bx.psu.edu To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.bx.psu.edu You can reach the person managing the list at galaxy-dev-owner(a)lists.bx.psu.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 2. Changes in admin menu - Upload files (Julien Daligault) 3. Re: Changes in admin menu - Upload files (Dannon Baker) 4. Re: New tool on TestToolShed still not tested (Greg Von Kuster) 5. Re: New tool on TestToolShed still not tested (Peter Cock) 6. Re: Main ToolShed wrong report: Repository does not have a test-data directory. (Greg Von Kuster) 7. Re: Installation failure on Test Tool Shed (Greg Von Kuster) 8. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 9. Re: ToolShed: Uploaded archives can only include regular directories and files (Greg Von Kuster) 10. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Peter Cock) 11. Re: ToolShed: Uploaded archives can only include regular directories and files (Peter Cock) 12. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 13. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 14. Uploading files to galaxy from a folder (Kandalaft, Iyad) 15. complex help for conditional param (Jun Fan) 16. Re: complex help for conditional param (Peter Cock) 17. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 18. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 19. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) ---------------------------------------------------------------------- Message: 1 Date: Tue, 27 May 2014 07:40:13 -0400 From: Dannon Baker <dannon.baker(a)gmail.com> To: "Paniagua, Eric" <epaniagu(a)cshl.edu> Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host Message-ID: <CAGn_WZn7WkihVYidHLOvL0J4cbU_xEeM02N1fKGGo8bcnYczCA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hey Eric, It looks like you have connection info for both tcp/ip connections and unix sockets in the connection strings. If you're logging in using "psql -h wigserv5.cshl.edu <snip>", then you only want the tcp/ip connection info. Drop the ?host=tmp off the third option you listed and I think you'll be up and running, so: postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric -Dannon On Sat, May 24, 2014 at 1:49 AM, Paniagua, Eric <epaniagu(a)cshl.edu> wrote: > Dear Galaxy Developers, > > I've been banging my head against this one for a few days now. > > I have two Galaxy instances. One resides on a server called "genomics", > which also hosts the corresponding PostgreSQL installation. The second > also resides on "genomics", but its database is hosted on "wigserv5". > > Based on the tests I just ran and code I just read, sqlalchemy (not > Galaxy) is ignoring the hostname/port part of the database_connection > string. For reference, the connection strings I've tried are: > > postgresql://glxeric:XXXXX@/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp > postgresql://glxeric:XXXXX@adgdgdfdflkhjfdhfkl/glxeric?host=/tmp > > All of these appear to result in Galaxy connecting to the PostgreSQL > installation on genomics, as determined by Galaxy schema version > discrepancies and other constraints. With each connection string, Galaxy > starts up normally. I force database activity by browsing saved histories. > It works every time. By all appearances, the second Galaxy instance is > using the PostgreSQL database hosted on "genomics", not on "wigserv5". > > All databases and roles exist, and the databases are populated. > > When I comment out the "database_connection" line in universe_wsgi.ini, I > get errors arising from the later configuration of PostgreSQL-specific > Galaxy options, as expected. > > I can connect to the database server on "wigserv5" using "psql -h > wigserv5.cshl.edu -d glxeric -U glxeric" from the server "genomics". > > Have you ever observed this behavior from Galaxy or sqlalchemy? > > Thanks, > Eric > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >

2 1