May 2014 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker)
by Kandalaft, Iyad 28 May '14

28 May '14

Hey Eric, The connection string dialect is as follows for TCP/IP connections: dialect+driver://username:password@host:port/database So, the host=/tmp is not necessary when are specifying a hostname (resolves to the IP) and a port because that is referring to UNIX sockets. If you were using SQL Alchemy with a unix socket to connect to the postgres server, then it would look something like this. "postgresql+psycopg2://user:password@/dbname?host=/var/lib/postgresql" Note that there's no hostname and port. Just the socket is specified. See http://docs.sqlalchemy.org/en/rel_0_9/core/engines.html for more info. Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228 ________________________________________ From: galaxy-dev-bounces(a)lists.bx.psu.edu [galaxy-dev-bounces(a)lists.bx.psu.edu] on behalf of galaxy-dev-request(a)lists.bx.psu.edu [galaxy-dev-request(a)lists.bx.psu.edu] Sent: May 27, 2014 12:00 PM To: galaxy-dev(a)lists.bx.psu.edu Subject: galaxy-dev Digest, Vol 95, Issue 25 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.bx.psu.edu To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.bx.psu.edu You can reach the person managing the list at galaxy-dev-owner(a)lists.bx.psu.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 2. Changes in admin menu - Upload files (Julien Daligault) 3. Re: Changes in admin menu - Upload files (Dannon Baker) 4. Re: New tool on TestToolShed still not tested (Greg Von Kuster) 5. Re: New tool on TestToolShed still not tested (Peter Cock) 6. Re: Main ToolShed wrong report: Repository does not have a test-data directory. (Greg Von Kuster) 7. Re: Installation failure on Test Tool Shed (Greg Von Kuster) 8. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 9. Re: ToolShed: Uploaded archives can only include regular directories and files (Greg Von Kuster) 10. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Peter Cock) 11. Re: ToolShed: Uploaded archives can only include regular directories and files (Peter Cock) 12. Re: Old Tool Shed URL http://community.g2.bx.psu.edu/ dead (Greg Von Kuster) 13. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 14. Uploading files to galaxy from a folder (Kandalaft, Iyad) 15. complex help for conditional param (Jun Fan) 16. Re: complex help for conditional param (Peter Cock) 17. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) 18. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Paniagua, Eric) 19. Re: Problem using Galaxy with a PostgreSQL database on a remote host (Dannon Baker) ---------------------------------------------------------------------- Message: 1 Date: Tue, 27 May 2014 07:40:13 -0400 From: Dannon Baker <dannon.baker(a)gmail.com> To: "Paniagua, Eric" <epaniagu(a)cshl.edu> Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: Re: [galaxy-dev] Problem using Galaxy with a PostgreSQL database on a remote host Message-ID: <CAGn_WZn7WkihVYidHLOvL0J4cbU_xEeM02N1fKGGo8bcnYczCA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hey Eric, It looks like you have connection info for both tcp/ip connections and unix sockets in the connection strings. If you're logging in using "psql -h wigserv5.cshl.edu <snip>", then you only want the tcp/ip connection info. Drop the ?host=tmp off the third option you listed and I think you'll be up and running, so: postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric -Dannon On Sat, May 24, 2014 at 1:49 AM, Paniagua, Eric <epaniagu(a)cshl.edu> wrote: > Dear Galaxy Developers, > > I've been banging my head against this one for a few days now. > > I have two Galaxy instances. One resides on a server called "genomics", > which also hosts the corresponding PostgreSQL installation. The second > also resides on "genomics", but its database is hosted on "wigserv5". > > Based on the tests I just ran and code I just read, sqlalchemy (not > Galaxy) is ignoring the hostname/port part of the database_connection > string. For reference, the connection strings I've tried are: > > postgresql://glxeric:XXXXX@/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp > postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp > postgresql://glxeric:XXXXX@adgdgdfdflkhjfdhfkl/glxeric?host=/tmp > > All of these appear to result in Galaxy connecting to the PostgreSQL > installation on genomics, as determined by Galaxy schema version > discrepancies and other constraints. With each connection string, Galaxy > starts up normally. I force database activity by browsing saved histories. > It works every time. By all appearances, the second Galaxy instance is > using the PostgreSQL database hosted on "genomics", not on "wigserv5". > > All databases and roles exist, and the databases are populated. > > When I comment out the "database_connection" line in universe_wsgi.ini, I > get errors arising from the later configuration of PostgreSQL-specific > Galaxy options, as expected. > > I can connect to the database server on "wigserv5" using "psql -h > wigserv5.cshl.edu -d glxeric -U glxeric" from the server "genomics". > > Have you ever observed this behavior from Galaxy or sqlalchemy? > > Thanks, > Eric > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >

2 1

complex help for conditional param
by Jun Fan 28 May '14

28 May '14

Hi all, In the conditional param, when sub-element allows you to write specific parameters for a particular value selected. I am wondering how to write the help context according to the value selection. For example, if there is a conditional param called blast_type, when "blastn" is selected, the help context will be "blast dna against dna" and when "blastp" is selected, the help context will be "blast protein against protein". Basically this selection-based help text provides a general description of the selection, which could help to set up the corresponding parameters. By the way, is there any document about formatting text in the help element? Best regards! Jun

2 3

RFC: Citations for tools
by Eric Rasche 28 May '14

28 May '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'd want to open up discussion on a feature I'd like to see. I'll try and implement it if I can find time this summer. I didn't see a trello card for anything like this yet, but please feel free to direct me there if I missed it. I'd like to see citations as a part of every tool. This would happen in the form of a <citation> block in the XML, which would contain sub-elements with text. These sub-elements could be based off of BibTeX, since they have existing specs for citing things: https://en.wikipedia.org/wiki/BibTeX These citations would then be accessible in the HTML generated tool pages, or via a View/Download button somewhere on the tool page. By storing as an XML tree, we could render these citations as BibTeX entries for the LaTeX users, and I believe there are ways to convert BibTeX to EndNote XML and so on. This could be extended for use in workflows so that when you run workflows, somehow a list of citations for all tools used could be generated. Anyone have thoughts or opinions on this? Using the example bibtex entry from the wikipedia page: @Book{abramowitz+stegun, author = "Milton {Abramowitz} and Irene A. {Stegun}", title = "Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables", publisher = "Dover", year = 1964, address = "New York", edition = "ninth Dover printing, tenth GPO printing" } I imagine it'd look like the following in a real-life tool: <tool> ... <citation type="book"> <author>Milton Abramowitz and Irene A. Stegun"</author> <title>Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables</title> <publisher>Dover</publisher> <year>1964</year> <address>New York</address> <edition>ninth Dover printing, tenth GPO printing</edition> </citation> </tool> Cheers, Eric - -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr(a)tamu.edu rasche.eric(a)yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAEBAgAGBQJThMEUAAoJEMqDXdrsMcpVe9AP/1BjPbP5JdK6KDybOeV2ElvC jvmUGetAjdQzkKO1Ikeuxb46yp0j4abAGGG92AccxlBYALsT3jsPv5dYjm505vcU IwlfTBE7gc5y19x3zx1CuAd7PB11tryODz2LwXNKI75f39bNdX6Qe5aA74Vn/k8a FBeFL/EvkZwI8Z6CvuTGtZrEHvDXVvTFoxMX875f59g2vNy7W8Fh5wQCQ8qgiogi 0SWGIGZ6OJdqX60h2hOjJ06NhzBcTsjQea+AwTo5wAkyGPFJy4DV78jKOvsd5PMG qkk4U6gBcmcFolCeItlLPa/C4uiyOK2wVWHyTnEeKjdpD6dwLuemRmwtZrModbQu PPPFkKzQhUnh5Wgq8TIP4kSc2t1/jSpdV8MHqM0piqOFnR52fExPrwLn82WEFcp8 tXfHHTaZj+6gL+IGswdAgqvxk1V+6QTjH57igWmYtcENpIDbhZVBh+HOD8SS2qdv ohwFg/Vn8VrCeTGGehNSBvkam0HXjXXq7mZ8W4xrOS8vxv8HPAFZHze2sPlUG/Ob WeOdiLgD2HCH95SXYDDBNFG6HwFkLETA3DjpfvgHqPdaLmEfIZW+ks1WF9RG+Pmo jhbWH8OmtTk5A/Zh63uRowMaDe7QSCruPIfIwlrgqLozOj8htMyJPBIW9oZGGkqC 7Vu5UFeL8gcigRmQv5NA =ceAo -----END PGP SIGNATURE-----

8 11

Problem using Galaxy with a PostgreSQL database on a remote host
by Paniagua, Eric 27 May '14

27 May '14

Dear Galaxy Developers, I've been banging my head against this one for a few days now. I have two Galaxy instances. One resides on a server called "genomics", which also hosts the corresponding PostgreSQL installation. The second also resides on "genomics", but its database is hosted on "wigserv5". Based on the tests I just ran and code I just read, sqlalchemy (not Galaxy) is ignoring the hostname/port part of the database_connection string. For reference, the connection strings I've tried are: postgresql://glxeric:XXXXX@/glxeric?host=/tmp postgresql://glxeric:XXXXX@wigserv5.cshl.edu/glxeric?host=/tmp postgresql://glxeric:XXXXX@wigserv5.cshl.edu:5432/glxeric?host=/tmp postgresql://glxeric:XXXXX@adgdgdfdflkhjfdhfkl/glxeric?host=/tmp All of these appear to result in Galaxy connecting to the PostgreSQL installation on genomics, as determined by Galaxy schema version discrepancies and other constraints. With each connection string, Galaxy starts up normally. I force database activity by browsing saved histories. It works every time. By all appearances, the second Galaxy instance is using the PostgreSQL database hosted on "genomics", not on "wigserv5". All databases and roles exist, and the databases are populated. When I comment out the "database_connection" line in universe_wsgi.ini, I get errors arising from the later configuration of PostgreSQL-specific Galaxy options, as expected. I can connect to the database server on "wigserv5" using "psql -h wigserv5.cshl.edu -d glxeric -U glxeric" from the server "genomics". Have you ever observed this behavior from Galaxy or sqlalchemy? Thanks, Eric

2 8

ToolShed: Uploaded archives can only include regular directories and files
by Peter Cock 27 May '14

27 May '14

Hi Greg, Dave, I just tried to upload an updated tar-ball to the Test ToolShed and got a red error message: <quote> Uploaded archives can only include regular directories and files (no symbolic links, devices, etc). Offender: </quote> Note no offending files were listed. When I double checked I had prepared my tar-ball from the wrong source directory and it did contain symlinks, but the Tool Shed error message failed to list these. Regards, Peter

2 2

Installation failure on Test Tool Shed
by Peter Cock 27 May '14

27 May '14

Hi Dave, Can you tell me any more about this failed tool dependency error: http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira4_assembler Installation errors Tool dependencies TypeNameVersion MIRA package 4.0 Error File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/install_util.py", line 345, in install_and_build_package_via_fabric tool_dependency = fabric_util.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py", line 93, in install_and_build_package initial_download=False ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 418, in execute_step initial_download=initial_download ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 1312, in execute_step return_output=False ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 247, in handle_command output = self.handle_complex_command( command ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 287, in handle_complex_command cwd=state.env[ 'lcwd' ] ) File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpxFCguW/tmp-toolshed-mtdt0b857/MIRA' I presume this was a Python stack trace but the whitespace has been lost when rendered as HTML. However, I don't quite see what is going wrong here... Thanks, Peter

2 3

Main ToolShed wrong report: Repository does not have a test-data directory.
by Peter Cock 27 May '14

27 May '14

Hi guys, I just noticed two of my tools on the main ToolShed were listed under "Latest revision: missing tool tests" unexpectedly. http://toolshed.g2.bx.psu.edu/view/peterjc/sample_seqs Test runs 2014-05-25 15:12:04 Automated test environment Tools missing tests or test data Tool id: sample_seqs Tool version: 0.0.1 Tool guid: toolshed.g2.bx.psu.edu/repos/peterjc/sample_seqs/sample_seqs/0.0.1 Missing components: Repository does not have a test-data directory. 2014-05-13 01:43:51 Automated test environment Tests that passed successfully Successful installations 2014-05-11 01:38:18 Automated test environment Tests that passed successfully Successful installations 2014-05-09 01:43:58 Automated test environment Tests that passed successfully Successful installations 2014-05-07 01:55:08 Automated test environment Tests that passed successfully Successful installations And, http://toolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss Test runs 2014-05-25 15:10:53 Automated test environment Tools missing tests or test data Tool id: get_orfs_or_cdss Tool version: 0.0.5 Tool guid: toolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5 Missing components: Repository does not have a test-data directory. 2014-05-13 05:11:12 Automated test environment Tests that passed successfully Successful installations 2014-05-11 05:06:15 Automated test environment Tests that passed successfully Successful installations 2014-05-09 05:17:53 Automated test environment Tests that passed successfully Successful installations 2014-05-07 05:52:29 Automated test environment Tests that passed successfully Successful installations In both cases the most recent test run (2014-05-25) has failed in some way, and the ToolShed *wrongly* reports the repository does not have a test-data directory. Regards, Peter

2 1

Changes in admin menu - Upload files
by Julien Daligault 27 May '14

27 May '14

Hi, I manage a galaxy instance on a local server and I want that users use only the "filesystem paths" to create a link to their data by the Admin Menu. Where can I find the source file to modify the web page in order to appear directly this option in first in the drop-down menu ? It will avoid to load directly big data on our computing machine. Thanks Julien -- Julien Daligault Plateforme Génomique-Transcriptomique (583B) INSTITUT JACQUES MONOD CNRS-Université Paris Diderot Bât. Buffon - 15 rue Hélène Brion 75205 Paris Cedex 13 - France tel : +33 1 57 27 81 31 daligault(a)ijm.univ-paris-diderot.fr

2 1

Galaxy queue
by Neil.Burdett＠csiro.au 26 May '14

26 May '14

Hi, is there anyway to find out (from the command line or api) what is on the Galaxy queue? When items are queued they appear in grey or processing in yellow can I get hold of these details from the command line as I could then email the user who has just submitted a job with his place on queue and expected wait time. Thanks Neil

3 3

uninstalling tools via web api
by Evan Bollig 23 May '14

23 May '14

I notice that the galaxy web api for installing tools from the toolshed does not include any entrypoint for uninstalling tools. This must be intentional, but what was the motivation to leave this out? Cheers, -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan(a)msi.umn.edu boll0107(a)umn.edu

2 3