Greg,

I was mistaken. It wasn't a GET, it was a POST. Here's the POST request:

POST /admin_toolshed/prepare_for_install? includes_tools_for_display_in_tool_panel=True&encoded_repo_info_dicts=72 2fdbb60840f8caa6f7dd248a9f08b927df4325% 3A7b22746573745f6c6f6e675f757269223a205b227465737420746f20726570726f6475 6365205c2272657175657374206661696c65643a2055524920746f6f206c6f6e6720286c 6f6e676572207468616e2038313930295c22222c2022687474703a2f2f64657673686564 2e62782e7073752e6564752f7265706f732f7269636f2f746573745f6c6f6e675f757269 222c2022353836396263663736653838222c202230222c20227269636f222c207b226465 736372697074696f6e223a206e756c6c2c2022687474703a2f2f646576736865642e6278 2e7073752e6564755f5f455345505f5f7061636b6167655f70686173745f315f335f5f45 5345505f5f7269636f5f5f455345505f5f6339626465626330376566615f5f455345505f 5f54727565223a205b5b22687474703a2f2f646576736865642e62782e7073752e656475 2f222c20227061636b6167655f636c617061636b5f335f325f31222c20227269636f222c 2022323366366135663363366466222c202254727565225d5d2c2022687474703a2f2f64 6576736865642e62782e7073752e6564755f5f455345505f5f7061636b6167655f666973 6865725f305f315f345f5f455345505f5f7269636f5f5f455345505f5f66303165643539 36393339345f5f455345505f5f54727565223a205b5b22687474703a2f2f646576736865 642e62782e7073752e6564752f222c20227061636b6167655f6e756d70795f315f37222c 20227269636f222c2022353231356136303134303666222c202254727565225d5d2c2022 687474703a2f2f646576736865642e62782e7073752e6564755f5f455345505f5f706163 6b6167655f6d6174706c6f746c69625f315f325f5f455345505f5f7269636f5f5f455345 505f5f3039303562393835323961345f5f455345505f5f54727565223a205b5b22687474 703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167655f6e 756d70795f315f37222c20227269636f222c2022353231356136303134303666222c2022 54727565225d2c205b22687474703a2f2f646576736865642e62782e7073752e6564752f 222c20227061636b6167655f66726565747970655f325f34222c20227269636f222c2022 346238383437333030643731222c202254727565225d5d2c2022726f6f745f6b6579223a 2022687474703a2f2f646576736865642e62782e7073752e6564755f5f455345505f5f74 6573745f6c6f6e675f7572695f5f455345505f5f7269636f5f5f455345505f5f35383639 62636637366538385f5f455345505f5f46616c7365222c2022687474703a2f2f64657673 6865642e62782e7073752e6564755f5f455345505f5f746573745f6c6f6e675f7572695f 5f455345505f5f7269636f5f5f455345505f5f3538363962636637366538385f5f455345 505f5f46616c7365223a205b5b22687474703a2f2f646576736865642e62782e7073752e 6564752f222c20227061636b6167655f62656175746966756c736f75705f335f325f3122 2c20227269636f222c2022356231666636653736323961222c202254727565225d2c205b 22687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b61 67655f656967656e736f66745f355f305f31222c20227269636f222c2022356538306361 383230666439222c202254727565225d2c205b22687474703a2f2f646576736865642e62 782e7073752e6564752f222c20227061636b6167655f6669736865725f305f315f34222c 20227269636f222c2022663031656435393639333934222c202254727565225d2c205b22 687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167 655f67645f635f746f6f6c735f305f31222c20227269636f222c20226436656335383933 34323263222c202254727565225d2c205b22687474703a2f2f646576736865642e62782e 7073752e6564752f222c20227061636b6167655f6d6174706c6f746c69625f315f32222c 20227269636f222c2022303930356239383532396134222c202254727565225d2c205b22 687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167 655f6d656368616e697a655f305f325f35222c20227269636f222c202234346139303064 6638303964222c202254727565225d2c205b22687474703a2f2f646576736865642e6278 2e7073752e6564752f222c20227061636b6167655f6d756e6b7265735f315f305f355f34 222c20227269636f222c2022346663366163336137643736222c202254727565225d2c20 5b22687474703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b 6167655f6e6574776f726b785f315f385f31222c20227269636f222c2022633565623436 373265323163222c202254727565225d2c205b22687474703a2f2f646576736865642e62 782e7073752e6564752f222c20227061636b6167655f70686173745f315f33222c202272 69636f222c2022633962646562633037656661222c202254727565225d2c205b22687474 703a2f2f646576736865642e62782e7073752e6564752f222c20227061636b6167655f71 7569636b747265655f315f31222c20227269636f222c2022373235353762356166656639 222c202254727565225d2c205b22687474703a2f2f646576736865642e62782e7073752e 6564752f222c20227061636b6167655f7261786d6c5f375f375f36222c20227269636f22 2c2022393565623066343665656263222c202254727565225d5d7d2c207b227068617374 2f312e33223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e 675f757269222c20226e616d65223a20227068617374222c202276657273696f6e223a20 22312e33222c20227265706f7369746f72795f6f776e6572223a20227269636f222c2022 6368616e67657365745f7265766973696f6e223a2022353836396263663736653838222c 202274797065223a20227061636b616765227d2c20227261786d6c2f372e372e36223a20 7b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269222c 20226e616d65223a20227261786d6c222c202276657273696f6e223a2022372e372e3622 2c20227265706f7369746f72795f6f776e6572223a20227269636f222c20226368616e67 657365745f7265766973696f6e223a2022353836396263663736653838222c2022747970 65223a20227061636b616765227d2c20226d756e6b7265732f312e302e352e34223a207b 227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269222c20 226e616d65223a20226d756e6b726573222c202276657273696f6e223a2022312e302e35 2e34222c20227265706f7369746f72795f6f776e6572223a20227269636f222c20226368 616e67657365745f7265766973696f6e223a2022353836396263663736653838222c2022 74797065223a20227061636b616765227d2c20226d656368616e697a652f302e322e3522 3a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f757269 222c20226e616d65223a20226d656368616e697a65222c202276657273696f6e223a2022 302e322e35222c20227265706f7369746f72795f6f776e6572223a20227269636f222c20 226368616e67657365745f7265766973696f6e223a202235383639626366373665383822 2c202274797065223a20227061636b616765227d2c20226e6574776f726b782f312e382e 31223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e675f75 7269222c20226e616d65223a20226e6574776f726b78222c202276657273696f6e223a20 22312e382e31222c20227265706f7369746f72795f6f776e6572223a20227269636f222c 20226368616e67657365745f7265766973696f6e223a2022353836396263663736653838 222c202274797065223a20227061636b616765227d2c20226d6174706c6f746c69622f31 2e322e31223a207b227265706f7369746f72795f6e616d65223a2022746573745f6c6f6e 675f757269222c20226e616d65223a20226d6174706c6f746c6962222c20227665727369 6f6e223a2022312e322e31222c20227265706f7369746f72795f6f776e6572223a202272 69636f222c20226368616e67657365745f7265766973696f6e223a202235383639626366 3736653838222c202274797065223a20227061636b616765227d2c202262656175746966 756c736f75702f332e322e31223a207b227265706f7369746f72795f6e616d65223a2022 746573745f6c6f6e675f757269222c20226e616d65223a202262656175746966756c736f 7570222c202276657273696f6e223a2022332e322e31222c20227265706f7369746f7279 5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e 223a2022353836396263663736653838222c202274797065223a20227061636b61676522 7d2c202267645f635f746f6f6c732f302e31223a207b227265706f7369746f72795f6e61 6d65223a2022746573745f6c6f6e675f757269222c20226e616d65223a202267645f635f 746f6f6c73222c202276657273696f6e223a2022302e31222c20227265706f7369746f72 795f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f 6e223a2022353836396263663736653838222c202274797065223a20227061636b616765 227d2c2022717569636b747265652f312e31223a207b227265706f7369746f72795f6e61 6d65223a2022746573745f6c6f6e675f757269222c20226e616d65223a2022717569636b 74726565222c202276657273696f6e223a2022312e31222c20227265706f7369746f7279 5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e 223a2022353836396263663736653838222c202274797065223a20227061636b61676522 7d2c20226669736865722f302e312e34223a207b227265706f7369746f72795f6e616d65 223a2022746573745f6c6f6e675f757269222c20226e616d65223a202266697368657222 2c202276657273696f6e223a2022302e312e34222c20227265706f7369746f72795f6f77 6e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e223a20 22353836396263663736653838222c202274797065223a20227061636b616765227d2c20 22656967656e736f66742f352e302e31223a207b227265706f7369746f72795f6e616d65 223a2022746573745f6c6f6e675f757269222c20226e616d65223a2022656967656e736f 6674222c202276657273696f6e223a2022352e302e31222c20227265706f7369746f7279 5f6f776e6572223a20227269636f222c20226368616e67657365745f7265766973696f6e 223a2022353836396263663736653838222c202274797065223a20227061636b61676522 7d7d5d7d&includes_tools=True&tool_shed_url=http%3A%2F% 2Fdevshed.bx.psu.edu%2F&includes_tool_dependencies=True HTTP/1.1

Here is the POSTed data:

install_repository_dependencies=true&install_repository_dependencies=tru e&install_tool_dependencies=true&install_tool_dependencies=true&shed_too l_conf=.% 2Fshed_tool_conf.xml&new_tool_panel_section=Test&select_tool_panel_secti on_button=Install

The POSTed URI is 8,320 bytes. The encoded_repo_info_dicts account for 8,113 (97.5%) of these bytes.

In this particular case, it appears that the relevant code can be found in install_repositories_by_revision() in lib/galaxy/webapps/ tool_shed/controllers/repository.py and in prepare_for_install() in lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py.

It's possible that there's code elsewhere that also passes the encoded_repo_info_dicts in the request, which would trigger the same issue.

-rico

On Sep 26, 2013, at 9:21 AM, Greg Von Kuster wrote:

Hi Rico,

Can you provide the specific GET request URL with which you encountered this problem? There are very few GET requests when installing repositories from the tool shed - the only one I can locate without a lot of digging is the following, which I don't think is causing the problem.

# Get the information necessary to install each repository. url = suc.url_join( tool_shed_url, 'repository/get_repository_information? repository_ids=%s&changeset_revisions=%s' % \ ( repository_ids, changeset_revisions ) )

Thanks,

Greg Von Kuster

On Sep 13, 2013, at 10:59 AM, Richard Burhans rico@bx.psu.edu wrote:

...

Folks,

After checking a little closer last night, I have one clarification. The Tool Shed does use POSTs as well as GETs, but even when it's POSTing, the encoded dictionaries (encoded_repo_info_dicts) are part of the query string. The encoded_repo_info_dicts should probably become part of the POSTed data.

-rico

On Sep 12, 2013, at 3:55 PM, Richard Burhans wrote:

...

When installing Galaxy Tool Shed repositories into a local Galaxy instance, tool shed dependecies are exchanged as an encoded python dictionary via the HTTP GET method.

If a tool shed repository contains enough dependencies, the length of this encoded dictionary can become extremely long. In my case, it became long enough for my Apache server to choke on it:

[Wed Sep 11 15:34:53 2013] [error] [client XXX.XXX.XXX.XXX] request failed: URI too long (longer than 8190)

The HTTP standard does not place a limit on the length of a URI, but the de facto limit is 2000 characters. See the following stackoverflow answer:

http://stackoverflow.com/a/417184

I would recommend either:

1. communicating toolshed dependencies via an HTTP POST
2. redesigning tool shed <-> instance communications

As a stopgap measure, I've modified lib/tool_shed/util/ encoding_util.py to use a more compact encoding scheme. I'm attaching a copy in case someone else runs into the same problem.

-rico<encoding_util.py>

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/