broken after upgrade to 17.09 : uploading files via the API
Dear All If I execute the following command: python import_with_dbkey.py *** file.bam hg19sub the bam files will be added to the user's history the code (import_with_dbkey.py) is as follows: import sys sys.path.append('***/bioblend-0.10.0-py2.7.egg') sys.path.append('***/requests_toolbelt-0.7.0-py2.7.egg') from bioblend.galaxy import GalaxyInstance from bioblend.galaxy.tools import ToolClient url = "http://galaxy-dev.fmi.ch" key = "***" history_id = sys.argv[1] file_path = sys.argv[2] dbkey= sys.argv[3] gi = GalaxyInstance(url=url, key=key) toolClient = ToolClient(gi) uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey) Now, if I use the virtual environment from Galaxy, I get: (.venv)-bash-4.2$ python import_with_dbkey.py *** file.bam hg19sub Traceback (most recent call last): File "/***/galaxy/tools/fmi__api_helpers/import_with_dbkey.py", line 40, in <module> uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 148, in upload_file return self._tool_post(payload, files_attached=True) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 218, in _tool_post return Client._post(self, payload, files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/client.py", line 171, in _post files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxyclient.py", line 131, in make_post_request r.status_code, body=r.text) bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> </p> </body></html> (.venv)-bash-4.2$ ...and the file does not get uploaded. I will continue investigating the difference between the python modules used, but maybe someone has a clue and could help me Thank you very much Hans-Rudolf
Thanks for the report, I suspect the breaking change would be this - https://github.com/galaxyproject/galaxy/pull/4563. I added a bunch of tests for these API changes that I though would ensure backward compatibility but perhaps there is some sort of breakage here. I opened a PR in response to this thread against bioblend to ensure setting the dbkey still works (https://github.com/galaxyproject/bioblend/pull/252) and it seems to work for me. Does that dbkey not properly exist for that user? Maybe there is a change in behavior for non-existent dbkeys? Or is the dbkey a custom build for that user? I could write some tests for that if that is the case. -John On Mon, Nov 20, 2017 at 7:38 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
Dear All
If I execute the following command:
python import_with_dbkey.py *** file.bam hg19sub
the bam files will be added to the user's history
the code (import_with_dbkey.py) is as follows:
import sys
sys.path.append('***/bioblend-0.10.0-py2.7.egg') sys.path.append('***/requests_toolbelt-0.7.0-py2.7.egg')
from bioblend.galaxy import GalaxyInstance from bioblend.galaxy.tools import ToolClient
url = "http://galaxy-dev.fmi.ch" key = "***"
history_id = sys.argv[1] file_path = sys.argv[2] dbkey= sys.argv[3]
gi = GalaxyInstance(url=url, key=key)
toolClient = ToolClient(gi)
uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey)
Now, if I use the virtual environment from Galaxy, I get:
(.venv)-bash-4.2$ python import_with_dbkey.py *** file.bam hg19sub Traceback (most recent call last): File "/***/galaxy/tools/fmi__api_helpers/import_with_dbkey.py", line 40, in <module> uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 148, in upload_file return self._tool_post(payload, files_attached=True) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 218, in _tool_post return Client._post(self, payload, files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/client.py", line 171, in _post files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxyclient.py", line 131, in make_post_request r.status_code, body=r.text) bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> </p> </body></html>
(.venv)-bash-4.2$
...and the file does not get uploaded.
I will continue investigating the difference between the python modules used, but maybe someone has a clue and could help me
Thank you very much
Hans-Rudolf
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Dear John Thank you very much for your quick response. Unfortunately, this doesn't work for me. Yes, the dbkey exists. It is not custom build for this user, and other datasets already exist in this history with this dbkey. The culprit is somewhere in the virtual env: (.venv)-bash-4.2$ which python /***/galaxy/.venv/bin/python (.venv)-bash-4.2$ (.venv)-bash-4.2$ python import_with_dbkey.py *** file.bam hg19sub => does not work ...but: (.venv)-bash-4.2$ /usr/bin/python import_with_dbkey.py *** file.bam hg19sub => does work I can reproduce this without the virtual env (the other way round) -bash-4.2$ which python /usr/bin/python -bash-4.2$ -bash-4.2$ python import_with_dbkey.py *** file.bam hg19sub => does work -bash-4.2$ /***/galaxy/.venv/bin/python import_with_dbkey.py *** file.bam hg19sub => does not work and if I use the old directory (used with release 17.01) -bash-4.2$ /***/galaxy_20171120/.venv/bin/python import_with_dbkey.py *** file.bam hg19sub => it does work as well Hans-Rudolf On 11/20/2017 02:02 PM, John Chilton wrote:
Thanks for the report,
I suspect the breaking change would be this - https://github.com/galaxyproject/galaxy/pull/4563. I added a bunch of tests for these API changes that I though would ensure backward compatibility but perhaps there is some sort of breakage here. I opened a PR in response to this thread against bioblend to ensure setting the dbkey still works (https://github.com/galaxyproject/bioblend/pull/252) and it seems to work for me.
Does that dbkey not properly exist for that user? Maybe there is a change in behavior for non-existent dbkeys? Or is the dbkey a custom build for that user? I could write some tests for that if that is the case.
-John
On Mon, Nov 20, 2017 at 7:38 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
Dear All
If I execute the following command:
python import_with_dbkey.py *** file.bam hg19sub
the bam files will be added to the user's history
the code (import_with_dbkey.py) is as follows:
import sys
sys.path.append('***/bioblend-0.10.0-py2.7.egg') sys.path.append('***/requests_toolbelt-0.7.0-py2.7.egg')
from bioblend.galaxy import GalaxyInstance from bioblend.galaxy.tools import ToolClient
url = "http://galaxy-dev.fmi.ch" key = "***"
history_id = sys.argv[1] file_path = sys.argv[2] dbkey= sys.argv[3]
gi = GalaxyInstance(url=url, key=key)
toolClient = ToolClient(gi)
uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey)
Now, if I use the virtual environment from Galaxy, I get:
(.venv)-bash-4.2$ python import_with_dbkey.py *** file.bam hg19sub Traceback (most recent call last): File "/***/galaxy/tools/fmi__api_helpers/import_with_dbkey.py", line 40, in <module> uploadedFile = toolClient.upload_file(file_path, history_id, dbkey=dbkey) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 148, in upload_file return self._tool_post(payload, files_attached=True) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/tools/__init__.py", line 218, in _tool_post return Client._post(self, payload, files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/client.py", line 171, in _post files_attached=files_attached) File "/***/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxyclient.py", line 131, in make_post_request r.status_code, body=r.text) bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> </p> </body></html>
(.venv)-bash-4.2$
...and the file does not get uploaded.
I will continue investigating the difference between the python modules used, but maybe someone has a clue and could help me
Thank you very much
Hans-Rudolf
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
participants (2)
-
Hans-Rudolf Hotz -
John Chilton