bioblend problem trying to use run_tool function
Dear Developers, I would like to use bioblend to export the result of a galaxy workflow. To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export. We tried to launch this tool using the run_tool function but it doesn't work. giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"}) I get an error message which is very long and attached to this email. Please help me on that it is the final step of several ones using bioblend. Have a nice day, -- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this: Internal Server Error Galaxy was unable to successfully complete your request URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:96 in __call__ return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:201 in decorator log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True) TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line: log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True) I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret. Hope this helps. -John On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hello John, Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation. <br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________ I'm currently trying to understand this error. Please tell me if you know anything about this error. Have a nice day, -- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:96 in __call__ return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:201 in decorator log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True) TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong. -John On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Yes, indeed, the log from galaxy are the following: The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump: payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs) However, there is another error afterwards which is: ___________________________________________________________________ /home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage. galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic" ___________________________________________________________________ Thanks for your help, Olivia Doppelt-Azeroual Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:96 in __call__ return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:201 in decorator log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True) TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet. the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding *history* to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, *history* ) rval.append( rval_dict ) return rval There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"}) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } } On the galaxy side, there is no error message. Sorry for the many mails but I prefer to keep you updated so you don"t waste time. -- Olivia Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc)
File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py",
line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py",
line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py",
line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1
Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:96 in __call__ return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:201 in decorator log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True) TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Yes, the get_intial_value bug was a bug that has been fixed in a recent Galaxy release (I think the most recent one, but maybe the previous one). Unfortunately though, I fixed other bugs related to running more sophisticated tools through the API at the same time - so I would again strongly recommend upgrading before proceeding or you going to find all the same bugs I did. The tool running API has significantly improved over the last couple releases. As for removing the dumps statement in bioblend - I think (though am not certain) that is the wrong thing to do. I think you are likely somehow doing this at a level above the client call and shouldn't be - i.e. my guess is you are passing in a string instead of a dictionary and the result is that dumps it dumping an escaped string - instead of jsonifing a dictionary. For instance, the API driven test framework runs by calling this API and does almost this exact dumps. https://bitbucket.org/galaxy/galaxy-central/src/f573461f3667e8f120dc05bbd11a... Hope this helps. -John On Mon, Feb 24, 2014 at 8:51 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet.
the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding history to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, history ) rval.append( rval_dict ) return rval
There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"})
File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } }
On the galaxy side, there is no error message.
Sorry for the many mails but I prefer to keep you updated so you don"t waste time.
-- Olivia
Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Dear John, I am doing tests on a local version of the last galaxy-dist version before updating our Galaxy server. And I have a question concernant the second remark you made in your last email. I've written a small script to test the run_tool function. When i put back the dumps(inputs_dict), the script fails whereas the type of my input_dict is a dictionnary. The script is the following: -------------- from bioblend.galaxy import GalaxyInstance giLocal=GalaxyInstance('http://127.0.0.1:8080/', key='43359c833b8977b01d6237d9b839dc27') tool_inputs_loc={'input':'f597429621d6eb2b','name':"data"} print type(tool_inputs_loc) giLocal.tools.run_tool(history_id='1cd8e2f6b131e891',tool_id='export',tool_inputs=tool_inputs_loc) --------------- when i print the payload it is: {'tool_id': 'export', 'history_id': '1cd8e2f6b131e891', 'inputs': '{"input": "f597429621d6eb2b", "name": "data"}'} On the server side, the error is the following: -------------- galaxy.web.framework ERROR 2014-02-25 11:35:34,865 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 200, in decorator rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 104, in create for k, v in inputs.iteritems(): AttributeError: 'unicode' object has no attribute 'iteritems' ---------------- Do you have an idea ? Have a nice day, Thank you for your help, -- Olivia Le 24/02/14 16:18, John Chilton a écrit :
Yes, the get_intial_value bug was a bug that has been fixed in a recent Galaxy release (I think the most recent one, but maybe the previous one). Unfortunately though, I fixed other bugs related to running more sophisticated tools through the API at the same time - so I would again strongly recommend upgrading before proceeding or you going to find all the same bugs I did. The tool running API has significantly improved over the last couple releases.
As for removing the dumps statement in bioblend - I think (though am not certain) that is the wrong thing to do. I think you are likely somehow doing this at a level above the client call and shouldn't be - i.e. my guess is you are passing in a string instead of a dictionary and the result is that dumps it dumping an escaped string - instead of jsonifing a dictionary.
For instance, the API driven test framework runs by calling this API and does almost this exact dumps.
https://bitbucket.org/galaxy/galaxy-central/src/f573461f3667e8f120dc05bbd11a...
Hope this helps.
-John
On Mon, Feb 24, 2014 at 8:51 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet.
the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding history to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, history ) rval.append( rval_dict ) return rval
There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"})
File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } }
On the galaxy side, there is no error message.
Sorry for the many mails but I prefer to keep you updated so you don"t waste time.
-- Olivia
Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File "/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module> giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File "/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> ___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL: http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
I think you are right, and I am right, but you more so :). I bet the inputs need to be dumped to json if files are being attached (i.e. upload is being used) but should not be otherwise (because content-type will be different). I wrote the client just to test uploads - so this is probably the problem. I have created a Trello card (https://trello.com/c/67HsrgVm) to fix bioblend to handle these details transparently and (per your other email) to test and provide examples of using a repeat with data parameters. I am certain the Galaxy API allows this - I am not sure the bioblend tool client is quite there yet - it shouldn't be hard to update it or provide examples - but I will need to find some time which can be difficult. -John On Tue, Feb 25, 2014 at 4:41 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Dear John,
I am doing tests on a local version of the last galaxy-dist version before updating our Galaxy server. And I have a question concernant the second remark you made in your last email.
I've written a small script to test the run_tool function. When i put back the dumps(inputs_dict), the script fails whereas the type of my input_dict is a dictionnary.
The script is the following:
-------------- from bioblend.galaxy import GalaxyInstance giLocal=GalaxyInstance('http://127.0.0.1:8080/', key='43359c833b8977b01d6237d9b839dc27') tool_inputs_loc={'input':'f597429621d6eb2b','name':"data"} print type(tool_inputs_loc) giLocal.tools.run_tool(history_id='1cd8e2f6b131e891',tool_id='export',tool_inputs=tool_inputs_loc) ---------------
when i print the payload it is:
{'tool_id': 'export', 'history_id': '1cd8e2f6b131e891', 'inputs': '{"input": "f597429621d6eb2b", "name": "data"}'}
On the server side, the error is the following: -------------- galaxy.web.framework ERROR 2014-02-25 11:35:34,865 Uncaught exception in exposed API method:
Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 200, in decorator
rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 104, in create for k, v in inputs.iteritems(): AttributeError: 'unicode' object has no attribute 'iteritems' ----------------
Do you have an idea ?
Have a nice day, Thank you for your help, -- Olivia Le 24/02/14 16:18, John Chilton a écrit :
Yes, the get_intial_value bug was a bug that has been fixed in a recent Galaxy release (I think the most recent one, but maybe the previous one). Unfortunately though, I fixed other bugs related to running more sophisticated tools through the API at the same time - so I would again strongly recommend upgrading before proceeding or you going to find all the same bugs I did. The tool running API has significantly improved over the last couple releases.
As for removing the dumps statement in bioblend - I think (though am not certain) that is the wrong thing to do. I think you are likely somehow doing this at a level above the client call and shouldn't be - i.e. my guess is you are passing in a string instead of a dictionary and the result is that dumps it dumping an escaped string - instead of jsonifing a dictionary.
For instance, the API driven test framework runs by calling this API and does almost this exact dumps.
https://bitbucket.org/galaxy/galaxy-central/src/f573461f3667e8f120dc05bbd11a...
Hope this helps.
-John
On Mon, Feb 24, 2014 at 8:51 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet.
the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding history to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, history ) rval.append( rval_dict ) return rval
There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"})
File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } }
On the galaxy side, there is no error message.
Sorry for the many mails but I prefer to keep you updated so you don"t waste time.
-- Olivia
Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html>
___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL:
http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Hello John, Sorry to bother you again with this. I saw the many improvments added but I still can't run my tool using bioblend. The inputs names are 'input' and 'prefix' and the tool is very simple. The problem is still a mapping issue. Parameters are not assigned correctly in the command line. options are there but no values are there. Could you give me a use case, so I can mime it to my project ? Thanks in advance, -- Olivia Doppelt-Azeroual CIB, Institut Pasteur, Paris Le 27/02/14 14:46, John Chilton a écrit :
I think you are right, and I am right, but you more so :). I bet the inputs need to be dumped to json if files are being attached (i.e. upload is being used) but should not be otherwise (because content-type will be different). I wrote the client just to test uploads - so this is probably the problem.
I have created a Trello card (https://trello.com/c/67HsrgVm) to fix bioblend to handle these details transparently and (per your other email) to test and provide examples of using a repeat with data parameters. I am certain the Galaxy API allows this - I am not sure the bioblend tool client is quite there yet - it shouldn't be hard to update it or provide examples - but I will need to find some time which can be difficult.
-John
On Tue, Feb 25, 2014 at 4:41 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Dear John,
I am doing tests on a local version of the last galaxy-dist version before updating our Galaxy server. And I have a question concernant the second remark you made in your last email.
I've written a small script to test the run_tool function. When i put back the dumps(inputs_dict), the script fails whereas the type of my input_dict is a dictionnary.
The script is the following:
-------------- from bioblend.galaxy import GalaxyInstance giLocal=GalaxyInstance('http://127.0.0.1:8080/', key='43359c833b8977b01d6237d9b839dc27') tool_inputs_loc={'input':'f597429621d6eb2b','name':"data"} print type(tool_inputs_loc) giLocal.tools.run_tool(history_id='1cd8e2f6b131e891',tool_id='export',tool_inputs=tool_inputs_loc) ---------------
when i print the payload it is:
{'tool_id': 'export', 'history_id': '1cd8e2f6b131e891', 'inputs': '{"input": "f597429621d6eb2b", "name": "data"}'}
On the server side, the error is the following: -------------- galaxy.web.framework ERROR 2014-02-25 11:35:34,865 Uncaught exception in exposed API method:
Traceback (most recent call last): File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 200, in decorator
rval = func( self, trans, *args, **kwargs) File "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 104, in create for k, v in inputs.iteritems(): AttributeError: 'unicode' object has no attribute 'iteritems' ----------------
Do you have an idea ?
Have a nice day, Thank you for your help, -- Olivia Le 24/02/14 16:18, John Chilton a écrit :
Yes, the get_intial_value bug was a bug that has been fixed in a recent Galaxy release (I think the most recent one, but maybe the previous one). Unfortunately though, I fixed other bugs related to running more sophisticated tools through the API at the same time - so I would again strongly recommend upgrading before proceeding or you going to find all the same bugs I did. The tool running API has significantly improved over the last couple releases.
As for removing the dumps statement in bioblend - I think (though am not certain) that is the wrong thing to do. I think you are likely somehow doing this at a level above the client call and shouldn't be - i.e. my guess is you are passing in a string instead of a dictionary and the result is that dumps it dumping an escaped string - instead of jsonifing a dictionary.
For instance, the API driven test framework runs by calling this API and does almost this exact dumps.
https://bitbucket.org/galaxy/galaxy-central/src/f573461f3667e8f120dc05bbd11a...
Hope this helps.
-John
On Mon, Feb 24, 2014 at 8:51 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet.
the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding history to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, history ) rval.append( rval_dict ) return rval
There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"})
File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } }
On the galaxy side, there is no error message.
Sorry for the many mails but I prefer to keep you updated so you don"t waste time.
-- Olivia
Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html>
___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL:
http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Hmm... the random_lines1 test case distributed with bioblend is kind of like this but a little more complex because it includes a conditional also: https://github.com/jmchilton/bioblend/blob/master/tests/TestGalaxyTools.py#L... Just using random_lines1 without specifying anything in the conditional (so just a numeric property and dataset input) the following snippet works for me: from bioblend import galaxy gi = galaxy.GalaxyInstance("localhost:8080", "1957431058be3aa3c57ca312f4a05521") history = gi.histories.create_history(name="TestRandomLines") dataset_id = gi.tools.paste_content("1\n2\n3", history["id"])['outputs'][0]["id"] from bioblend.galaxy.tools.inputs import inputs, dataset random_lines_inputs = inputs().set("num_lines", 1).set("input", dataset(dataset_id)) tool_outputs = gi.tools.run_tool(history_id=history["id"], tool_id="random_lines1", tool_inputs=random_lines_inputs) This seems very comparable to the two input tool execution you are describing. Cleaner version of above script @ https://gist.github.com/jmchilton/38c33d365e116c750c21. If this is not working, please make sure you are using a relatively recent version of Galaxy (probably either of the last two releases should work - one before that might as well) and the latest development version of bioblend (right from github - don't know if a full version has been released since I added tool inputs helpers). If none of that works - please make sure the tool works through the GUI and if it does can you share the tool XML with me? Thanks, -John On Mon, Jun 30, 2014 at 9:49 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Sorry to bother you again with this. I saw the many improvments added but I still can't run my tool using bioblend.
The inputs names are 'input' and 'prefix' and the tool is very simple.
The problem is still a mapping issue. Parameters are not assigned correctly in the command line. options are there but no values are there.
Could you give me a use case, so I can mime it to my project ?
Thanks in advance, -- Olivia Doppelt-Azeroual CIB, Institut Pasteur, Paris
Le 27/02/14 14:46, John Chilton a écrit :
I think you are right, and I am right, but you more so :). I bet the inputs need to be dumped to json if files are being attached (i.e. upload is being used) but should not be otherwise (because content-type will be different). I wrote the client just to test uploads - so this is probably the problem.
I have created a Trello card (https://trello.com/c/67HsrgVm) to fix bioblend to handle these details transparently and (per your other email) to test and provide examples of using a repeat with data parameters. I am certain the Galaxy API allows this - I am not sure the bioblend tool client is quite there yet - it shouldn't be hard to update it or provide examples - but I will need to find some time which can be difficult.
-John
On Tue, Feb 25, 2014 at 4:41 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Dear John,
I am doing tests on a local version of the last galaxy-dist version before updating our Galaxy server. And I have a question concernant the second remark you made in your last email.
I've written a small script to test the run_tool function. When i put back the dumps(inputs_dict), the script fails whereas the type of my input_dict is a dictionnary.
The script is the following:
-------------- from bioblend.galaxy import GalaxyInstance giLocal=GalaxyInstance('http://127.0.0.1:8080/', key='43359c833b8977b01d6237d9b839dc27') tool_inputs_loc={'input':'f597429621d6eb2b','name':"data"} print type(tool_inputs_loc)
giLocal.tools.run_tool(history_id='1cd8e2f6b131e891',tool_id='export',tool_inputs=tool_inputs_loc) ---------------
when i print the payload it is:
{'tool_id': 'export', 'history_id': '1cd8e2f6b131e891', 'inputs': '{"input": "f597429621d6eb2b", "name": "data"}'}
On the server side, the error is the following: -------------- galaxy.web.framework ERROR 2014-02-25 11:35:34,865 Uncaught exception in exposed API method:
Traceback (most recent call last): File
"/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 200, in decorator
rval = func( self, trans, *args, **kwargs) File
"/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 104, in create for k, v in inputs.iteritems(): AttributeError: 'unicode' object has no attribute 'iteritems' ----------------
Do you have an idea ?
Have a nice day, Thank you for your help, -- Olivia Le 24/02/14 16:18, John Chilton a écrit :
Yes, the get_intial_value bug was a bug that has been fixed in a recent Galaxy release (I think the most recent one, but maybe the previous one). Unfortunately though, I fixed other bugs related to running more sophisticated tools through the API at the same time - so I would again strongly recommend upgrading before proceeding or you going to find all the same bugs I did. The tool running API has significantly improved over the last couple releases.
As for removing the dumps statement in bioblend - I think (though am not certain) that is the wrong thing to do. I think you are likely somehow doing this at a level above the client call and shouldn't be - i.e. my guess is you are passing in a string instead of a dictionary and the result is that dumps it dumping an escaped string - instead of jsonifing a dictionary.
For instance, the API driven test framework runs by calling this API and does almost this exact dumps.
https://bitbucket.org/galaxy/galaxy-central/src/f573461f3667e8f120dc05bbd11a...
Hope this helps.
-John
On Mon, Feb 24, 2014 at 8:51 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
I detected another problem which enables me to go further, however doesn't resolve the whole problem yet.
the function get_initial value didn't use the history in the recursive call, which led to an empty history. I changed it juste adding history to the input.get_initial_value
def get_initial_value( self, trans, context, history=None ): rval = [] for i in range( self.default ): rval_dict = { '__index__': i} for input in self.inputs.itervalues(): rval_dict[ input.name ] = input.get_initial_value( trans, context, history ) rval.append( rval_dict ) return rval
There is another problem though.
On the bioblend side: _____________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs={'input':'8ad39366bdad8699','name':"data"})
File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 400: { "message": { "data": { "inputs": [ { "input": "History does not include a dataset of the required format / build" } ] }, "type": "error" } }
On the galaxy side, there is no error message.
Sorry for the many mails but I prefer to keep you updated so you don"t waste time.
-- Olivia
Le 24/02/14 15:29, Olivia Doppelt-Azeroual a écrit :
Yes, indeed,
the log from galaxy are the following:
The first error was due to the fact that bioblend function run_tools; gave galaxy a dump of the input_dicts which type is string where galaxy was waiting for an item structure. I removed the dump:
payload["inputs"] = tool_inputs instead of payload["inputs"] = dumps(tool_inputs)
However, there is another error afterwards which is:
___________________________________________________________________
/home/odoppelt/localStuff_VM/tools/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py:215: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
galaxy.web.framework ERROR 2014-02-24 15:15:15,374 Uncaught exception in exposed API method: Traceback (most recent call last): File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 203, in decorator rval = func( self, trans, *args, **kwargs) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 123, in create template, vars = tool.handle_input( trans, params.__dict__, history=target_history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1921, in handle_input state = self.new_state( trans, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1846, in new_state self.fill_in_new_state( trans, inputs, state.inputs, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/__init__.py", line 1855, in fill_in_new_state state[ input.name ] = input.get_initial_value( trans, context, history=history ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/grouping.py", line 116, in get_initial_value rval_dict[ input.name ] = input.get_initial_value( trans, context ) File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1657, in get_initial_value return self.get_initial_value_from_history_prevent_repeats(trans, context, None, history=history); File
"/home/odoppelt/localStuff_VM/tools/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1670, in get_initial_value_from_history_prevent_repeats assert history is not None, "DataToolParameter requires a history" AssertionError: DataToolParameter requires a history 127.0.0.1 - - [24/Feb/2014:15:15:15 +0200] "POST /api/tools?key=cd065330048989af4ae266cf94685e6b HTTP/1.1" 500 - "-" "python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-15-generic"
___________________________________________________________________
Thanks for your help,
Olivia Doppelt-Azeroual
Le 24/02/14 15:21, John Chilton a écrit :
Hard to infer much from this, usually when I get this error there is a full stack trace for this error in the Galaxy logs giving some indication what went wrong.
-John
On Mon, Feb 24, 2014 at 4:36 AM, Olivia Doppelt-Azeroual <olivia.doppelt@pasteur.fr> wrote:
Hello John,
Yes it helps a little but I still have the following error: _______________________________________________________________________ Traceback (most recent call last): File "script_testBioblend_distant.py", line 16, in <module>
giLocal.tools.run_tool(history_id='0a248a1f62a0cc04',tool_id='export',tool_inputs=tool_inputs_loc) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/tools/__init__.py", line 26, in run_tool return Client._post(self, payload) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxy/client.py", line 181, in _post r = self.gi.make_post_request(url, payload=payload, files_attached=files_attached) File
"/home/odoppelt/localStuff_VM/galaxy_work/SBW_project/bioblend/bioblend/galaxyclient.py", line 94, in make_post_request r.status_code, body=r.text) # @see self.body for HTTP response body bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy: 500: <html> <head><title>Internal Server Error</title></head> <body> <h1>Internal Server Error</h1> <p>The server has either erred or is incapable of performing the requested operation.
<br/> <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html>
___________________________________________________________________________________________
I'm currently trying to understand this error. Please tell me if you know anything about this error.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
Le 21/02/14 15:49, John Chilton a écrit :
I copied the HTML into a new document, opened it, and copied the stacktrace produced by Galaxy - it looks like this:
Internal Server Error
Galaxy was unable to successfully complete your request
URL:
http://galaxy.web.pasteur.fr/api/tools?key=c23d8cb98671911813305051710d4ae1 Module galaxy.web.framework.middleware.error:149 in __call__
app_iter = self.application(environ, sr_checker)
Module paste.recursive:84 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.middleware.remoteuser:96 in __call__
return self.app( environ, start_response )
Module paste.httpexceptions:633 in __call__
return self.application(environ, start_response)
Module galaxy.web.framework.base:132 in __call__
return self.handle_request( environ, start_response )
Module galaxy.web.framework.base:190 in handle_request
body = method( trans, **kwargs )
Module galaxy.web.framework:201 in decorator
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
TypeError: exception() got an unexpected keyword argument 'exc_info'
So what is happening here is and error is occurring when Galaxy is trying to log another error - causing the API to break down. I don't think the newest release of Galaxy has this line:
log.exception( 'Uncaught exception in exposed API method:'+ str(err),exc_info=True)
I am not sure what exc_info is - but it looks like it shouldn't be there. You may want to remove that and see if you get a cleaner error message that is easier to interpret.
Hope this helps.
-John
On Fri, Feb 21, 2014 at 3:50 AM, Olivia Doppelt <olivia.doppelt@pasteur.fr> wrote:
Dear Developers,
I would like to use bioblend to export the result of a galaxy workflow.
To do that, we want to use an implemented tool called export_data; which enables the copy of a result file from galaxy arborescence to a local directory which belong to the user making the analysis and the export.
We tried to launch this tool using the run_tool function but it doesn't work.
giOlivia.tools.run_tool(history_id='xxxxxxxxxxx',tool_id='yyyyyyyy/export_data/export/2.2',tool_inputs={'input':'id_data','name':"data"})
I get an error message which is very long and attached to this email.
Please help me on that it is the final step of several ones using bioblend.
Have a nice day,
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
-- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur
participants (4)
-
John Chilton -
John Chilton -
Olivia Doppelt -
Olivia Doppelt-Azeroual