Problem with LWR (input size limitation)
Hi everybody, In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size
1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server? Thank you very much in advance, Bests regards, Cheers, Misharl Monsoor
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack. I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax). Hope this helps, -John On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi John , First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml). Thank you again for your help, Misharl Monsoor. Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml).
Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter. Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi John, I can confirm you that your advice putting "LWR_CURL_TRANSPORT=1" works very fine, thanks a lot! Sorry, i haven't expressed myself correctly for my second question, thanks for answering, it helps me. I have another problem with LWR, when executing the jobs on the Windows Server, i have the following bug (it happens a lot of time, and i must restart the LWR server in order for it to works): File "C:\Python2764\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver py", line 287, in wsgi_execute self.wsgi_start_response) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 37, in __call__ return controller(environ, start_response, **request_args) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 141, in controller_replacement resp = self.__build_response(result) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 129, in __build_response resp = Response(body=self.body(result)) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 155, in body body = dumps(result) File "C:\Python2764\lib\json\__init__.py", line 243, in dumps return _default_encoder.encode(obj) File "C:\Python2764\lib\json\encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Python2764\lib\json\encoder.py", line 270, in iterencode return _iterencode(o, 0) nicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 718: invalid start byte Bests regards and thank you very much for your help, Misharl Monsoor. Le 21/11/2014 17:44, John Chilton a écrit :
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml). Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter.
Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
I have had one other report of this error previously - but I have not been able to reproduce it. I have tried so hard to understand these unicode/utf-8/Python 2 v 3/etc... issues and I must admit I simply don't :(. Reading through http://farmdev.com/talks/unicode/ I found a hack that might work, if you place: import sys sys.setdefaultencoding('utf-8') at the top of lwr/web/framework.py does that fix the problem? -John On Wed, Nov 26, 2014 at 2:53 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
I can confirm you that your advice putting "LWR_CURL_TRANSPORT=1" works very fine, thanks a lot! Sorry, i haven't expressed myself correctly for my second question, thanks for answering, it helps me. I have another problem with LWR, when executing the jobs on the Windows Server, i have the following bug (it happens a lot of time, and i must restart the LWR server in order for it to works):
File "C:\Python2764\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver py", line 287, in wsgi_execute self.wsgi_start_response) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 37, in __call__ return controller(environ, start_response, **request_args) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 141, in controller_replacement resp = self.__build_response(result) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 129, in __build_response resp = Response(body=self.body(result)) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 155, in body body = dumps(result) File "C:\Python2764\lib\json\__init__.py", line 243, in dumps return _default_encoder.encode(obj) File "C:\Python2764\lib\json\encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Python2764\lib\json\encoder.py", line 270, in iterencode return _iterencode(o, 0) nicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 718: invalid start byte
Bests regards and thank you very much for your help,
Misharl Monsoor.
Le 21/11/2014 17:44, John Chilton a écrit :
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml).
Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter.
Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi John, Thanks again for you help. So i place "sys.setdefaultencoding('utf-8')" at the top of the "lwr/web/framework.py", and i still have this bug. I use Python version "2.7 64 bits" to executes LWR on the windows server, and i encounter a "common" problem with the "setdefaultencoding" function which can't be loaded when doing "import sys" and "sys.setdefaultencoding('utf-8')". The hack that i found was to do "reload(sys)" then "sys.setdefaultencoding('utf-8')". But i still get the "utf8' codec can't decode byte" bug. Can it come from my input data which contains utf8 characters? Thank you very much, Bests regards, Misharl Monsoor Le 04/12/2014 06:25, John Chilton a écrit :
I have had one other report of this error previously - but I have not been able to reproduce it. I have tried so hard to understand these unicode/utf-8/Python 2 v 3/etc... issues and I must admit I simply don't :(.
Reading through http://farmdev.com/talks/unicode/ I found a hack that might work, if you place:
import sys sys.setdefaultencoding('utf-8')
at the top of lwr/web/framework.py does that fix the problem?
-John
On Wed, Nov 26, 2014 at 2:53 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
I can confirm you that your advice putting "LWR_CURL_TRANSPORT=1" works very fine, thanks a lot! Sorry, i haven't expressed myself correctly for my second question, thanks for answering, it helps me. I have another problem with LWR, when executing the jobs on the Windows Server, i have the following bug (it happens a lot of time, and i must restart the LWR server in order for it to works):
File "C:\Python2764\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver py", line 287, in wsgi_execute self.wsgi_start_response) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 37, in __call__ return controller(environ, start_response, **request_args) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 141, in controller_replacement resp = self.__build_response(result) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 129, in __build_response resp = Response(body=self.body(result)) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 155, in body body = dumps(result) File "C:\Python2764\lib\json\__init__.py", line 243, in dumps return _default_encoder.encode(obj) File "C:\Python2764\lib\json\encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Python2764\lib\json\encoder.py", line 270, in iterencode return _iterencode(o, 0) nicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 718: invalid start byte
Bests regards and thank you very much for your help,
Misharl Monsoor.
Le 21/11/2014 17:44, John Chilton a écrit :
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml). Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter.
Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi everybody,
In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function:
data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory
I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server?
Thank you very much in advance,
Bests regards,
Cheers,
Misharl Monsoor ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
I do have a test case that should exercise interesting UTF-8 stuff. Perhaps it would help to narrow down whether this is a problem of the test case being too weak or my system having some sort of configuration that is different than yours. Can you try to run python run_client_tests.py --url=http://localhost:8913 (replacing the URL with whatever it should actually be based on your setup.) Does that pass or fail? It should cause some logging on the LWR side but if the above call complete without throwing an error than the test case passed. Thanks for your continued patience and sorry I have not been able to reproduce the problem. -John On Mon, Dec 8, 2014 at 3:48 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
Thanks again for you help. So i place "sys.setdefaultencoding('utf-8')" at the top of the "lwr/web/framework.py", and i still have this bug.
I use Python version "2.7 64 bits" to executes LWR on the windows server, and i encounter a "common" problem with the "setdefaultencoding" function which can't be loaded when doing "import sys" and "sys.setdefaultencoding('utf-8')". The hack that i found was to do "reload(sys)" then "sys.setdefaultencoding('utf-8')". But i still get the "utf8' codec can't decode byte" bug. Can it come from my input data which contains utf8 characters?
Thank you very much,
Bests regards,
Misharl Monsoor
Le 04/12/2014 06:25, John Chilton a écrit :
I have had one other report of this error previously - but I have not been able to reproduce it. I have tried so hard to understand these unicode/utf-8/Python 2 v 3/etc... issues and I must admit I simply don't :(.
Reading through http://farmdev.com/talks/unicode/ I found a hack that might work, if you place:
import sys sys.setdefaultencoding('utf-8')
at the top of lwr/web/framework.py does that fix the problem?
-John
On Wed, Nov 26, 2014 at 2:53 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
I can confirm you that your advice putting "LWR_CURL_TRANSPORT=1" works very fine, thanks a lot! Sorry, i haven't expressed myself correctly for my second question, thanks for answering, it helps me. I have another problem with LWR, when executing the jobs on the Windows Server, i have the following bug (it happens a lot of time, and i must restart the LWR server in order for it to works):
File "C:\Python2764\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver py", line 287, in wsgi_execute self.wsgi_start_response) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 37, in __call__ return controller(environ, start_response, **request_args) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 141, in controller_replacement resp = self.__build_response(result) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 129, in __build_response resp = Response(body=self.body(result)) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 155, in body body = dumps(result) File "C:\Python2764\lib\json\__init__.py", line 243, in dumps return _default_encoder.encode(obj) File "C:\Python2764\lib\json\encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Python2764\lib\json\encoder.py", line 270, in iterencode return _iterencode(o, 0) nicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 718: invalid start byte
Bests regards and thank you very much for your help,
Misharl Monsoor.
Le 21/11/2014 17:44, John Chilton a écrit :
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml).
Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter.
Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
If you install pycurl (globally or in a Galaxy's virtualenv) and set LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream large files with CURL instead of the mmap hack.
I don't understand your second question - the LWR should stream whatever data inputs are selected by the user. If you want the tool to be able to take many inputs you can use multiple="true" on your data parameter or use a repeat block. (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax).
Hope this helps, -John
On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote: > > Hi everybody, > > In our lab, we are trying to connect our Galaxy instance to a Windows > 7 > 64 > bits server, in order to executes programs that need to be run within > Windows. However we have a problem concerning the size of the input > that > is > uploaded to the Windows server, it doesn't accept inputs with a size > > > 1.3 > Gb ? We have the following problem with mmap function: > > data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) > error: [Errno 12] Cannot allocate memory > > > I have another question concerning the inputs, is there a way to use > several > inputs that be uploaded to a Windows Server? > > Thank you very much in advance, > > Bests regards, > > Cheers, > > Misharl Monsoor > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/
Hi John, Thank you again for your help. So i tested the "python run_client_tests.py --url=http://localhost:8913" command on the lwr server, and i don't have any error message (i guess that the test has passed). Bests regards, Misharl Monsoor. Le 18/12/2014 22:50, John Chilton a écrit :
I do have a test case that should exercise interesting UTF-8 stuff.
Perhaps it would help to narrow down whether this is a problem of the test case being too weak or my system having some sort of configuration that is different than yours. Can you try to run
python run_client_tests.py --url=http://localhost:8913
(replacing the URL with whatever it should actually be based on your setup.) Does that pass or fail? It should cause some logging on the LWR side but if the above call complete without throwing an error than the test case passed.
Thanks for your continued patience and sorry I have not been able to reproduce the problem.
-John
On Mon, Dec 8, 2014 at 3:48 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
Thanks again for you help. So i place "sys.setdefaultencoding('utf-8')" at the top of the "lwr/web/framework.py", and i still have this bug.
I use Python version "2.7 64 bits" to executes LWR on the windows server, and i encounter a "common" problem with the "setdefaultencoding" function which can't be loaded when doing "import sys" and "sys.setdefaultencoding('utf-8')". The hack that i found was to do "reload(sys)" then "sys.setdefaultencoding('utf-8')". But i still get the "utf8' codec can't decode byte" bug. Can it come from my input data which contains utf8 characters?
Thank you very much,
Bests regards,
Misharl Monsoor
Le 04/12/2014 06:25, John Chilton a écrit :
I have had one other report of this error previously - but I have not been able to reproduce it. I have tried so hard to understand these unicode/utf-8/Python 2 v 3/etc... issues and I must admit I simply don't :(.
Reading through http://farmdev.com/talks/unicode/ I found a hack that might work, if you place:
import sys sys.setdefaultencoding('utf-8')
at the top of lwr/web/framework.py does that fix the problem?
-John
On Wed, Nov 26, 2014 at 2:53 AM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John,
I can confirm you that your advice putting "LWR_CURL_TRANSPORT=1" works very fine, thanks a lot! Sorry, i haven't expressed myself correctly for my second question, thanks for answering, it helps me. I have another problem with LWR, when executing the jobs on the Windows Server, i have the following bug (it happens a lot of time, and i must restart the LWR server in order for it to works):
File "C:\Python2764\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver py", line 287, in wsgi_execute self.wsgi_start_response) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 37, in __call__ return controller(environ, start_response, **request_args) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 141, in controller_replacement resp = self.__build_response(result) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 129, in __build_response resp = Response(body=self.body(result)) File "C:\Desktop\lwr_master\lwr\web\framework.py", l ne 155, in body body = dumps(result) File "C:\Python2764\lib\json\__init__.py", line 243, in dumps return _default_encoder.encode(obj) File "C:\Python2764\lib\json\encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Python2764\lib\json\encoder.py", line 270, in iterencode return _iterencode(o, 0) nicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 718: invalid start byte
Bests regards and thank you very much for your help,
Misharl Monsoor.
Le 21/11/2014 17:44, John Chilton a écrit :
On Wed, Nov 19, 2014 at 12:58 PM, Misharl Monsoor <mmonsoor@sb-roscoff.fr> wrote:
Hi John ,
First all, thank you very much for your quick reply. I will test your suggestion about pycurl, and tell you if it has worked. For the second question, i was wondering if it is possible with LWR to define a path (where are located the inputs) as an input (string input in the wrapper xml). Still not certain I understand - but I do not believe this is possible for datasets - this is something that must be configured for the LWR itself. If you have inputs to the tool that are not datasets - such as reference or index data - those could potentially be specified this way as just <data type="text"... in your tool. Those parameters will be passed through on the command-line just like any other parameter.
Hope this helps. -John
Thank you again for your help,
Misharl Monsoor.
Le 19/11/2014 15:30, John Chilton a écrit :
> If you install pycurl (globally or in a Galaxy's virtualenv) and set > LWR_CURL_TRANSPORT=1 in Galaxy's environment than the LWR will stream > large files with CURL instead of the mmap hack. > > I don't understand your second question - the LWR should stream > whatever data inputs are selected by the user. If you want the tool to > be able to take many inputs you can use multiple="true" on your data > parameter or use a repeat block. > (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax). > > Hope this helps, > -John > > > On Mon, Nov 17, 2014 at 3:04 AM, Misharl Monsoor > <mmonsoor@sb-roscoff.fr> > wrote: >> Hi everybody, >> >> In our lab, we are trying to connect our Galaxy instance to a Windows >> 7 >> 64 >> bits server, in order to executes programs that need to be run within >> Windows. However we have a problem concerning the size of the input >> that >> is >> uploaded to the Windows server, it doesn't accept inputs with a size >> 1.3 >> Gb ? We have the following problem with mmap function: >> >> data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) >> error: [Errno 12] Cannot allocate memory >> >> >> I have another question concerning the inputs, is there a way to use >> several >> inputs that be uploaded to a Windows Server? >> >> Thank you very much in advance, >> >> Bests regards, >> >> Cheers, >> >> Misharl Monsoor >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/
participants (2)
-
John Chilton -
Misharl Monsoor