2011/5/19 Mariusz Mamoński
<mamonski@man.poznan.pl>
Hi,
> ------------------------------
>
> Message: 13
> Date: Wed, 18 May 2011 17:15:06 +0200
> From: Leandro Hermida <softdev@leandrohermida.com>
> To: Galaxy Dev <galaxy-dev@bx.psu.edu>
> Subject: [galaxy-dev] wait thread: <1002:Not enough memory.> error
> after enabling DRMAA
> Message-ID: <BANLkTi=J4-ct04U=8dm6ra=sJcR_eDEG1A@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
>
> I enabled DRMAA on my test Galaxy server installation and in the server
> startup output I get the following strange "E #14ca [ 1.10] * wait
are you using LSF?
> thread: <1002:Not enough memory.>" lines after loading the job manager:
>
> ...
> galaxy.jobs.runners.local INFO 2011-05-18 17:10:23,025 starting workers
> galaxy.jobs.runners.local DEBUG 2011-05-18 17:10:23,026 5 workers ready
> galaxy.jobs DEBUG 2011-05-18 17:10:23,026 Loaded job runner:
> galaxy.jobs.runners.local:LocalJobRunner
> galaxy.jobs.runners.drmaa DEBUG 2011-05-18 17:10:23,130 3 workers ready
> galaxy.jobs DEBUG 2011-05-18 17:10:23,130 Loaded job runner:
> galaxy.jobs.runners.drmaa:DRMAAJobRunner
> galaxy.jobs INFO 2011-05-18 17:10:23,131 job manager started
> E #14ca [ 0.00] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.02] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.05] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.07] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.10] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.12] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.15] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.17] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.20] * wait thread: <1002:Not enough memory.>
> galaxy.jobs INFO 2011-05-18 17:10:23,373 job stopper started
> E #14ca [ 0.22] * wait thread: <1002:Not enough memory.>
> galaxy.sample_tracking.external_service_types DEBUG 2011-05-18 17:10:23,383
> Loaded external_service_type: Simple unknown sequencer 1.0.0
> galaxy.sample_tracking.external_service_types DEBUG 2011-05-18 17:10:23,386
> Loaded external_service_type: Applied Biosystems SOLiD 1.0.0
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,400 Enabling 'admin'
> controller, class: AdminGalaxy
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,402 Enabling 'async'
> controller, class: ASync
> E #14ca [ 0.25] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.27] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,428 Enabling 'dataset'
> controller, class: DatasetInterface
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,432 Enabling 'error'
> controller, class: Error
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,436 Enabling
> 'external_service' controller, class: ExternalService
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,437 Enabling
> 'external_services' controller, class: ExternalServiceController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,439 Enabling 'forms'
> controller, class: Forms
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,441 Enabling 'history'
> controller, class: HistoryController
> E #14ca [ 0.30] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,460 Enabling 'library'
> controller, class: Library
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,461 Enabling
> 'library_admin' controller, class: LibraryAdmin
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,462 Enabling
> 'library_common' controller, class: LibraryCommon
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,475 Enabling 'mobile'
> controller, class: Mobile
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,478 Enabling 'page'
> controller, class: PageController
> E #14ca [ 0.32] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,482 Enabling
> 'request_type' controller, class: RequestType
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,483 Enabling 'requests'
> controller, class: Requests
> E #14ca [ 0.35] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,508 Enabling
> 'requests_admin' controller, class: RequestsAdmin
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,509 Enabling
> 'requests_common' controller, class: RequestsCommon
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,511 Enabling 'root'
> controller, class: RootController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,514 Enabling 'tag'
> controller, class: TagsController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,519 Enabling
> 'tool_runner' controller, class: ToolRunner
> E #14ca [ 0.37] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.40] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,558 Enabling 'tracks'
> controller, class: TracksController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,562 Enabling
> 'ucsc_proxy' controller, class: UCSCProxy
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,564 Enabling 'user'
> controller, class: User
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,567 Enabling
> 'visualization' controller, class: VisualizationController
> E #14ca [ 0.42] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.45] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.47] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.50] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.53] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.55] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.58] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,747 Enabling 'workflow'
> controller, class: WorkflowController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,755 Enabling 'contents'
> API controller, class: ContentsController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,757 Enabling 'forms' API
> controller, class: FormDefinitionAPIController
> E #14ca [ 0.60] * wait thread: <1002:Not enough memory.>
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,762 Enabling 'libraries'
> API controller, class: LibrariesController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,767 Enabling
> 'request_types' API controller, class: RequestTypeAPIController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,768 Enabling 'requests'
> API controller, class: RequestsAPIController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,770 Enabling 'roles' API
> controller, class: RoleAPIController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,772 Enabling 'samples'
> API controller, class: SamplesAPIController
> galaxy.web.framework.base DEBUG 2011-05-18 17:10:23,773 Enabling 'users' API
> controller, class: UserAPIController
> E #14ca [ 0.63] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.65] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.67] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.70] * wait thread: <1002:Not enough memory.>
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:23,862 Enabling 'httpexceptions'
> middleware
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:23,869 Enabling 'remote user'
> middleware
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:23,871 Enabling 'recursive'
> middleware
> E #14ca [ 0.72] * wait thread: <1002:Not enough memory.>
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:23,882 Enabling 'print debug'
> middleware
> E #14ca [ 0.74] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.77] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.79] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.82] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.85] * wait thread: <1002:Not enough memory.>
> /gpfshpc/home/lhermida/soft/galaxy/galaxy_dist/eggs/WebError-0.8a-py2.6.egg/weberror/exceptions/serial_number_generator.py:11:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead
> import md5
> E #14ca [ 0.87] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.90] * wait thread: <1002:Not enough memory.>
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:24,058 Enabling 'eval exceptions'
> middleware
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:24,062 Enabling 'trans logger'
> middleware
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:24,062 Enabling 'config'
> middleware
> galaxy.web.buildapp DEBUG 2011-05-18 17:10:24,064 Enabling
> 'x-forwarded-host' middleware
> Starting server in PID 5229.
> serving on http://10.155.10.23:8082
> E #14ca [ 0.92] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.95] * wait thread: <1002:Not enough memory.>
> E #14ca [ 0.97] * wait thread: <1002:Not enough memory.>
> ...
>
> Do I need to increase some kind of running memory somewhere or is it another
> problem?
i think it is a bug. Could you try to edit the
LSF_TOP/conf/lsf_drmaa.conf
and set there
wait_thread: 0
?
Unfortunately i do not have access to any LSF cluster so i can not dig
further into this problem.
>
> thanks,
> Leandro
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.bx.psu.edu/pipermail/galaxy-dev/attachments/20110518/7dd7663d/attachment-0001.html>
>
> ------------------------------
>
> _______________________________________________
> galaxy-dev mailing list
> galaxy-dev@lists.bx.psu.edu
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
>
> End of galaxy-dev Digest, Vol 59, Issue 17
> ******************************************
>
Cheers,
--
Mariusz