galaxy doesn't start with drmaa job runner
I am trying to configure galaxy with sge/drmaa scheduler. The galaxy process is starting up fine without any drmaa configuration. However the galaxy daemon doesn't start properly when I add drmaa configuration lines as shown below. I have set SGE_ROOT and DRMAA_LIBRARY_PATH variables properly. Am I missing something important here? # drmaa configuration: {{{ start_job_runners = drmaa default_cluster_job_runner = drmaa:/// }} # paster.log file - with drmaa config the process dies with "Loaded job runner: galaxy.jobs.runners.local:LocalJobRunner" as last line in the log file. With local job runner the galaxy starts fine as shown below: {{{ galaxy.jobs DEBUG 2011-05-17 09:59:58,959 Loaded job runner: galaxy.jobs.runners.local:LocalJobRunner galaxy.jobs INFO 2011-05-17 09:59:58,959 job manager started galaxy.jobs INFO 2011-05-17 09:59:59,325 job stopper started galaxy.sample_tracking.external_service_types DEBUG 2011-05-17 09:59:59,330 Loaded external_service_type: Simple unknown sequencer 1.0.0 galaxy.sample_tracking.external_service_types DEBUG 2011-05-17 09:59:59,333 Loaded external_service_type: Applied Biosystems SOLiD 1.0.0 galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,349 Enabling 'mobile' controller, class: Mobile galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,417 Enabling 'library_common' controller, class: LibraryCommon galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,435 Enabling 'admin' controller, class: AdminGalaxy galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,451 Enabling 'requests' controller, class: Requests galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,455 Enabling 'external_services' controller, class: ExternalServiceController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,474 Enabling 'page' controller, class: PageController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,480 Enabling 'visualization' controller, class: VisualizationController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,549 Enabling 'tracks' controller, class: TracksController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,550 Enabling 'requests_common' controller, class: RequestsCommon galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,556 Enabling 'request_type' controller, class: RequestType galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,561 Enabling 'external_service' controller, class: ExternalService galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,562 Enabling 'library' controller, class: Library galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,746 Enabling 'workflow' controller, class: WorkflowController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,752 Enabling 'library_admin' controller, class: LibraryAdmin galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,756 Enabling 'async' controller, class: ASync galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,026 Enabling 'history' controller, class: HistoryController galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,031 Enabling 'error' controller, class: Error galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,090 Enabling 'requests_admin' controller, class: RequestsAdmin galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,095 Enabling 'ucsc_proxy' controller, class: UCSCProxy galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,104 Enabling 'forms' controller, class: Forms galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,137 Enabling 'dataset' controller, class: DatasetInterface galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,143 Enabling 'tool_runner' controller, class: ToolRunner galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,148 Enabling 'tag' controller, class: TagsController galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,158 Enabling 'user' controller, class: User galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,165 Enabling 'root' controller, class: RootController galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,189 Enabling 'httpexceptions' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,195 Enabling 'recursive' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,220 Enabling 'print debug' middleware /share/apps/galaxy/shantanu-temp-workspace/galaxy-cluster-test/galaxy-dist-50e249442c5a/eggs/WebError-0.8a-py2.6.egg/weberror/exceptions/serial_number_generator.py:11: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,726 Enabling 'eval exceptions' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,740 Enabling 'trans logger' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,740 Enabling 'config' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,744 Enabling 'x-forwarded-host' middleware Starting server in PID 10170. serving on 0.0.0.0:8081 view at http://127.0.0.1:8081 }}} -- Thanks, Shantanu.
Shantanu Pavgi wrote:
I am trying to configure galaxy with sge/drmaa scheduler. The galaxy process is starting up fine without any drmaa configuration. However the galaxy daemon doesn't start properly when I add drmaa configuration lines as shown below. I have set SGE_ROOT and DRMAA_LIBRARY_PATH variables properly. Am I missing something important here?
Could you provide the output of starting Galaxy with the drmaa options enabled? You may want to start it in the foreground (without --daemon) to capture any messages which are not output to paster.log --nate
# drmaa configuration: {{{ start_job_runners = drmaa default_cluster_job_runner = drmaa:/// }}
# paster.log file - with drmaa config the process dies with "Loaded job runner: galaxy.jobs.runners.local:LocalJobRunner" as last line in the log file. With local job runner the galaxy starts fine as shown below:
{{{ galaxy.jobs DEBUG 2011-05-17 09:59:58,959 Loaded job runner: galaxy.jobs.runners.local:LocalJobRunner galaxy.jobs INFO 2011-05-17 09:59:58,959 job manager started galaxy.jobs INFO 2011-05-17 09:59:59,325 job stopper started galaxy.sample_tracking.external_service_types DEBUG 2011-05-17 09:59:59,330 Loaded external_service_type: Simple unknown sequencer 1.0.0 galaxy.sample_tracking.external_service_types DEBUG 2011-05-17 09:59:59,333 Loaded external_service_type: Applied Biosystems SOLiD 1.0.0 galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,349 Enabling 'mobile' controller, class: Mobile galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,417 Enabling 'library_common' controller, class: LibraryCommon galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,435 Enabling 'admin' controller, class: AdminGalaxy galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,451 Enabling 'requests' controller, class: Requests galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,455 Enabling 'external_services' controller, class: ExternalServiceController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,474 Enabling 'page' controller, class: PageController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,480 Enabling 'visualization' controller, class: VisualizationController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,549 Enabling 'tracks' controller, class: TracksController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,550 Enabling 'requests_common' controller, class: RequestsCommon galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,556 Enabling 'request_type' controller, class: RequestType galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,561 Enabling 'external_service' controller, class: ExternalService galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,562 Enabling 'library' controller, class: Library galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,746 Enabling 'workflow' controller, class: WorkflowController galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,752 Enabling 'library_admin' controller, class: LibraryAdmin galaxy.web.framework.base DEBUG 2011-05-17 09:59:59,756 Enabling 'async' controller, class: ASync galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,026 Enabling 'history' controller, class: HistoryController galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,031 Enabling 'error' controller, class: Error galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,090 Enabling 'requests_admin' controller, class: RequestsAdmin galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,095 Enabling 'ucsc_proxy' controller, class: UCSCProxy galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,104 Enabling 'forms' controller, class: Forms galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,137 Enabling 'dataset' controller, class: DatasetInterface galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,143 Enabling 'tool_runner' controller, class: ToolRunner galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,148 Enabling 'tag' controller, class: TagsController galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,158 Enabling 'user' controller, class: User galaxy.web.framework.base DEBUG 2011-05-17 10:00:00,165 Enabling 'root' controller, class: RootController galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,189 Enabling 'httpexceptions' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,195 Enabling 'recursive' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,220 Enabling 'print debug' middleware /share/apps/galaxy/shantanu-temp-workspace/galaxy-cluster-test/galaxy-dist-50e249442c5a/eggs/WebError-0.8a-py2.6.egg/weberror/exceptions/serial_number_generator.py:11: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,726 Enabling 'eval exceptions' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,740 Enabling 'trans logger' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,740 Enabling 'config' middleware galaxy.web.buildapp DEBUG 2011-05-17 10:00:00,744 Enabling 'x-forwarded-host' middleware Starting server in PID 10170. serving on 0.0.0.0:8081 view at http://127.0.0.1:8081
}}}
-- Thanks, Shantanu. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On May 17, 2011, at 11:35 AM, Nate Coraor wrote:
Shantanu Pavgi wrote:
I am trying to configure galaxy with sge/drmaa scheduler. The galaxy process is starting up fine without any drmaa configuration. However the galaxy daemon doesn't start properly when I add drmaa configuration lines as shown below. I have set SGE_ROOT and DRMAA_LIBRARY_PATH variables properly. Am I missing something important here?
Could you provide the output of starting Galaxy with the drmaa options enabled? You may want to start it in the foreground (without --daemon) to capture any messages which are not output to paster.log
It's working now. I hadn't set SGE_QMASTER_PORT env variable earlier. I am guessing galaxy-start process got hung while attempting to connect with 'unknown/null' SGE port. I didn't see any error related to SGE_QMASTER_PORT in the log file, however I did notice that galaxy checks for SGE_ROOT variable value. We are using a fresh SGE submit host install here and hence we may see some initial environment configuration issues. Another community member Ross Lazarus suggested me to try sge as job_runner setting instead of pbs or drmaa. This printed out error message related to SGE_QMASTER_PORT in the logs. I set this variable and then tried drmaa setting again which worked fine this time. Thanks for the help Ross and Nate. -- Shantanu.
participants (2)
-
Nate Coraor -
Shantanu Pavgi