Hi,
trying to bring up a new instance.
1, remove old cm from S3 bucket 2, bring up new 3, get console 4, SGE and fs running 5,Database and galaxy are not running 6, on /mnt i see /mnt and galaxyData
East 1c
xy Cloudman Admin | Report bugs | Wiki | Screencast Only up to the most recent 200 lines of the file (paster.log) are shown. Show all | Show less | Show more | Back to admin view
[DEBUG] sge:311 2011-10-14 08:54:00,368: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.24 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:54:00,445: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:54:16,706: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.24 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:54:16,759: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:54:33,018: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.24 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:54:33,085: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:54:49,421: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.20 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:54:49,853: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:55:06,110: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.20 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:55:06,161: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:55:22,440: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.19 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:55:22,754: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:55:39,011: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.19 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:55:39,063: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:55:55,326: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.19 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:55:55,375: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:56:11,634: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.19 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:56:11,684: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:56:27,948: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.19 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:56:28,002: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:56:44,264: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.20 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:56:44,356: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:57:00,623: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.20 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:57:00,672: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:57:16,920: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.17 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:57:16,971: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:57:33,229: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.17 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:57:33,276: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:57:49,529: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.17 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:57:49,579: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:58:05,844: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.18 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:58:05,938: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:58:22,207: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.18 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:58:22,258: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:58:38,512: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.14 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:58:38,562: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:58:54,832: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.14 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:58:54,896: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:59:11,159: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.14 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:59:11,223: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:59:27,542: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.09 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:59:27,631: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 08:59:43,920: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.09 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 08:59:43,968: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:00:00,232: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.06 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:00:00,285: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:00:16,552: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.06 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:00:16,636: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] master:468 2011-10-14 09:00:17,717: List of services before shutdown: ['SGE', 'FS-galaxyData'] [DEBUG] master:237 2011-10-14 09:00:17,718: Not stopping autoscaling because it is not on. [INFO] sge:28 2011-10-14 09:00:17,718: Removing SGE service [DEBUG] misc:511 2011-10-14 09:00:17,945: Successfully stopped SGE master [INFO] master:498 2011-10-14 09:00:17,946: Cluster shut down at 2011-10-14 09:00:17.946341 (uptime: 6:16:36.067737). If not done automatically, manually terminate the master instance (and any remaining instances associated with this cluster) from the AWS console. Python version: (2, 6) [DEBUG] app:42 2011-10-14 09:01:05,719: Initializing app [DEBUG] ec2:207 2011-10-14 09:01:05,719: Getting boto S3 connection [DEBUG] ec2:209 2011-10-14 09:01:05,719: No S3 Connection, creating a new one. [DEBUG] ec2:212 2011-10-14 09:01:05,719: Got boto S3 connection. [INFO] misc:378 2011-10-14 09:01:05,792: Retrieved file 'persistent_data.yaml' from bucket 'cm-cefdad8733f8efe0be7f3c3f5059d9b8' to 'pd.yaml'. [INFO] app:52 2011-10-14 09:01:05,794: Master starting [DEBUG] master:50 2011-10-14 09:01:05,826: Initializing console manager - cluster start time: 2011-10-14 09:01:05.826253 [DEBUG] connection:648 2011-10-14 09:01:05,828: Start from server, version: 8.0, properties: {u'platform': 'Erlang/OTP', u'product': 'RabbitMQ', u'version': '1.7.2', u'copyright': 'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': 'Licensed under the MPL. See http://www.rabbitmq.com/%27%7D, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US'] [DEBUG] connection:494 2011-10-14 09:01:05,865: Open OK! known_hosts [ip-10-191-205-72.ec2.internal:5672] [DEBUG] channel:70 2011-10-14 09:01:05,865: using channel_id: 1 [DEBUG] channel:484 2011-10-14 09:01:05,866: Channel open [DEBUG] comm:35 2011-10-14 09:01:05,867: Successfully established AMQP connection [DEBUG] master:436 2011-10-14 09:01:05,867: Trying to discover any worker instances associated with this cluster... [DEBUG] ec2:177 2011-10-14 09:01:05,867: Establishing boto EC2 connection [DEBUG] ec2:85 2011-10-14 09:01:05,868: Gathering instance zone, attempt 0 [DEBUG] ec2:90 2011-10-14 09:01:05,870: Instance zone is 'us-east-1c' [DEBUG] ec2:198 2011-10-14 09:01:06,078: Got boto EC2 connection for region 'us-east-1' [DEBUG] ec2:66 2011-10-14 09:01:06,128: Gathering instance id, attempt 0 [DEBUG] ec2:71 2011-10-14 09:01:06,130: Instance ID is 'i-2e4f734e' [DEBUG] ec2:228 2011-10-14 09:01:06,215: Adding tag 'clusterName:WASP' to resource 'i-2e4f734e' [DEBUG] ec2:228 2011-10-14 09:01:06,257: Adding tag 'role:master' to resource 'i-2e4f734e' [DEBUG] master:147 2011-10-14 09:01:06,299: ud at manager start: {'access_key': 'AKIAI4G3E4I2722APZZA', 'data_filesystems': {'galaxyData': [{'vol_id': 'vol-872314ed', 'size': 700}]}, 'cloudman_home': '/mnt/cm', 'cluster_name': 'WASP', 'services': [{'service': 'SGE'}], 'bucket_default': 'cloudman', 'boot_script_name': 'cm_boot.py', 'bucket_cluster': 'cm-cefdad8733f8efe0be7f3c3f5059d9b8', 'boot_script_path': '/tmp/cm', 'role': 'master', 'secret_key': 'ygWVriMaS0osJC6Ix3QZvKue19EdLtL5yUV7EuAY', 'password': 'WASP'} [DEBUG] master:455 2011-10-14 09:01:06,299: Trying to discover any volumes attached to this instance... [DEBUG] base:55 2011-10-14 09:01:06,303: Enabling 'root' controller, class: CM [DEBUG] buildapp:80 2011-10-14 09:01:06,305: Enabling 'httpexceptions' middleware [DEBUG] buildapp:86 2011-10-14 09:01:06,307: Enabling 'recursive' middleware [DEBUG] buildapp:106 2011-10-14 09:01:06,320: Enabling 'print debug' middleware [DEBUG] buildapp:120 2011-10-14 09:01:06,338: Enabling 'error' middleware [DEBUG] buildapp:130 2011-10-14 09:01:06,338: Enabling 'config' middleware [DEBUG] buildapp:134 2011-10-14 09:01:06,339: Enabling 'x-forwarded-host' middleware Starting server in PID 1009. [DEBUG] master:461 2011-10-14 09:01:06,369: Attached volumes: [Volume:vol-1b251271, Volume:vol-872314ed] [DEBUG] master:185 2011-10-14 09:01:06,369: Adding a previously existing data filesystem: 'galaxyData' [DEBUG] master:193 2011-10-14 09:01:06,369: Adding service: 'SGE' [INFO] master:155 2011-10-14 09:01:06,369: Completed initial cluster configuration. [DEBUG] master:1488 2011-10-14 09:01:06,369: Monitor started; manager started serving on 0.0.0.0:42284 view at http://127.0.0.1:42284 [DEBUG] master:1515 2011-10-14 09:01:10,372: Monitor adding service 'SGE' [DEBUG] __init__:31 2011-10-14 09:01:10,372: Trying to add service 'SGE' [INFO] __init__:49 2011-10-14 09:01:10,372: Prerequisites OK; starting service 'SGE' [DEBUG] sge:61 2011-10-14 09:01:10,372: Unpacking SGE from '/opt/galaxy/pkg/ge6.2u5' [DEBUG] __init__:42 2011-10-14 09:01:10,434: 'sge' daemon is NOT running any more (expected pid: '2530'). [DEBUG] sge:76 2011-10-14 09:01:10,435: Cleaning '/opt/sge' directory. [DEBUG] sge:82 2011-10-14 09:01:10,934: Unpacking SGE to '/opt/sge'. [INFO] sge:96 2011-10-14 09:01:12,949: Configuring SGE... [DEBUG] ec2:136 2011-10-14 09:01:12,949: Gathering instance private hostname, attempt 0 [DEBUG] sge:104 2011-10-14 09:01:12,952: Created SGE install template as file '/opt/sge/galaxyEC2.conf' [DEBUG] sge:112 2011-10-14 09:01:12,952: Setting up SGE. [DEBUG] misc:511 2011-10-14 09:01:20,181: Successfully set up SGE [INFO] sge:114 2011-10-14 09:01:20,181: Successfully setup SGE; configuring SGE [DEBUG] sge:119 2011-10-14 09:01:20,182: Created SGE all.q template as file '/opt/sge/all.q.conf' [DEBUG] misc:511 2011-10-14 09:01:20,200: Successfully modified all.q [DEBUG] misc:511 2011-10-14 09:01:20,211: Modified /opt/sge/util/arch [DEBUG] misc:511 2011-10-14 09:01:20,217: Successfully chmod /opt/sge/util/arch [DEBUG] sge:125 2011-10-14 09:01:20,218: Configuring users' SGE profiles [DEBUG] master:1515 2011-10-14 09:01:20,220: Monitor adding service 'FS-galaxyData' [DEBUG] filesystem:311 2011-10-14 09:01:20,220: Trying to add service 'FS-galaxyData' [DEBUG] filesystem:143 2011-10-14 09:01:20,397: Tried to create a volume but it is in state 'attached' (volume ID: vol-872314ed) [DEBUG] ec2:228 2011-10-14 09:01:20,397: Adding tag 'clusterName:WASP' to resource 'vol-872314ed' [DEBUG] ec2:228 2011-10-14 09:01:20,435: Adding tag 'filesystem:galaxyData' to resource 'vol-872314ed' [DEBUG] filesystem:214 2011-10-14 09:01:20,641: Tried to attach a volume but the volume 'vol-872314ed' is already attached (as device /dev/sdg1) [DEBUG] filesystem:426 2011-10-14 09:01:20,726: Path '/dev/sdg1' checked and exists. [DEBUG] misc:511 2011-10-14 09:01:20,862: Successfully mounted file system '/mnt/galaxyData' from '/dev/sdg1' [DEBUG] misc:511 2011-10-14 09:01:20,874: Successfully edited '/mnt/galaxyData' in '/etc/exports' for NFS. [DEBUG] misc:511 2011-10-14 09:01:21,962: As part of filesystem 'Filesystem-galaxyData' update, successfully restarted NFS server [DEBUG] ec2:207 2011-10-14 09:01:22,014: Getting boto S3 connection [INFO] misc:394 2011-10-14 09:01:22,109: Saved file 'persistent_data.yaml' to bucket 'cm-cefdad8733f8efe0be7f3c3f5059d9b8' [DEBUG] master:1462 2011-10-14 09:01:22,139: Instance post start script (/mnt/cm/post_start_script) does not exist? [DEBUG] sge:311 2011-10-14 09:01:26,429: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.08 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:01:26,485: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:01:42,730: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.08 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:01:42,785: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:01:59,035: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.08 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:01:59,089: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:02:15,330: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.24 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:02:15,407: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:02:31,656: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.24 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:02:31,706: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:02:47,950: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.36 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:02:48,002: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:03:04,261: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.36 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:03:04,312: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:03:20,568: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.36 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:03:20,614: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:03:36,870: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.44 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:03:36,914: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:03:53,167: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.44 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:03:53,252: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:04:09,499: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.59 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:04:09,550: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:04:25,800: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.59 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:04:25,847: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:04:42,094: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.68 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:04:42,150: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:04:58,405: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.68 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:04:58,482: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:05:14,736: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.68 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:05:14,784: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:05:31,037: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.76 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:05:31,087: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:05:47,336: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.76 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:05:47,383: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:06:03,697: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.87 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:06:03,748: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:06:19,994: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.87 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:06:20,045: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:06:36,297: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 0.87 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:06:36,369: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:06:52,621: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:06:52,677: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:07:08,931: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:07:08,982: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:07:25,231: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:07:25,286: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:07:41,534: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:07:41,617: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:07:57,865: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:07:57,911: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:08:14,164: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.01 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:08:14,211: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:08:30,453: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.01 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:08:30,499: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:08:46,746: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.06 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:08:46,796: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:09:03,051: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.06 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:09:03,107: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:09:19,360: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.06 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:09:19,442: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:09:35,698: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:09:35,751: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:09:52,063: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.00 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:09:52,106: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:10:08,360: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.05 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:10:08,419: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:10:24,674: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.05 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:10:24,754: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:10:40,999: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.05 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:10:41,046: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:10:57,295: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.05 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:10:57,344: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:11:13,595: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.05 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:11:13,646: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:11:29,895: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.03 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:11:29,947: S&S: SGE..OK; FS-galaxyData..OK; [DEBUG] sge:311 2011-10-14 09:11:46,190: qstat: ['all.q@ip-10-191-205-72.ec2.int BIP 0/0/4 1.03 lx24-amd64 '] [DEBUG] master:1510 2011-10-14 09:11:46,234: S&S: SGE..OK; FS-galaxyData..OK;