'pd.yaml' not found at paths.py load!
Set paths.P_GALAXY_HOME as '/mnt/galaxyTools/galaxy-central'
Python version:  (2, 6)
Image configuration suports: {'apps': ['cloudman', 'galaxy']}
[DEBUG] app:54 2013-08-01 20:07:40,138: Initializing app
[DEBUG] app:55 2013-08-01 20:07:40,138: Running on 'ec2' type of cloud.
[DEBUG] app:72 2013-08-01 20:07:40,138: Getting pd.yaml
[DEBUG] ec2:275 2013-08-01 20:07:40,138: No S3 Connection, creating a new one.
[DEBUG] ec2:278 2013-08-01 20:07:40,139: Got boto S3 connection.
[DEBUG] misc:113 2013-08-01 20:07:40,229: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not.
[DEBUG] misc:444 2013-08-01 20:07:40,229: Bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' does not exist, did not get remote file 'persistent_data.yaml'
[INFO] app:78 2013-08-01 20:07:40,229: Master starting
[DEBUG] master:70 2013-08-01 20:07:40,229: Initializing console manager - cluster start time: 2013-08-01 20:07:40.229549
[DEBUG] connection:648 2013-08-01 20:07:40,231: Start from server, version: 8.0, properties: {u'platform': 'Erlang/OTP', u'product': 'RabbitMQ', u'version': '1.7.2', u'copyright': 'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[DEBUG] connection:494 2013-08-01 20:07:40,263: Open OK! known_hosts [ip-10-179-15-41.ec2.internal:5672]
[DEBUG] channel:70 2013-08-01 20:07:40,263: using channel_id: 1
[DEBUG] channel:484 2013-08-01 20:07:40,264: Channel open
[DEBUG] comm:35 2013-08-01 20:07:40,266: Successfully established AMQP connection
[DEBUG] master:510 2013-08-01 20:07:40,266: Trying to discover any worker instances associated with this cluster...
[DEBUG] ec2:246 2013-08-01 20:07:40,267: Establishing boto EC2 connection
[DEBUG] ec2:103 2013-08-01 20:07:40,267: Gathering instance zone, attempt 0
[DEBUG] ec2:108 2013-08-01 20:07:40,270: Instance zone is 'us-east-1d'
[DEBUG] ec2:263 2013-08-01 20:07:40,516: Got boto EC2 connection for region 'us-east-1'
[DEBUG] ec2:66 2013-08-01 20:07:40,574: Gathering instance id, attempt 0
[DEBUG] ec2:71 2013-08-01 20:07:40,575: Instance ID is 'i-7b386714'
[DEBUG] ec2:295 2013-08-01 20:07:40,639: Adding tag 'clusterName:gal_oldami_form2' to resource 'i-7b386714'
[DEBUG] ec2:295 2013-08-01 20:07:41,066: Adding tag 'role:master' to resource 'i-7b386714'
[DEBUG] master:174 2013-08-01 20:07:41,223: ud at manager start: {'access_key': 'AKIAIUSCRANAJESP3GFQ', 'cloudman_home': '/mnt/cm', 'cluster_name': 'gal_oldami_form2', 'bucket_default': 'gxy-workshop', 'boot_script_name': 'cm_boot.py', 'bucket_cluster': 'cm-35f4d40b627583b6949bc5de11bcfecc', 'boot_script_path': '/tmp/cm', 'role': 'master', 'secret_key': 'EV12T4PMGSbM1IZmS7ZUJVi9nCvMncnrxXvytacJ', 'password': 'google'}
[DEBUG] pss:25 2013-08-01 20:07:41,224: Configured PSS as master
[DEBUG] master:204 2013-08-01 20:07:41,225: Cheking for and adding any preconfigured services
[DEBUG] master:529 2013-08-01 20:07:41,225: Trying to discover any volumes attached to this instance...
[DEBUG] base:55 2013-08-01 20:07:41,236: Enabling 'root' controller, class: CM
[DEBUG] buildapp:80 2013-08-01 20:07:41,239: Enabling 'httpexceptions' middleware
[DEBUG] buildapp:86 2013-08-01 20:07:41,469: Enabling 'recursive' middleware
[DEBUG] buildapp:106 2013-08-01 20:07:41,719: Enabling 'print debug' middleware
[DEBUG] buildapp:120 2013-08-01 20:07:41,734: Enabling 'error' middleware
[DEBUG] buildapp:130 2013-08-01 20:07:41,735: Enabling 'config' middleware
[DEBUG] buildapp:134 2013-08-01 20:07:41,736: Enabling 'x-forwarded-host' middleware
[DEBUG] master:535 2013-08-01 20:07:41,737: Attached volumes: [Volume:vol-fe5b7abe]
[INFO] master:198 2013-08-01 20:07:41,758: Completed initial cluster configuration. This seems to be a new cluster; waiting to configure the type
[DEBUG] master:1627 2013-08-01 20:07:41,758: Monitor started; manager started
Starting server in PID 1252.
serving on 0.0.0.0:42284 view at http://127.0.0.1:42284
[DEBUG] master:1247 2013-08-01 20:07:45,657: Checking for new version of CloudMan
[DEBUG] misc:512 2013-08-01 20:07:45,657: Getting metadata 'revision' for file 'cm.tar.gz' from bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] misc:113 2013-08-01 20:07:45,740: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not.
[DEBUG] misc:512 2013-08-01 20:07:45,740: Getting metadata 'revision' for file 'cm.tar.gz' from bucket 'gxy-workshop'
[DEBUG] master:1676 2013-08-01 20:07:45,763: Monitor adding service 'SGE'
[DEBUG] __init__:31 2013-08-01 20:07:45,763: Trying to add service 'SGE'
[INFO] __init__:50 2013-08-01 20:07:45,763: SGE prerequisites OK; starting the service
[DEBUG] sge:62 2013-08-01 20:07:45,763: Unpacking SGE from '/opt/galaxy/pkg/ge6.2u5'
[DEBUG] __init__:42 2013-08-01 20:07:45,793: 	'sge' daemon is NOT running any more (expected pid: '2521').
[DEBUG] sge:77 2013-08-01 20:07:45,793: Cleaning '/opt/sge' directory.
[DEBUG] master:1251 2013-08-01 20:07:45,804: Revision number for user's CloudMan: 'None'; revision number for default CloudMan: 'None'
[DEBUG] ec2:48 2013-08-01 20:07:45,805: Gathering instance type, attempt 0
[DEBUG] sge:83 2013-08-01 20:07:46,195: Unpacking SGE to '/opt/sge'.
[DEBUG] ec2:218 2013-08-01 20:07:46,267: Gathering instance public hostname, attempt 0
[INFO] master:817 2013-08-01 20:07:50,102: Initializing a 'Galaxy' cluster.
[INFO] misc:438 2013-08-01 20:07:51,212: Retrieved file 'snaps.yaml' from bucket 'gxy-workshop' to 'cm_snaps.yaml'.
Error - : 'static_filesystems'
[INFO] sge:97 2013-08-01 20:07:51,551: Configuring SGE...
[DEBUG] ec2:162 2013-08-01 20:07:51,552: Gathering instance private IP, attempt 0
[DEBUG] sge:105 2013-08-01 20:07:51,554: Created SGE install template as file '/opt/sge/galaxyEC2.conf'
[DEBUG] sge:119 2013-08-01 20:07:51,554: Setting up SGE.
[DEBUG] misc:577 2013-08-01 20:07:51,567: Modified /opt/sge/util/arch
[DEBUG] misc:577 2013-08-01 20:07:51,573: Successfully chmod /opt/sge/util/arch
[DEBUG] misc:577 2013-08-01 20:07:51,581: 'sed -i.bak '/^127.0.1./s/^/# (Commented by CloudMan) /' /etc/hosts' command OK
[DEBUG] ec2:142 2013-08-01 20:07:57,465: Gathering instance public keys (i.e., key pairs), attempt 0
[DEBUG] ec2:148 2013-08-01 20:07:57,468: Got key pair: 'rav_kp2'
[DEBUG] misc:577 2013-08-01 20:07:59,992: Successfully set up SGE
[INFO] sge:122 2013-08-01 20:07:59,993: Successfully setup SGE; configuring SGE
[DEBUG] sge:123 2013-08-01 20:07:59,993: Adding parallel environments
[DEBUG] misc:577 2013-08-01 20:08:00,009: 'cd /opt/sge; ./bin/lx24-amd64/qconf -Ap /tmp/SMP_PE' command OK
[DEBUG] misc:577 2013-08-01 20:08:00,026: 'cd /opt/sge; ./bin/lx24-amd64/qconf -Ap /tmp/MPI_PE' command OK
[DEBUG] sge:130 2013-08-01 20:08:00,026: Creating queue 'all.q'
[DEBUG] sge:135 2013-08-01 20:08:00,027: Created SGE all.q template as file '/opt/sge/all.q.conf'
[DEBUG] misc:577 2013-08-01 20:08:00,044: Successfully modified all.q
[DEBUG] sge:137 2013-08-01 20:08:00,045: Configuring users' SGE profiles
[DEBUG] master:1676 2013-08-01 20:08:00,045: Monitor adding service 'post_start_script'
[DEBUG] pss:51 2013-08-01 20:08:00,045: Custom-checking 'post_start_script' service prerequisites
[DEBUG] pss:58 2013-08-01 20:08:00,046: SGE not running (Starting), post_start_script service prerequisites not met afterall,not starting the service yet
[DEBUG] misc:113 2013-08-01 20:08:00,121: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not.
[DEBUG] misc:124 2013-08-01 20:08:00,312: Created bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'.
[ERROR] misc:139 2013-08-01 20:08:00,434: Problem connecting to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc', attempt 1/5
[INFO] misc:455 2013-08-01 20:08:02,976: Saved file 'persistent_data.yaml' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] misc:113 2013-08-01 20:08:03,041: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not.
[DEBUG] master:1601 2013-08-01 20:08:03,042: Saving current instance boot script (/tmp/cm/cm_boot.py) to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'cm_boot.py'
[INFO] misc:455 2013-08-01 20:08:03,293: Saved file 'cm_boot.py' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] master:1605 2013-08-01 20:08:03,508: Saving CloudMan source (/mnt/cm/cm.tar.gz) to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'cm.tar.gz'
[INFO] misc:455 2013-08-01 20:08:04,987: Saved file 'cm.tar.gz' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] misc:527 2013-08-01 20:08:04,987: Setting metadata 'revision' for file 'cm.tar.gz' in bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] master:1619 2013-08-01 20:08:05,781: Saving '/mnt/cm/gal_oldami_form2.clusterName' file to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'gal_oldami_form2.clusterName'
[INFO] misc:455 2013-08-01 20:08:06,027: Saved file 'gal_oldami_form2.clusterName' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] sge:405 2013-08-01 20:08:10,466: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.65     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:08:10,467: S&S: SGE..OK; post_start_script..Unstarted; 
[DEBUG] master:1676 2013-08-01 20:08:10,467: Monitor adding service 'post_start_script'
[DEBUG] pss:51 2013-08-01 20:08:10,467: Custom-checking 'post_start_script' service prerequisites
[DEBUG] pss:70 2013-08-01 20:08:10,468: Starting post_start_script service
[DEBUG] pss:74 2013-08-01 20:08:10,468: post_start_script service prerequisites OK (i.e., all other services running), checking if post_start_script was provided...
[DEBUG] pss:91 2013-08-01 20:08:10,535: Cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' found; looking for post start script 'post_start_script'
[DEBUG] misc:440 2013-08-01 20:08:10,856: Failed to get file 'post_start_script' from bucket 'cm-35f4d40b627583b6949bc5de11bcfecc': S3ResponseError: 404 Not Found

NoSuchKeyThe specified key does not exist.post_start_scriptA459857A0CC9D8BFqfo3h0QIZYyRSsubkp1GKCbDWljS5GNMSw0zxCOSrcO/CS5wcMU08G44u5oVdu/G
[DEBUG] pss:102 2013-08-01 20:08:10,857: post_start_script does not exist or could not be downloaded; continuing without running it.
[DEBUG] pss:40 2013-08-01 20:08:10,857: Priming local data variables...
[DEBUG] pss:107 2013-08-01 20:08:10,857: post_start_script service done and marked as 'Shut down'
[DEBUG] ec2:33 2013-08-01 20:08:10,858: Gathering instance ami, attempt 0
[DEBUG] pss:136 2013-08-01 20:08:10,858: Removing post_start_script service from master list of services
[DEBUG] ec2:122 2013-08-01 20:08:10,861: Gathering instance security group, attempt 0
[DEBUG] ec2:181 2013-08-01 20:08:10,862: Gathering instance local hostname, attempt 0
[INFO] misc:455 2013-08-01 20:08:12,333: Saved file 'persistent_data.yaml' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'
[DEBUG] sge:405 2013-08-01 20:08:25,375: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.65     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:08:25,376: S&S: SGE..OK; 
[DEBUG] sge:405 2013-08-01 20:08:37,826: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.65     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:08:37,827: S&S: SGE..OK; 
[DEBUG] sge:405 2013-08-01 20:08:50,281: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.83     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:08:50,282: S&S: SGE..OK; 
[DEBUG] sge:405 2013-08-01 20:09:02,751: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.83     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:09:02,752: S&S: SGE..OK; 
[DEBUG] sge:405 2013-08-01 20:09:15,209: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.83     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:09:15,209: S&S: SGE..OK; 
[DEBUG] sge:405 2013-08-01 20:09:27,665: qstat: ['all.q@ip-10-179-15-41.ec2.inte BIP   0/0/2          0.91     lx24-amd64    ']
[DEBUG] master:1650 2013-08-01 20:09:27,666: S&S: SGE..OK;