August 2013 - galaxy-dev - lists.galaxyproject.org

Problems with Tophat & uploading files to Cloudman by FTP
by Anto Praveen Rajkumar Rajamani 11 Aug '13

11 Aug '13

Hello, I started a new cloudman instance two days ago. I am stuck with two problems. 1. When I try to upload my fastq files to FTP server using Filezilla (I typed my ec2... address as the host and entered my cloudman registered email address and password), Filezilla connects to 220-Galaxy Cloudman FTP but then rejects my password and displays "Could not connect to server". I could see some command line based suggestions in this forum (http://dev.list.galaxyproject.org/Connecting-to-new-Galaxy-Cloudman-by-FTP-…) but I do not know how to employ them. 2. When I try tophat on my pilot data, Cloudman fails and gives a message, "Could not determine Tophat version /mnt/galaxy/tmp/job_working_directory/000/30/galaxy_30.sh: 13: /mnt/galaxy/tmp/job_working_directory/000/30/galaxy_30.sh: tophat: not found /bin/sh: 1: tophat: not found Error indexing reference sequence /bin/sh: 1: bow" I repeated running tophat many times without any success. I tried uploading fastq files to my Google drive and provided the file download url to the upload file tool. It did not work. Kindly help me to come out of these problems. Thanking you in advance. Best wishes, Anto

1 1

Fwd: Public production instance page is not rendering properly
by Vipin TS 09 Aug '13

09 Aug '13

--/Vipin ---------- Forwarded message ---------- From: Vipin TS <vipin.ts(a)gmail.com> Date: Fri, Aug 9, 2013 at 5:04 PM Subject: Re: [galaxy-dev] Public production instance page is not rendering properly To: Daniel Blankenberg <dan(a)bx.psu.edu> Hi Dan, Thanks for your support, Hmm, I can see that there are changes: but here is the out come from the below commands: $$ hg diff templates/webapps/galaxy/root/history.mako empty $$ hg summary parent: 10218:ff0bd4a811d9 tip merged the changes from stable release version July 2013 from bx.psu.edu branch: default commit: 5 modified, 5 removed, 452 deleted update: 1663 new changesets, 2 branch heads (merge) $$hg config Sorry, it says unknown option. Do you think I have mixed up something ? Thank you, Vipin On Fri, Aug 9, 2013 at 4:41 PM, Daniel Blankenberg <dan(a)bx.psu.edu> wrote: > Hi Vipin, > > Strange, are you running the latest galaxy-dist? That line doesn't exist > in e.g. > https://bitbucket.org/galaxy/galaxy-dist/src/ebe87051fadff5750ec226490b31e4…. > Can you compare your file to the file in the repository? What is the output > of: > > hg diff templates/webapps/galaxy/root/history.mako > > and > > hg summary > > and > > hg config > > > > > Thanks for using Galaxy, > > Dan > > On Aug 9, 2013, at 3:56 PM, Vipin TS wrote: > > Hi Dan, > > I deleted the > contents from > /codebase/database/complied_templates/* and restarted the instance, still > no luck. Sorry. > > Thanks, Vipin > > > On Fri, Aug 9, 2013 at 3:43 PM, Daniel Blankenberg <dan(a)bx.psu.edu> wrote: > >> Hi Vipin, >> >> Can you try deleting the contents of your compiled templates directory >> (i.e. >> /codebase/database/compiled_templates/*)? >> >> Thanks for using Galaxy, >> >> Dan >> >> >> On Aug 9, 2013, at 3:22 PM, Vipin TS wrote: >> >> Hi Dan, >> >> Thanks for the information, I have updated this to my repository >> manually. Now I can invoke the page but the frames were not loaded >> correctly. >> >> The log message as follows: >> /codebase/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/sql/expression.py:1927: >> SAWarning: The IN-predicate on "request_type.id" was invoked with an >> empty sequence. This results in a contradiction, which nonetheless can be >> expensive to evaluate. Consider alternative strategies for improved >> performance. >> X.X.X.X - - [09/Aug/2013:15:09:55 -0400] "GET / HTTP/1.1" 200 - "-" >> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.26.17 >> (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17" >> X.X.X.X - - [09/Aug/2013:15:09:56 -0400] "GET /root/tool_menu HTTP/1.1" >> 404 - "https://galaxy.cbio.mskcc.org/" "Mozilla/5.0 (Macintosh; Intel >> Mac OS X 10_7_5) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 >> Safari/536.26.17" >> X.X.X.X - - [09/Aug/2013:15:09:56 -0400] "GET /history HTTP/1.1" 200 - " >> https://galaxy.cbio.mskcc.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X >> 10_7_5) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 >> Safari/536.26.17" >> X.X.X.X - - [09/Aug/2013:15:09:56 -0400] "GET /history HTTP/1.1" 500 - " >> https://galaxy.cbio.mskcc.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X >> 10_7_5) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 >> Safari/536.26.17" >> ---------------------------------------- >> Exception happened during processing of request from ('127.0.0.1', 43370) >> Traceback (most recent call last): >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line >> 1068, in process_request_in_thread >> self.finish_request(request, client_address) >> File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request >> self.RequestHandlerClass(request, client_address, self) >> File "/usr/lib64/python2.6/SocketServer.py", line 617, in __init__ >> self.handle() >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line >> 442, in handle >> BaseHTTPRequestHandler.handle(self) >> File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle >> self.handle_one_request() >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line >> 437, in handle_one_request >> self.wsgi_execute() >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line >> 287, in wsgi_execute >> self.wsgi_start_response) >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/urlmap.py", line >> 203, in __call__ >> return app(environ, start_response) >> File "/codebase/lib/galaxy/web/framework/middleware/request_id.py", >> line 12, in __call__ >> return self.app( environ, start_response ) >> File "/codebase/lib/galaxy/web/framework/middleware/xforwardedhost.py", >> line 21, in __call__ >> return self.app( environ, start_response ) >> File "/codebase/lib/galaxy/web/framework/middleware/translogger.py", >> line 68, in __call__ >> return self.application(environ, replacement_start_response) >> File "/codebase/lib/galaxy/web/framework/middleware/error.py", line >> 159, in __call__ >> exc_info) >> File "/codebase/lib/galaxy/web/framework/middleware/translogger.py", >> line 67, in replacement_start_response >> return start_response( status, headers, exc_info ) >> File "/codebase/lib/galaxy/web/framework/middleware/error.py", line >> 149, in __call__ >> app_iter = self.application(environ, sr_checker) >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py", line >> 84, in __call__ >> return self.application(environ, start_response) >> File "/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py", >> line 633, in __call__ >> return self.application(environ, start_response) >> File "/codebase/lib/galaxy/web/framework/base.py", line 132, in __call__ >> return self.handle_request( environ, start_response ) >> File "/codebase/lib/galaxy/web/framework/base.py", line 199, in >> handle_request >> return body( environ, start_response ) >> File "/codebase/lib/galaxy/web/framework/__init__.py", line 985, in >> render >> template.render_context( context ) >> File "/codebase/eggs/Mako-0.4.1-py2.6.egg/mako/template.py", line 319, >> in render_context >> **kwargs) >> File "/codebase/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py", line 692, >> in _render_context >> _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) >> File "/codebase/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py", line 718, >> in _exec_template >> callable_(context, *args, **kwargs) >> File "/codebase/database/compiled_templates/root/history.mako.py", >> line 73, in render_body >> if bool( [ data for data in history.active_datasets if data.state in >> ['running', 'queued', '', None ] ] ): >> AttributeError: 'Undefined' object has no attribute 'active_datasets' >> >> Do you have any quick way to fix the same? Or update to galaxy-central >> stable branch? >> >> Thank you, Vipin >> >> On Fri, Aug 9, 2013 at 2:57 PM, Daniel Blankenberg <dan(a)bx.psu.edu>wrote: >> >>> Hi Vipin, >>> >>> This has been fixed in the stable branch of galaxy-central >>> https://bitbucket.org/galaxy/galaxy-central/commits/cd6fec2f5cba but >>> has not made it on to galaxy-dist yet. >>> >>> >>> Thanks for using Galaxy, >>> >>> Dan >>> >>> >>> >>> On Aug 9, 2013, at 2:33 PM, Vipin TS wrote: >>> >>> Hello dev-team, >>> >>> I am experiencing a strange problem with my production instance, after I >>> carefully merged remote changes from galaxy-dist release to my code >>> repository. While restarting, It fails to load the the page properly, the >>> message from log file stating that "serving on 0.0.0.0:8080" >>> >>> Here are some message found in the log file: >>> >>> X.X.X.X - - [09/Aug/2013:14:30:56 -0400] "GET / HTTP/1.1" 500 - "-" >>> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) >>> rekonq Safari/534.34" >>> Error - <type 'exceptions.ValueError'>: expected only letters, got ' en' >>> URL: https://galaxy.cbio.mskcc.org/ >>> File '/codebase/lib/galaxy/web/framework/middleware/error.py', line 149 >>> in __call__ >>> app_iter = self.application(environ, sr_checker) >>> File '/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line >>> 84 in __call__ >>> return self.application(environ, start_response) >>> File '/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', >>> line 633 in __call__ >>> return self.application(environ, start_response) >>> File '/codebase/lib/galaxy/web/framework/base.py', line 132 in __call__ >>> return self.handle_request( environ, start_response ) >>> File '/codebase/lib/galaxy/web/framework/base.py', line 159 in >>> handle_request >>> trans = self.transaction_factory( environ ) >>> File '/codebase/lib/galaxy/web/framework/__init__.py', line 238 in >>> <lambda> >>> self.set_transaction_factory( lambda e: self.transaction_chooser( e, >>> galaxy_app, session_cookie ) ) >>> File '/codebase/lib/galaxy/web/framework/__init__.py', line 269 in >>> transaction_chooser >>> return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie >>> ) >>> File '/codebase/lib/galaxy/web/framework/__init__.py', line 332 in >>> __init__ >>> self.setup_i18n() >>> File '/codebase/lib/galaxy/web/framework/__init__.py', line 370 in >>> setup_i18n >>> locales.append( Locale.parse( locale.split( ';' )[0], sep='-' >>> ).language ) >>> File '/codebase/eggs/Babel-0.9.4-py2.6.egg/babel/core.py', line 212 in >>> parse >>> return cls(*parse_locale(identifier, sep=sep)) >>> File '/codebase/eggs/Babel-0.9.4-py2.6.egg/babel/core.py', line 763 in >>> parse_locale >>> raise ValueError('expected only letters, got %r' % lang) >>> ValueError: expected only letters, got ' en' >>> >>> >>> CGI Variables >>> ------------- >>> CONTENT_LENGTH: '0' >>> HTTP_ACCEPT: >>> 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' >>> HTTP_ACCEPT_CHARSET: 'utf-8,*;q=0.5' >>> HTTP_ACCEPT_ENCODING: 'gzip, deflate, x-gzip, x-deflate' >>> HTTP_ACCEPT_LANGUAGE: 'en-US, en-US; q=0.8, en; q=0.6' >>> HTTP_CACHE_CONTROL: 'max-age=0' >>> HTTP_CONNECTION: 'Keep-Alive' >>> HTTP_HOST: 'galaxy.cbio.mskcc.org' >>> HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 >>> (KHTML, like Gecko) rekonq Safari/534.34' >>> HTTP_X_FORWARDED_FOR: '172.21.50.129' >>> HTTP_X_FORWARDED_HOST: 'galaxy.cbio.mskcc.org' >>> HTTP_X_FORWARDED_SERVER: 'galaxy.cbio.mskcc.org' >>> HTTP_X_URL_SCHEME: 'https' >>> ORGINAL_HTTP_HOST: 'localhost:8081' >>> ORGINAL_REMOTE_ADDR: '127.0.0.1' >>> PATH_INFO: '/' >>> REMOTE_ADDR: '172.21.50.129' >>> REQUEST_METHOD: 'GET' >>> SERVER_NAME: '0.0.0.0' >>> SERVER_PORT: '8081' >>> SERVER_PROTOCOL: 'HTTP/1.1' >>> >>> >>> WSGI Variables >>> -------------- >>> application: <paste.recursive.RecursiveMiddleware object at 0x7f78910> >>> is_api_request: False >>> original_wsgi.url_scheme: 'http' >>> paste.expected_exceptions: [<class >>> 'paste.httpexceptions.HTTPException'>] >>> paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler >>> object at 0x7f78210> >>> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at >>> 0x8066d90> >>> paste.recursive.forward: <paste.recursive.Forwarder from /> >>> paste.recursive.include: <paste.recursive.Includer from /> >>> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter >>> from /> >>> paste.recursive.script_name: '' >>> paste.throw_errors: True >>> request_id: 'd4f643a8012111e3819090b11c087d99' >>> wsgi process: 'Multithreaded' >>> ------------------------------------------------------------ >>> >>> Any idea what is going wrong here? Please let me know if you need more >>> get more details. >>> >>> Many thanks, >>> --/Vipin >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >>> >>> >> >> > >

1 1

Public production instance page is not rendering properly
by Vipin TS 09 Aug '13

09 Aug '13

Hello dev-team, I am experiencing a strange problem with my production instance, after I carefully merged remote changes from galaxy-dist release to my code repository. While restarting, It fails to load the the page properly, the message from log file stating that "serving on 0.0.0.0:8080" Here are some message found in the log file: X.X.X.X - - [09/Aug/2013:14:30:56 -0400] "GET / HTTP/1.1" 500 - "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) rekonq Safari/534.34" Error - <type 'exceptions.ValueError'>: expected only letters, got ' en' URL: https://galaxy.cbio.mskcc.org/ File '/codebase/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/codebase/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/codebase/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/codebase/lib/galaxy/web/framework/base.py', line 159 in handle_request trans = self.transaction_factory( environ ) File '/codebase/lib/galaxy/web/framework/__init__.py', line 238 in <lambda> self.set_transaction_factory( lambda e: self.transaction_chooser( e, galaxy_app, session_cookie ) ) File '/codebase/lib/galaxy/web/framework/__init__.py', line 269 in transaction_chooser return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie ) File '/codebase/lib/galaxy/web/framework/__init__.py', line 332 in __init__ self.setup_i18n() File '/codebase/lib/galaxy/web/framework/__init__.py', line 370 in setup_i18n locales.append( Locale.parse( locale.split( ';' )[0], sep='-' ).language ) File '/codebase/eggs/Babel-0.9.4-py2.6.egg/babel/core.py', line 212 in parse return cls(*parse_locale(identifier, sep=sep)) File '/codebase/eggs/Babel-0.9.4-py2.6.egg/babel/core.py', line 763 in parse_locale raise ValueError('expected only letters, got %r' % lang) ValueError: expected only letters, got ' en' CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_CHARSET: 'utf-8,*;q=0.5' HTTP_ACCEPT_ENCODING: 'gzip, deflate, x-gzip, x-deflate' HTTP_ACCEPT_LANGUAGE: 'en-US, en-US; q=0.8, en; q=0.6' HTTP_CACHE_CONTROL: 'max-age=0' HTTP_CONNECTION: 'Keep-Alive' HTTP_HOST: 'galaxy.cbio.mskcc.org' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) rekonq Safari/534.34' HTTP_X_FORWARDED_FOR: '172.21.50.129' HTTP_X_FORWARDED_HOST: 'galaxy.cbio.mskcc.org' HTTP_X_FORWARDED_SERVER: 'galaxy.cbio.mskcc.org' HTTP_X_URL_SCHEME: 'https' ORGINAL_HTTP_HOST: 'localhost:8081' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/' REMOTE_ADDR: '172.21.50.129' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8081' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f78910> is_api_request: False original_wsgi.url_scheme: 'http' paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f78210> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x8066d90> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'd4f643a8012111e3819090b11c087d99' wsgi process: 'Multithreaded' ------------------------------------------------------------ Any idea what is going wrong here? Please let me know if you need more get more details. Many thanks, --/Vipin

2 5

Error running oasesoptimizer rev. 53e887dda799 (konradpaszkiewicz) on Galaxy fd4113962c32+ (stable)
by Rodolfo Aramayo 09 Aug '13

09 Aug '13

Hi, Regarding the error I experienced with [galaxy-dev] Error running oasesoptimizer rev. 53e887dda799 (konradpaszkiewicz) on Galaxy fd4113962c32+ (stable) Two things: FIRST The error I reported, in short, it was my fault as I failed to to set up the: cmdline = '/usr/local/galaxy/bioinfosoft/shed_tools/toolshed.g2.bx.psu.edu/repos/konradpaszkiewicz/oasesoptimiser/53e887dda799/oasesoptimiser/oases_optimiser.sh %s %s \'%s\' %s %s 2&1>/dev/null' line properly in the oases_optimiser.py script properly I guess it would help for developers to state exactly the customization required for each package installed via ToolShed for the tool to work properly or better yet, it would be really nice if we could set up environmental variables like: TOOL_SHED_REPOS=/usr/local/galaxy/bioinfosoft/shed_tools/toolshed.g2.bx.psu… so that scripts could invoke them upon installation: cmdline = '$TOOL_SHED_REPOS/konradpaszkiewicz/oasesoptimiser/53e887dda799/oasesoptimiser/oases_optimiser.sh %s %s \'%s\' %s %s 2&1>/dev/null' SECOND I still am experiencing errors with this tool When I run oases_optimiser on either fasta or fastq files the jobs start, run and then fail In all cases the problem is the lack of an ''splicing_events.txt' file Help will be appreciated Thanks --R ##### The stderr is as follows: Traceback (most recent call last): File "/usr/local/galaxy/bioinfosoft/shed_tools/toolshed.g2.bx.psu.edu/repos/konradpaszkiewicz/oasesoptimiser/53e887dda799/oasesoptimiser/oases_optimiser.py", line 120, in <module> if __name__ == "__main__": __main__() File "/usr/local/galaxy/bioinfosoft/shed_tools/toolshed.g2.bx.psu.edu/repos/konradpaszkiewicz/oasesoptimiser/53e887dda799/oasesoptimiser/oases_optimiser.py", line 76, in __main__ for line in open(path ): IOError: [Errno 2] No such file or directory: 'splicing_events.txt' ##### The stdout is as follows: [0.000000] Reading FastQ file /usr/local/galaxy/galaxy-dist/database/files/003/dataset_3687.dat; [13.354877] 8855297 sequences found [13.354879] Done [13.354940] Reading read set file ./Sequences; [14.405160] 8855297 sequences found [20.956147] Done [20.956153] 8855297 sequences in total. [20.956229] Writing into roadmap file ./Roadmaps... [23.008558] Inputting sequences... [23.008571] Inputting sequence 0 / 8855297 [25.835351] Inputting sequence 1000000 / 8855297 [28.835964] Inputting sequence 2000000 / 8855297 [31.938277] Inputting sequence 3000000 / 8855297 [35.129436] Inputting sequence 4000000 / 8855297 [38.384093] Inputting sequence 5000000 / 8855297 [41.660153] Inputting sequence 6000000 / 8855297 [44.905326] Inputting sequence 7000000 / 8855297 [48.168615] Inputting sequence 8000000 / 8855297 [50.969068] === Sequences loaded in 28.188442 s [50.969101] Done inputting sequences [50.969103] Destroying splay table [51.018112] Splay table destroyed [0.000000] Reading roadmap file ./Roadmaps [13.140830] 8855297 roadmaps read [13.144978] Creating insertion markers [15.796233] Ordering insertion markers [21.652798] Counting preNodes [22.631404] 2753107 preNodes counted, creating them now [24.516794] Sequence 1000000 / 8855297 [25.609456] Sequence 2000000 / 8855297 [26.585862] Sequence 3000000 / 8855297 [27.501523] Sequence 4000000 / 8855297 [28.375045] Sequence 5000000 / 8855297 [29.217391] Sequence 6000000 / 8855297 [30.037355] Sequence 7000000 / 8855297 [30.838062] Sequence 8000000 / 8855297 [31.514315] Adjusting marker info... [32.498044] Connecting preNodes [32.734023] Connecting 1000000 / 8855297 [33.029694] Connecting 2000000 / 8855297 [33.360000] Connecting 3000000 / 8855297 [33.712200] Connecting 4000000 / 8855297 [34.080947] Connecting 5000000 / 8855297 [34.464629] Connecting 6000000 / 8855297 [34.853754] Connecting 7000000 / 8855297 [35.256476] Connecting 8000000 / 8855297 [35.604610] Cleaning up memory [35.622205] Done creating preGraph [35.622208] Concatenation... [36.944164] Renumbering preNodes [36.944174] Initial preNode count 2753107 [36.992738] Destroyed 2139761 preNodes [36.992745] Concatenation over! [36.992747] Clipping short tips off preGraph [37.241017] Concatenation... [37.561728] Renumbering preNodes [37.561735] Initial preNode count 613346 [37.564816] Destroyed 602327 preNodes [37.564818] Concatenation over! [37.564819] 565869 tips cut off [37.564820] 11019 nodes left [37.564908] Writing into pregraph file ./PreGraph... [37.628994] Reading read set file ./Sequences; [38.666026] 8855297 sequences found [45.262765] Done [47.287358] Reading pre-graph file ./PreGraph [47.287449] Graph has 11019 nodes and 8855297 sequences [47.320337] Scanning pre-graph file ./PreGraph for k-mers [47.345965] 587492 kmers found [47.410094] Sorting kmer occurence table ... [47.807351] Sorting done. [47.807356] Computing acceleration table... [47.839961] Computing offsets... [47.856571] Ghost Threading through reads 0 / 8855297 [49.327252] Ghost Threading through reads 1000000 / 8855297 [50.789941] Ghost Threading through reads 2000000 / 8855297 [52.251861] Ghost Threading through reads 3000000 / 8855297 [53.719122] Ghost Threading through reads 4000000 / 8855297 [55.175811] Ghost Threading through reads 5000000 / 8855297 [56.640149] Ghost Threading through reads 6000000 / 8855297 [58.103628] Ghost Threading through reads 7000000 / 8855297 [59.570354] Ghost Threading through reads 8000000 / 8855297 [60.865193] === Ghost-Threaded in 13.008623 s [60.865203] Threading through reads 0 / 8855297 [62.545417] Threading through reads 1000000 / 8855297 [64.224281] Threading through reads 2000000 / 8855297 [65.912903] Threading through reads 3000000 / 8855297 [67.591620] Threading through reads 4000000 / 8855297 [69.271917] Threading through reads 5000000 / 8855297 [70.956665] Threading through reads 6000000 / 8855297 [72.633673] Threading through reads 7000000 / 8855297 [74.323969] Threading through reads 8000000 / 8855297 [75.752202] === Threaded in 14.886999 s [76.213422] Correcting graph with cutoff 0.200000 [76.215727] Determining eligible starting points [76.223605] Done listing starting nodes [76.223606] Initializing todo lists [76.224193] Done with initilization [76.224195] Activating arc lookup table [76.224796] Done activating arc lookup table [76.246815] 10000 / 11019 nodes visited [76.247077] Concatenation... [76.247364] Renumbering nodes [76.247365] Initial node count 11019 [76.247654] Removed 140 null nodes [76.247655] Concatenation over! [76.247655] Clipping short tips off graph, drastic [76.251632] Concatenation... [76.363538] Renumbering nodes [76.363546] Initial node count 10879 [76.363860] Removed 3988 null nodes [76.363861] Concatenation over! [76.363862] 6891 nodes left [76.363931] Writing into graph file ./Graph2... [77.954766] WARNING: NO COVERAGE CUTOFF PROVIDED [77.954770] Velvet will probably leave behind many detectable errors [77.954771] See manual for instructions on how to set the coverage cutoff parameter [77.954779] Removing contigs with coverage < -1.000000... [77.959164] Concatenation... [77.959591] Renumbering nodes [77.959591] Initial node count 6891 [77.959603] Removed 0 null nodes [77.959604] Concatenation over! [77.959817] Concatenation... [77.959947] Renumbering nodes [77.959947] Initial node count 6891 [77.959958] Removed 0 null nodes [77.959958] Concatenation over! [77.959962] Clipping short tips off graph, drastic [77.960338] Concatenation... [77.960467] Renumbering nodes [77.960468] Initial node count 6891 [77.960479] Removed 0 null nodes [77.960479] Concatenation over! [77.960479] 6891 nodes left [77.960481] WARNING: NO EXPECTED COVERAGE PROVIDED [77.960481] Velvet will be unable to resolve any repeats [77.960481] See manual for instructions on how to set the expected coverage parameter [77.960482] Concatenation... [77.960610] Renumbering nodes [77.960610] Initial node count 6891 [77.960621] Removed 0 null nodes [77.960621] Concatenation over! [77.960622] Removing reference contigs with coverage < -1.000000... [77.960918] Concatenation... [77.961047] Renumbering nodes [77.961047] Initial node count 6891 [77.961058] Removed 0 null nodes [77.961059] Concatenation over! [78.028474] Writing contigs into ./contigs.fa... [79.423669] Writing into stats file ./stats.txt... [79.559581] Writing into graph file ./LastGraph... Final graph has 6891 nodes and n50 of 96, max 2769, total 569496, using 5202858/8855297 reads [0.000000] Reading graph file ./Graph2 [0.000070] Graph has 6891 nodes and 8855297 sequences [2.414712] Reading read set file ./Sequences; [3.454420] 8855297 sequences found [9.983101] Done [12.073890] Removing contigs with coverage < 3.000000... [12.077227] Concatenation... [12.110841] Renumbering nodes [12.110844] Initial node count 6891 [12.111045] Removed 112 null nodes [12.111046] Concatenation over! [12.111295] Concatenation... [12.111423] Renumbering nodes [12.111423] Initial node count 6779 [12.111436] Removed 0 null nodes [12.111436] Concatenation over! [12.111441] Clipping short tips off graph, drastic [12.112174] Concatenation... [12.112311] Renumbering nodes [12.112312] Initial node count 6779 [12.112405] Removed 7 null nodes [12.112406] Concatenation over! [12.112406] 6772 nodes left [12.112407] Correcting graph with cutoff 0.200000 [12.112413] Removing low coverage edges from the graph [12.116981] Concatenation... [12.205746] Renumbering nodes [12.205752] Initial node count 6772 [12.205956] Removed 32 null nodes [12.205957] Concatenation over! [12.206236] Determining eligible starting points [12.209874] Done listing starting nodes [12.209876] Initializing todo lists [12.210092] Done with initilization [12.210092] Activating arc lookup table [12.210256] Done activating arc lookup table [12.211125] 1000 nodes visited [12.211314] 2000 nodes visited [12.211725] 3000 nodes visited [12.211981] 4000 nodes visited [12.212184] 5000 nodes visited [12.212412] 6000 nodes visited [12.212988] 7000 nodes visited [12.214581] 8000 nodes visited [12.214905] 9000 nodes visited [12.215178] 10000 nodes visited [12.215483] 11000 nodes visited [12.215855] 12000 nodes visited [12.216435] 13000 nodes visited [12.217246] Concatenation... [12.239617] Renumbering nodes [12.239620] Initial node count 6740 [12.239808] Removed 78 null nodes [12.239809] Concatenation over! [12.239810] Clipping short tips off graph, drastic [12.240542] Concatenation... [12.269659] Renumbering nodes [12.269667] Initial node count 6662 [12.269860] Removed 21 null nodes [12.269861] Concatenation over! [12.269861] 6641 nodes left [12.269863] Removing low coverage edges from the graph [12.270134] Concatenation... [12.270253] Renumbering nodes [12.270254] Initial node count 6641 [12.270266] Removed 0 null nodes [12.270267] Concatenation over! [12.270750] Writing into stats file ./stats.txt... [12.418659] Writing into graph file ./LastGraph... Computing local connections [18.310196] Activating arc lookup table [18.310878] Done activating arc lookup table Extracting loci from connection graph... [18.532657] Counted 3577 mRNA loci Transitive reduction of graph Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings [18.623163] Exporting transcripts to ./transcripts.fa [18.637751] Exporting transcript contigs to ./contig-ordering.txt [18.773863] Finished extracting transcripts, used 4747390/8855297 reads [0.000000] Reading FastQ file /usr/local/galaxy/galaxy-dist/database/files/003/dataset_3687.dat; [12.768650] 8855297 sequences found [12.768653] Done [12.945838] Reading read set file ./Sequences; [13.863345] 8855297 sequences found [19.497520] Done [19.497531] 8855297 sequences in total. [19.541544] Writing into roadmap file ./Roadmaps... [21.360277] Inputting sequences... [21.360288] Inputting sequence 0 / 8855297 [23.204556] Inputting sequence 1000000 / 8855297 [25.109676] Inputting sequence 2000000 / 8855297 [27.073594] Inputting sequence 3000000 / 8855297 [29.074912] Inputting sequence 4000000 / 8855297 [31.101859] Inputting sequence 5000000 / 8855297 [33.295043] Inputting sequence 6000000 / 8855297 [35.624233] Inputting sequence 7000000 / 8855297 [37.969184] Inputting sequence 8000000 / 8855297 [39.983558] === Sequences loaded in 18.844462 s [40.106935] Done inputting sequences [40.106938] Destroying splay table [40.147227] Splay table destroyed [0.000000] Reading roadmap file ./Roadmaps [10.688511] 8855297 roadmaps read [10.694034] Creating insertion markers [12.646929] Ordering insertion markers [16.397709] Counting preNodes [17.105913] 1802122 preNodes counted, creating them now [18.580180] Sequence 1000000 / 8855297 [19.551491] Sequence 2000000 / 8855297 [20.443988] Sequence 3000000 / 8855297 [21.288288] Sequence 4000000 / 8855297 [22.100638] Sequence 5000000 / 8855297 [22.889490] Sequence 6000000 / 8855297 [23.659644] Sequence 7000000 / 8855297 [24.414121] Sequence 8000000 / 8855297 [25.053583] Adjusting marker info... [25.710606] Connecting preNodes [25.814606] Connecting 1000000 / 8855297 [25.936316] Connecting 2000000 / 8855297 [26.072741] Connecting 3000000 / 8855297 [26.215108] Connecting 4000000 / 8855297 [26.368631] Connecting 5000000 / 8855297 [26.522818] Connecting 6000000 / 8855297 [26.681469] Connecting 7000000 / 8855297 [26.845782] Connecting 8000000 / 8855297 [26.987872] Cleaning up memory [27.003705] Done creating preGraph [27.003708] Concatenation... [27.767289] Renumbering preNodes [27.767294] Initial preNode count 1802122 [27.793408] Destroyed 1115129 preNodes [27.793419] Concatenation over! [27.793421] Clipping short tips off preGraph [27.933941] Concatenation... [27.943271] Renumbering preNodes [27.943273] Initial preNode count 686993 [27.946796] Destroyed 685336 preNodes [27.946798] Concatenation over! [27.946799] 676719 tips cut off [27.946799] 1657 nodes left [27.946865] Writing into pregraph file ./PreGraph... [27.958115] Reading read set file ./Sequences; [29.008103] 8855297 sequences found [35.730429] Done [37.535139] Reading pre-graph file ./PreGraph [37.750964] Graph has 1657 nodes and 8855297 sequences [37.755970] Scanning pre-graph file ./PreGraph for k-mers [37.774919] 85345 kmers found [37.784901] Sorting kmer occurence table ... [37.821440] Sorting done. [37.821442] Computing acceleration table... [37.839000] Computing offsets... [37.841254] Ghost Threading through reads 0 / 8855297 [38.725065] Ghost Threading through reads 1000000 / 8855297 [39.601268] Ghost Threading through reads 2000000 / 8855297 [40.473358] Ghost Threading through reads 3000000 / 8855297 [41.347286] Ghost Threading through reads 4000000 / 8855297 [42.220368] Ghost Threading through reads 5000000 / 8855297 [43.096893] Ghost Threading through reads 6000000 / 8855297 [43.969893] Ghost Threading through reads 7000000 / 8855297 [44.844439] Ghost Threading through reads 8000000 / 8855297 [45.596543] === Ghost-Threaded in 7.755290 s [45.596549] Threading through reads 0 / 8855297 [46.518374] Threading through reads 1000000 / 8855297 [47.444445] Threading through reads 2000000 / 8855297 [48.366937] Threading through reads 3000000 / 8855297 [49.288411] Threading through reads 4000000 / 8855297 [50.211439] Threading through reads 5000000 / 8855297 [51.138604] Threading through reads 6000000 / 8855297 [52.061485] Threading through reads 7000000 / 8855297 [52.985071] Threading through reads 8000000 / 8855297 [53.776191] === Threaded in 8.179642 s [54.119435] Correcting graph with cutoff 0.200000 [54.119719] Determining eligible starting points [54.120670] Done listing starting nodes [54.120672] Initializing todo lists [54.120754] Done with initilization [54.120755] Activating arc lookup table [54.120825] Done activating arc lookup table [54.123725] Concatenation... [54.123766] Renumbering nodes [54.123766] Initial node count 1657 [54.123811] Removed 12 null nodes [54.123812] Concatenation over! [54.123813] Clipping short tips off graph, drastic [54.124252] Concatenation... [54.224979] Renumbering nodes [54.224989] Initial node count 1645 [54.225027] Removed 685 null nodes [54.225028] Concatenation over! [54.225029] 960 nodes left [54.225116] Writing into graph file ./Graph2... [55.367296] WARNING: NO COVERAGE CUTOFF PROVIDED [55.367299] Velvet will probably leave behind many detectable errors [55.367299] See manual for instructions on how to set the coverage cutoff parameter [55.367310] Removing contigs with coverage < -1.000000... [55.369762] Concatenation... [55.369800] Renumbering nodes [55.369800] Initial node count 960 [55.369804] Removed 0 null nodes [55.369804] Concatenation over! [55.369836] Concatenation... [55.369852] Renumbering nodes [55.369852] Initial node count 960 [55.369854] Removed 0 null nodes [55.369855] Concatenation over! [55.369859] Clipping short tips off graph, drastic [55.369917] Concatenation... [55.369933] Renumbering nodes [55.369933] Initial node count 960 [55.369935] Removed 0 null nodes [55.369936] Concatenation over! [55.369936] 960 nodes left [55.369937] WARNING: NO EXPECTED COVERAGE PROVIDED [55.369938] Velvet will be unable to resolve any repeats [55.369938] See manual for instructions on how to set the expected coverage parameter [55.369939] Concatenation... [55.369955] Renumbering nodes [55.369955] Initial node count 960 [55.369957] Removed 0 null nodes [55.369958] Concatenation over! [55.369958] Removing reference contigs with coverage < -1.000000... [55.370004] Concatenation... [55.370020] Renumbering nodes [55.370020] Initial node count 960 [55.370023] Removed 0 null nodes [55.370023] Concatenation over! [55.422086] Writing contigs into ./contigs.fa... [56.445270] Writing into stats file ./stats.txt... [56.509837] Writing into graph file ./LastGraph... Final graph has 960 nodes and n50 of 97, max 2466, total 83275, using 3349500/8855297 reads [0.000000] Reading graph file ./Graph2 [0.000036] Graph has 960 nodes and 8855297 sequences [1.504128] Reading read set file ./Sequences; [2.507512] 8855297 sequences found [9.014853] Done [11.103883] Removing contigs with coverage < 3.000000... [11.106359] Concatenation... [11.106391] Renumbering nodes [11.106392] Initial node count 960 [11.106405] Removed 0 null nodes [11.106406] Concatenation over! [11.106437] Concatenation... [11.106451] Renumbering nodes [11.106452] Initial node count 960 [11.106454] Removed 0 null nodes [11.106454] Concatenation over! [11.106457] Clipping short tips off graph, drastic [11.106511] Concatenation... [11.106526] Renumbering nodes [11.106526] Initial node count 960 [11.106528] Removed 0 null nodes [11.106529] Concatenation over! [11.106529] 960 nodes left [11.106530] Correcting graph with cutoff 0.200000 [11.106533] Removing low coverage edges from the graph [11.107536] Concatenation... [11.143393] Renumbering nodes [11.143395] Initial node count 960 [11.143419] Removed 2 null nodes [11.143420] Concatenation over! [11.143472] Determining eligible starting points [11.143963] Done listing starting nodes [11.143965] Initializing todo lists [11.143992] Done with initilization [11.143993] Activating arc lookup table [11.144018] Done activating arc lookup table [11.144227] 1000 nodes visited [11.144573] Concatenation... [11.170620] Renumbering nodes [11.170625] Initial node count 958 [11.170649] Removed 3 null nodes [11.170650] Concatenation over! [11.170650] Clipping short tips off graph, drastic [11.170755] Concatenation... [11.170770] Renumbering nodes [11.170770] Initial node count 955 [11.170777] Removed 1 null nodes [11.170777] Concatenation over! [11.170778] 954 nodes left [11.170778] Removing low coverage edges from the graph [11.170807] Concatenation... [11.170821] Renumbering nodes [11.170822] Initial node count 954 [11.170824] Removed 0 null nodes [11.170825] Concatenation over! [11.170964] Writing into stats file ./stats.txt... [11.230812] Writing into graph file ./LastGraph... Computing local connections [14.746804] Activating arc lookup table [14.746902] Done activating arc lookup table Extracting loci from connection graph... [14.901713] Counted 680 mRNA loci Transitive reduction of graph Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings [14.943051] Exporting transcripts to ./transcripts.fa [14.945550] Exporting transcript contigs to ./contig-ordering.txt [15.018696] Finished extracting transcripts, used 2812567/8855297 reads [0.000000] Reading FastA file transcripts_19.fa; [0.010226] 1570 sequences found [0.010227] Done [0.010247] Reading FastA file transcripts_21.fa; [0.011900] 344 sequences found [0.011901] Done [0.011933] Reading FastQ file /usr/local/galaxy/galaxy-dist/database/files/003/dataset_3687.dat; [12.198483] 8855297 sequences found [12.198486] Done [12.401859] Reading read set file ./Sequences; [13.230828] 8857211 sequences found [18.721759] Done [18.721771] 8857211 sequences in total. [18.757685] Writing into roadmap file ./Roadmaps... [20.584822] Inputting sequences... [20.584833] Inputting sequence 0 / 8857211 [22.366728] Inputting sequence 1000000 / 8857211 [24.064109] Inputting sequence 2000000 / 8857211 [25.810546] Inputting sequence 3000000 / 8857211 [27.594079] Inputting sequence 4000000 / 8857211 [29.489010] Inputting sequence 5000000 / 8857211 [31.550585] Inputting sequence 6000000 / 8857211 [33.630592] Inputting sequence 7000000 / 8857211 [35.727077] Inputting sequence 8000000 / 8857211 [37.556610] === Sequences loaded in 17.186429 s [37.658648] Done inputting sequences [37.658651] Destroying splay table [37.699062] Splay table destroyed [0.000000] Reading roadmap file ./Roadmaps [9.685475] 8857211 roadmaps read [9.691067] Creating insertion markers [10.875179] Ordering insertion markers [15.994335] Counting preNodes [16.591386] 1819394 preNodes counted, creating them now [18.120056] Sequence 1000000 / 8857211 [19.096796] Sequence 2000000 / 8857211 [20.019315] Sequence 3000000 / 8857211 [20.903522] Sequence 4000000 / 8857211 [21.760984] Sequence 5000000 / 8857211 [22.597284] Sequence 6000000 / 8857211 [23.416069] Sequence 7000000 / 8857211 [24.216209] Sequence 8000000 / 8857211 [24.899732] Adjusting marker info... [25.431336] Connecting preNodes [25.558999] Connecting 1000000 / 8857211 [25.675256] Connecting 2000000 / 8857211 [25.804199] Connecting 3000000 / 8857211 [25.937344] Connecting 4000000 / 8857211 [26.076245] Connecting 5000000 / 8857211 [26.221485] Connecting 6000000 / 8857211 [26.374118] Connecting 7000000 / 8857211 [26.527566] Connecting 8000000 / 8857211 [26.663482] Cleaning up memory [26.676121] Done creating preGraph [26.676124] Concatenation... [27.334684] Renumbering preNodes [27.334691] Initial preNode count 1819394 [27.360193] Destroyed 1162378 preNodes [27.360203] Concatenation over! [27.360204] Clipping short tips off preGraph [27.472827] Concatenation... [27.482889] Renumbering preNodes [27.482890] Initial preNode count 657016 [27.486213] Destroyed 654610 preNodes [27.486215] Concatenation over! [27.486215] 642103 tips cut off [27.486216] 2406 nodes left [27.486304] Writing into pregraph file ./PreGraph... [27.632873] Reading read set file ./Sequences; [28.652946] 8857211 sequences found [35.138336] Done [36.942242] Reading pre-graph file ./PreGraph [37.162349] Graph has 2406 nodes and 8857211 sequences [37.177742] Scanning pre-graph file ./PreGraph for k-mers [37.202831] 323188 kmers found [37.235987] Sorting kmer occurence table ... [37.412622] Sorting done. [37.412631] Computing acceleration table... [37.436981] Computing offsets... [37.446554] Ghost Threading through reads 0 / 8857211 [38.426036] Ghost Threading through reads 1000000 / 8857211 [39.403194] Ghost Threading through reads 2000000 / 8857211 [40.382459] Ghost Threading through reads 3000000 / 8857211 [41.359848] Ghost Threading through reads 4000000 / 8857211 [42.338263] Ghost Threading through reads 5000000 / 8857211 [43.312651] Ghost Threading through reads 6000000 / 8857211 [44.288659] Ghost Threading through reads 7000000 / 8857211 [45.262101] Ghost Threading through reads 8000000 / 8857211 [46.097808] === Ghost-Threaded in 8.651254 s [46.097819] Threading through reads 0 / 8857211 [47.257090] Threading through reads 1000000 / 8857211 [48.297172] Threading through reads 2000000 / 8857211 [49.336488] Threading through reads 3000000 / 8857211 [50.376047] Threading through reads 4000000 / 8857211 [51.415893] Threading through reads 5000000 / 8857211 [52.455649] Threading through reads 6000000 / 8857211 [53.494621] Threading through reads 7000000 / 8857211 [54.534154] Threading through reads 8000000 / 8857211 [55.426228] === Threaded in 9.328408 s [55.878521] Correcting graph with cutoff 0.200000 [55.878882] Determining eligible starting points [55.880237] Done listing starting nodes [55.880238] Initializing todo lists [55.880345] Done with initilization [55.880346] Activating arc lookup table [55.880433] Done activating arc lookup table [55.882259] Concatenation... [55.882315] Renumbering nodes [55.882316] Initial node count 2406 [55.882381] Removed 8 null nodes [55.882391] Concatenation over! [55.882391] Clipping short tips off graph, drastic [55.883006] Concatenation... [55.986679] Renumbering nodes [55.986689] Initial node count 2398 [55.986751] Removed 757 null nodes [55.986752] Concatenation over! [55.986753] 1641 nodes left [55.986847] Writing into graph file ./Graph2... [57.586219] WARNING: NO COVERAGE CUTOFF PROVIDED [57.586222] Velvet will probably leave behind many detectable errors [57.586223] See manual for instructions on how to set the coverage cutoff parameter [57.586229] Removing contigs with coverage < -1.000000... [57.588755] Concatenation... [57.588822] Renumbering nodes [57.588823] Initial node count 1641 [57.588828] Removed 0 null nodes [57.588828] Concatenation over! [57.588882] Concatenation... [57.588910] Renumbering nodes [57.588910] Initial node count 1641 [57.588914] Removed 0 null nodes [57.588914] Concatenation over! [57.588918] Clipping short tips off graph, drastic [57.589010] Concatenation... [57.589038] Renumbering nodes [57.589038] Initial node count 1641 [57.589042] Removed 0 null nodes [57.589042] Concatenation over! [57.589043] 1641 nodes left [57.589044] WARNING: NO EXPECTED COVERAGE PROVIDED [57.589044] Velvet will be unable to resolve any repeats [57.589045] See manual for instructions on how to set the expected coverage parameter [57.589046] Concatenation... [57.589073] Renumbering nodes [57.589074] Initial node count 1641 [57.589077] Removed 0 null nodes [57.589077] Concatenation over! [57.589078] Removing reference contigs with coverage < -1.000000... [57.589313] Concatenation... [57.589342] Renumbering nodes [57.589343] Initial node count 1641 [57.589346] Removed 0 null nodes [57.589347] Concatenation over! [57.642005] Writing contigs into ./contigs.fa... [59.115401] Writing into stats file ./stats.txt... [59.194880] Writing into graph file ./LastGraph... Final graph has 1641 nodes and n50 of 253, max 2780, total 321253, using 4663132/8857211 reads [0.000000] Reading graph file ./Graph2 [0.000035] Graph has 1641 nodes and 8857211 sequences [2.107753] Reading read set file ./Sequences; [3.038393] 8857211 sequences found [9.441180] Done [11.533317] Removing contigs with coverage < 3.000000... [11.536633] Concatenation... [11.536765] Renumbering nodes [11.536766] Initial node count 1641 [11.536809] Removed 38 null nodes [11.536810] Concatenation over! [11.536862] Concatenation... [11.536889] Renumbering nodes [11.536889] Initial node count 1603 [11.536892] Removed 0 null nodes [11.536893] Concatenation over! [11.536896] Clipping short tips off graph, drastic [11.536986] Concatenation... [11.537012] Renumbering nodes [11.537013] Initial node count 1603 [11.537016] Removed 0 null nodes [11.537017] Concatenation over! [11.537017] 1603 nodes left [11.537018] Correcting graph with cutoff 0.200000 [11.537021] Removing low coverage edges from the graph [11.537755] Concatenation... [11.570878] Renumbering nodes [11.570880] Initial node count 1603 [11.570922] Removed 2 null nodes [11.570923] Concatenation over! [11.570992] Determining eligible starting points [11.571857] Done listing starting nodes [11.571859] Initializing todo lists [11.571906] Done with initilization [11.571907] Activating arc lookup table [11.571953] Done activating arc lookup table [11.572158] 1000 nodes visited [11.572367] 2000 nodes visited [11.572720] 3000 nodes visited [11.574751] Concatenation... [11.658520] Renumbering nodes [11.658526] Initial node count 1601 [11.658573] Removed 13 null nodes [11.658575] Concatenation over! [11.658576] Clipping short tips off graph, drastic [11.658669] Concatenation... [11.658696] Renumbering nodes [11.658696] Initial node count 1588 [11.658699] Removed 0 null nodes [11.658700] Concatenation over! [11.658700] 1588 nodes left [11.658701] Removing low coverage edges from the graph [11.658754] Concatenation... [11.658779] Renumbering nodes [11.658779] Initial node count 1588 [11.658783] Removed 0 null nodes [11.658783] Concatenation over! [11.659006] Writing into stats file ./stats.txt... [11.739723] Writing into graph file ./LastGraph... Computing local connections [16.410447] Activating arc lookup table [16.410585] Done activating arc lookup table Extracting loci from connection graph... [16.589680] Counted 1507 mRNA loci Transitive reduction of graph Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings Preparing to correct graph with cutoff 0.200000 Deactivating local correction settings [16.653772] Exporting transcripts to ./transcripts.fa [16.668614] Exporting transcript contigs to ./contig-ordering.txt [16.783990] Finished extracting transcripts, used 4672425/8857211 reads [16.787745] Printing unused reads into ./UnusedReads.fa [24.411594] Writing into AMOS file ./oases_asm.afg... Thanks Again --R

1 0

One small problem for experts, one giant question for me..
by Can Holyavkin 09 Aug '13

09 Aug '13

Hello everyone, Firstly, I am beginner. I am a new Galaxy user that installed the Galaxy locally into my ubuntu (13.04) installed computer. Everything is working perfectly right now. Galaxy is really awesome. To learn the basics, I started the basic tutorial (Galaxy101 https://main.g2.bx.psu.edu/u/aun1/p/galaxy101) that helps too much. This tutorial consist a step for explaning "Compare Two Datasets" function. I did exactly everthing the instructor said however that module become red in History with following error: tool error An error occured with dataset: Unable to finish job (screenshot of error: http://i.imgur.com/xHqhK72.png) I wrote the bug report at end of the mail. Interestingly, when I press "View" button, I get the data that I wanted. However, any metadata is seen. So I can not see the final results on the UCSC browser. I replicate the same module at public server, it didn't gave an error. What might cause this kind of problem? Does reinstalling solve this problem? Would you please help me? Thank you everyone. Traceback (most recent call last): File "/home/can/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 94, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/can/galaxy-dist/lib/galaxy/jobs/__init__.py", line 1009, in finish self.sa_session.flush() File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush self._flush(objects) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1789, in _flush flush_context.execute() File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 331, in execute rec.execute(self) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 475, in execute uow File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 59, in save_obj mapper, table, update) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 485, in _emit_update_statements execute(statement, params) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "/home/can/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ('2013-08-09 09:00:48.464260', 'join (GNU coreutils) 8.20\nCopyright (C) 2012 Free Software Foundation, Inc.\nLisans GPLv3+: GNU GPL s\xc3\xbcr\xc3\xbcm 3 ve sonras\xc4\xb1 <http://gnu.org/licenses/gpl.html>.\nBu \xc3\x96zg\xc3\xbcr bir yaz\xc4\xb1l\xc4\xb1md\xc4\xb1r: de\xc4\x9fi\xc5\x9ftirmek ve da\xc4\x9f\xc4\xb1tmakta \xc3\xb6zg\xc3\xbcrs\xc3\xbcn\xc3\xbcz.\nYasalar\xc4\xb1n izin verdi\xc4\x9fi \xc3\xb6l\xc3\xa7\xc3\xbcde, herhangi bir GARANT\xc4\xb0 sunulmaz.\n\nMike Haertel taraf\xc4\xb1ndan yaz\xc4\xb1ld\xc4\xb1.\n', '', 41) Can Holyavkin PhD Student Istanbul Technical University Molecular Biology and Genetics Dept.

2 2

Galaxy cannot start last [server:handlerxx]
by Rodolfo Aramayo 09 Aug '13

09 Aug '13

Hi, I have a 80 core/1TB RAM Dell server. In order improve Galaxy performance I modified the universe_wsgi.ini file so as to improve the Scaling and Load Balancing as follows: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 ################################################################################ [server:web0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web2] use = egg:Paste#http port = 8092 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web3] use = egg:Paste#http port = 8093 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web4] use = egg:Paste#http port = 8094 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web5] use = egg:Paste#http port = 8095 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web6] use = egg:Paste#http port = 8096 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web7] use = egg:Paste#http port = 8097 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web8] use = egg:Paste#http port = 8098 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web9] use = egg:Paste#http port = 8099 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web10] use = egg:Paste#http port = 8100 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ################################################################################ [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 ################################################################################ [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler2] use = egg:Paste#http port = 8302 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler3] use = egg:Paste#http port = 8303 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler4] use = egg:Paste#http port = 8304 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler5] use = egg:Paste#http port = 8305 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler6] use = egg:Paste#http port = 8306 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler7] use = egg:Paste#http port = 8307 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler8] use = egg:Paste#http port = 8308 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler9] use = egg:Paste#http port = 8309 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler10] use = egg:Paste#http port = 8310 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler11] use = egg:Paste#http port = 8311 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler12] use = egg:Paste#http port = 8312 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler13] use = egg:Paste#http port = 8313 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler14] use = egg:Paste#http port = 8314 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler15] use = egg:Paste#http port = 8315 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler16] use = egg:Paste#http port = 8316 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler17] use = egg:Paste#http port = 8317 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler18] use = egg:Paste#http port = 8318 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler19] # use = egg:Paste#http # port = 8319 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 ################################################################################ job_manager = manager job_handlers = handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 ################################################################################ Now the problem I have is that when I start galaxy I cannot get the last [server:handler18] to load correclt I get the following error: galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'library_contents' API controller, class: LibraryContentsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'group_users' API controller, class: GroupUsersAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'configuration' API controller, class: ConfigurationController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,237 Enabling 'requests' API controller, class: RequestsAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling 'folders' API controller, class: FoldersController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling 'quotas' API controller, class: QuotaAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling 'folder_contents' API controller, class: FolderContentsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling 'roles' API controller, class: RoleAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling 'request_types' API controller, class: RequestTypeAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling 'users' API controller, class: UserAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'samples' API controller, class: SamplesAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'forms' API controller, class: FormDefinitionAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'permissions' API controller, class: PermissionsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,242 Enabling 'workflows' API controller, class: WorkflowsAPIController galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,242 app.config.api_folders: False galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling 'httpexceptions' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling 'recursive' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,700 Enabling 'error' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'trans logger' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'x-forwarded-host' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'Request ID' middleware Starting server in PID 3899. Traceback (most recent call last): File "./scripts/paster.py", line 33, in <module> serve.run() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 1049, in run invoke(command, command_name, options, args[1:]) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 1055, in invoke exit_code = runner.run(args) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 220, in run result = self.command() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 670, in command serve() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 654, in serve server(app) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 292, in server_wrapper **context.local_conf) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 100, in fix_call reraise(*exc_info) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 37, in reraise exec('raise t, e, tb', dict(t=t, e=e, tb=tb)) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 97, in fix_call val = callable(*args, **kw) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1342, in server_runner serve(wsgi_app, **kwargs) TypeError: serve() got an unexpected keyword argument 'job_manager' Removing PID file handler18.pid If I use activate [server:handler20] for example, this one would act up as well. It is always the last one and in all cases the port used is totally incorrect and was never assigned Am I doing something terribly wrong? I tried to use ports not 'assigned' and tried to add web servers and handlers according to my best understanding of the instructions (http://wiki.galaxyproject.org/Admin/Config/Performance/Web%20Application%20…) Please any suggestions would be most welcome Thanks --Rodolfo

2 4

Galaxy on cloud config issues
by Ulyanov_Anatoly 09 Aug '13

09 Aug '13

Hello, Does anybody use the last Galaxy image? I am running Galaxy image on EC2: Instance ID i-1b2d3773 Image ID (AMI) ami-118bfc78 It looks as those three tools have configuration problems: EBOSS Blast+ meme For all three I am getting a message that looks like that: /mnt/galaxy/tmp/job_working_directory/000/2/galaxy_2.sh: blastp: not found Error invoking command: blastp Is there a way to fix configuration files for those tools? Thank you. Best Regards, Anatoly </pre> This e-mail, including any attachments, is meant only for the intended recipient and may be a confidential communication or a communication privileged by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of this e-mail is strictly prohibited. Please notify the sender immediately of the error by return e-mail and please delete this message from your system. Thank you in advance for your cooperation.

3 8

Re: [galaxy-dev] support pbkdf2 in proftpd 1.3.5rc3
by Benjamin Schuster-Boeckler 09 Aug '13

09 Aug '13

I previously tried to get this to work, too. After a lot of trial-and-error, I gave up because proFTPd always calculated a different string after salting and hashing the password with PBKDF2 than what is stored in the database. I'm sure that it extracted the correct salt and password, but something about the salt&hash algorithm in proFTPd is different from what Galaxy does. I ended up reverting to SHA1 for our internal instance which is behind a firewall so pw hashing isn't as critical. Just my 5c, cheers, Ben -- Benjamin Schuster-Böckler Leadership Fellow for Bioinformatics ☎ +44 1865 617481 ✉ benjamin.schuster-boeckler(a)ludwig.ox.ac.uk Ludwig Institute for Cancer Research University of Oxford Old Road Campus Research Building Oxford OX3 7DQ United Kingdom

1 0

support pbkdf2 in proftpd 1.3.5rc3
by Leon Mei 09 Aug '13

09 Aug '13

Dear galaxy developers, We have tried today to upgrade our proftpd configuration to make uploading for our galaxy users possible again, both for users with "old" as well as "new" style hashed passwords. We upgraded proftpd on the server to 1.3.5rc3 and have the following SQL part in our configuration file based on the post of http://dev.list.galaxyproject.org/ProFTPD-integration-with-Galaxy-td4660295… SQLEngine on SQLLogFile /var/log/proftpd-sql.log SQLBackend postgres SQLConnectInfo galaxy@localhost:5840 galaxyftp <[ourpassword]> SQLAuthTypes SHA1 SHA256 PBKDF2 SQLPasswordPBKDF2 SHA256 1000 24 SQLPasswordUserSalt sql:/GetUserSalt SQLAuthenticate users SQLDefaultUID 108 SQLDefaultGID 116 SQLDefaultHomedir /opt/cloudman/pkg/proftpd/var SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38 for 32) ELSE password END) AS password2,'108','116','/mnt/galaxyData/tmp/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt FROM galaxy_user WHERE email='%U'" We have executed the LookupGalaxyUser and GetUserSalt commands manually, and the results look good. Now, old users can login via ftp, but for a new user, the authentication still fails: 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: >>> cmd_check 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: checking password using SQLAuthType 'sha1' 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: 'sha1' SQLAuthType handler reports failure 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: checking password using SQLAuthType 'pbkdf2' 2013-07-26 13:15:06,993 mod_sql/4.3[31761]: 'pbkdf2' SQLAuthType handler reports failure What are we missing? Thanks! Rob and Leon -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - http://ngs.nbic.nl<https://wiki.nbic.nl/index.php/Next_Generation_Sequencing> Skype: leon_mei Mobile: +31 6 41709231

3 5

Re: [galaxy-dev] Cloudman - When will the symlinks be resolved ?
by Ravpreet Setia 08 Aug '13

08 Aug '13

Here it is: 'pd.yaml' not found at paths.py load! Set paths.P_GALAXY_HOME as '/mnt/galaxyTools/galaxy-central' Python version: (2, 6) Image configuration suports: {'apps': ['cloudman', 'galaxy']} [DEBUG] app:54 2013-08-01 20:07:40,138: Initializing app [DEBUG] app:55 2013-08-01 20:07:40,138: Running on 'ec2' type of cloud. [DEBUG] app:72 2013-08-01 20:07:40,138: Getting pd.yaml [DEBUG] ec2:275 2013-08-01 20:07:40,138: No S3 Connection, creating a new one. [DEBUG] ec2:278 2013-08-01 20:07:40,139: Got boto S3 connection. [DEBUG] misc:113 2013-08-01 20:07:40,229: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not. [DEBUG] misc:444 2013-08-01 20:07:40,229: Bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' does not exist, did not get remote file 'persistent_data.yaml' [INFO] app:78 2013-08-01 20:07:40,229: Master starting [DEBUG] master:70 2013-08-01 20:07:40,229: Initializing console manager - cluster start time: 2013-08-01 20:07:40.229549 [DEBUG] connection:648 2013-08-01 20:07:40,231: Start from server, version: 8.0, properties: {u'platform': 'Erlang/OTP', u'product': 'RabbitMQ', u'version': '1.7.2', u'copyright': 'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': 'Licensed under the MPL. See http://www.rabbitmq.com/'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US'] [DEBUG] connection:494 2013-08-01 20:07:40,263: Open OK! known_hosts [ip-10-179-15-41.ec2.internal:5672] [DEBUG] channel:70 2013-08-01 20:07:40,263: using channel_id: 1 [DEBUG] channel:484 2013-08-01 20:07:40,264: Channel open [DEBUG] comm:35 2013-08-01 20:07:40,266: Successfully established AMQP connection [DEBUG] master:510 2013-08-01 20:07:40,266: Trying to discover any worker instances associated with this cluster... [DEBUG] ec2:246 2013-08-01 20:07:40,267: Establishing boto EC2 connection [DEBUG] ec2:103 2013-08-01 20:07:40,267: Gathering instance zone, attempt 0 [DEBUG] ec2:108 2013-08-01 20:07:40,270: Instance zone is 'us-east-1d' [DEBUG] ec2:263 2013-08-01 20:07:40,516: Got boto EC2 connection for region 'us-east-1' [DEBUG] ec2:66 2013-08-01 20:07:40,574: Gathering instance id, attempt 0 [DEBUG] ec2:71 2013-08-01 20:07:40,575: Instance ID is 'i-7b386714' [DEBUG] ec2:295 2013-08-01 20:07:40,639: Adding tag 'clusterName:gal_oldami_form2' to resource 'i-7b386714' [DEBUG] ec2:295 2013-08-01 20:07:41,066: Adding tag 'role:master' to resource 'i-7b386714' [DEBUG] master:174 2013-08-01 20:07:41,223: ud at manager start: {'access_key': 'AKIAIUSCRANAJESP3GFQ', 'cloudman_home': '/mnt/cm', 'cluster_name': 'gal_oldami_form2', 'bucket_default': 'gxy-workshop', 'boot_script_name': 'cm_boot.py', 'bucket_cluster': 'cm-35f4d40b627583b6949bc5de11bcfecc', 'boot_script_path': '/tmp/cm', 'role': 'master', 'secret_key': 'EV12T4PMGSbM1IZmS7ZUJVi9nCvMncnrxXvytacJ', 'password': 'google'} [DEBUG] pss:25 2013-08-01 20:07:41,224: Configured PSS as master [DEBUG] master:204 2013-08-01 20:07:41,225: Cheking for and adding any preconfigured services [DEBUG] master:529 2013-08-01 20:07:41,225: Trying to discover any volumes attached to this instance... [DEBUG] base:55 2013-08-01 20:07:41,236: Enabling 'root' controller, class: CM [DEBUG] buildapp:80 2013-08-01 20:07:41,239: Enabling 'httpexceptions' middleware [DEBUG] buildapp:86 2013-08-01 20:07:41,469: Enabling 'recursive' middleware [DEBUG] buildapp:106 2013-08-01 20:07:41,719: Enabling 'print debug' middleware [DEBUG] buildapp:120 2013-08-01 20:07:41,734: Enabling 'error' middleware [DEBUG] buildapp:130 2013-08-01 20:07:41,735: Enabling 'config' middleware [DEBUG] buildapp:134 2013-08-01 20:07:41,736: Enabling 'x-forwarded-host' middleware [DEBUG] master:535 2013-08-01 20:07:41,737: Attached volumes: [Volume:vol-fe5b7abe] [INFO] master:198 2013-08-01 20:07:41,758: Completed initial cluster configuration. This seems to be a new cluster; waiting to configure the type [DEBUG] master:1627 2013-08-01 20:07:41,758: Monitor started; manager started Starting server in PID 1252. serving on 0.0.0.0:42284 view at http://127.0.0.1:42284 [DEBUG] master:1247 2013-08-01 20:07:45,657: Checking for new version of CloudMan [DEBUG] misc:512 2013-08-01 20:07:45,657: Getting metadata 'revision' for file 'cm.tar.gz' from bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] misc:113 2013-08-01 20:07:45,740: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not. [DEBUG] misc:512 2013-08-01 20:07:45,740: Getting metadata 'revision' for file 'cm.tar.gz' from bucket 'gxy-workshop' [DEBUG] master:1676 2013-08-01 20:07:45,763: Monitor adding service 'SGE' [DEBUG] __init__:31 2013-08-01 20:07:45,763: Trying to add service 'SGE' [INFO] __init__:50 2013-08-01 20:07:45,763: SGE prerequisites OK; starting the service [DEBUG] sge:62 2013-08-01 20:07:45,763: Unpacking SGE from '/opt/galaxy/pkg/ge6.2u5' [DEBUG] __init__:42 2013-08-01 20:07:45,793: 'sge' daemon is NOT running any more (expected pid: '2521'). [DEBUG] sge:77 2013-08-01 20:07:45,793: Cleaning '/opt/sge' directory. [DEBUG] master:1251 2013-08-01 20:07:45,804: Revision number for user's CloudMan: 'None'; revision number for default CloudMan: 'None' [DEBUG] ec2:48 2013-08-01 20:07:45,805: Gathering instance type, attempt 0 [DEBUG] sge:83 2013-08-01 20:07:46,195: Unpacking SGE to '/opt/sge'. [DEBUG] ec2:218 2013-08-01 20:07:46,267: Gathering instance public hostname, attempt 0 [INFO] master:817 2013-08-01 20:07:50,102: Initializing a 'Galaxy' cluster. [INFO] misc:438 2013-08-01 20:07:51,212: Retrieved file 'snaps.yaml' from bucket 'gxy-workshop' to 'cm_snaps.yaml'. Error - : 'static_filesystems' [INFO] sge:97 2013-08-01 20:07:51,551: Configuring SGE... [DEBUG] ec2:162 2013-08-01 20:07:51,552: Gathering instance private IP, attempt 0 [DEBUG] sge:105 2013-08-01 20:07:51,554: Created SGE install template as file '/opt/sge/galaxyEC2.conf' [DEBUG] sge:119 2013-08-01 20:07:51,554: Setting up SGE. [DEBUG] misc:577 2013-08-01 20:07:51,567: Modified /opt/sge/util/arch [DEBUG] misc:577 2013-08-01 20:07:51,573: Successfully chmod /opt/sge/util/arch [DEBUG] misc:577 2013-08-01 20:07:51,581: 'sed -i.bak '/^127.0.1./s/^/# (Commented by CloudMan) /' /etc/hosts' command OK [DEBUG] ec2:142 2013-08-01 20:07:57,465: Gathering instance public keys (i.e., key pairs), attempt 0 [DEBUG] ec2:148 2013-08-01 20:07:57,468: Got key pair: 'rav_kp2' [DEBUG] misc:577 2013-08-01 20:07:59,992: Successfully set up SGE [INFO] sge:122 2013-08-01 20:07:59,993: Successfully setup SGE; configuring SGE [DEBUG] sge:123 2013-08-01 20:07:59,993: Adding parallel environments [DEBUG] misc:577 2013-08-01 20:08:00,009: 'cd /opt/sge; ./bin/lx24-amd64/qconf -Ap /tmp/SMP_PE' command OK [DEBUG] misc:577 2013-08-01 20:08:00,026: 'cd /opt/sge; ./bin/lx24-amd64/qconf -Ap /tmp/MPI_PE' command OK [DEBUG] sge:130 2013-08-01 20:08:00,026: Creating queue 'all.q' [DEBUG] sge:135 2013-08-01 20:08:00,027: Created SGE all.q template as file '/opt/sge/all.q.conf' [DEBUG] misc:577 2013-08-01 20:08:00,044: Successfully modified all.q [DEBUG] sge:137 2013-08-01 20:08:00,045: Configuring users' SGE profiles [DEBUG] master:1676 2013-08-01 20:08:00,045: Monitor adding service 'post_start_script' [DEBUG] pss:51 2013-08-01 20:08:00,045: Custom-checking 'post_start_script' service prerequisites [DEBUG] pss:58 2013-08-01 20:08:00,046: SGE not running (Starting), post_start_script service prerequisites not met afterall,not starting the service yet [DEBUG] misc:113 2013-08-01 20:08:00,121: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not. [DEBUG] misc:124 2013-08-01 20:08:00,312: Created bucket 'cm-35f4d40b627583b6949bc5de11bcfecc'. [ERROR] misc:139 2013-08-01 20:08:00,434: Problem connecting to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc', attempt 1/5 [INFO] misc:455 2013-08-01 20:08:02,976: Saved file 'persistent_data.yaml' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] misc:113 2013-08-01 20:08:03,041: Checking if bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' exists... it does not. [DEBUG] master:1601 2013-08-01 20:08:03,042: Saving current instance boot script (/tmp/cm/cm_boot.py) to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'cm_boot.py' [INFO] misc:455 2013-08-01 20:08:03,293: Saved file 'cm_boot.py' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] master:1605 2013-08-01 20:08:03,508: Saving CloudMan source (/mnt/cm/cm.tar.gz) to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'cm.tar.gz' [INFO] misc:455 2013-08-01 20:08:04,987: Saved file 'cm.tar.gz' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] misc:527 2013-08-01 20:08:04,987: Setting metadata 'revision' for file 'cm.tar.gz' in bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] master:1619 2013-08-01 20:08:05,781: Saving '/mnt/cm/gal_oldami_form2.clusterName' file to cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' as 'gal_oldami_form2.clusterName' [INFO] misc:455 2013-08-01 20:08:06,027: Saved file 'gal_oldami_form2.clusterName' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] sge:405 2013-08-01 20:08:10,466: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.65 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:08:10,467: S&S: SGE..OK; post_start_script..Unstarted; [DEBUG] master:1676 2013-08-01 20:08:10,467: Monitor adding service 'post_start_script' [DEBUG] pss:51 2013-08-01 20:08:10,467: Custom-checking 'post_start_script' service prerequisites [DEBUG] pss:70 2013-08-01 20:08:10,468: Starting post_start_script service [DEBUG] pss:74 2013-08-01 20:08:10,468: post_start_script service prerequisites OK (i.e., all other services running), checking if post_start_script was provided... [DEBUG] pss:91 2013-08-01 20:08:10,535: Cluster bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' found; looking for post start script 'post_start_script' [DEBUG] misc:440 2013-08-01 20:08:10,856: Failed to get file 'post_start_script' from bucket 'cm-35f4d40b627583b6949bc5de11bcfecc': S3ResponseError: 404 Not Found NoSuchKeyThe specified key does not exist.post_start_scriptA459857A0CC9D8BFqfo3h0QIZYyRSsubkp1GKCbDWljS5GNMSw0zxCOSrcO/CS5wcMU08G44u5oVdu/G [DEBUG] pss:102 2013-08-01 20:08:10,857: post_start_script does not exist or could not be downloaded; continuing without running it. [DEBUG] pss:40 2013-08-01 20:08:10,857: Priming local data variables... [DEBUG] pss:107 2013-08-01 20:08:10,857: post_start_script service done and marked as 'Shut down' [DEBUG] ec2:33 2013-08-01 20:08:10,858: Gathering instance ami, attempt 0 [DEBUG] pss:136 2013-08-01 20:08:10,858: Removing post_start_script service from master list of services [DEBUG] ec2:122 2013-08-01 20:08:10,861: Gathering instance security group, attempt 0 [DEBUG] ec2:181 2013-08-01 20:08:10,862: Gathering instance local hostname, attempt 0 [INFO] misc:455 2013-08-01 20:08:12,333: Saved file 'persistent_data.yaml' to bucket 'cm-35f4d40b627583b6949bc5de11bcfecc' [DEBUG] sge:405 2013-08-01 20:08:25,375: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.65 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:08:25,376: S&S: SGE..OK; [DEBUG] sge:405 2013-08-01 20:08:37,826: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.65 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:08:37,827: S&S: SGE..OK; [DEBUG] sge:405 2013-08-01 20:08:50,281: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.83 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:08:50,282: S&S: SGE..OK; [DEBUG] sge:405 2013-08-01 20:09:02,751: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.83 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:09:02,752: S&S: SGE..OK; [DEBUG] sge:405 2013-08-01 20:09:15,209: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.83 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:09:15,209: S&S: SGE..OK; [DEBUG] sge:405 2013-08-01 20:09:27,665: qstat: ['all.q(a)ip-10-179-15-41.ec2.inte BIP 0/0/2 0.91 lx24-amd64 '] [DEBUG] master:1650 2013-08-01 20:09:27,666: S&S: SGE..OK; ________________________________ From: Ravpreet Setia Sent: Thursday, August 01, 2013 4:01 PM To: Dannon Baker Subject: RE: [galaxy-dev] Cloudman - When will the symlinks be resolved ? The volumes are not mounting ; I do not see them in /mnt after I ssh in. Here is the entirety of my log: * 19:52:27 - Master starting * 19:52:28 - Completed initial cluster configuration. This seems to be a new cluster; waiting to configure the type * 19:52:32 - SGE prerequisites OK; starting the service * 19:52:39 - Configuring SGE... * 19:52:50 - Successfully setup SGE; configuring SGE * 19:52:50 - Saved file 'persistent_data.yaml' to bucket 'cm-e0d3b778388b8c2eebfbc397f8757075' * 19:52:50 - Saved file 'cm_boot.py' to bucket 'cm-e0d3b778388b8c2eebfbc397f8757075' * 19:52:51 - Saved file 'cm.tar.gz' to bucket 'cm-e0d3b778388b8c2eebfbc397f8757075' * 19:52:52 - Saved file 'gal_oldami_fromform.clusterName' to bucket 'cm-e0d3b778388b8c2eebfbc397f8757075' * 19:52:53 - Initializing a 'Galaxy' cluster. * 19:52:55 - Retrieved file 'snaps.yaml' from bucket 'gxy-workshop' to 'cm_snaps.yaml'. * 19:52:57 - Saved file 'persistent_data.yaml' to bucket 'cm-e0d3b778388b8c2eebfbc397f8757075' ________________________________ From: Dannon Baker [dannon.baker(a)gmail.com] Sent: Monday, July 29, 2013 10:54 AM To: Ravpreet Setia Cc: galaxy-dev(a)lists.bx.psu.edu Subject: Re: [galaxy-dev] Cloudman - When will the symlinks be resolved ? We're working on it but don't have a specific ETA for the release of the new volume. For now, you can continue to create and run non-migrated clusters using the previous version of Cloudman using this launch form: https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-… On Mon, Jul 29, 2013 at 10:49 AM, Ravpreet Setia <Ravpreet.Setia(a)oicr.on.ca<mailto:Ravpreet.Setia@oicr.on.ca>> wrote: I am referring to the following bug: https://bitbucket.org/galaxy/cloudman/issue/6/tool-default-symbolic-links-t… We are waiting for this bug to resolve before we fix any other issues with our Cloudman scripts. I am wondering when you think this bug will be resolved so I can plan my time. Thanks. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

2 4