On Thu, Jan 31, 2013 at 2:40 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I was hoping to make some enhancements to the ToolShed, so first I need to be able to run my own ToolShed locally:
http://wiki.galaxyproject.org/HostingALocalToolShed
I have the latest code for bitbucket for galaxy-central:
$ hg heads default changeset: 9117:04e221996871 user: Greg Von Kuster <greg@bx.psu.edu> date: Wed Jan 30 16:47:20 2013 -0500 summary: Dont' assume a complex repository dependency was properly defined.
The following cryptic error is not helpful:
$ sh run_community.sh Command 'serve' not known (you may need to run setup.py egg_info) No commands registered. Have you installed Paste Script? (try running python setup.py develop)
I then noticed that there was no community_wsgi.ini file, and tried using a copy of the sample:
$ cp community_wsgi.ini.sample community_wsgi.ini $ sh run_community.sh (runs with no output, finished in 3s)
Does it make sense to generate community_wsgi.ini automatically? Or give an explicit error if it is missing?
Looking at the log file, I have a Python depenancy issue here:
$ more community_webapp.log ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Traceback (most recent call last): File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 56, in app_factory from galaxy.webapps.community.app import UniverseApplication File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/app.py", line 2, in <module> from galaxy import tools File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 36, in <module> import galaxy.util.shed_util_common File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/shed_util_common.py", line 27, in <module> eggs.require( 'markupsafe' ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 413, in require return pkg_resources.working_set.require( req_str ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/pkg_resources.py", line 666, in require needed = self.resolve(parse_requirements(requirements)) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here DistributionNotFound: markupsafe Removing PID file community_webapp.pid
Should this work under Mac OS X? I suspect there are some eggs missing on the Galaxy site which would be needed:
$ python2.6 ./scripts/fetch_eggs.py Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Warning: ssh (a dependent egg of Fabric) cannot be fetched Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 37, in <module> c.resolve() # Only fetch eggs required by the config File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: numpy==1.6.0
Looks like Galaxy was using Python 2.7, and that does seem to be able to fetch all the eggs. Progress: $ tail -n 50 community_webapp.log galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: SNP/WGA: QC; LD; Plots galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: SNP/WGA: Statistical Models galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: Phenotype Association galaxy.tools DEBUG 2013-01-31 14:43:30,553 Loading section: VCF Tools Tool shed hgweb.config file is: /Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/hgweb.config galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,903 Enabling 'admin' controller, class: AdminController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,910 Enabling 'hg' controller, class: HgController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,910 Enabling 'repository' controller, class: RepositoryController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,918 Enabling 'repository_review' controller, class: RepositoryReviewController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,926 Enabling 'upload' controller, class: UploadController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,927 Enabling 'user' controller, class: User galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,930 Enabling 'httpexceptions' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,930 Enabling 'recursive' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,932 Enabling 'print debug' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,946 Enabling 'error' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'trans logger' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'config' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'x-forwarded-host' middleware mercurial version is: 2.2.3 Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 225, in loadobj return context.create() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 625, in create return self.object_type.invoke(self) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 110, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 72, in app_factory webapp = wrap_in_middleware( webapp, global_conf, **kwargs ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 157, in wrap_in_middleware app = hg.Hg( app, conf ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/framework/middleware/hg.py", line 30, in __init__ self.db_url = "sqlite:///%s?isolation_level=IMMEDIATE" % self.config[ 'database_file' ] KeyError: 'database_file' Removing PID file community_webapp.pid I noticed that the community_wsgi.ini.sample file had database_file as a commented out line: #database_file = database/community.sqlite I expected that to be used as the default value - commenting it got the tool shed to run. Perhaps the wiki page needs clarifying on this? http://wiki.galaxyproject.org/HostingALocalToolShed Regards, Peter