Thanks Peter- I'll make sure to update the tool shed wiki accordingly as soon as possible. Greg Von Kuster On Jan 31, 2013, at 9:48 AM, Peter Cock wrote:

On Thu, Jan 31, 2013 at 2:40 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

...

Hi all,

I was hoping to make some enhancements to the ToolShed, so first I need to be able to run my own ToolShed locally:

http://wiki.galaxyproject.org/HostingALocalToolShed

I have the latest code for bitbucket for galaxy-central:

$ hg heads default changeset: 9117:04e221996871 user: Greg Von Kuster <greg@bx.psu.edu> date: Wed Jan 30 16:47:20 2013 -0500 summary: Dont' assume a complex repository dependency was properly defined.

The following cryptic error is not helpful:

$ sh run_community.sh Command 'serve' not known (you may need to run setup.py egg_info) No commands registered. Have you installed Paste Script? (try running python setup.py develop)

I then noticed that there was no community_wsgi.ini file, and tried using a copy of the sample:

$ cp community_wsgi.ini.sample community_wsgi.ini $ sh run_community.sh (runs with no output, finished in 3s)

Does it make sense to generate community_wsgi.ini automatically? Or give an explicit error if it is missing?

Looking at the log file, I have a Python depenancy issue here:

$ more community_webapp.log ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Traceback (most recent call last): File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 56, in app_factory from galaxy.webapps.community.app import UniverseApplication File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/app.py", line 2, in <module> from galaxy import tools File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 36, in <module> import galaxy.util.shed_util_common File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/util/shed_util_common.py", line 27, in <module> eggs.require( 'markupsafe' ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 413, in require return pkg_resources.working_set.require( req_str ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/pkg_resources.py", line 666, in require needed = self.resolve(parse_requirements(requirements)) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here DistributionNotFound: markupsafe Removing PID file community_webapp.pid

Should this work under Mac OS X? I suspect there are some eggs missing on the Galaxy site which would be needed:

$ python2.6 ./scripts/fetch_eggs.py Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Warning: ssh (a dependent egg of Fabric) cannot be fetched Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 37, in <module> c.resolve() # Only fetch eggs required by the config File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: numpy==1.6.0

Looks like Galaxy was using Python 2.7, and that does seem to be able to fetch all the eggs. Progress:

$ tail -n 50 community_webapp.log galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: SNP/WGA: QC; LD; Plots galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: SNP/WGA: Statistical Models galaxy.tools DEBUG 2013-01-31 14:43:30,552 Loading section: Phenotype Association galaxy.tools DEBUG 2013-01-31 14:43:30,553 Loading section: VCF Tools Tool shed hgweb.config file is: /Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/hgweb.config galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,903 Enabling 'admin' controller, class: AdminController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,910 Enabling 'hg' controller, class: HgController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,910 Enabling 'repository' controller, class: RepositoryController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,918 Enabling 'repository_review' controller, class: RepositoryReviewController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,926 Enabling 'upload' controller, class: UploadController galaxy.web.framework.base DEBUG 2013-01-31 14:43:30,927 Enabling 'user' controller, class: User galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,930 Enabling 'httpexceptions' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,930 Enabling 'recursive' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,932 Enabling 'print debug' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,946 Enabling 'error' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'trans logger' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'config' middleware galaxy.webapps.community.buildapp DEBUG 2013-01-31 14:43:30,947 Enabling 'x-forwarded-host' middleware mercurial version is: 2.2.3 Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 225, in loadobj return context.create() File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 625, in create return self.object_type.invoke(self) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 110, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 72, in app_factory webapp = wrap_in_middleware( webapp, global_conf, **kwargs ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/buildapp.py", line 157, in wrap_in_middleware app = hg.Hg( app, conf ) File "/Volumes/Hitachi-3TB-mirrored/repositories/galaxy-central/lib/galaxy/webapps/community/framework/middleware/hg.py", line 30, in __init__ self.db_url = "sqlite:///%s?isolation_level=IMMEDIATE" % self.config[ 'database_file' ] KeyError: 'database_file' Removing PID file community_webapp.pid

I noticed that the community_wsgi.ini.sample file had database_file as a commented out line:

#database_file = database/community.sqlite

I expected that to be used as the default value - commenting it got the tool shed to run. Perhaps the wiki page needs clarifying on this? http://wiki.galaxyproject.org/HostingALocalToolShed

Regards,

Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/