On Aug 26, 2016 6:49 AM, "Steve Cassidy" <steve.cassidy@mq.edu.au> wrote:

Ok, probing further trying to understand this. It looks like the job runner is trying to call set_meta() from galaxy_ext/metadata/set_metadata.py.  In there there’s a line that tries to set up the PATH:

# insert *this* galaxy before all others on sys.path
sys.path.insert( 1, os.path.abspath( os.path.join( os.path.dirname( __file__ ), os.pardir, os.pardir ) ) )

In my case the result is:

['/export/galaxy-central/database/job_working_directory/000/3’,
'/galaxy-central/lib’,
'/galaxy-central/lib’,
 '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python27.zip’,
 '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7’,
 '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/plat-linux2’,
 '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-tk’,
'/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-old’,
 '/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/lib-dynload’,
'/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/site-packages’,
'/export/galaxy-central/database/job_working_directory/000/3/conda-env/lib/python2.7/site-packages/setuptools-25.1.6-py2.7.egg’
]

Looking for sqlalchemy, I find it installed at:

/galaxy-central/.venv/lib/python2.7/site-packages/sqlalchemy

so clearly that line of code is not doing what it should. I’m guessing that the Conda resolver is running the script within a conda env and the BETA magic hasn’t quite made it to the right place yet…

BTW I was trying to understand the flags you mentioned but I can’t find GALAXY_CONFIG_ENABLE_BETA_TOOL_COMMAND_ISOLATION anywhere other than in sample Dockerfiles - is this some kind of magic incantation that creates a rift in space-time…enquiring minds want to know!!!

Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy

> On 25 Aug 2016, at 9:54 PM, Steve Cassidy <steve.cassidy@mq.edu.au> wrote:
>
> Nah, definitely baby steps…
>
> so, in the repo you point to there seems to be an error in the Dockerfile, the ENV line should use the var=value syntax to have more than one setting on one line (maybe that’s changed recently in docker).
>
> with this I built a new docker image, when I run my first tool it takes an age while it’s installing the deps, then it crashes with:
>
> Traceback (most recent call last):
>  File "/export/galaxy-central/database/job_working_directory/000/1/set_metadata_QUJQLD.py", line 1, in <module>
>    from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
>  File "/galaxy-central/lib/galaxy_ext/metadata/set_metadata.py", line 23, in <module>
>    from sqlalchemy.orm import clear_mappers
> ImportError: No module named sqlalchemy.orm
>
> and the output:
>
> discarding /galaxy-central/tool_deps/_conda/bin from PATH
> prepending /export/galaxy-central/database/job_working_directory/000/1/conda-env/bin to PATH
> discarding /galaxy-central/tool_deps/_conda/bin from PATH
> prepending /export/galaxy-central/database/job_working_directory/000/1/conda-env/bin to PATH
>
> I’m guessing this is some kind of conflict between python versions in and out of conda environments? Surely sqlalchemy would be installed for Galaxy to work?
>
> I’ll try to dig around this in the morning but if it rings a bell…
>
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
>> On 25 Aug 2016, at 8:43 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
>>
>> Hi Steve,
>>
>> you call this baby-steps? I think this is huge! :)
>>
>> All what you are missing is to enable conda in Galaxy.
>> Look at Gregs new flavour which is entirely Conda/Galaxy based.
>>
>> You need to enable these env vars to make Galaxy conda enabled:
>>
>> https://github.com/gregvonkuster/docker-galaxy-csg/blob/master/Dockerfile#L9
>>
>> Hope this helps,
>> Bjoern
>>
>> Am 25.08.2016 um 12:32 schrieb Steve Cassidy:
>>> Hi all,
>>> I’m making baby steps towards having a repeatable installation for my
>>> tools.  But I’m now stuck, so help would be appreciated.
>>>
>>> I have a tool that works and is in the test toolshed (alveoimport in
>>> Data Sources).
>>>
>>> It depends on my python package which is now part of bioconda (pyalveo,
>>> version 0.6).
>>>
>>> I can run my tool via planemo, which works I think because I have
>>> pyalveo installed in a local venv.
>>>
>>> If I try to run the docker image (derived from  bgruening/galaxy-stable
>>> but with the testtoolshed added) I am able to install my tool, but it
>>> doesn’t pick up the dependency, so it doesn’t work.
>>>
>>> I tried running with planemo turning on conda dependency resolution
>>> (following https://pypi.python.org/pypi/planemo/):
>>>
>>> planemo serve --galaxy_branch release_16.01 --conda_dependency_resolution .
>>>
>>> It seems to have a go, but fails:
>>>
>>> galaxy.tools.deps DEBUG 2016-08-25 17:32:22,449 Building dependency
>>> shell command for dependency 'pyalveo'
>>> galaxy.tools.deps DEBUG 2016-08-25 17:32:22,449 Find dependency pyalveo
>>> version 0.6
>>> galaxy.tools.deps DEBUG 2016-08-25 17:32:22,450 Resolver conda returned
>>> <galaxy.tools.deps.resolvers.NullDependency object at 0x105bef5d0>
>>> (isnull? True)
>>> galaxy.tools.deps DEBUG 2016-08-25 17:32:22,450 Resolver conda returned
>>> <galaxy.tools.deps.resolvers.NullDependency object at 0x105bef5d0>
>>> (isnull? True)
>>> galaxy.tools.deps WARNING 2016-08-25 17:32:22,451 Failed to resolve
>>> dependency on 'pyalveo', ignoring
>>>
>>> So, what’s the easiest route to a Galaxy deployment with my tool
>>> installed. The Docker route would be best I think, but what do I have to
>>> add to bgruening/galaxy-stable to have conda find my dependencies.
>>>
>>> Thanks in advance,
>>>
>>> Steve
>>>
>>> —
>>> Department of Computing, Macquarie University
>>> http://web.science.mq.edu.au/~cassidy
>>> <http://web.science.mq.edu.au/%7Ecassidy>
>>>
>>>
>>>
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>> http://galaxyproject.org/search/mailinglists/
>>>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/