Hello Enis,

I am trying to add PyCogent to the PYTHONPATH, but it seems the Fabric run command does not recognize it. I tried: run("export PYTHONPATH=/mnt/galaxyTools/tools/PyCogent/1.5.1/lib/python2.6/site-packages/:$PYTHONPATH") and then use cmd_install() to install Galaxy, but I keep getting the message that PyCogent is not installed.

The normal strategy to use sys.path will not work because that will adjust the PYTHONPATH on my local machine, not in the image if I am correct. Do you know a way to set the PYTHONPATH for Fabric' run() command?

Thanks,

Mattias

On Thu, 2011-07-14 at 09:05 +0000, Enis Afgan wrote:

Hi Mattias, PyCogent is not currently being installed as part of mi-deployment, so the method for installing it should be added (if you do so, please consider issuing a pull request on bitbucket so it can be added to the project).

When it comes to composing $PYTHONPATH to include the reference to PyCogent via Qiime's env.sh - this should be done as part of the Qiime installation method in mi-deployment (something like export PYHTONPATH=<path where PyCogent was installed>; export PATH=...). Sourcing of the tool's env.sh script is prepended to the execution command when running the tool so the appropriate environment settings will be loaded before the tool is run.

Enis

On Wed, Jul 13, 2011 at 12:09 PM, Mattias de Hollander m.dehollander@nioo.knaw.nl wrote: Hello,

    I am trying to add tools (PyCogent, Qiime) to a Cloudman image
    using the
    mi-deployment scripts. Qiime depends on PyCogent and it needs
    to be
    accessible on the python path, otherwise I get an error:
    PyCogent not installed but required. (Is it installed? Is it
    in the
    current user's $PYTHONPATH or site-packages?)

    Is there a way to add the paths in env.sh to the sys.path
    during
    installation using tools_fabfile.py?
    All the tools are in /mnt/galaxyTools/tools.
    I tried the DependencyManager from galaxy but then I only
    receive the
    path to the env.sh file:
    In [43]: dependency_manager.find_dep('blast')
    Out[43]:
    ('/mnt/galaxyTools/tools/blast/2.2.25/env.sh',
     '/mnt/galaxyTools/tools/blast/2.2.25',
     '2.2.25')

    Is there a smarter way to add files to the PYTHONPATH during
    the
    execution of tools_fabfile.py?

    Thanks,

    Mattias

    --
    Bioinformatician
    Netherlands Institute of Ecology (NIOO-KNAW)
    Wageningen, the Netherlands

I just found out the former approach you mentioned. Its now installing perfectly. The latter approach won't work for a Qiime installation because its needs PyCogent on its path during the setup.py phase.

Thanks for clarification. If I finished adding Qiime' dependencies I will put it on bitbucket.

Greetz,

Mattias

On Fri, 2011-07-15 at 13:44 +0000, Enis Afgan wrote:

Each use of fabric's `run` method is independent so they cannot be chained; you would either need to prepend the export command to the command that depends on the fact PYTHONPATH is set (e.g., run("export PYTHONPATH=...; <command that requires given PP>") or (better approach) create an env.sh file in the Qiime's install dir so that it includes the required 'export PYTHONPATH=...'. When Galaxy invokes Qiime, it will automatically source that env.sh (assuming you have config variable `tool_dependency_dir = /mnt/galaxyTools/tools` set in Galaxy's universe_wsgi.ini).

Enis

On Fri, Jul 15, 2011 at 12:19 PM, Mattias de Hollander m.dehollander@nioo.knaw.nl wrote: Hello Enis,

    I am trying to add PyCogent to the PYTHONPATH, but it seems
    the Fabric
    run command does not recognize it.
    I tried:                 run("export
    PYTHONPATH=/mnt/galaxyTools/tools/PyCogent/1.5.1/lib/python2.6/site-packages/:$PYTHONPATH")
    and then use cmd_install() to install Galaxy, but I keep
    getting the
    message that PyCogent is not installed.

    The normal strategy to use sys.path will not work because that
    will
    adjust the PYTHONPATH on my local machine, not in the image if
    I am
    correct.
    Do you know a way to set the PYTHONPATH for Fabric' run()
    command?

    Thanks,

    Mattias


    On Thu, 2011-07-14 at 09:05 +0000, Enis Afgan wrote:
    > Hi Mattias,
    > PyCogent is not currently being installed as part of
    mi-deployment, so
    > the method for installing it should be added (if you do so,
    please
    > consider issuing a pull request on bitbucket so it can be
    added to the
    > project).
    >
    >
    > When it comes to composing $PYTHONPATH to include the
    reference to
    > PyCogent via Qiime's env.sh - this should be done as part of
    the Qiime
    > installation method in mi-deployment (something like export
    > PYHTONPATH=<path where PyCogent was installed>; export
    PATH=...).
    > Sourcing of the tool's env.sh script is prepended to the
    execution
    > command when running the tool so the appropriate environment
    settings
    > will be loaded before the tool is run.
    >
    >
    > Enis
    >
    > On Wed, Jul 13, 2011 at 12:09 PM, Mattias de Hollander
    > <m.dehollander@nioo.knaw.nl> wrote:
    >         Hello,
    >
    >         I am trying to add tools (PyCogent, Qiime) to a
    Cloudman image
    >         using the
    >         mi-deployment scripts. Qiime depends on PyCogent and
    it needs
    >         to be
    >         accessible on the python path, otherwise I get an
    error:
    >         PyCogent not installed but required. (Is it
    installed? Is it
    >         in the
    >         current user's $PYTHONPATH or site-packages?)
    >
    >         Is there a way to add the paths in env.sh to the
    sys.path
    >         during
    >         installation using tools_fabfile.py?
    >         All the tools are in /mnt/galaxyTools/tools.
    >         I tried the DependencyManager from galaxy but then I
    only
    >         receive the
    >         path to the env.sh file:
    >         In [43]: dependency_manager.find_dep('blast')
    >         Out[43]:
    >         ('/mnt/galaxyTools/tools/blast/2.2.25/env.sh',
    >          '/mnt/galaxyTools/tools/blast/2.2.25',
    >          '2.2.25')
    >
    >         Is there a smarter way to add files to the
    PYTHONPATH during
    >         the
    >         execution of tools_fabfile.py?
    >
    >         Thanks,
    >
    >         Mattias
    >
    >         --
    >         Bioinformatician
    >         Netherlands Institute of Ecology (NIOO-KNAW)
    >         Wageningen, the Netherlands
    >
    >



    --

    Bioinformatician
    Netherlands Institute of Ecology (NIOO-KNAW)
    Wageningen, the Netherlands

Hello Enis,

I would like to give you another update about Galaxy on Opennebula. I haven't been working on it that much recently, but nevertheless I think it's time to merge it with the repository. Off course there is a new file opennebula.py, but I also had to make some changes to master.py, app.py, galaxy.py and comm.py. Should I make patches for these files? I hope the are not causing backward compatibility problems. Maybe you can check it first? I added comments with my name or placed if statements to check if its running opennebula. See the attached files for details.

The Cloud I am using is reaching its official release on 4th October (http://sara.nl/systems/hpc-cloud). The asked me if I could give a demonstration or poster/presentation on that day to show how I am using the cloud. I think this is a good opportunity to showcase Galaxy Cloud. But off course I need your permission for that because you wrote it and the only thing I did was make it running in a different environment. I don't want to show off with others work.

Hope to hear from you,

Mattias

On Fri, 2011-07-15 at 14:41 +0000, Enis Afgan wrote:

Great! If you fork mi-deployment project on bitbucket and issue a pull request, the new tool can be added to upstream (or send a patch via email and I'll add it manually).

Thanks, Enis

On Fri, Jul 15, 2011 at 2:03 PM, Mattias de Hollander m.dehollander@nioo.knaw.nl wrote: I just found out the former approach you mentioned. Its now installing perfectly. The latter approach won't work for a Qiime installation because its needs PyCogent on its path during the setup.py phase.

    Thanks for clarification.
    If I finished adding Qiime' dependencies I will put it on
    bitbucket.

    Greetz,

    Mattias


    On Fri, 2011-07-15 at 13:44 +0000, Enis Afgan wrote:
    > Each use of fabric's `run` method is independent so they
    cannot be
    > chained; you would either need to prepend the export command
    to the
    > command that depends on the fact PYTHONPATH is set (e.g.,
    run("export
    > PYTHONPATH=...; <command that requires given PP>") or
    (better
    > approach) create an env.sh file in the Qiime's install dir
    so that it
    > includes the required 'export PYTHONPATH=...'. When Galaxy
    invokes
    > Qiime, it will automatically source that env.sh (assuming
    you have
    > config variable `tool_dependency_dir
    = /mnt/galaxyTools/tools` set in
    > Galaxy's universe_wsgi.ini).
    >
    > Enis
    >
    > On Fri, Jul 15, 2011 at 12:19 PM, Mattias de Hollander
    > <m.dehollander@nioo.knaw.nl> wrote:
    >         Hello Enis,
    >
    >         I am trying to add PyCogent to the PYTHONPATH, but
    it seems
    >         the Fabric
    >         run command does not recognize it.
    >         I tried:                 run("export
    >
    PYTHONPATH=/mnt/galaxyTools/tools/PyCogent/1.5.1/lib/python2.6/site-packages/:$PYTHONPATH")
    >         and then use cmd_install() to install Galaxy, but I
    keep
    >         getting the
    >         message that PyCogent is not installed.
    >
    >         The normal strategy to use sys.path will not work
    because that
    >         will
    >         adjust the PYTHONPATH on my local machine, not in
    the image if
    >         I am
    >         correct.
    >         Do you know a way to set the PYTHONPATH for Fabric'
    run()
    >         command?
    >
    >         Thanks,
    >
    >         Mattias
    >
    >
    >         On Thu, 2011-07-14 at 09:05 +0000, Enis Afgan wrote:
    >         > Hi Mattias,
    >         > PyCogent is not currently being installed as part
    of
    >         mi-deployment, so
    >         > the method for installing it should be added (if
    you do so,
    >         please
    >         > consider issuing a pull request on bitbucket so it
    can be
    >         added to the
    >         > project).
    >         >
    >         >
    >         > When it comes to composing $PYTHONPATH to include
    the
    >         reference to
    >         > PyCogent via Qiime's env.sh - this should be done
    as part of
    >         the Qiime
    >         > installation method in mi-deployment (something
    like export
    >         > PYHTONPATH=<path where PyCogent was installed>;
    export
    >         PATH=...).
    >         > Sourcing of the tool's env.sh script is prepended
    to the
    >         execution
    >         > command when running the tool so the appropriate
    environment
    >         settings
    >         > will be loaded before the tool is run.
    >         >
    >         >
    >         > Enis
    >         >
    >         > On Wed, Jul 13, 2011 at 12:09 PM, Mattias de
    Hollander
    >         > <m.dehollander@nioo.knaw.nl> wrote:
    >         >         Hello,
    >         >
    >         >         I am trying to add tools (PyCogent, Qiime)
    to a
    >         Cloudman image
    >         >         using the
    >         >         mi-deployment scripts. Qiime depends on
    PyCogent and
    >         it needs
    >         >         to be
    >         >         accessible on the python path, otherwise I
    get an
    >         error:
    >         >         PyCogent not installed but required. (Is
    it
    >         installed? Is it
    >         >         in the
    >         >         current user's $PYTHONPATH or
    site-packages?)
    >         >
    >         >         Is there a way to add the paths in env.sh
    to the
    >         sys.path
    >         >         during
    >         >         installation using tools_fabfile.py?
    >         >         All the tools are
    in /mnt/galaxyTools/tools.
    >         >         I tried the DependencyManager from galaxy
    but then I
    >         only
    >         >         receive the
    >         >         path to the env.sh file:
    >         >         In [43]:
    dependency_manager.find_dep('blast')
    >         >         Out[43]:
    >         >
    ('/mnt/galaxyTools/tools/blast/2.2.25/env.sh',
    >         >          '/mnt/galaxyTools/tools/blast/2.2.25',
    >         >          '2.2.25')
    >         >
    >         >         Is there a smarter way to add files to the
    >         PYTHONPATH during
    >         >         the
    >         >         execution of tools_fabfile.py?
    >         >
    >         >         Thanks,
    >         >
    >         >         Mattias
    >         >
    >         >         --
    >         >         Bioinformatician
    >         >         Netherlands Institute of Ecology
    (NIOO-KNAW)
    >         >         Wageningen, the Netherlands
    >         >
    >         >
    >
    >
    >
    >         --
    >
    >         Bioinformatician
    >         Netherlands Institute of Ecology (NIOO-KNAW)
    >         Wageningen, the Netherlands
    >
    >
    >



    --

    Bioinformatician
    Netherlands Institute of Ecology (NIOO-KNAW)
    Wageningen, the Netherlands