Hello Kwon,

Most of this sounds correct. The only issue is with compressed files. Galaxy uses uncompressed liftOver data at our site and this is how we instruct that it be set up (see galaxy-central/tool-data/liftOver.loc.sample).

Would you be able to run a test to see if uncompressing resolves the issue? Uncompress one of the over.chain files, modify the .loc file so that the name also reflects the uncompressed naming, then try to use the UI version of liftOver with this uncompressed data. Please let us know how this works out (cc-ing the list, so that others can learn of the confirmed dependency).

For now, there is no automated way to enter the .loc data entries. There is some development activity going on right now to address data setup (including liftOver), but nothing is ready yet to be shared. When complete, any new tools will be announced (loudly - we know that they are highly desired).

Thanks!

Jen Galaxy team

On 7/3/12 10:21 PM, Kwon SoonJae wrote:

Hi Everyone,

I installed a local instance of galaxy and some tools using the command line, and for some reason liftover will work in the command line but won't work in galaxy. Here's what I've done so far:

• Downloaded liftover standalone executable from the tool-dependencies

page in the Gwiki.

• Changed permissions on liftover executable to make it usable in the

command prompt (terminal, macosx) -> otherwise I get permission denied.

• Copied executable over to /usr/local/bin likewise to use in command

prompt and in Galaxy.

• Modified liftover.loc to include all the conversion types and

directories of chain.gz files I downloaded to use with the tool (I believe liftover can make use of both .chain and .chain.gz files).

After that I tried to run liftover in Galaxy and it didn't work, giving me some error that said hg19ToMm9 mapping is currently not available. It seems to work fine using the command line though.

Since my goal is to setup galaxy locally, I really want to get this working correctly. Can anyone guide me through what's gone wrong? Also, there must be a better way of installing all these tools than manually downloading every single one and using the command line, surely? I had to type out the format of liftover.loc file manually for around 130 chain files... I'm convinced there has to be a better way.

Thanks in advance,

SJ

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

Hello Kwon,

There are no known issues with the liftOver wrapper (tools / extract / liftOver_wrapper.py and liftOver_wrapper.xml).

Three more troubleshooting ideas:

1 - This may sound simple, but have you restarted the database since you updated the .loc files? http://wiki.g2.bx.psu.edu/Admin/Data%20Integration (final step)

2 - Remove the relative path from the .loc file entries (the "~/")) and replace with an explicit full path. I personally have not seen this used before in .loc files. Perhaps others on the list can comment about whether they have been able to use this successfully and if any special configuration was necessary.

3- Confirm that the genomes are in your builds list (also covered in wiki above). Note that UCSC "dbkey" values have special capitalization rules depending on where they are used. Your example .loc file has this correct, but I recommend that you also check your builds list entries against the wiki examples and your actual data. If you make any changes, be sure to restart before testing.

For example, the liftOver file pair: hg19ToDanRer6.over.chain & danRer6ToHg19.over.chain human dbkey = hg19 zebrafish dbkey = danRer6

Please let us know how this works out,

Jen Galaxy team

On 7/4/12 7:35 PM, Kwon SoonJae wrote:

Hi Jen,

Thanks for looking into this. I have unzipped all the .gz files so that the directory contains all .chain files. The liftover.loc file has also been modified to refer to all the .chain files instead, as such: http://pastebin.com/xSKGvUtG

I still run into the same error if I try to run the tool in Galaxy, though (hg19ToMm9 mapping currently not available). The liftover command still works fine through the command line.

I have a feeling the error has to do with the issue with Galaxy's job runner returning anything written to stderr as an error, mentioned here: http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr

Seems like it was an issue back in 2010. Has this been rectified?

Again, thank you for your help,

SJ

On Wed, Jul 4, 2012 at 9:42 PM, Jennifer Jackson <jen@bx.psu.edu mailto:jen@bx.psu.edu> wrote:

Hello Kwon,

Most of this sounds correct. The only issue is with compressed
files. Galaxy uses uncompressed liftOver data at our site and this
is how we instruct that it be set up (see
galaxy-central/tool-data/liftOver.loc.sample).

Would you be able to run a test to see if uncompressing resolves
the issue? Uncompress one of the over.chain files, modify the .loc
file so that the name also reflects the uncompressed naming, then
try to use the UI version of liftOver with this uncompressed data.
Please let us know how this works out (cc-ing the list, so that
others can learn of the confirmed dependency).

For now, there is no automated way to enter the .loc data entries.
There is some development activity going on right now to address
data setup (including liftOver), but nothing is ready yet to be
shared. When complete, any new tools will be announced (loudly -
we know that they are highly desired).

Thanks!

Jen
Galaxy team


On 7/3/12 10:21 PM, Kwon SoonJae wrote:

...

Hi Everyone,

I installed a local instance of galaxy and some tools using the
command line, and for some reason liftover will work in the
command line but won't work in galaxy. Here's what I've done so far:

- Downloaded liftover standalone executable from the
tool-dependencies page in the Gwiki.

- Changed permissions on liftover executable to make it usable in
the command prompt (terminal, macosx) -> otherwise I get
permission denied.

- Copied executable over to /usr/local/bin likewise to use in
command prompt and in Galaxy.

- Modified liftover.loc to include all the conversion types and
directories of chain.gz files I downloaded to use with the tool
(I believe liftover can make use of both .chain and .chain.gz files).

After that I tried to run liftover in Galaxy and it didn't work,
giving me some error that said hg19ToMm9 mapping is currently not
available. It seems to work fine using the command line though.

Since my goal is to setup galaxy locally, I really want to get
this working correctly. Can anyone guide me through what's gone
wrong?
Also, there must be a better way of installing all these tools
than manually downloading every single one and using the command
line, surely? I had to type out the format of liftover.loc file
manually for around 130 chain files... I'm convinced there has to
be a better way.

Thanks in advance,

SJ


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/

-- 
Jennifer Jackson
http://galaxyproject.org

Great news, very glad this worked. I'll add liftOver to our wiki with more specific instructions and add in the bit about explicit paths to the .loc file section as well. Should help others going forward.

Thanks for taking the time to report back the solution to the list!

Jen Galaxy team

On 7/5/12 1:47 AM, Kwon SoonJae wrote:

Hi Jen!

Definitely didn't forget to reset the server everytime I tried. The problem was solved when I fixed the relative path to the full path.. can't believe I was stuck forever because of such a small miss. Fixed! and everything working so far =)

Thank you so much,

SJ

On Thu, Jul 5, 2012 at 4:43 PM, Jennifer Jackson <jen@bx.psu.edu mailto:jen@bx.psu.edu> wrote:

Hello Kwon,

There are no known issues with the liftOver wrapper (tools / extract
/ liftOver_wrapper.py and liftOver_wrapper.xml).

Three more troubleshooting ideas:

1 - This may sound simple, but have you restarted the database since
you updated the .loc files?
http://wiki.g2.bx.psu.edu/Admin/Data%20Integration (final step)

2 - Remove the relative path from the .loc file entries (the "~/"))
and replace with an explicit full path. I personally have not seen
this used before in .loc files. Perhaps others on the list can
comment about whether they have been able to use this successfully
and if any special configuration was necessary.

3- Confirm that the genomes are in your builds list (also covered in
wiki above). Note that UCSC "dbkey" values have special
capitalization rules depending on where they are used. Your example
.loc file has this correct, but I recommend that you also check your
builds list entries against the wiki examples and your actual data.
If you make any changes, be sure to restart before testing.

For example, the liftOver file pair: hg19ToDanRer6.over.chain &
danRer6ToHg19.over.chain
human dbkey = hg19
zebrafish dbkey = danRer6

Please let us know how this works out,

Jen
Galaxy team


On 7/4/12 7:35 PM, Kwon SoonJae wrote:

...

Hi Jen,

Thanks for looking into this. I have unzipped all the .gz files so
that the directory contains all .chain files. The liftover.loc
file has also been modified to refer to all the .chain files
instead, as such: http://pastebin.com/xSKGvUtG

I still run into the same error if I try to run the tool in
Galaxy, though (hg19ToMm9 mapping currently not available). The
liftover command still works fine through the command line.

I have a feeling the error has to do with the issue with Galaxy's
job runner returning anything written to stderr as an error,
mentioned here:
http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr

Seems like it was an issue back in 2010. Has this been rectified?

Again, thank you for your help,

SJ

On Wed, Jul 4, 2012 at 9:42 PM, Jennifer Jackson <jen@bx.psu.edu
<mailto:jen@bx.psu.edu>> wrote:

    Hello Kwon,

    Most of this sounds correct. The only issue is with compressed
    files. Galaxy uses uncompressed liftOver data at our site and
    this is how we instruct that it be set up (see
    galaxy-central/tool-data/liftOver.loc.sample).

    Would you be able to run a test to see if uncompressing
    resolves the issue? Uncompress one of the over.chain files,
    modify the .loc file so that the name also reflects the
    uncompressed naming, then try to use the UI version of
    liftOver with this uncompressed data. Please let us know how
    this works out (cc-ing the list, so that others can learn of
    the confirmed dependency).

    For now, there is no automated way to enter the .loc data
    entries. There is some development activity going on right now
    to address data setup (including liftOver), but nothing is
    ready yet to be shared. When complete, any new tools will be
    announced (loudly - we know that they are highly desired).

    Thanks!

    Jen
    Galaxy team


    On 7/3/12 10:21 PM, Kwon SoonJae wrote:

...

    Hi Everyone,

    I installed a local instance of galaxy and some tools using
    the command line, and for some reason liftover will work in
    the command line but won't work in galaxy. Here's what I've
    done so far:

    - Downloaded liftover standalone executable from the
    tool-dependencies page in the Gwiki.

    - Changed permissions on liftover executable to make it
    usable in the command prompt (terminal, macosx) -> otherwise
    I get permission denied.

    - Copied executable over to /usr/local/bin likewise to use in
    command prompt and in Galaxy.

    - Modified liftover.loc to include all the conversion types
    and directories of chain.gz files I downloaded to use with
    the tool (I believe liftover can make use of both .chain and
    .chain.gz files).

    After that I tried to run liftover in Galaxy and it didn't
    work, giving me some error that said hg19ToMm9 mapping is
    currently not available. It seems to work fine using the
    command line though.

    Since my goal is to setup galaxy locally, I really want to
    get this working correctly. Can anyone guide me through
    what's gone wrong?
    Also, there must be a better way of installing all these
    tools than manually downloading every single one and using
    the command line, surely? I had to type out the format of
    liftover.loc file manually for around 130 chain files... I'm
    convinced there has to be a better way.

    Thanks in advance,

    SJ


    ___________________________________________________________
    Please keep all replies on the list by using"reply all"
    in your mail client.  To manage your subscriptions to this
    and other Galaxy lists, please use the interface at:

       http://lists.bx.psu.edu/

    --
    Jennifer Jackson
    http://galaxyproject.org

--
Jennifer Jackson
http://galaxyproject.org