[galaxy-dev] [Spam:*****] Re: How galaxy workflow work ?

18 Dec 2014

      I am also having problems using the Galaxy API, with a similar error message to the one reported by Bongsoo Park a few weeks ago (Uncaught exception in exposed API method). We noticed it a few days ago and have not found the cause yet. What we do know:
- the problem occurs on several Galaxy servers, including the main instance at https://usegalaxy.org/;
- the problem seems to occur whilte using BioBlend, blend4j, and api Python scripts;
- even some of the blend4j unit tests are failing.

One of my colleagues noticed a lot of changes in for example workflows.py (in lib/galaxy/webapps/galaxy/api): https://bitbucket.org/galaxy/galaxy-dist/annotate/579d211c2b0fa8ef4195930cb9...

Did something change in the Galaxy API? Should we adjust the libraries and the programs that use them?

Cheers, Freek

________________________________________
From: galaxy-dev [galaxy-dev-bounces@lists.galaxyproject.org] on behalf of galaxy-dev-request@lists.galaxyproject.org [galaxy-dev-request@lists.galaxyproject.org]
Sent: Tuesday, December 16, 2014 18:00
To: galaxy-dev@lists.galaxyproject.org
Subject: galaxy-dev Digest, Vol 102, Issue 17

Send galaxy-dev mailing list submissions to
        galaxy-dev@lists.galaxyproject.org

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.galaxyproject.org/listinfo/galaxy-dev
or, via email, send a message with subject or body 'help' to
        galaxy-dev-request@lists.galaxyproject.org

You can reach the person managing the list at
        galaxy-dev-owner@lists.galaxyproject.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of galaxy-dev digest..."

HEY!  This is important!  If you reply to a thread in a digest, please
1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread.
2. Strip out everything else in the digest that is not part of the thread you are responding to.

Why?
1. This will keep the subject meaningful.  People will have some idea from the subject line if they should read it or not.
2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.

Today's Topics:

   1. Re: How galaxy workflow work ? (John Chilton)
   2. Re: Metadata error in uploading files to data libraries
      (John Chilton)
   3. Re: package_numpy_1_7 can't be installed (Wang, Yu)
   4. Re: "join" produces an import error (Wang, Yu)
   5. Re: package_numpy_1_7 can't be installed (Björn Grüning)

----------------------------------------------------------------------

Message: 1
Date: Mon, 15 Dec 2014 14:24:13 -0500
From: John Chilton <jmchilton@gmail.com>
To: Bongsoo Park <bxp12@psu.edu>
Cc: galaxy-dev <galaxy-dev@lists.bx.psu.edu>
Subject: Re: [galaxy-dev] How galaxy workflow work ?
Message-ID:
        <CANwbokcggD3W7Tkyd5fy-9d6qedLT+ta1gzzmkfE1Apa2R8w1A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Somehow your user object is not being set. Are you sure you have
initialized bioblend with the correct API key? Do other operations
work with that key?

-John

On Wed, Dec 3, 2014 at 11:00 AM, Bongsoo Park <bxp12@psu.edu> wrote:
...
Hi All,
I faced a problem after I tested the Bio-Blend API. I cannot access the
history anymore with the below error messages. Although, I tried
installation from the scratch, but I got the same error message. Please let
me know how to fix this problem. Thanks!
(GALAXY User Interface)
=======================================================
"xhr": {
"readyState": 4,
"responseText": "{\"err_msg\": \"Uncaught exception in exposed API
method:\", \"err_code\": 0}",
"responseJSON": {
"err_msg": "Uncaught exception in exposed API method:",
"err_code": 0
},
"status": 500,
"statusText": "Internal Server Error",
"responseHeaders": {
"Date": "Wed, 03 Dec 2014 15:50:02 GMT\r",
"cache-control": "max-age=0,no-cache,no-store\r",
"Server": "PasteWSGIServer/0.5 Python/2.7.8\r",
"Connection": "close\r",
"content-type": "application/json\r"
}
=======================================================
I found the error message in the Galaxy instance shell interface.
=======================================================
galaxy.web.framework.decorators ERROR 2014-12-03 10:50:02,279 Uncaught
exception in exposed API method:
Traceback (most recent call last):
File "/home/bxp12/galaxy-dist/lib/galaxy/web/framework/decorators.py",
line 244, in decorator
rval = func( self, trans, *args, **kwargs)
File
"/home/bxp12/galaxy-ist/lib/galaxy/webapps/galaxy/api/history_contents.py",
line 76, in index
and ( history_id == trans.security.encode_id( trans.history.id ) ) ):
AttributeError: 'NoneType' object has no attribute 'id'
=======================================================
Best,
Bongsoo
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
------------------------------

Message: 2
Date: Mon, 15 Dec 2014 14:31:20 -0500
From: John Chilton <jmchilton@gmail.com>
To: Jelle Scholtalbers <j.scholtalbers@gmail.com>
Cc: galaxy-dev <galaxy-dev@lists.bx.psu.edu>
Subject: Re: [galaxy-dev] Metadata error in uploading files to data
        libraries
Message-ID:
        <CANwbokdzjumt4abseN3b6rxOkaXH+9fB23+_OhqmUtFqnxbxWw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hello,

  Sorry we haven't made progress on this - and thanks for creating a
Trello card (https://trello.com/c/tw75nq1U).

  It looks like samtools is not on your path, can you use it from the
command-line? If not you should probably install it - I would suggest
install samtools 0.1.19 with homebrew (looks like you are a on Mac).
If you can run samtools from the command-line - can you do a which
`samtools` from your command-line and see where it is coming from and
then add that directory explicitly to your Galaxy PATH - say at the
top of run.sh in your Galaxy root (let me know if you need more
details on that).

-John

On Tue, Dec 9, 2014 at 3:02 AM, Jelle Scholtalbers
<j.scholtalbers@gmail.com> wrote:
...
Hi all,
after an update to the following changeset(14859:7ba05957588a, stable,
05.12.14), our bam files that are uploaded(linked) to a data library, are no
longer indexed. The metadata_xxx.dat is created, but it stays empty.
The following error message appears in the log, although the state of the
dataset is 'ok':
galaxy.jobs WARNING 2014-12-05 12:47:02,218 Error accessing /g/K/K27.bam,
will retry: [Errno 1] Operation not permitted: '/g/K/K27.bam'
galaxy.jobs WARNING 2014-12-08 13:38:57,045 Error accessing /g/K/K2.bam,
will retry: [Errno 1] Operation not permitted: '/g/K/K2.bam'
All file permissions are correct (i.e. galaxy owns them). Furthermore,
executing samtools index, just works on those files:
samtools index /g/K/K2.bam
/g/galaxy/galaxy_data/files/_metadata_files/006/metadata_6598.dat
When uploading the file - "copy files into galaxy" - the samtools index just
works.
==============
Now, on a clean local install(14874:885f940bff64, stable, 05.12.14) and
samtools installed globally and with the bam file sorted, I get the
following situation:
When I try to upload this bam to a data library by linking the following
error is shown on the dataset (note: here the dataset is set in error state,
which does not happen on our server)
Uploaded by: scholtal@embl.de
Date uploaded: Mon Dec 8 17:42:39 2014 (UTC)
File size: 2.6 GB
UUID: d23cf11a-0372-41cb-939a-7c8761d78b73
Data type: auto
Build: ?
Miscellaneous information: Traceback (most recent call last): File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in __main__() File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in _
Job Standard Error
Traceback (most recent call last):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in
    __main__()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 294, in add_file
    if datatype.dataset_content_needs_grooming( dataset.path ):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
stdout=subprocess.PIPE ).communicate()[1]
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 711, in __init__
    errread, errwrite)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
error
Database/Build: ?
Number of data lines: None
Disk file: /Users/scholtalbers/workspace/idr_data/WT1.sort.bam
===============================
When uploading the bam without linking, I see the following processes:
Upload->set meta->samtools index->'error state'
Miscellaneous information: uploaded bam file Traceback (most recent call
last): File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in __main__() File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.p
Job Standard Error
Traceback (most recent call last):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in
    __main__()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 324, in add_file
    if link_data_only == 'copy_files' and
datatype.dataset_content_needs_grooming( output_path ):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
stdout=subprocess.PIPE ).communicate()[1]
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 711, in __init__
    errread, errwrite)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
error
Database/Build: ?
Number of data lines: None
Disk file:
/Users/scholtalbers/workspace/galaxy-dist-new/database/files/000/dataset_6.dat
=================================
Although the error messages are different, it might be related?
Cheers,
Jelle
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
------------------------------

Message: 3
Date: Mon, 15 Dec 2014 17:16:41 +0000
From: "Wang, Yu" <Yu.Wang@lrz.de>
To: John Chilton <jmchilton@gmail.com>
Cc: "Wang, Yu" <Yu.Wang@lrz.de>, "galaxy-dev@lists.galaxyproject.org"
        <galaxy-dev@lists.galaxyproject.org>
Subject: Re: [galaxy-dev] package_numpy_1_7 can't be installed
Message-ID:
        <f4c6170a-5f28-4871-971b-2881fc739510@BADWLRZ-SW13MB1.ads.mwn.de>
Content-Type: text/plain; charset="utf-8"

Hi, John,thanks for your answer. I installed svnversion, and compiled/installed ATLAS/Blas/ LAPACK. I still got error messages like ATLAS/Blas/ LAPACK are not found.
I could install package_numpy_1_9 by using galaxy main tool shed. There is something wrong with package_numpy_1_7 on my environment.

Cheers,
Yu

di29her
wang@lrz.de<mailto:wang@lrz.de>

On 15 Dec 2014, at 16:12, John Chilton <jmchilton@gmail.com<mailto:jmchilton@gmail.com>> wrote:

Looks like this - https://github.com/clarete/curdling/issues/19

I think you somehow need to install the package svn on your machine - if you have root on that machine the command might be something like "sudo zypper install svn" or may it is "sudo zypper install subversion". If you don't have root on the machine I would ask the operating system maintainer or try to compile and install svn locally and make it available to Galaxy.

Hope this helps,

-John

On Fri, Dec 12, 2014 at 7:14 AM, Wang, Yu <Yu.Wang@lrz.de<mailto:Yu.Wang@lrz.de>> wrote:
Hi, I am running galaxy on SLSE11, when I tried to install bumpy 1.7.1, I got the following error.
Can anyone point what goes wrong here?

Error   Running from numpy source directory.
/bin/sh: svnversion: command not found
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1494: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1503: UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  warnings.warn(BlasNotFoundError.__doc__)
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1506: UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  warnings.warn(BlasSrcNotFoundError.__doc__)
/bin/sh: svnversion: command not found
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1408: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1419: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  warnings.warn(LapackNotFoundError.__doc__)
/home/wang/galaxy-dist/database/tmp/tmp-toolshed-mtdMR9ITC/numpy-1.7.1/numpy/distutils/system_info.py:1422: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  warnings.warn(LapackSrcNotFoundError.__doc__)
error: must supply either home or prefix/exec-prefix -- not both

Cheers,
Yu

di29her
wang@lrz.de<mailto:wang@lrz.de>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] [Spam:*****] Re: How galaxy workflow work ?

Bruijn, Freek de