Hi, Some of our users are experiencing problems with (mainly) tabular data on our local install of Galaxy (changeset 8368:0042b30216fc, Nov 06 2012). I'm presuming it's some kind of meta-data problem. The first strange behavior is that they are getting green history items, but when the history item is expanded it has 'no peek' in the data preview and 'empty' as the line-count. When they click on the eye icon, their data appears and the output is as expected. Sometimes the data preview panel (in the history item) has the number of columns across the top of it (but still with 'no peek') and this data can be used downstream, but often the columns are missing or incorrect and although the output file is correctly tabulated, it cannot be used downstream. All of these problems can be addressed by using the Auto-detect in the Edit Attributes. This provides the correct column count, gives the line count and provides a peek. This happens with lots of different tools, usually (but not exclusively) with tabular data. I'm wondering if anyone has ever encountered this problem before and what they did to address it. Many thanks,
Graham
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
On Feb 7, 2013, at 6:04 AM, graham etherington (TSL) wrote:
Hi, Some of our users are experiencing problems with (mainly) tabular data on our local install of Galaxy (changeset 8368:0042b30216fc, Nov 06 2012). I'm presuming it's some kind of meta-data problem. The first strange behavior is that they are getting green history items, but when the history item is expanded it has 'no peek' in the data preview and 'empty' as the line-count. When they click on the eye icon, their data appears and the output is as expected. Sometimes the data preview panel (in the history item) has the number of columns across the top of it (but still with 'no peek') and this data can be used downstream, but often the columns are missing or incorrect and although the output file is correctly tabulated, it cannot be used downstream. All of these problems can be addressed by using the Auto-detect in the Edit Attributes. This provides the correct column count, gives the line count and provides a peek. This happens with lots of different tools, usually (but not exclusively) with tabular data. I'm wondering if anyone has ever encountered this problem before and what they did to address it. Many thanks,
Hi Graham,
Is this a sporadic problem, and are you using a cluster (and a shared filesystem)?
--nate
Graham
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Nate, It's a sporadic problem (in that it happens quite often, but not all the time) and yes, the Galaxy jobs are dispatched to a cluster. I'm not sure about the shared file system. Galaxy is defined as a user on our cluster, along with many other users. Does that answer your question? Cheers, Graham
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
On 07/02/2013 13:39, "Nate Coraor" nate@bx.psu.edu wrote:
On Feb 7, 2013, at 6:04 AM, graham etherington (TSL) wrote:
Hi, Some of our users are experiencing problems with (mainly) tabular data on our local install of Galaxy (changeset 8368:0042b30216fc, Nov 06 2012). I'm presuming it's some kind of meta-data problem. The first strange behavior is that they are getting green history items, but when the history item is expanded it has 'no peek' in the data preview and 'empty' as the line-count. When they click on the eye icon, their data appears and the output is as expected. Sometimes the data preview panel (in the history item) has the number of columns across the top of it (but still with 'no peek') and this data can be used downstream, but often the columns are missing or incorrect and although the output file is correctly tabulated, it cannot be used downstream. All of these problems can be addressed by using the Auto-detect in the Edit Attributes. This provides the correct column count, gives the line count and provides a peek. This happens with lots of different tools, usually (but not exclusively) with tabular data. I'm wondering if anyone has ever encountered this problem before and what they did to address it. Many thanks,
Hi Graham,
Is this a sporadic problem, and are you using a cluster (and a shared filesystem)?
--nate
Graham
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Feb 7, 2013, at 9:07 AM, graham etherington (TSL) wrote:
Hi Nate, It's a sporadic problem (in that it happens quite often, but not all the time) and yes, the Galaxy jobs are dispatched to a cluster. I'm not sure about the shared file system. Galaxy is defined as a user on our cluster, along with many other users. Does that answer your question? Cheers, Graham
Thanks Graham, that information helps. It's not really possible to use Galaxy on a cluster without a shared filesystem at this point. My guess as to what's going on here is that the filesystem is caching attributes (e.g. that the file is empty) on job outputs. This can be disabled via specific mount options, there is some discussion of it at the bottom of this section in the documentation:
http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Unified_Metho...
--nate
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
On 07/02/2013 13:39, "Nate Coraor" nate@bx.psu.edu wrote:
On Feb 7, 2013, at 6:04 AM, graham etherington (TSL) wrote:
Hi, Some of our users are experiencing problems with (mainly) tabular data on our local install of Galaxy (changeset 8368:0042b30216fc, Nov 06 2012). I'm presuming it's some kind of meta-data problem. The first strange behavior is that they are getting green history items, but when the history item is expanded it has 'no peek' in the data preview and 'empty' as the line-count. When they click on the eye icon, their data appears and the output is as expected. Sometimes the data preview panel (in the history item) has the number of columns across the top of it (but still with 'no peek') and this data can be used downstream, but often the columns are missing or incorrect and although the output file is correctly tabulated, it cannot be used downstream. All of these problems can be addressed by using the Auto-detect in the Edit Attributes. This provides the correct column count, gives the line count and provides a peek. This happens with lots of different tools, usually (but not exclusively) with tabular data. I'm wondering if anyone has ever encountered this problem before and what they did to address it. Many thanks,
Hi Graham,
Is this a sporadic problem, and are you using a cluster (and a shared filesystem)?
--nate
Graham
Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
galaxy-dev@lists.galaxyproject.org
-
graham etherington (TSL) -
Nate Coraor