automatic adding of a tab in a fastq header
Hi, We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character. from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added.Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr <mailto:fabien.mareuil@pasteur.fr> | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
Hi Fabien, There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem. Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes. Peter On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Peter, Thank a lot for this information, I didn't found this option for the new version of the Data Libraries. This could explain why there is this conversion. I tested it with an automatic format detection and when I set explicitly the format but in the two cases a tab was added. Fabien Le 18/03/2016 17:04, Peter Cock a écrit :
Hi Fabien,
There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem.
Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes.
Peter
On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Can you provide more details - in particular the version of Galaxy you are running? A step-by-step description of how you did the data import might be useful too for the Galaxy team to try to reproduce this. Peter On Fri, Mar 18, 2016 at 5:34 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
Thank a lot for this information, I didn't found this option for the new version of the Data Libraries. This could explain why there is this conversion. I tested it with an automatic format detection and when I set explicitly the format but in the two cases a tab was added.
Fabien
Le 18/03/2016 17:04, Peter Cock a écrit :
Hi Fabien,
There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem.
Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes.
Peter
On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Peter, I tested it on the last version of the dev and master branches. step-by-step : I click on the "shared data/Data Libraries", I select my library and I click on the button to upload a file and I select the option "from User Directory". I select my fastqsanger file (sequences headers are like this : @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC with a space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC and I import it (with or without the auto-detect option). When I look the file in a history, the space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC has been replaced by a tab. I didn't have this problem with "shared data/ Data Libraries deprecated". Thank a lot for your help, Best, Fabien On 18/03/2016 18:59, Peter Cock wrote:
Can you provide more details - in particular the version of Galaxy you are running? A step-by-step description of how you did the data import might be useful too for the Galaxy team to try to reproduce this.
Peter
On Fri, Mar 18, 2016 at 5:34 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
Thank a lot for this information, I didn't found this option for the new version of the Data Libraries. This could explain why there is this conversion. I tested it with an automatic format detection and when I set explicitly the format but in the two cases a tab was added.
Fabien
Le 18/03/2016 17:04, Peter Cock a écrit :
Hi Fabien,
There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem.
Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes.
Peter
On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Peter, I worked on it today and I found the reason of this bug. In the upload.py tool, the existence of dataset.space_to_tab is tested. In the old version of the data library, when the option was unchecked, the value space_to_tab was None and the test worked. But in the new version the value of space_to_tab is "False", it's a string and the test "if dataset.space_to_tab" returns True. It is possible to fix it with a little correction of the "/lib/galaxy/webapps/galaxy/api/lda_datasets.py". I join a patch with this email. I can propose a pull request if you want. Best, Fabien -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr <mailto:fabien.mareuil@pasteur.fr> | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France On 21/03/2016 09:45, fmareuil wrote:
Hi Peter,
I tested it on the last version of the dev and master branches.
step-by-step : I click on the "shared data/Data Libraries", I select my library and I click on the button to upload a file and I select the option "from User Directory". I select my fastqsanger file (sequences headers are like this : @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC with a space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC and I import it (with or without the auto-detect option). When I look the file in a history, the space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC has been replaced by a tab.
I didn't have this problem with "shared data/ Data Libraries deprecated".
Thank a lot for your help, Best,
Fabien
On 18/03/2016 18:59, Peter Cock wrote:
Can you provide more details - in particular the version of Galaxy you are running? A step-by-step description of how you did the data import might be useful too for the Galaxy team to try to reproduce this.
Peter
On Fri, Mar 18, 2016 at 5:34 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
Thank a lot for this information, I didn't found this option for the new version of the Data Libraries. This could explain why there is this conversion. I tested it with an automatic format detection and when I set explicitly the format but in the two cases a tab was added.
Fabien
Le 18/03/2016 17:04, Peter Cock a écrit :
Hi Fabien,
There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem.
Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes.
Peter
On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thank you very much for solving this Fabien, The PR with your fix has been created here: https://github.com/galaxyproject/galaxy/pull/1985 Martin On Mon, Mar 21, 2016 at 5:43 PM fmareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
I worked on it today and I found the reason of this bug. In the upload.py tool, the existence of dataset.space_to_tab is tested. In the old version of the data library, when the option was unchecked, the value space_to_tab was None and the test worked. But in the new version the value of space_to_tab is "False", it's a string and the test "if dataset.space_to_tab" returns True. It is possible to fix it with a little correction of the "/lib/galaxy/webapps/galaxy/api/lda_datasets.py". I join a patch with this email. I can propose a pull request if you want.
Best,
Fabien
-- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
On 21/03/2016 09:45, fmareuil wrote:
Hi Peter,
I tested it on the last version of the dev and master branches.
step-by-step : I click on the "shared data/Data Libraries", I select my library and I click on the button to upload a file and I select the option "from User Directory". I select my fastqsanger file (sequences headers are like this : @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC with a space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC and I import it (with or without the auto-detect option). When I look the file in a history, the space between @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 and 1:N:0:ATGAGC has been replaced by a tab.
I didn't have this problem with "shared data/ Data Libraries deprecated".
Thank a lot for your help, Best,
Fabien
On 18/03/2016 18:59, Peter Cock wrote:
Can you provide more details - in particular the version of Galaxy you are running? A step-by-step description of how you did the data import might be useful too for the Galaxy team to try to reproduce this.
Peter
On Fri, Mar 18, 2016 at 5:34 PM, Fabien Mareuil <fmareuil@pasteur.fr> <fmareuil@pasteur.fr> wrote:
Hi Peter,
Thank a lot for this information, I didn't found this option for the new version of the Data Libraries. This could explain why there is this conversion. I tested it with an automatic format detection and when I set explicitly the format but in the two cases a tab was added.
Fabien
Le 18/03/2016 17:04, Peter Cock a écrit :
Hi Fabien,
There is a Galaxy upload option to turn spaces into tabs, useful for tables of data. Is that being ticked by mistake? That would explain this problem.
Also I would check if the FASTQ file format is being detected automatically, or set explicitly (e.g. fastqsanger)? Doing that ought to prevent any changes.
Peter
On Fri, Mar 18, 2016 at 3:26 PM, Fabien Mareuil <fmareuil@pasteur.fr> <fmareuil@pasteur.fr> wrote:
Hi,
We got a problem with the new version of Data libraries. When we upload a fastq file from a User Directory, a tabulation character is added in the header in replacement of a space character.
from @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC to @HWI-D00395:188:HYGNFADXX:2:1108:1315:1965 1:N:0:ATGAGC
This behavior causes an error with cutadapt 1.6 (maybe with other tools) because it doesn't work with tab character. With the direct upload tool or the deprecated version of Data Libraries no tab character is added. Do you have some idea to correct this problem? Thanks a lot Best regards -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil@pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Well done for solving this Fabien, with only a small hint from me. I'm glad to see your suggested fix has already been applied to the Galaxy repository so should be in the next stable release. Thank you! Peter (Also thank you Martin, Nicola, Dan, etc) On Tue, Mar 22, 2016 at 8:43 PM, Martin Čech <marten@bx.psu.edu> wrote:
Thank you very much for solving this Fabien, The PR with your fix has been created here: https://github.com/galaxyproject/galaxy/pull/1985
Martin
On Mon, Mar 21, 2016 at 5:43 PM fmareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
I worked on it today and I found the reason of this bug. In the upload.py tool, the existence of dataset.space_to_tab is tested. In the old version of the data library, when the option was unchecked, the value space_to_tab was None and the test worked. But in the new version the value of space_to_tab is "False", it's a string and the test "if dataset.space_to_tab" returns True. It is possible to fix it with a little correction of the "/lib/galaxy/webapps/galaxy/api/lda_datasets.py". I join a patch with this email. I can propose a pull request if you want.
Best,
Fabien
Thank you for all the help Peter! Also this is already backported and fixed in 16.01 Sorry for causing this bug in the first place. M. On Tue, Mar 22, 2016, 18:29 Peter Cock <p.j.a.cock@googlemail.com> wrote:
Well done for solving this Fabien, with only a small hint from me. I'm glad to see your suggested fix has already been applied to the Galaxy repository so should be in the next stable release.
Thank you!
Peter
(Also thank you Martin, Nicola, Dan, etc)
On Tue, Mar 22, 2016 at 8:43 PM, Martin Čech <marten@bx.psu.edu> wrote:
Thank you very much for solving this Fabien, The PR with your fix has been created here: https://github.com/galaxyproject/galaxy/pull/1985
Martin
On Mon, Mar 21, 2016 at 5:43 PM fmareuil <fmareuil@pasteur.fr> wrote:
Hi Peter,
I worked on it today and I found the reason of this bug. In the
upload.py
tool, the existence of dataset.space_to_tab is tested. In the old version of the data library, when the option was unchecked, the value space_to_tab was None and the test worked. But in the new version the value of space_to_tab is "False", it's a string and the test "if dataset.space_to_tab" returns True. It is possible to fix it with a little correction of the "/lib/galaxy/webapps/galaxy/api/lda_datasets.py". I join a patch with this email. I can propose a pull request if you want.
Best,
Fabien
Thank :)! Fabien Le 23/03/2016 00:01, Martin Čech a écrit :
Thank you for all the help Peter!
Also this is already backported and fixed in 16.01
Sorry for causing this bug in the first place. M.
On Tue, Mar 22, 2016, 18:29 Peter Cock <p.j.a.cock@googlemail.com <mailto:p.j.a.cock@googlemail.com>> wrote:
Well done for solving this Fabien, with only a small hint from me. I'm glad to see your suggested fix has already been applied to the Galaxy repository so should be in the next stable release.
Thank you!
Peter
(Also thank you Martin, Nicola, Dan, etc)
On Tue, Mar 22, 2016 at 8:43 PM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote: > Thank you very much for solving this Fabien, The PR with your fix has been > created here: https://github.com/galaxyproject/galaxy/pull/1985 > > Martin > > > On Mon, Mar 21, 2016 at 5:43 PM fmareuil <fmareuil@pasteur.fr <mailto:fmareuil@pasteur.fr>> wrote: >> >> Hi Peter, >> >> I worked on it today and I found the reason of this bug. In the upload.py >> tool, the existence of dataset.space_to_tab is tested. In the old version of >> the data library, when the option was unchecked, the value space_to_tab was >> None and the test worked. But in the new version the value of space_to_tab >> is "False", it's a string and the test "if dataset.space_to_tab" returns >> True. >> It is possible to fix it with a little correction of the >> "/lib/galaxy/webapps/galaxy/api/lda_datasets.py". >> I join a patch with this email. >> I can propose a pull request if you want. >> >> Best, >> >> Fabien >>
participants (4)
-
Fabien Mareuil -
fmareuil
-
Martin Čech -
Peter Cock