corrupted binary file
Dear all, I found one of my binary files was corrupted after its uploading through "Get data'. I used editplus to check its hex code and found some "0D" is changed to "0A" and also one byte is lost. May I ask how to solve this problem? Thank you for your help. Best Regards, HU Yingwei
On Fri, Feb 24, 2012 at 4:03 PM, Yingwei HU <husince@gmail.com> wrote:
Dear all,
I found one of my binary files was corrupted after its uploading through "Get data'. I used editplus to check its hex code and found some "0D" is changed to "0A" and also one byte is lost. May I ask how to solve this problem?
Thank you for your help.
Best Regards, HU Yingwei
There has been a similar discussion on the archives - those hex codes are ASCII carriage return and line feed characters, meaning the changes were made assuming the file was plain text to standardize the new lines. What file type is it, and how is this defined on your Galaxy? i.e. Is it a custom binary file type? Peter
Dear Peter, Thank you for your response. Yes. it is a custom binary file type. I attached the sample binary files. humanK562C_d2.splib is the original one. temp.splib is the uploaded and corrupted one. For definition in Galaxy, I just simply inherit the binary.Binary. The definition is below: In Python script: class Splib( binary.Binary ): """ Class describing a spectral library (splib)""" file_ext = "splib" I did not add the sniffer but specify the file format manually in the page of "get data". Best Regards, HU Yingwei 2012/2/25 Peter Cock <p.j.a.cock@googlemail.com>
On Fri, Feb 24, 2012 at 4:03 PM, Yingwei HU <husince@gmail.com> wrote:
Dear all,
I found one of my binary files was corrupted after its uploading through "Get data'. I used editplus to check its hex code and found some "0D" is changed to "0A" and also one byte is lost. May I ask how to solve this problem?
Thank you for your help.
Best Regards, HU Yingwei
There has been a similar discussion on the archives - those hex codes are ASCII carriage return and line feed characters, meaning the changes were made assuming the file was plain text to standardize the new lines.
What file type is it, and how is this defined on your Galaxy? i.e. Is it a custom binary file type?
Peter
On Saturday, February 25, 2012, Yingwei HU wrote:
Dear Peter,
Thank you for your response. Yes. it is a custom binary file type. I attached the sample binary files. humanK562C_d2.splib is the original one. temp.splib is the uploaded and corrupted one. For definition in Galaxy, I just simply inherit the binary.Binary. The definition is below: In Python script: class Splib( binary.Binary ): """ Class describing a spectral library (splib)""" file_ext = "splib" I did not add the sniffer but specify the file format manually in the page of "get data".
Which version of Galaxy (hg revision) are you using? I'd have to go back over the email archive but I think were some changes made to help user defined formats. Peter
Dear Peter, I am sorry I do not know the version of my galaxy by using hg. I directly downloaded the zip file from the address: http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip The zipped files are compressed on 2011-06-29. Is this version too old? Thank you for your help. Best Regards, HU Yingwei 2012/2/25 Peter Cock <p.j.a.cock@googlemail.com>
On Saturday, February 25, 2012, Yingwei HU wrote:
Dear Peter,
Thank you for your response. Yes. it is a custom binary file type. I attached the sample binary files. humanK562C_d2.splib is the original one. temp.splib is the uploaded and corrupted one. For definition in Galaxy, I just simply inherit the binary.Binary. The definition is below: In Python script: class Splib( binary.Binary ): """ Class describing a spectral library (splib)""" file_ext = "splib" I did not add the sniffer but specify the file format manually in the page of "get data".
Which version of Galaxy (hg revision) are you using? I'd have to go back over the email archive but I think were some changes made to help user defined formats.
Peter
On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <husince@gmail.com> wrote:
Dear Peter,
I am sorry I do not know the version of my galaxy by using hg. I directly downloaded the zip file from the address: http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
That would be fine for a one-off test installation, but it will make doing an upgrade extremely difficult. That is why the Galaxy recommend using the mercurial (hg) source code system.
The zipped files are compressed on 2011-06-29. Is this version too old?
The changes I am thinking of were earlier this month (Feb 2012) by Greg on galaxy-central and have not yet been released on galaxy-dist. Hopefully he can provide more information next week. e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d Regards, Peter
It is highly recommended that you get the latest source from our Galaxy distribution repository using mercurial. As Peter stated, this will enable you to get regular updates as new Galaxy distribution releases occur. Your downloaded zip file contains what we consider a very old release. The Galaxy distribution is available using mercurial at this url. https://bitbucket.org/galaxy/galaxy-dist The changes I made to the Galaxy datatypes components were implemented to provide support for proprietary datatypes contained in installed tool shed repositories, so are not related to problems that may occur when uploading. If I understand the issue correctly, it sounds like there is a problem when you upload your file with contents that are a custom datatype which you've subclassed from the Galaxy binary datatype. If your file is being changed by the Galaxy sniff process, then there may be a bug in that process. What do your paster logs show? Also, your data attachments have been lost in the mail thread - did only Peter get them? Greg Von Kuster On Feb 25, 2012, at 5:20 AM, Peter Cock wrote:
On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <husince@gmail.com> wrote:
Dear Peter,
I am sorry I do not know the version of my galaxy by using hg. I directly downloaded the zip file from the address: http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
That would be fine for a one-off test installation, but it will make doing an upgrade extremely difficult. That is why the Galaxy recommend using the mercurial (hg) source code system.
The zipped files are compressed on 2011-06-29. Is this version too old?
The changes I am thinking of were earlier this month (Feb 2012) by Greg on galaxy-central and have not yet been released on galaxy-dist. Hopefully he can provide more information next week.
e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d
Regards,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Dear Greg and Peter, Thank you for your reply. I think I had better try the newest version first. I need to ask our administrator to install mercurial in our server. I think only Peter have the binary files now. Let me try to attach them again for you. Best Regards, HU Yingwei 2012/2/25 Greg Von Kuster <greg@bx.psu.edu>
It is highly recommended that you get the latest source from our Galaxy distribution repository using mercurial. As Peter stated, this will enable you to get regular updates as new Galaxy distribution releases occur. Your downloaded zip file contains what we consider a very old release. The Galaxy distribution is available using mercurial at this url.
https://bitbucket.org/galaxy/galaxy-dist
The changes I made to the Galaxy datatypes components were implemented to provide support for proprietary datatypes contained in installed tool shed repositories, so are not related to problems that may occur when uploading.
If I understand the issue correctly, it sounds like there is a problem when you upload your file with contents that are a custom datatype which you've subclassed from the Galaxy binary datatype. If your file is being changed by the Galaxy sniff process, then there may be a bug in that process. What do your paster logs show?
Also, your data attachments have been lost in the mail thread - did only Peter get them?
Greg Von Kuster
On Feb 25, 2012, at 5:20 AM, Peter Cock wrote:
Dear Peter,
I am sorry I do not know the version of my galaxy by using hg. I
On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <husince@gmail.com> wrote: directly
downloaded the zip file from the address: http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
That would be fine for a one-off test installation, but it will make doing an upgrade extremely difficult. That is why the Galaxy recommend using the mercurial (hg) source code system.
The zipped files are compressed on 2011-06-29. Is this version too old?
The changes I am thinking of were earlier this month (Feb 2012) by Greg on galaxy-central and have not yet been released on galaxy-dist. Hopefully he can provide more information next week.
e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d
Regards,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
Greg Von Kuster -
Peter Cock -
Yingwei HU