upload file format is data - type not recognized in galaxy
Hello, do you have any idea why my galaxy 16.07 stop recognizing uploaded files and format of uploaded files is set to the format 'data'? After using older version of database backup, galaxy stop recognizing format of uploaded files (e.g. fasta, fastqc) and everything is set to format 'data'. Uploaded files are OK, I have tested them on other public galaxy servers, where they were recognized well. Before changing database (after the technical problems with the server), recognizing of the files were OK. Even in debugging level I do not see any errors or problems in the log (I have compared logs before changing database with new ones, and there are no differences). From the investigating of upload.py, I guess something happen with galaxy.datatypes.registry (settings and code of galaxy were not changed). Do you have any ideas? I cannot switch the database back and changing file format in Edit attributes is annoying. Best, Pavel
Hi Pavel what happens if you connect your galaxy instance to an empty PostgreSQL database, let galaxy build all the tables and then try to upload files. Regards, Hans-Rudolf On 09/22/2017 10:51 AM, Pavel Fibich wrote:
Hello,
do you have any idea why my galaxy 16.07 stop recognizing uploaded files and format of uploaded files is set to the format 'data'?
After using older version of database backup, galaxy stop recognizing format of uploaded files (e.g. fasta, fastqc) and everything is set to format 'data'. Uploaded files are OK, I have tested them on other public galaxy servers, where they were recognized well. Before changing database (after the technical problems with the server), recognizing of the files were OK. Even in debugging level I do not see any errors or problems in the log (I have compared logs before changing database with new ones, and there are no differences). From the investigating of upload.py, I guess something happen with galaxy.datatypes.registry (settings and code of galaxy were not changed). Do you have any ideas? I cannot switch the database back and changing file format in Edit attributes is annoying.
Best, Pavel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
My first thought would be to double check the Galaxy startup logs (which normally tell you about the datatypes as they are setup). That ought to give some clues. Also check the XML files where the datatypes are defined are all still fine - do you have any of these and do they look fine? datatypes_conf.xml config/datatypes_conf.xml config/datatypes_conf.xml.sample (I know you said the settings and code were not changed, but it seems worth double checking) Peter On Fri, Sep 22, 2017 at 12:31 PM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
Hi Pavel
what happens if you connect your galaxy instance to an empty PostgreSQL database, let galaxy build all the tables and then try to upload files.
Regards, Hans-Rudolf
On 09/22/2017 10:51 AM, Pavel Fibich wrote:
Hello,
do you have any idea why my galaxy 16.07 stop recognizing uploaded files and format of uploaded files is set to the format 'data'?
After using older version of database backup, galaxy stop recognizing format of uploaded files (e.g. fasta, fastqc) and everything is set to format 'data'. Uploaded files are OK, I have tested them on other public galaxy servers, where they were recognized well. Before changing database (after the technical problems with the server), recognizing of the files were OK. Even in debugging level I do not see any errors or problems in the log (I have compared logs before changing database with new ones, and there are no differences). From the investigating of upload.py, I guess something happen with galaxy.datatypes.registry (settings and code of galaxy were not changed). Do you have any ideas? I cannot switch the database back and changing file format in Edit attributes is annoying.
Best, Pavel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Hi Hans-Rudolf and Peter, thank you for replies. I tried both suggestions, but no luck. I made empty database, let galaxy run from it. Than, uploading fasta file again finished in the format 'data', not recognized again. Also, I did not find anything interesting in the startup logs ( logs are here: https://repeatexplorer-elixir.cerit-sc.cz/galstart2.log ), there are just fine information about datatypes, eg. for fasta: galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,810 Retrieved datatype module galaxy.datatypes.sequence:Fasta from the datatype registry. ... galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,929 Loaded sniffer for datatype 'galaxy.datatypes.sequence:Fasta' I have only config/datatypes_conf.xml.sample, so I expect that it is default. If you have any other ideas, I will be glad for them. Best, Pavel On Fri, Sep 22, 2017 at 2:25 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
My first thought would be to double check the Galaxy startup logs (which normally tell you about the datatypes as they are setup). That ought to give some clues.
Also check the XML files where the datatypes are defined are all still fine - do you have any of these and do they look fine?
datatypes_conf.xml config/datatypes_conf.xml config/datatypes_conf.xml.sample
(I know you said the settings and code were not changed, but it seems worth double checking)
Peter
On Fri, Sep 22, 2017 at 12:31 PM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
Hi Pavel
what happens if you connect your galaxy instance to an empty PostgreSQL database, let galaxy build all the tables and then try to upload files.
Regards, Hans-Rudolf
On 09/22/2017 10:51 AM, Pavel Fibich wrote:
Hello,
do you have any idea why my galaxy 16.07 stop recognizing uploaded files and format of uploaded files is set to the format 'data'?
After using older version of database backup, galaxy stop recognizing format of uploaded files (e.g. fasta, fastqc) and everything is set to format 'data'. Uploaded files are OK, I have tested them on other public galaxy servers, where they were recognized well. Before changing database (after the technical problems with the server), recognizing of the files were OK. Even in debugging level I do not see any errors or problems in the log (I have compared logs before changing database with new ones, and there are no differences). From the investigating of upload.py, I guess something happen with galaxy.datatypes.registry (settings and code of galaxy were not changed). Do you have any ideas? I cannot switch the database back and changing file format in Edit attributes is annoying.
Best, Pavel
That's a shame - I don't see anything in the log either at first reading - it was hoping to see problems with missing datatypes or sniffers, which might explain this. Sorry, I am out of ideas. Peter On Fri, Sep 22, 2017 at 2:41 PM, Pavel Fibich <pavel.fibich@prf.jcu.cz> wrote:
Hi Hans-Rudolf and Peter,
thank you for replies. I tried both suggestions, but no luck.
I made empty database, let galaxy run from it. Than, uploading fasta file again finished in the format 'data', not recognized again.
Also, I did not find anything interesting in the startup logs ( logs are here: https://repeatexplorer-elixir.cerit-sc.cz/galstart2.log ), there are just fine information about datatypes, eg. for fasta:
galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,810 Retrieved datatype module galaxy.datatypes.sequence:Fasta from the datatype registry. ... galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,929 Loaded sniffer for datatype 'galaxy.datatypes.sequence:Fasta'
I have only config/datatypes_conf.xml.sample, so I expect that it is default.
If you have any other ideas, I will be glad for them.
Best, Pavel
I got the solution by debugging upload.py. Recently, I tried to force galaxy to run jobs in SCRATCHDIR (directory in the local disks of computer), so I put cd $SCRATCHDIR at the end of the environmental settings file. Generated tool script contains line like python .../upload.py .../registry.xml ./files.. but before that environmental settings are applied (including cd $SCRATCHDIR). upload.py is using absolute paths except json_file = open( 'galaxy.json', 'w' ) that breaks further recognition of the uploaded file (I guess due to calling dataset = loads( line ), where loads is from json library). Removing cd command from the settings solved the problem. Thank you for all suggestions. Best regards, Pavel
participants (3)
-
Hans-Rudolf Hotz -
Pavel Fibich -
Peter Cock