Hi Galaxy Devs, We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types. Anyone else run into this? Any idea of what's going on? Thanks much, Cris
Hey Cris, Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools? Thanks, Sam On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi Galaxy Devs,
We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types. Anyone else run into this? Any idea of what's going on?
Thanks much, Cris
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Sam, I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........) I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes. What I found so far on our test instance is the following: - the problem exists with all tools on the instance, not just the few that are set to take in several datatypes. - the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up. - the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes. thanks for your help! -Cris On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?
Thanks, Sam
On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi Galaxy Devs,
We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types. Anyone else run into this? Any idea of what's going on?
Thanks much, Cris
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thanks for the quick response. What is the set of datatypes you are specifying? It is possible that one of the specified datatypes can be implicitly converted to html and text e.g. a fasta file will be listed even if only tabular has been specifed as valid datatype. In other words, Galaxy will implicitly convert the fasta file to a tabular file and present it as a valid option in the select field. On Mon, Dec 5, 2016 at 11:48 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi Sam, I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........) I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes. What I found so far on our test instance is the following: - the problem exists with all tools on the instance, not just the few that are set to take in several datatypes. - the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up. - the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes. thanks for your help! -Cris
On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?
Thanks, Sam
On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi Galaxy Devs,
We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types. Anyone else run into this? Any idea of what's going on?
Thanks much, Cris
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
there are 8 datatypes, 7 are tabular / text files, one is binary. The binary format also shows up in the dropdown. I can send you the datatype specification python code if you'd like, Im not sure what you need to see. On Mon, Dec 5, 2016 at 12:01 PM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Thanks for the quick response. What is the set of datatypes you are specifying? It is possible that one of the specified datatypes can be implicitly converted to html and text e.g. a fasta file will be listed even if only tabular has been specifed as valid datatype. In other words, Galaxy will implicitly convert the fasta file to a tabular file and present it as a valid option in the select field.
On Mon, Dec 5, 2016 at 11:48 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi Sam, I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........) I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes. What I found so far on our test instance is the following: - the problem exists with all tools on the instance, not just the few that are set to take in several datatypes. - the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up. - the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes. thanks for your help! -Cris
On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?
Thanks, Sam
On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <cristel.thomas@gmail.com
wrote:
Hi Galaxy Devs,
We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types. Anyone else run into this? Any idea of what's going on?
Thanks much, Cris
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Ok, so this seems to work properly, and implicit conversion is the reason why you see html and text, but not png and pdf files. Currently there is no option to disable implicit conversion but its an enhancement we can suggest to be implemented by creating a github issue.
If it's implicit conversion, why does it pick up the FCS bianry format along with the other text formats? and why does it get triggered on every single tool instead of just the few that have the multiple datatypes? On Mon, Dec 5, 2016 at 12:31 PM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Ok, so this seems to work properly, and implicit conversion is the reason why you see html and text, but not png and pdf files. Currently there is no option to disable implicit conversion but its an enhancement we can suggest to be implemented by creating a github issue.
Ok, let me look a little bit further then. Can you send me the corresponding tool xml file and let me know which datatypes appear in the dropdown list and which of those you would not expect to see? Ideally we can reproduce it with datatypes available to everyone, so I can test it locally.
Hey Cris, Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports: https://github.com/ImmPortDB/immport-galaxy/blob/release_16.07/lib/galaxy/da... Removing them fixes the issue. The error for this is shown in the server log. Thanks, Sam
Hey Sam, thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R). We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted). In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there. Thanks again, Cris On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports:
https://github.com/ImmPortDB/immport-galaxy/blob/release_ 16.07/lib/galaxy/datatypes/flow.py#L13
Removing them fixes the issue. The error for this is shown in the server log.
Thanks, Sam
Hi again, ok, ran some more tests - we switched to a new dev server recently and we were having the issue because of the R/rpy2 thing. With that taken care of though, we still have files in the dropdown that shouldn't be there - no more FCS, but .ypnb still show up. I guess this is because of the implicit conversion? I would love some input on what you guys think we should do about adding the FCS datatype to Galaxy. Thanks again, Cris On Tue, Dec 13, 2016 at 8:52 PM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hey Sam, thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R). We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted).
In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there.
Thanks again, Cris
On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports:
https://github.com/ImmPortDB/immport-galaxy/blob/release_16. 07/lib/galaxy/datatypes/flow.py#L13
Removing them fixes the issue. The error for this is shown in the server log.
Thanks, Sam
Hi Chris, I am glad that we figured this out. So I am not sure what the best strategy for datatypes with code dependencies is, we should open a new thread or git issue and see if someone can help with that. Thanks, Sam On Wed, Dec 14, 2016 at 10:59 AM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hi again, ok, ran some more tests - we switched to a new dev server recently and we were having the issue because of the R/rpy2 thing.
With that taken care of though, we still have files in the dropdown that shouldn't be there - no more FCS, but .ypnb still show up. I guess this is because of the implicit conversion?
I would love some input on what you guys think we should do about adding the FCS datatype to Galaxy. Thanks again, Cris
On Tue, Dec 13, 2016 at 8:52 PM, Cristel Thomas <cristel.thomas@gmail.com> wrote:
Hey Sam, thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R). We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted).
In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there.
Thanks again, Cris
On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <aysam.guerler@gmail.com> wrote:
Hey Cris,
Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports:
https://github.com/ImmPortDB/immport-galaxy/blob/release_16. 07/lib/galaxy/datatypes/flow.py#L13
Removing them fixes the issue. The error for this is shown in the server log.
Thanks, Sam
participants (2)
-
Aysam Guerler -
Cristel Thomas