galaxy-dev June 2015

galaxy-dev@lists.galaxyproject.org

52 participants
83 discussions

Data Manager - BWA-MEM index
by Alexander Vowinkel 11 Jun '15

11 Jun '15

Hi, I'm trying to index some fasta files. They are uploaded in data library and I also loaded them into Data Manager history and my local history. But when I open the Data Manager, I can't choose my file. It says in "Source FASTA Sequence" only "No options available". Can somebody help? Best, Alexander

2 1

job queue problem with cloud man
by Alexander Vowinkel 11 Jun '15

11 Jun '15

Hi list-members, I have a Cloud Man instance on amazon. I did minimal adaption (installing some tools, adding an index) I'm currently running only a master, which is allowed to run jobs. I have now 3 jobs saying: "This job is waiting to run" But in the CloudMan console it says: "Worker status: Idle: 0 Available: 0 Requested: 0" Also condor_q gives 0 jobs. Where is the problem? I already restarted the galaxy service via the admin console... Can someone help me to find whats going wrong? Thanks! Alexander PS: I'm running different job types (one is just "Add column") PPS: "Upload File" jobs are working

2 2

Bismark doesn't end and blocks the server
by C. Ch. 10 Jun '15

10 Jun '15

Hi all, I have a clean local install of galaxy from git (commit: 40b69cb948eed2cece82a32dfc5c6116414ab97c).I've installed bismark and I get an error message (see below) if I do not choose the 'sort' option.I have the impression that this might be related more to galaxy than to bismark itself. The result is that the server gets blocked and the only solution is to restart galaxy. Any help is appreciated!Cristian The error message:galaxy.datatypes.metadata DEBUG 2015-06-05 11:14:12,514 setting metadata externally failed for HistoryDatasetAssociation 1305: External set_meta() not called[bam_index_core] the alignment is not sorted (HWI-ST278R:398:C1WUTACXX:3:1101:9378:2574): 2-th chr > 1-th chr[bam_index_build2] fail to index the BAM file.

2 3

Trackster bug
by C. Ch. 10 Jun '15

10 Jun '15

Hi all, I have uploaded a gff annotation to a shared library and when I try to load it into trackster it tries consistently to load an older dataset that does not exist anymore and does not correspond to the real gff dataset file. If I load the gff into a history, then this file is correctly loaded into trackster. Any ideas what is going on? Any workaround so that we can load visualisations without creating histories with all the data beforehand? Cheers,Cristian

2 6

install packagess without toolshed
by Alexander Vowinkel 09 Jun '15

09 Jun '15

Hi, I would like to create a wrapper for a program. For that I would like to test it, but I just found how to create a tool in galaxy. I couldn't find anything about how to create a package as a dependency. Can someone give me some help here? Best, Alexander

3 4

filtering html tables in a history item
by Rose Mark USRE 09 Jun '15

09 Jun '15

Hello Here is what I would like to do. Any suggestions are appreciated. I have a history item that is an html page with an html table. This table has a checkbox on the end of each table row. I would like it if a user could check which rows he would like to keep and then be able to filter out all unchecked rows. I would like for this filtered table to be a persistent, new history item. Can this be done and how? Thanks Mark ________________________________ This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited.

2 1

errors and questions with run_tests.sh and install_and_test_tool_shed_repositories.sh
by Loraine Guéguen 09 Jun '15

09 Jun '15

Hi, I have investigated the testing possibilities within a Galaxy local instance and within a toolshed. I met some difficulties and I have 4 questions listed below. 1- Tests on tools installed in a local Galaxy instance (with run_tests.sh) : It seems that it is only possible to test the tools that have been manually installed (run_test.sh -id ...), not the tools that have been installed through a toolshed. Is that true ? When I try to launch the tests of a toolshed installed tool, I have the following error message : ######### Failure: ValueError (No such test TestForTool_sartools_edger) ... ERROR ====================================================================== ERROR: Failure: ValueError (No such test TestForTool_sartools_edger) ---------------------------------------------------------------------- Traceback (most recent call last): File "/projet/htdocs/galaxy-dev/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/failure.py", line 39, in runTest raise self.exc_class(self.exc_val) ValueError: No such test TestForTool_sartools_edger ---------------------------------------------------------------------- Ran 1 test in 0.002s FAILED (errors=1) ######### 2- Tests on tools installed in a local Galaxy instance (with run_tests.sh) : Is it possible to test tools which have repository or tool dependencies ? I have a tool (manually installed in galaxy-dist/tools/) with a dependence to a datatype installed from a toolshed. The datatype (no_unzip.zip) is defined in the <param> input tag (in <test>) with attribute "ftype". When running run_tests.sh, it seems that the dataype is not known : ######### ====================================================================== ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_sartools_edger) SARTools edgeR ( sartools_edger ) > Test-2 ---------------------------------------------------------------------- Traceback (most recent call last): File "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line 36, in stage_data_in_history upload_waits.append( galaxy_interactor.stage_data_async( test_data, history, shed_tool_id ) ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line 182, in stage_data_async raise Exception(submit_response) Exception: {u'message': {u'type': u'error', u'data': {u'file_type': u"An invalid option was selected for file_type, u'no_unzip.zip', please verify.", u'files_metadata': [u"An invalid option was selected for file_type, u'no_unzip.zip', please verify."]}}} ######### 3- Toolshed Install and Test framework ( install_and_test_tool_shed_repositories.sh) : For my tool, the test fails if I don't explicitely defined the dbkey attribute to "unknown" (dbkey="?") in the <param> input tag (in <test>). The default dbkey value seems to be "hg17" (whereas hg17 is not defined on the server running my toolshed). So I have the following error message if I don't add dbkey="?" in <test> : ######### ====================================================================== ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209) SARTools edgeR ( galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209 ) > Test-2 ---------------------------------------------------------------------- Traceback (most recent call last): File "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line 36, in stage_data_in_history upload_waits.append( galaxy_interactor.stage_data_async( test_data, history, shed_tool_id ) ) File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line 182, in stage_data_async raise Exception(submit_response) Exception: {u'message': {u'type': u'error', u'data': {u'dbkey': u"An invalid option was selected for dbkey, u'hg17', please verify."}}} ######### 4- Is it possible to launch the tests against an external server ? That would be very useful to be able run the tests on a production Galaxy instance in order to test the tools in real condition, within the whole environment. Thanks in advance for your answers. Loraine -- Loraine Guéguen Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences) Service Informatique et Bioinformatique Station Biologique de Roscoff FR 2424 CNRS UPMC Place Georges Teissier CS 90074 29688 Roscoff cedex tel: ++33 (0)2 98 29 56 46 (interne : 415) http://abims.sb-roscoff.fr/ http://www.sb-roscoff.fr/

3 4

Problem with set_user_disk_usage.py script
by Shane Sturrock 08 Jun '15

08 Jun '15

I recently updated our server to the latest release and have just had a user who ran up a lot of space and isn’t seeing it come back when she purges her data. I’ve tried using the set_user_disk_usage.py script as I’ve previously done to resolve this but now it is failing with the following: (galaxy_env)[galaxy@galaxy scripts]$ ./set_user_disk_usage.py Loading Galaxy model... Traceback (most recent call last): File "./set_user_disk_usage.py", line 85, in <module> model, object_store, engine = init() File "./set_user_disk_usage.py", line 43, in init for key, value in config_parser.items( "app:main" ): File "/usr/lib64/python2.6/ConfigParser.py", line 565, in items raise NoSectionError(section) ConfigParser.NoSectionError: No section: ‘app:main' Do I need a newer python or is this a bug/regression? Shane Dr. Shane Sturrock Senior Scientist shane.sturrock(a)biomatters.com | P: +64 9 379 5064 BIOMATTERS

2 2

Re: [galaxy-dev] galaxy-dev Digest, Vol 108, Issue 8
by ZMarc 08 Jun '15

08 Jun '15

errors and questions with run_tests.sh and install_and_test_tool_shed_repositories.sh > 在 2015年6月8日，17:00，galaxy-dev-request@lists.galaxyproject.org 写道： > > Send galaxy-dev mailing list submissions to > galaxy-dev(a)lists.galaxyproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.galaxyproject.org/listinfo/galaxy-dev > or, via email, send a message with subject or body 'help' to > galaxy-dev-request(a)lists.galaxyproject.org > > You can reach the person managing the list at > galaxy-dev-owner(a)lists.galaxyproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of galaxy-dev digest..." > > > HEY! This is important! If you reply to a thread in a digest, please > 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. > 2. Strip out everything else in the digest that is not part of the thread you are responding to. > > Why? > 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. > 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. > > Today's Topics: > > 1. Re: Multi-level datatype conversions (Keith Suderman) > 2. Re: Bismark doesn't end and blocks the server (Björn Grüning) > 3. Re: errors and questions with run_tests.sh and > install_and_test_tool_shed_repositories.sh (Loraine Guéguen) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 7 Jun 2015 16:37:08 -0400 > From: Keith Suderman <suderman(a)cs.vassar.edu> > To: David Kelly <davidkelly(a)uchicago.edu> > Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> > Subject: Re: [galaxy-dev] Multi-level datatype conversions > Message-ID: <12AB4920-B9A0-4993-B320-B225394C237E(a)cs.vassar.edu> > Content-Type: text/plain; charset="utf-8" > > Hi David, > >> On Jun 5, 2015, at 2:10 PM, David Kelly <davidkelly(a)uchicago.edu> wrote: >> >> Hi Keith, >> >> I am able to manually convert from a->b and then separately b->c, but not directly from a->c. I'm still thinking about how to approach this as well, but I'll share my thoughts so far. > > Does Galaxy automatically do the conversions for you in the workflow editor? For example, if you have a tool A that produces format 'a' and another tool B that consumes format 'b' you should be able to drag the "noodle" from A to B and Galaxy will do an implicit format conversion. If this is working for you then someone from the Galaxy team will have to jump in. If Galaxy is not doing implicit format conversions then it likely means your datatype(s) are not fully defined. I didn't get the implicit format conversion working properly until I implemented the format sniffers. > >> In our use case, we have a collection of climate tools we'd like to use. The tools are written by different groups, so most use their own file formats. Ideally we would like users to be able to upload their data in any of these file formats and use all tools without having to explicitly convert their data. >> >> Let's say there are 10 climate datatypes. My understanding is that in order to achieve this kind of interoperability, I'd have to write 90 converters. >> >> If Galaxy could chain converters together, I could write 1 converter per datatype and chain them together like this: >> a -> b -> c -> d -> e -> f -> g -> h -> i -> j -> a >> >> Another scenario I've been thinking about is picking one data type to be the 'standard'. Let's say it's 'a' in this example. You could write 18 converters: >> a->b, a->c, a->d,... >> b->a, c->a, d->a,... > > This is what we do. We have a standard "pivot format" then converters from the other formats to/from the pivot format. > > Cheers, > Keith > >> >> Then suppose you wanted to plug a 'd' file into a 'c' tool. You could first manually convert d->a, then use the automatic conversion for a->c. Not fully automatic, but not too bad. >> >> On Thu, Jun 4, 2015 at 5:20 PM, Keith Suderman <suderman(a)anc.org> wrote: >> Hi David, >> >> We just got it working for the single conversions, that is, getting Galaxy to automatically convert from a -> b when needed. Are you able to do the conversion, i.e a -> b and b -> c individually? >> >> We are just coming to a similar use case and I'm definitely interested in how you make out, even if I can't be much help. >> >> Cheers, >> Keith >> >>> On Jun 3, 2015, at 4:15 PM, David Kelly <davidkelly(a)uchicago.edu> wrote: >>> >>> Hello, >>> >>> Suppose I have three custom datatypes defined: a, b, and c >>> >>> I have a datatype converter that converts from a -> b >>> I have a datatype converter that converts from b -> c >>> >>> Can I then send 'a' files as input into tools that accept type 'c'? >>> >>> This doesn't seem to work in my test case, but I thought it might be worth asking to see if anyone has tried this before. Thanks! >>> >>> Regards, >>> David >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> >> ------------------------------ >> Research Associate >> Department of Computer Science >> Vassar College >> Poughkeepsie, NY > > ------------------------------ > Research Associate > Department of Computer Science > Vassar College > Poughkeepsie, NY > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20150607/0…> > > ------------------------------ > > Message: 2 > Date: Mon, 08 Jun 2015 11:22:44 +0200 > From: Björn Grüning <bjoern.gruening(a)gmail.com> > To: "C. Ch." <tuto345(a)hotmail.com>, > "galaxy-dev(a)lists.galaxyproject.org" > <galaxy-dev(a)lists.galaxyproject.org> > Subject: Re: [galaxy-dev] Bismark doesn't end and blocks the server > Message-ID: <55755EE4.1080902(a)gmail.com> > Content-Type: text/plain; charset=utf-8 > > Hi, > > I think this is a general problem you encountered here with unsorted BAM > files. We can reproduce it locally. Will let you know if I find a > workaround. > > Cheers, > Bjoer > >> Am 05.06.2015 um 14:23 schrieb C. Ch.: >> Hi all, I have a clean local install of galaxy from git (commit: >> 40b69cb948eed2cece82a32dfc5c6116414ab97c).I've installed bismark and >> I get an error message (see below) if I do not choose the 'sort' >> option.I have the impression that this might be related more to >> galaxy than to bismark itself. The result is that the server gets >> blocked and the only solution is to restart galaxy. Any help is >> appreciated!Cristian The error message:galaxy.datatypes.metadata >> DEBUG 2015-06-05 11:14:12,514 setting metadata externally failed for >> HistoryDatasetAssociation 1305: External set_meta() not >> called[bam_index_core] the alignment is not sorted >> (HWI-ST278R:398:C1WUTACXX:3:1101:9378:2574): 2-th chr > 1-th >> chr[bam_index_build2] fail to index the BAM file. >> >> >> >> ___________________________________________________________ Please >> keep all replies on the list by using "reply all" in your mail >> client. To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > > ------------------------------ > > Message: 3 > Date: Mon, 08 Jun 2015 12:30:29 +0200 > From: Loraine Guéguen <loraine.gueguen(a)sb-roscoff.fr> > To: John Chilton <jmchilton(a)gmail.com> > Cc: Galaxy Dev List <galaxy-dev(a)lists.galaxyproject.org> > Subject: Re: [galaxy-dev] errors and questions with run_tests.sh and > install_and_test_tool_shed_repositories.sh > Message-ID: <55756EC5.70303(a)sb-roscoff.fr> > Content-Type: text/plain; charset=utf-8; format=flowed > > Thanks for your answers ! > > I will try "./run_tests.sh -installed" and see if it can solve the 2 > first points. By the way, I didn't know this option (not displayed with > run_tests.sh --help) or documented on the wiki if I'm not mistaken. > > For point 3, I'll try the new "planemo shed_test" subcommand. The > subcommand "planemo test" uses the run_tests.sh script, isn't it ? Does > the "planemo shed_test" subcommand use all or any part of the Tool Shed > Install and Test Framework > (https://wiki.galaxyproject.org/AutomatedToolTests) or is it completely > different ? > > Is there any way to launch the tool tests against an external server > (point 4) with Planemo ? > > Loraine > > > Le 02/06/2015 15:16, John Chilton a écrit : >> On Tue, Jun 2, 2015 at 4:14 AM, Loraine Guéguen >> <loraine.gueguen(a)sb-roscoff.fr> wrote: >>> Hi, >>> >>> I have investigated the testing possibilities within a Galaxy local instance >>> and within a toolshed. I met some difficulties and I have 4 questions listed >>> below. >>> >>> >>> 1- Tests on tools installed in a local Galaxy instance (with run_tests.sh) : >>> It seems that it is only possible to test the tools that have been manually >>> installed (run_test.sh -id ...), not the tools that have been installed >>> through a toolshed. Is that true ? When I try to launch the tests of a >>> toolshed installed tool, I have the following error message : >>> >>> ######### >>> >>> Failure: ValueError (No such test TestForTool_sartools_edger) ... ERROR >>> ====================================================================== >>> ERROR: Failure: ValueError (No such test TestForTool_sartools_edger) >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "/projet/htdocs/galaxy-dev/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/failure.py", >>> line 39, in runTest >>> raise self.exc_class(self.exc_val) >>> ValueError: No such test TestForTool_sartools_edger >>> >>> ---------------------------------------------------------------------- >>> Ran 1 test in 0.002s >>> FAILED (errors=1) >> ./run_tests.sh -installed runs the tests of all installed tool shed >> tools. It might be possible to combine that with -id - for a specific >> tool but I am not sure. It may be possible to construct that test even >> if -id doesn't work - but it is probably pretty hacky. Let me know if >> -installed works for you but not when combined with -id and we can >> work out the details. >>> ######### >>> >>> >>> 2- Tests on tools installed in a local Galaxy instance (with run_tests.sh) : >>> Is it possible to test tools which have repository or tool dependencies ? I >>> have a tool (manually installed in galaxy-dist/tools/) with a dependence to >>> a datatype installed from a toolshed. The datatype (no_unzip.zip) is defined >>> in the <param> input tag (in <test>) with attribute "ftype". When running >>> run_tests.sh, it seems that the dataype is not known : >>> >>> ######### >>> >>> ====================================================================== >>> ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_sartools_edger) >>> SARTools edgeR ( sartools_edger ) > Test-2 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line >>> 268, in test_tool >>> self.do_it( td ) >>> File >>> "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line >>> 37, in do_it >>> stage_data_in_history( galaxy_interactor, testdef.test_data(), >>> test_history, shed_tool_id ) >>> File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line >>> 36, in stage_data_in_history >>> upload_waits.append( galaxy_interactor.stage_data_async( test_data, >>> history, shed_tool_id ) ) >>> File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line >>> 182, in stage_data_async >>> raise Exception(submit_response) >>> Exception: {u'message': {u'type': u'error', u'data': {u'file_type': u"An >>> invalid option was selected for file_type, u'no_unzip.zip', please verify.", >>> u'files_metadata': [u"An invalid option was selected for file_type, >>> u'no_unzip.zip', please verify."]}}} >>> >>> ######### >> It is possible it is the case that tool shed datatypes are not loaded >> unless running with the -installed flag. I would either setup the >> tests to run locally (manually add datatypes and tools) or run >> installed tools with -installed. >> >>> >>> 3- Toolshed Install and Test framework ( >>> install_and_test_tool_shed_repositories.sh) : For my tool, the test fails if >>> I don't explicitely defined the dbkey attribute to "unknown" (dbkey="?") in >>> the <param> input tag (in <test>). The default dbkey value seems to be >>> "hg17" (whereas hg17 is not defined on the server running my toolshed). So I >>> have the following error message if I don't add dbkey="?" in <test> : >>> >>> ######### >>> >>> ====================================================================== >>> ERROR: test_tool_000001 >>> (functional.test_toolbox.TestForTool_galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209) >>> SARTools edgeR ( >>> galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209 ) >>>> Test-2 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line >>> 268, in test_tool >>> self.do_it( td ) >>> File >>> "/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py", line >>> 37, in do_it >>> stage_data_in_history( galaxy_interactor, testdef.test_data(), >>> test_history, shed_tool_id ) >>> File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line >>> 36, in stage_data_in_history >>> upload_waits.append( galaxy_interactor.stage_data_async( test_data, >>> history, shed_tool_id ) ) >>> File "/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line >>> 182, in stage_data_async >>> raise Exception(submit_response) >>> Exception: {u'message': {u'type': u'error', u'data': {u'dbkey': u"An invalid >>> option was selected for dbkey, u'hg17', please verify."}}} >>> >>> ######### >> Well that is frustrating. I am not aware of the internals of that >> script or why that is happening - but my personal recommendation would >> be not to use the install and test framework at this point - it seems >> rather un-maintained. I recently added a subcommnd to planemo called >> shed_test (http://planemo.readthedocs.org/en/master/commands.html#shed-test-command) >> - that will install and test a tool shed repository. It probably >> requires modifying your tool shed development process to use planemo >> (http://planemo.readthedocs.org/en/latest/publishing.html#shed). >> >>> >>> 4- Is it possible to launch the tests against an external server ? That >>> would be very useful to be able run the tests on a production Galaxy >>> instance in order to test the tools in real condition, within the whole >>> environment. >> This would be great - but as I understand it - it is not possible at >> this time. The test framework needs local access to some aspects of >> the Galaxy app. Last year I outlined a Trello card to add this option >> back to Galaxy (https://trello.com/c/8K86l4Qk) - it was available >> years ago. >> >>> >>> >>> Thanks in advance for your answers. >>> >>> Loraine >>> >>> >>> >>> >>> -- >>> Loraine Guéguen >>> >>> Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences) >>> Service Informatique et Bioinformatique >>> Station Biologique de Roscoff >>> FR 2424 CNRS UPMC >>> Place Georges Teissier >>> CS 90074 >>> 29688 Roscoff cedex >>> tel: ++33 (0)2 98 29 56 46 (interne : 415) >>> http://abims.sb-roscoff.fr/ >>> http://www.sb-roscoff.fr/ >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ > > -- > Loraine Guéguen > > Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences) > Service Informatique et Bioinformatique > Station Biologique de Roscoff > FR 2424 CNRS UPMC > Place Georges Teissier > 29680 Roscoff > tel: ++33 (0)2 98 29 25 50 > http://abims.sb-roscoff.fr/ > http://www.sb-roscoff.fr/ > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > galaxy-dev mailing list > galaxy-dev(a)lists.galaxyproject.org > https://lists.galaxyproject.org/listinfo/galaxy-dev > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > ------------------------------ > > End of galaxy-dev Digest, Vol 108, Issue 8 > ******************************************

1 0

Re: [galaxy-dev] Multi-level datatype conversions
by Keith Suderman 07 Jun '15

07 Jun '15

Hi David, On Jun 5, 2015, at 2:10 PM, David Kelly <davidkelly(a)uchicago.edu> wrote: > Hi Keith, > > I am able to manually convert from a->b and then separately b->c, but not directly from a->c. I'm still thinking about how to approach this as well, but I'll share my thoughts so far. Does Galaxy automatically do the conversions for you in the workflow editor? For example, if you have a tool A that produces format 'a' and another tool B that consumes format 'b' you should be able to drag the "noodle" from A to B and Galaxy will do an implicit format conversion. If this is working for you then someone from the Galaxy team will have to jump in. If Galaxy is not doing implicit format conversions then it likely means your datatype(s) are not fully defined. I didn't get the implicit format conversion working properly until I implemented the format sniffers. > In our use case, we have a collection of climate tools we'd like to use. The tools are written by different groups, so most use their own file formats. Ideally we would like users to be able to upload their data in any of these file formats and use all tools without having to explicitly convert their data. > > Let's say there are 10 climate datatypes. My understanding is that in order to achieve this kind of interoperability, I'd have to write 90 converters. > > If Galaxy could chain converters together, I could write 1 converter per datatype and chain them together like this: > a -> b -> c -> d -> e -> f -> g -> h -> i -> j -> a > > Another scenario I've been thinking about is picking one data type to be the 'standard'. Let's say it's 'a' in this example. You could write 18 converters: > a->b, a->c, a->d,... > b->a, c->a, d->a,... This is what we do. We have a standard "pivot format" then converters from the other formats to/from the pivot format. Cheers, Keith > > Then suppose you wanted to plug a 'd' file into a 'c' tool. You could first manually convert d->a, then use the automatic conversion for a->c. Not fully automatic, but not too bad. > > On Thu, Jun 4, 2015 at 5:20 PM, Keith Suderman <suderman(a)anc.org> wrote: > Hi David, > > We just got it working for the single conversions, that is, getting Galaxy to automatically convert from a -> b when needed. Are you able to do the conversion, i.e a -> b and b -> c individually? > > We are just coming to a similar use case and I'm definitely interested in how you make out, even if I can't be much help. > > Cheers, > Keith > > On Jun 3, 2015, at 4:15 PM, David Kelly <davidkelly(a)uchicago.edu> wrote: > > > Hello, > > > > Suppose I have three custom datatypes defined: a, b, and c > > > > I have a datatype converter that converts from a -> b > > I have a datatype converter that converts from b -> c > > > > Can I then send 'a' files as input into tools that accept type 'c'? > > > > This doesn't seem to work in my test case, but I thought it might be worth asking to see if anyone has tried this before. Thanks! > > > > Regards, > > David > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > ------------------------------ > Research Associate > Department of Computer Science > Vassar College > Poughkeepsie, NY > > > ------------------------------ Research Associate Department of Computer Science Vassar College Poughkeepsie, NY

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev June 2015