Re: [galaxy-dev] Fwd: Re: Help about zip upload (esd files)
Dear Nate, We're trying to implement different tools to handle sequences generated by Megabace/AB1 sequencers. The very first step in this process is to load the chromatograms provided by the machines. The thing is these machines produce an individual file per sequence and the phred software (base caller) receive as parameters a compressed file or a directory path. Since galaxy 'get data' feature loads only individual files and we're not able to implement phred to do such things, what options are left? Thank you so much, Adhemar On 07/29/2010 01:57 PM, Eric Aguiar wrote:
-------- Original Message -------- Subject: Re: [galaxy-dev] Help about zip upload (esd files) Date: Wed, 28 Jul 2010 12:09:06 -0400 From: To: Eric Aguiar <eric@cebio.org> CC: List
Eric Aguiar wrote:
Dear,
I followed all steps in the galaxy tutorial, but I didn't have success. I'm trying to create a datatype for megabase chromatograms (.esd) very similar to the Ab1 ones.
Here is my configurations.
*1 - Creating datatypes in datatypes_conf.xml*
<datatype extension="zip" type="galaxy.datatypes.binary:Esd" mimetype="application/zip" display_in_upload="true"/> <datatype extension="esd" type="galaxy.datatypes.binary:Esd" mimetype="application/octet-stream" display_in_upload="true"/>
*2 - Defining types in lib/galaxy/datatypes/binary.py*
class Esd( Binary ): """Class describing an ab1 binary sequence file""" file_ext = "esd"
def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = "Binary chromatograms sequence file" dataset.blurb = data.nice_size( dataset.get_size() ) else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' def display_peek( self, dataset ): try: return dataset.peek except: return "Binary esd sequence file (%s)" % ( data.nice_size( dataset.get_size() ) )
class Zip( Binary ): """Class describing a zip archive of binary sequence files""" file_ext = "zip"
def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: zip_file = zipfile.ZipFile( dataset.file_name, "r" ) num_files = len( zip_file.namelist() ) dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) dataset.blurb = data.nice_size( dataset.get_size() ) else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' def display_peek( self, dataset ): try: return dataset.peek except: return "Binary sequence file archive (%s)" % ( data.nice_size( dataset.get_size() ) ) def get_mime( self ): """Returns the mime type of the datatype""" return 'application/zip'
When I'm going to send the file in zip format (.esd files compressed),it shows me the following error: *"An error occurred running this job: Invalid 'File Format' for archive consisting of binary files - use 'Binseq.zip'"*
Hi Eric,
Have you updated to a recent revision of Galaxy? Binseq.zip and the code that prevents uploading zip files has been removed. Currently, though, a zip file can only contain one member.
--nate
I tried somethings, but I don't have success.
Thank you, --
------------------------------------------------------------------------
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
-- Adhemar Zerlotini Neto, PhD FIOCRUZ - Fundação Oswaldo Cruz CPqRR - Centro de Pesquisas René Rachou http://www.cpqrr.fiocruz.br http://www.cebio.org +55 31 3337 3863 +55 31 3337 3649
Adhemar wrote:
Dear Nate, We're trying to implement different tools to handle sequences generated by Megabace/AB1 sequencers. The very first step in this process is to load the chromatograms provided by the machines. The thing is these machines produce an individual file per sequence and the phred software (base caller) receive as parameters a compressed file or a directory path. Since galaxy 'get data' feature loads only individual files and we're not able to implement phred to do such things, what options are left? Thank you so much, Adhemar
Hi Adhemar, It sounds like you may be able to make good use of Galaxy Data Libraries in this case: http://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/Libraries Specifically, adding datasets to a library from a directory, which can be configured in universe_wsgi.ini. But to clarify, you need for these files to remain compressed once uploaded to Galaxy? This is not currently possible, although there is an issue created for it: http://bitbucket.org/galaxy/galaxy-central/issue/280/allow-zip-file-uploads There are a number of people out there in the community who've hacked in support for their specific stay-zipped formats, as well, and maybe they'd be willing to share their code. Thanks, --nate
On 07/29/2010 01:57 PM, Eric Aguiar wrote:
-------- Original Message -------- Subject: Re: [galaxy-dev] Help about zip upload (esd files) Date: Wed, 28 Jul 2010 12:09:06 -0400 From: To: Eric Aguiar <eric@cebio.org> CC: List
Eric Aguiar wrote:
Dear,
I followed all steps in the galaxy tutorial, but I didn't have success. I'm trying to create a datatype for megabase chromatograms (.esd) very similar to the Ab1 ones.
Here is my configurations.
*1 - Creating datatypes in datatypes_conf.xml*
<datatype extension="zip" type="galaxy.datatypes.binary:Esd" mimetype="application/zip" display_in_upload="true"/> <datatype extension="esd" type="galaxy.datatypes.binary:Esd" mimetype="application/octet-stream" display_in_upload="true"/>
*2 - Defining types in lib/galaxy/datatypes/binary.py*
class Esd( Binary ): """Class describing an ab1 binary sequence file""" file_ext = "esd"
def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: dataset.peek = "Binary chromatograms sequence file" dataset.blurb = data.nice_size( dataset.get_size() ) else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' def display_peek( self, dataset ): try: return dataset.peek except: return "Binary esd sequence file (%s)" % ( data.nice_size( dataset.get_size() ) )
class Zip( Binary ): """Class describing a zip archive of binary sequence files""" file_ext = "zip"
def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: zip_file = zipfile.ZipFile( dataset.file_name, "r" ) num_files = len( zip_file.namelist() ) dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) dataset.blurb = data.nice_size( dataset.get_size() ) else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' def display_peek( self, dataset ): try: return dataset.peek except: return "Binary sequence file archive (%s)" % ( data.nice_size( dataset.get_size() ) ) def get_mime( self ): """Returns the mime type of the datatype""" return 'application/zip'
When I'm going to send the file in zip format (.esd files compressed),it shows me the following error: *"An error occurred running this job: Invalid 'File Format' for archive consisting of binary files - use 'Binseq.zip'"*
Hi Eric,
Have you updated to a recent revision of Galaxy? Binseq.zip and the code that prevents uploading zip files has been removed. Currently, though, a zip file can only contain one member.
--nate
I tried somethings, but I don't have success.
Thank you, --
------------------------------------------------------------------------
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
-- Adhemar Zerlotini Neto, PhD FIOCRUZ - Fundação Oswaldo Cruz CPqRR - Centro de Pesquisas René Rachou http://www.cpqrr.fiocruz.br http://www.cebio.org +55 31 3337 3863 +55 31 3337 3649
participants (2)
-
Adhemar -
Nate Coraor