How to make Galaxy link to a local installed UCSC database?
by xue dong
Hallo every one,
I am new to Galaxy, I have a idea which maybe not new to people here. I work
on arabidopsis. maybe you know that UCSC do not offer the genome assembly
for plant.
so i will install UCSC locally and built the genome assembly for
arabidopsis. this work is half done now.
I also installed Galaxy, if i understand correctly, Galaxy has the link to
UCSC so the user can get data from UCSC. since what i interested is
arabidopsis information, I hope Galaxy can retrive data form the locally
installed UCSC in future. I do not know if it is possible to make this idea
come ture.maybe some of you have done it. if yes, I am gratefully if you
can tell me the how to get it.
many thanks,
Xue
11 years
Round-Robin Scheduling
by Assaf Gordon
Hello,
I have another question regarding the local job scheduler:
Is it possible to limit the number of jobs *per user* ?
That is - any given user can have up to X number of jobs running
concurrently, regardless of the value of local_job_queue_workers ?
Imagine the following situation:
local_job_queue_worker = 5
job_scheduler_policy =
galaxy.jobs.schedulingpolicy.roundrobin:UserRoundRobin
Which means that at any given moment, galaxy can run only five jobs.
Now, Galaxy is completely Idle, no jobs are running.
One users starts 7 very long running jobs (each jobs will take about two
hours).
If I understand correctly - since no jobs are running, 5 of the user's
job will be started immediately, even with the round-robin policy, right ?
And this means that for the next two hours, every other user which
starts a job - his/her job will be either new or limbo-running, but none
will actually be started, right ?
I think I'm experiencing this situation on my galaxy server.
Users are complaining their jobs have been running 'forever' or not even
starting for a long long time.
Close examination shows that there are running 5 jobs (all from the same
user) which have been running for three hours, and they are kind of
hogging all the worker threads.
To make a long story short -
I would like to make sure a single user can't hog Galaxy.
Is it possible with the local job runner, and if not - is it possible
with the SGE job runner ?
Thanks for reading so far,
Gordon.
11 years
[hg] galaxy 2390: Reintroduce the ability to copy a history item...
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/ee9a29c9efc1
changeset: 2390:ee9a29c9efc1
user: Dan Blankenberg <dan(a)bx.psu.edu>
date: Thu Apr 30 16:12:59 2009 -0400
description:
Reintroduce the ability to copy a history item between histories (and within the same).
3 file(s) affected in this change:
lib/galaxy/web/controllers/dataset.py
templates/dataset/copy_view.mako
templates/dataset/edit_attributes.mako
diffs (53 lines):
diff -r 54be726b7711 -r ee9a29c9efc1 lib/galaxy/web/controllers/dataset.py
--- a/lib/galaxy/web/controllers/dataset.py Thu Apr 30 15:37:37 2009 -0400
+++ b/lib/galaxy/web/controllers/dataset.py Thu Apr 30 16:12:59 2009 -0400
@@ -157,6 +157,7 @@
user = trans.get_user()
history = trans.get_history()
create_new_history = False
+ refresh_frames = []
if source_dataset_ids:
if not isinstance( source_dataset_ids, list ):
source_dataset_ids = source_dataset_ids.split( "," )
@@ -203,6 +204,8 @@
else:
for hist in target_histories:
hist.add_dataset( data.copy( copy_children = True ) )
+ if history in target_histories:
+ refresh_frames = ['history']
trans.app.model.flush()
done_msg = "%i datasets copied to %i histories." % ( len( source_dataset_ids ) - invalid_datasets, len( target_histories ) )
history.refresh()
@@ -220,4 +223,5 @@
target_histories = target_histories,
new_history_name = new_history_name,
done_msg = done_msg,
- error_msg = error_msg )
+ error_msg = error_msg,
+ refresh_frames = refresh_frames )
diff -r 54be726b7711 -r ee9a29c9efc1 templates/dataset/copy_view.mako
--- a/templates/dataset/copy_view.mako Thu Apr 30 15:37:37 2009 -0400
+++ b/templates/dataset/copy_view.mako Thu Apr 30 16:12:59 2009 -0400
@@ -1,5 +1,8 @@
<%inherit file="/base.mako"/>
+<%namespace file="/message.mako" import="javascripts" />
<%def name="title()">Copy History Items</%def>
+
+${javascripts()}
%if error_msg:
<p>
diff -r 54be726b7711 -r ee9a29c9efc1 templates/dataset/edit_attributes.mako
--- a/templates/dataset/edit_attributes.mako Thu Apr 30 15:37:37 2009 -0400
+++ b/templates/dataset/edit_attributes.mako Thu Apr 30 16:12:59 2009 -0400
@@ -155,3 +155,10 @@
</div>
</div>
%endif
+<p/>
+<div class="toolForm">
+<div class="toolFormTitle">Copy History Item</div>
+<div class="toolFormBody">
+ Click <a href="${h.url_for( controller='dataset', action='copy_datasets', source_dataset_ids=data.id, target_history_ids=data.history_id )}" target="galaxy_main">here</a> to make a copy of this history item.
+</div>
+</div>
11 years, 2 months
[hg] galaxy 2389: Fix for HID incrementing by two when importing...
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/54be726b7711
changeset: 2389:54be726b7711
user: Dan Blankenberg <dan(a)bx.psu.edu>
date: Thu Apr 30 15:37:37 2009 -0400
description:
Fix for HID incrementing by two when importing from a library.
2 file(s) affected in this change:
lib/galaxy/model/__init__.py
lib/galaxy/web/controllers/library.py
diffs (44 lines):
diff -r ff2fca65799e -r 54be726b7711 lib/galaxy/model/__init__.py
--- a/lib/galaxy/model/__init__.py Thu Apr 30 11:07:34 2009 -0400
+++ b/lib/galaxy/model/__init__.py Thu Apr 30 15:37:37 2009 -0400
@@ -777,8 +777,7 @@
self.copied_from_library_dataset_dataset_association = copied_from_library_dataset_dataset_association
self.library_dataset = library_dataset
self.user = user
- def to_history_dataset_association( self, target_history, parent_id=None ):
- hid = target_history._next_hid()
+ def to_history_dataset_association( self, target_history, parent_id = None, add_to_history = False ):
hda = HistoryDatasetAssociation( name=self.name,
info=self.info,
blurb=self.blurb,
@@ -790,12 +789,13 @@
deleted=self.deleted,
parent_id=parent_id,
copied_from_library_dataset_dataset_association=self,
- history=target_history,
- hid=hid )
+ history=target_history )
hda.flush()
hda.metadata = self.metadata #need to set after flushed, as MetadataFiles require dataset.id
+ if add_to_history and target_history:
+ target_history.add_dataset( hda )
for child in self.children:
- child_copy = child.to_history_dataset_association( target_history=target_history, parent_id=hda.id )
+ child_copy = child.to_history_dataset_association( target_history = target_history, parent_id = hda.id, add_to_history = False )
if not self.datatype.copy_safe_peek:
hda.set_peek() #in some instances peek relies on dataset_id, i.e. gmaj.zip for viewing MAFs
hda.flush()
diff -r ff2fca65799e -r 54be726b7711 lib/galaxy/web/controllers/library.py
--- a/lib/galaxy/web/controllers/library.py Thu Apr 30 11:07:34 2009 -0400
+++ b/lib/galaxy/web/controllers/library.py Thu Apr 30 15:37:37 2009 -0400
@@ -175,9 +175,7 @@
history = trans.get_history()
for ldda_id in ldda_ids:
ldda = trans.app.model.LibraryDatasetDatasetAssociation.get( ldda_id )
- hda = ldda.to_history_dataset_association( target_history=history )
- history.add_dataset( hda )
- hda.flush()
+ hda = ldda.to_history_dataset_association( target_history=history, add_to_history = True )
history.flush()
msg = "%i dataset(s) have been imported into your history" % len( ldda_ids )
return trans.response.send_redirect( web.url_for( controller='library',
11 years, 2 months
[hg] galaxy 2387: Added test datasets for qualityscore unit tests.
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/609ee7376d0d
changeset: 2387:609ee7376d0d
user: guru
date: Thu Apr 30 10:54:43 2009 -0400
description:
Added test datasets for qualityscore unit tests.
2 file(s) affected in this change:
lib/galaxy/datatypes/test/sequence.qual454
lib/galaxy/datatypes/test/sequence.qualsolid
diffs (38 lines):
diff -r 799a333342c4 -r 609ee7376d0d lib/galaxy/datatypes/test/sequence.qual454
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/lib/galaxy/datatypes/test/sequence.qual454 Thu Apr 30 10:54:43 2009 -0400
@@ -0,0 +1,16 @@
+>EYKX4VC04IWAEA length=68 xy=3531_0528 region=4 run=R_2007_11_07_16_15_57_
+22 13 9 6 4 3 2 2 1 1 1 1 24 44 33 23 16 11 7 2 28 33 23 18 28 27 27 28 20 21 42 35 21 6 24 25 31 21 28 27 41 34 15 28 28 27 28 28 33 24 27 28 28 24 27 36 27 28 28 28
+28 28 36 30 8 34 25 18
+>EYKX4VC04JKOGH length=48 xy=3808_3903 region=4 run=R_2007_11_07_16_15_57_
+28 28 27 28 38 31 10 28 28 27 27 34 25 28 24 26 27 28 27 37 29 34 25 31 21 28 21 36 28 31 20 24 27 37 28 28 34 27 3 34 25 24 28 28 26 28 35 28
+>EYKX4VC04JIUVK length=84 xy=3788_0830 region=4 run=R_2007_11_07_16_15_57_
+29 20 14 11 8 6 3 1 25 27 25 28 28 27 28 27 28 28 36 28 27 28 36 29 7 28 28 28 27 27 27 35 26 35 26 27 36 28 28 28 38 32 11 28 36 28 27 26 35 25 28 38 31 11 27 28 37 28 27 27
+28 36 29 8 33 24 41 34 19 3 26 28 28 28 35 26 36 29 8 38 32 11 28 28
+>EYKX4VC04JWDRY length=78 xy=3942_1068 region=4 run=R_2007_11_07_16_15_57_
+36 24 14 5 27 20 28 27 28 32 22 28 27 43 36 23 11 27 28 28 28 32 23 36 27 28 28 26 38 32 11 34 25 27 43 36 23 11 38 31 11 37 28 28 28 27 28 30 20 28 32 22 28 36 27 37 30 9 27 28
+28 27 28 42 35 20 5 28 28 28 35 26 27 27 26 39 32 12
+>EYKX4VC04JWMUW length=55 xy=3945_0550 region=4 run=R_2007_11_07_16_15_57_
+36 24 14 4 28 17 34 25 35 25 31 20 28 28 36 27 28 28 24 27 28 28 37 28 27 27 35 25 31 21 27 39 32 12 28 36 28 28 26 27 28 27 26 28 42 35 20 6 28 27 28 28 28 28 28
+>EYKX4VC04JH4RG length=85 xy=3779_3850 region=4 run=R_2007_11_07_16_15_57_
+37 28 35 26 38 31 10 27 37 28 28 38 31 10 27 35 25 25 28 28 28 28 28 28 28 28 27 28 33 23 28 32 22 35 25 31 20 34 25 31 21 26 28 27 26 26 15 36 29 7 27 27 24 36 27 28 37 28 36 28
+27 28 28 28 37 28 28 40 34 14 37 28 28 26 28 36 28 26 28 37 28 28 28 28 27
\ No newline at end of file
diff -r 799a333342c4 -r 609ee7376d0d lib/galaxy/datatypes/test/sequence.qualsolid
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/lib/galaxy/datatypes/test/sequence.qualsolid Thu Apr 30 10:54:43 2009 -0400
@@ -0,0 +1,12 @@
+>946_21_302_F3
+2 10 2 2 4 2 21 2 2 4 4 17 5 2 5 2 11 4 2 2 2 10 7 3 15 2 2 2 3 19 3 2 6 3 2
+>946_21_659_F3
+3 31 3 2 2 2 34 3 2 2 2 31 2 2 3 4 31 8 3 2 2 30 4 8 3 3 2 2 6 9 4 4 6 2 2
+>946_21_1071_F3
+5 5 2 2 2 8 5 3 2 3 7 7 2 3 4 6 5 2 2 2 5 5 2 2 2 3 8 2 3 3 3 8 2 3 2
+>946_21_1115_F3
+21 5 2 8 13 31 6 2 17 24 10 27 4 21 29 8 20 2 11 21 13 24 5 5 6 24 31 2 13 6 22 17 6 27 10
+>946_21_1218_F3
+11 21 2 13 13 16 27 16 19 27 22 28 14 26 24 23 29 10 15 13 6 4 7 16 26 22 11 6 16 22 21 6 4 7 21
+>946_21_1232_F3
+17 16 2 28 21 31 15 16 10 11 8 20 6 5 18 6 13 23 7 13 4 12 19 8 6 9 10 19 7 10 6 10 20 14 8
\ No newline at end of file
11 years, 2 months
[hg] galaxy 2386: Change variable names in validation from min/m...
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/799a333342c4
changeset: 2386:799a333342c4
user: Dan Blankenberg <dan(a)bx.psu.edu>
date: Thu Apr 30 10:24:41 2009 -0400
description:
Change variable names in validation from min/max to range/length_min/max.
1 file(s) affected in this change:
lib/galaxy/tools/parameters/validation.py
diffs (40 lines):
diff -r add0911007d9 -r 799a333342c4 lib/galaxy/tools/parameters/validation.py
--- a/lib/galaxy/tools/parameters/validation.py Wed Apr 29 17:37:22 2009 -0400
+++ b/lib/galaxy/tools/parameters/validation.py Thu Apr 30 10:24:41 2009 -0400
@@ -102,10 +102,10 @@
@classmethod
def from_element( cls, param, elem ):
return cls( elem.get( 'message', None ), elem.get( 'min' ), elem.get( 'max' ) )
- def __init__( self, message, min, max ):
- self.message = message or ( "Value must be between %f and %f" % ( min, max ) )
- self.min = float( min )
- self.max = float( max )
+ def __init__( self, message, range_min, range_max ):
+ self.message = message or ( "Value must be between %f and %f" % ( range_min, range_max ) )
+ self.min = float( range_min )
+ self.max = float( range_max )
def validate( self, value, history=None ):
if not( self.min <= float( value ) <= self.max ):
raise ValueError( self.message )
@@ -134,14 +134,14 @@
@classmethod
def from_element( cls, param, elem ):
return cls( elem.get( 'message', None ), elem.get( 'min', None ), elem.get( 'max', None ) )
- def __init__( self, message, min, max ):
+ def __init__( self, message, length_min, length_max ):
self.message = message
- if min is not None:
- min = int( min )
- if max is not None:
- max = int( max )
- self.min = min
- self.max = max
+ if length_min is not None:
+ length_min = int( length_min )
+ if length_max is not None:
+ length_max = int( length_max )
+ self.min = length_min
+ self.max = length_max
def validate( self, value, history=None ):
if self.min is not None and len( value ) < self.min:
raise ValueError( self.message or ( "Must have length of at least %d" % self.min ) )
11 years, 2 months
[hg] galaxy 2388: A tiny bug fix for linear regression tool.
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/ff2fca65799e
changeset: 2388:ff2fca65799e
user: guru
date: Thu Apr 30 11:07:34 2009 -0400
description:
A tiny bug fix for linear regression tool.
1 file(s) affected in this change:
tools/regVariation/linear_regression.py
diffs (12 lines):
diff -r 609ee7376d0d -r ff2fca65799e tools/regVariation/linear_regression.py
--- a/tools/regVariation/linear_regression.py Thu Apr 30 10:54:43 2009 -0400
+++ b/tools/regVariation/linear_regression.py Thu Apr 30 11:07:34 2009 -0400
@@ -17,7 +17,7 @@
print "Predictor columns: %s; Response column: %d" %(x_cols,y_col+1)
fout = open(outfile,'w')
-
+elems = []
for i, line in enumerate( file ( infile )):
line = line.rstrip('\r\n')
if len( line )>0 and not line.startswith( '#' ):
11 years, 2 months
[hg] galaxy 2380: Replacing \"qual\" datatype with 3 new datatyp...
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/22b08d47f7ba
changeset: 2380:22b08d47f7ba
user: guru
date: Wed Apr 29 11:53:22 2009 -0400
description:
Replacing \"qual\" datatype with 3 new datatypes: qualsolid, qual454 and qualsolexa.
10 file(s) affected in this change:
datatypes_conf.xml.sample
lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml
lib/galaxy/datatypes/qualityscore.py
lib/galaxy/datatypes/registry.py
lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py
tools/metag_tools/fastqsolexa_to_fasta_qual.xml
tools/metag_tools/rmapq_wrapper.xml
tools/metag_tools/short_reads_figure_high_quality_length.xml
tools/metag_tools/short_reads_figure_score.xml
tools/metag_tools/short_reads_trim_seq.xml
diffs (409 lines):
diff -r 5a4aac327bad -r 22b08d47f7ba datatypes_conf.xml.sample
--- a/datatypes_conf.xml.sample Wed Apr 29 10:27:57 2009 -0400
+++ b/datatypes_conf.xml.sample Wed Apr 29 11:53:22 2009 -0400
@@ -19,7 +19,7 @@
</datatype>
<datatype extension="fastqsolexa" type="galaxy.datatypes.sequence:FastqSolexa" display_in_upload="true">
<converter file="fastqsolexa_to_fasta_converter.xml" target_datatype="fasta"/>
- <converter file="fastqsolexa_to_qual_converter.xml" target_datatype="qual"/>
+ <converter file="fastqsolexa_to_qual_converter.xml" target_datatype="qualsolexa"/>
</datatype>
<datatype extension="genetrack" type="galaxy.datatypes.tracks:GeneTrack"/>
<datatype extension="gff" type="galaxy.datatypes.interval:Gff" display_in_upload="true">
@@ -42,7 +42,9 @@
</datatype>
<datatype extension="pdf" type="galaxy.datatypes.images:Image" mimetype="application/pdf"/>
<datatype extension="png" type="galaxy.datatypes.images:Image" mimetype="image/png"/>
- <datatype extension="qual" type="galaxy.datatypes.qualityscore:QualityScore" display_in_upload="true"/>
+ <datatype extension="qualsolexa" type="galaxy.datatypes.qualityscore:QualityScoreSolexa" display_in_upload="true"/>
+ <datatype extension="qualsolid" type="galaxy.datatypes.qualityscore:QualityScoreSOLiD" display_in_upload="true"/>
+ <datatype extension="qual454" type="galaxy.datatypes.qualityscore:QualityScore454" display_in_upload="true"/>
<datatype extension="scf" type="galaxy.datatypes.images:Scf" mimetype="application/octet-stream" display_in_upload="true"/>
<datatype extension="taxonomy" type="galaxy.datatypes.tabular:Taxonomy" display_in_upload="true"/>
<datatype extension="tabular" type="galaxy.datatypes.tabular:Tabular" display_in_upload="true"/>
@@ -186,7 +188,9 @@
<sniffer type="galaxy.datatypes.sequence:Maf"/>
<sniffer type="galaxy.datatypes.sequence:Lav"/>
<sniffer type="galaxy.datatypes.sequence:csFasta"/>
- <sniffer type="galaxy.datatypes.qualityscore:QualityScore"/>
+ <sniffer type="galaxy.datatypes.qualityscore:QualityScoreSolexa"/>
+ <sniffer type="galaxy.datatypes.qualityscore:QualityScoreSOLiD"/>
+ <sniffer type="galaxy.datatypes.qualityscore:QualityScore454"/>
<sniffer type="galaxy.datatypes.sequence:Fasta"/>
<sniffer type="galaxy.datatypes.sequence:FastqSolexa"/>
<sniffer type="galaxy.datatypes.interval:Wiggle"/>
diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml
--- a/lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml Wed Apr 29 11:53:22 2009 -0400
@@ -4,7 +4,7 @@
<param format="fastqsolexa" name="input1" type="data" label="Choose Fastqsolexa file"/>
</inputs>
<outputs>
- <data format="qual" name="output1" />
+ <data format="qualsolexa" name="output1" />
</outputs>
<help>
</help>
diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/qualityscore.py
--- a/lib/galaxy/datatypes/qualityscore.py Wed Apr 29 10:27:57 2009 -0400
+++ b/lib/galaxy/datatypes/qualityscore.py Wed Apr 29 11:53:22 2009 -0400
@@ -9,11 +9,11 @@
log = logging.getLogger(__name__)
-class QualityScore ( data.Text ):
+class QualityScoreSOLiD ( data.Text ):
"""
until we know more about quality score formats
"""
- file_ext = "qual"
+ file_ext = "qualsolid"
def set_peek( self, dataset, line_count=None ):
if not dataset.dataset.purged:
@@ -21,7 +21,7 @@
if line_count is None:
dataset.blurb = data.nice_size( dataset.get_size() )
else:
- dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) )
+ dataset.blurb = "%s lines, SOLiD Quality score file" % util.commaify( str( line_count ) )
else:
dataset.peek = 'file does not exist'
dataset.blurb = 'file purged from disk'
@@ -30,15 +30,80 @@
try:
return dataset.peek
except:
- return "Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) )
+ return "SOLiD Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) )
def sniff( self, filename ):
"""
>>> fname = get_test_fname( 'sequence.fasta' )
- >>> QualityScore().sniff( fname )
+ >>> QualityScoreSOLiD().sniff( fname )
False
- >>> fname = get_test_fname( 'sequence.qual' )
- >>> QualityScore().sniff( fname )
+ >>> fname = get_test_fname( 'sequence.qualsolid' )
+ >>> QualityScoreSOLiD().sniff( fname )
+ True
+ """
+ try:
+ fh = open( filename )
+ readlen = None
+ goodblock = 0
+ while True:
+ line = fh.readline()
+ if not line:
+ if goodblock > 0:
+ return True
+ else:
+ break #EOF
+ line = line.strip()
+ if line and not line.startswith( '#' ): #first non-empty non-comment line
+ if line.startswith( '>' ):
+ line = fh.readline().strip()
+ if line == '' or line.startswith( '>' ):
+ break
+ try:
+ [ int( x ) for x in line.split() ]
+ if not(readlen):
+ readlen = len(line.split())
+ assert len(line.split()) == readlen #SOLiD reads should be of the same length
+ except:
+ break
+ goodblock += 1
+ if goodblock > 10:
+ return True
+ else:
+ break #we found a non-empty line, but it's not a header
+ except:
+ pass
+ return False
+
+class QualityScore454 ( data.Text ):
+ """
+ until we know more about quality score formats
+ """
+ file_ext = "qual454"
+
+ def set_peek( self, dataset, line_count=None ):
+ if not dataset.dataset.purged:
+ dataset.peek = data.get_file_peek( dataset.file_name )
+ if line_count is None:
+ dataset.blurb = data.nice_size( dataset.get_size() )
+ else:
+ dataset.blurb = "%s lines, 454 Quality score file" % util.commaify( str( line_count ) )
+ else:
+ dataset.peek = 'file does not exist'
+ dataset.blurb = 'file purged from disk'
+
+ def display_peek(self, dataset):
+ try:
+ return dataset.peek
+ except:
+ return "454 Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) )
+
+ def sniff( self, filename ):
+ """
+ >>> fname = get_test_fname( 'sequence.fasta' )
+ >>> QualityScore454().sniff( fname )
+ False
+ >>> fname = get_test_fname( 'sequence.qual454' )
+ >>> QualityScore454().sniff( fname )
True
"""
try:
@@ -63,3 +128,59 @@
except:
pass
return False
+
+class QualityScoreSolexa ( data.Text ):
+ """
+ until we know more about quality score formats
+ """
+ file_ext = "qualsolexa"
+
+ def set_peek( self, dataset, line_count=None ):
+ if not dataset.dataset.purged:
+ dataset.peek = data.get_file_peek( dataset.file_name )
+ if line_count is None:
+ dataset.blurb = data.nice_size( dataset.get_size() )
+ else:
+ dataset.blurb = "%s lines, Solexa Quality score file" % util.commaify( str( line_count ) )
+ else:
+ dataset.peek = 'file does not exist'
+ dataset.blurb = 'file purged from disk'
+
+ def display_peek(self, dataset):
+ try:
+ return dataset.peek
+ except:
+ return "Solexa Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) )
+
+ def sniff( self, filename ):
+ """
+ >>> fname = get_test_fname( 'sequence.fasta' )
+ >>> QualityScoreSolexa().sniff( fname )
+ False
+ >>> fname = get_test_fname( 'sequence.qualsolexa' )
+ >>> QualityScoreSolexa().sniff( fname )
+ True
+ """
+ try:
+ fh = open( filename )
+ readlen = None
+ while True:
+ line = fh.readline()
+ if not line:
+ break #EOF
+ line = line.strip()
+ if line and not line.startswith( '#' ):
+ if len(line.split('\t')) > 1:
+ break
+ try:
+ [ int( x ) for x in line.split() ]
+ if not(readlen):
+ readlen = len(line.split())
+ assert len(line.split()) == readlen #Solexa reads should be of the same length
+ except:
+ break
+
+ except:
+ pass
+ return False
+
diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/registry.py
--- a/lib/galaxy/datatypes/registry.py Wed Apr 29 10:27:57 2009 -0400
+++ b/lib/galaxy/datatypes/registry.py Wed Apr 29 11:53:22 2009 -0400
@@ -116,7 +116,9 @@
'laj' : images.Laj(),
'lav' : sequence.Lav(),
'maf' : sequence.Maf(),
- 'qual' : qualityscore.QualityScore(),
+ 'qualsolid' : qualityscore.QualityScoreSOLiD(),
+ 'qualsolexa' : qualityscore.QualityScoreSolexa(),
+ 'qual454' : qualityscore.QualityScore454(),
'scf' : images.Scf(),
'tabular' : tabular.Tabular(),
'taxonomy' : tabular.Taxonomy(),
@@ -140,7 +142,9 @@
'laj' : 'text/plain',
'lav' : 'text/plain',
'maf' : 'text/plain',
- 'qual' : 'text/plain',
+ 'qualsolid' : 'text/plain',
+ 'qualsolexa' : 'text/plain',
+ 'qual454' : 'text/plain',
'scf' : 'application/octet-stream',
'tabular' : 'text/plain',
'taxonomy' : 'text/plain',
diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py Wed Apr 29 11:53:22 2009 -0400
@@ -0,0 +1,47 @@
+from sqlalchemy import *
+from sqlalchemy.orm import *
+from migrate import *
+import sys, logging
+
+log = logging.getLogger( __name__ )
+log.setLevel(logging.DEBUG)
+handler = logging.StreamHandler( sys.stdout )
+format = "%(name)s %(levelname)s %(asctime)s %(message)s"
+formatter = logging.Formatter( format )
+handler.setFormatter( formatter )
+log.addHandler( handler )
+
+metadata = MetaData( migrate_engine )
+db_session = scoped_session( sessionmaker( bind=migrate_engine, autoflush=False, transactional=False ) )
+HistoryDatasetAssociation_table = Table( "history_dataset_association", metadata, autoload=True )
+
+def upgrade():
+ # Load existing tables
+ metadata.reflect()
+ # Add 2 indexes to the galaxy_user table
+ i = Index( 'ix_hda_extension', HistoryDatasetAssociation_table.c.extension )
+ try:
+ i.create()
+ except Exception, e:
+ log.debug( "Adding index 'ix_hda_extension' to history_dataset_association table failed: %s" % ( str( e ) ) )
+
+ # Set the default data in the galaxy_user table, but only for null values
+ cmd = "UPDATE history_dataset_association SET extension = 'qual454' WHERE extension = 'qual' and peek like \'>%%\'"
+ try:
+ db_session.execute( cmd )
+ except Exception, e:
+ log.debug( "Resetting extension qual to qual454 in history_dataset_association failed: %s" % ( str( e ) ) )
+ cmd = "UPDATE history_dataset_association SET extension = 'qualsolexa' WHERE extension = 'qual' and peek not like \'>%%\'"
+ try:
+ db_session.execute( cmd )
+ except Exception, e:
+ log.debug( "Resetting extension qual to qualsolexa in history_dataset_association failed: %s" % ( str( e ) ) )
+ # Add 1 index to the history_dataset_association table
+ try:
+ i.drop()
+ except Exception, e:
+ log.debug( "Dropping index 'ix_hda_extension' to history_dataset_association table failed: %s" % ( str( e ) ) )
+
+
+def downgrade():
+ pass
diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/fastqsolexa_to_fasta_qual.xml
--- a/tools/metag_tools/fastqsolexa_to_fasta_qual.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/tools/metag_tools/fastqsolexa_to_fasta_qual.xml Wed Apr 29 11:53:22 2009 -0400
@@ -6,7 +6,7 @@
</inputs>
<outputs>
<data name="output1" format="fasta"/>
- <data name="output2" format="qual"/>
+ <data name="output2" format="qualsolexa"/>
</outputs>
<tests>
<!-- NOTE: this tool generates 2 output files, but our functional tests currently only handle the last one generated -->
diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/rmapq_wrapper.xml
--- a/tools/metag_tools/rmapq_wrapper.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/tools/metag_tools/rmapq_wrapper.xml Wed Apr 29 11:53:22 2009 -0400
@@ -13,7 +13,7 @@
</options>
</param>
<param name="input_seq" type="data" format="fasta" label="Sequence file"/>
- <param name="input_score" type="data" format="qual" label="Quality score file"/>
+ <param name="input_score" type="data" format="qualsolexa" label="Quality score file"/>
<param name="high_score" type="float" size="15" value="40" label="Minimum score for high-quality base (-q)"/>
<param name="high_len" type="integer" size="15" value="36" label="Minimal high-quality bases (-M)"/>
<param name="align_len" type="integer" size="15" value="11" label="Minimal length of a hit (-h)" help="seed"/>
@@ -46,7 +46,7 @@
<test>
<param name="database" value="/depot/data2/galaxy/faseq/test" />
<param name="input_seq" value="rmapq_wrapper_test1.fasta" ftype="fasta"/>
- <param name="input_score" value="rmapq_wrapper_test1.qual" ftype="qual" />
+ <param name="input_score" value="rmapq_wrapper_test1.qual" ftype="qualsolexa" />
<param name="high_score" value="40" />
<param name="high_len" value="36" />
<param name="read_len" value="36" />
diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_figure_high_quality_length.xml
--- a/tools/metag_tools/short_reads_figure_high_quality_length.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/tools/metag_tools/short_reads_figure_high_quality_length.xml Wed Apr 29 11:53:22 2009 -0400
@@ -5,7 +5,7 @@
<inputs>
<page>
- <param name="input1" type="data" format="qual,txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/>
+ <param name="input1" type="data" format="qualsolexa,qual454,txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/>
<param name="input2" type="integer" size="5" value="20" label="Quality score threshold" />
</page>
</inputs>
@@ -17,12 +17,12 @@
</requirements>
<tests>
<test>
- <param name="input1" value="solexa.qual" ftype="qual" />
+ <param name="input1" value="solexa.qual" ftype="qualsolexa" />
<param name="input2" value="5" />
<output name="output1" file="solexa_high_quality_hist.pdf" ftype="pdf"/>
</test>
<test>
- <param name="input1" value="454.qual" ftype="qual" />
+ <param name="input1" value="454.qual" ftype="qual454" />
<param name="input2" value="5" />
<output name="output1" file="454_high_quality_hist.pdf" ftype="pdf"/>
</test>
diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_figure_score.xml
--- a/tools/metag_tools/short_reads_figure_score.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/tools/metag_tools/short_reads_figure_score.xml Wed Apr 29 11:53:22 2009 -0400
@@ -5,7 +5,7 @@
<inputs>
<page>
- <param name="input1" type="data" format="qual, txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/>
+ <param name="input1" type="data" format="qualsolexa, qual454, txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/>
</page>
</inputs>
@@ -17,11 +17,11 @@
</requirements>
<tests>
<test>
- <param name="input1" value="solexa.qual" ftype="qual" />
+ <param name="input1" value="solexa.qual" ftype="qualsolexa" />
<output name="output1" file="solexaScore.png" ftype="png" />
</test>
<test>
- <param name="input1" value="454.qual" ftype="qual" />
+ <param name="input1" value="454.qual" ftype="qual454" />
<output name="output1" file="454Score.png" ftype="png" />
</test>
</tests>
diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_trim_seq.xml
--- a/tools/metag_tools/short_reads_trim_seq.xml Wed Apr 29 10:27:57 2009 -0400
+++ b/tools/metag_tools/short_reads_trim_seq.xml Wed Apr 29 11:53:22 2009 -0400
@@ -7,7 +7,7 @@
<inputs>
<page>
<param name="input1" type="data" format="fasta,txtseq.zip" label="Reads" />
- <param name="input2" type="data" format="qual,txtseq.zip" label="Quality scores" />
+ <param name="input2" type="data" format="qualsolexa,qual454,txtseq.zip" label="Quality scores" />
<param name="trim" type="integer" size="5" value="20" label="Minimal quality score" help="bases scoring below this value will trigger splitting"/>
<param name="length" type="integer" size="5" value="100" label="Minimal length of contiguous segment" help="report all high quality segments above this length. Setting this option to '0' will cause the program to return a single longest run of high quality bases per read" />
<conditional name="sequencing_method_choice">
@@ -36,7 +36,7 @@
<test>
<param name="sequencer" value="454" />
<param name="input1" value="454.fasta" ftype="fasta" />
- <param name="input2" value="454.qual" ftype="qual" />
+ <param name="input2" value="454.qual" ftype="qual454" />
<param name="input3" value="no" />
<param name="trim" value="20" />
<param name="length" value="0" />
@@ -45,7 +45,7 @@
<test>
<param name="sequencer" value="Solexa" />
<param name="input1" value="solexa.fasta" ftype="fasta" />
- <param name="input2" value="solexa.qual" ftype="qual" />
+ <param name="input2" value="solexa.qual" ftype="qualsolexa" />
<param name="input3" value="0" />
<param name="trim" value="20" />
<param name="length" value="0" />
11 years, 2 months
[hg] galaxy 2382: Fix for reporting job errors - somehow necessa...
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/89541ac4983f
changeset: 2382:89541ac4983f
user: Greg Von Kuster <greg(a)bx.psu.edu>
date: Wed Apr 29 13:56:59 2009 -0400
description:
Fix for reporting job errors - somehow necessary imported stuff got eliminated.
1 file(s) affected in this change:
lib/galaxy/web/controllers/dataset.py
diffs (24 lines):
diff -r dae0313bf5bb -r 89541ac4983f lib/galaxy/web/controllers/dataset.py
--- a/lib/galaxy/web/controllers/dataset.py Wed Apr 29 13:46:02 2009 -0400
+++ b/lib/galaxy/web/controllers/dataset.py Wed Apr 29 13:56:59 2009 -0400
@@ -1,7 +1,7 @@
-import logging, os, mimetypes, smtplib
+import logging, os, sets, string, shutil, re, socket, mimetypes, smtplib
from galaxy.web.base.controller import *
-from galaxy import web, model
+from galaxy import util, datatypes, jobs, web, model
from cgi import escape, FieldStorage
from email.MIMEText import MIMEText
@@ -89,8 +89,8 @@
s.sendmail( frm, [ to ], msg.as_string() )
s.close()
return trans.show_ok_message( "Your error report has been sent" )
- except:
- return trans.show_error_message( "An error occurred sending the report by email" )
+ except Exception, e:
+ return trans.show_error_message( "An error occurred sending the report by email: %s" % str( e ) )
@web.expose
def default(self, trans, dataset_id=None, **kwd):
11 years, 2 months
[hg] galaxy 2383: Accidently commited a testing change, removed
by Greg Von Kuster
details: http://www.bx.psu.edu/hg/galaxy/rev/656f1df80b46
changeset: 2383:656f1df80b46
user: James Taylor <james(a)jamestaylor.org>
date: Wed Apr 29 14:47:18 2009 -0400
description:
Accidently commited a testing change, removed
1 file(s) affected in this change:
tools/maf/interval2maf.xml
diffs (11 lines):
diff -r 89541ac4983f -r 656f1df80b46 tools/maf/interval2maf.xml
--- a/tools/maf/interval2maf.xml Wed Apr 29 13:56:59 2009 -0400
+++ b/tools/maf/interval2maf.xml Wed Apr 29 14:47:18 2009 -0400
@@ -6,7 +6,6 @@
#end if
</command>
<inputs>
- <param type="text" name="TEST" label="Test" />
<param format="interval" name="input1" type="data" label="Choose intervals">
<validator type="unspecified_build" />
</param>
11 years, 2 months
