stupid logging question
by Michael Pheasant
Using the run.sh script, and the default 'log_level = DEBUG'
universe_wsgi.ini file setting gives a huge amount of output.
Changing the ini to either log_level = INFO, WARNING or ERROR and i
get the same output, including the INFO messages every second.
I assumed that WARNING or ERROR would not give the INFO messages; is
there some way to not see these?
Cheers
m
--
Michael Pheasant
Software Engineer
Queensland Facility for Advanced Bioinformatics
Level 6, QBP
University of Queensland, QLD 4072
T: +61 (0)7 3346 2070
F: +61 (0)7 3346 2101
www.qfab.org
10 years
Server error from history view
by simon andrews (BI)
I'm making pretty good progress setting up our galaxy server, but have hit a snag. Since moving from an SQLite to a MySQL backend I'm getting an internal server error from the history panel on our galaxy instance. Running the galaxy server from a console gives me the following python traceback:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 60795)
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib64/python2.6/SocketServer.py", line 320, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib64/python2.6/SocketServer.py", line 615, in __init__
self.handle()
File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 442, in handle
BaseHTTPRequestHandler.handle(self)
File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
self.handle_one_request()
File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 437, in handle_one_request
self.wsgi_execute()
File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 287, in wsgi_execute
self.wsgi_start_response)
File "/usr/lib/python2.6/site-packages/paste/urlmap.py", line 203, in __call__
return app(environ, start_response)
File "/data/private/galaxy/galaxy_dist/lib/galaxy/web/framework/middleware/xforwardedhost.py", line 21, in __call__
return self.app( environ, start_response )
File "/usr/lib/python2.6/site-packages/paste/deploy/config.py", line 164, in __call__
app_iter = self.application(environ, start_response)
File "/usr/lib/python2.6/site-packages/paste/translogger.py", line 68, in __call__
return self.application(environ, replacement_start_response)
File "/usr/lib/python2.6/site-packages/paste/exceptions/errormiddleware.py", line 154, in __call__
exc_info)
File "/usr/lib/python2.6/site-packages/paste/translogger.py", line 67, in replacement_start_response
return start_response(status, headers)
File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 166, in wsgi_start_response
assert 0, "Attempt to set headers a second time w/o an exc_info"
AssertionError: Attempt to set headers a second time w/o an exc_info
----------------------------------------
Any ideas?
Thanks
Simon.
10 years, 1 month
Sort tab error?
by Minh Bui
Hi,
I always get this error when I use Sort feature
*"An error occurred running this job: sort: multi-character tab `$\t'"*
Even I already use Text Manipulation->Convert to convert my table to Tabs
format.
Here is the screenshot of what happen (I want to sort C2 on job #12) :
http://img408.imageshack.us/img408/7906/screenshotbu.png
Could someone please show me how to fix this?
Thank you.
10 years, 1 month
adding another UCSC mirror to display
by Davide Cittaro
Hi again, this may be a FAQ, sorry for that... How can I add an alternate UCSC mirror site (our local mirror) to galaxy so that everything that could be displayed at UCSC-main will have the UCSC-localmirror option? Besides... how can I remove the bx main mirror?
I've tried to add a ${GALAXYROOT}/tool-data/shared/campus/campus_build_sites.txt but that's not enough... I've modified the ucsc/bam.xml file adding another dynamic_link section but that's not enough... any hint?
Thanks again
d
/*
Davide Cittaro
Cogentech - Consortium for Genomic Technologies
via adamello, 16
20139 Milano
Italy
tel.: +39(02)574303007
e-mail: davide.cittaro(a)ifom-ieo-campus.it
*/
10 years, 1 month
Upload issue: 'Upload a directory of files' and compressed files
by padioleau
Dear Galaxy developer,
Using the data-library upload, 'Upload a directory of files', I encounter
an issue. When uploading compressed files, without copying them in galaxy
folder, they are uncompressed in my folder but the file name extension is
not changed. This mean that I have files with 'gz' extension that are in
fact uncompressed.
Is there a possibility to avoid this issue?
I'm using tools which can use compressed files, so for me the best would
be to just add the compressed file in my data-library.
I believe galaxy can accept compressed files, but I didn't found any
explanation in your wiki (index: compress, gzip or zip) to configure galaxy
this way.
Thanks,
Ismael
--
Padioleau Ismael
EMBL Heidelberg
( V205)
10 years, 1 month
Re: [galaxy-dev] Fwd: Re: Help about zip upload (esd files)
by Adhemar
Dear Nate,
We're trying to implement different tools to handle sequences generated
by Megabace/AB1 sequencers.
The very first step in this process is to load the chromatograms
provided by the machines.
The thing is these machines produce an individual file per sequence and
the phred software (base caller) receive as parameters a compressed file
or a directory path.
Since galaxy 'get data' feature loads only individual files and we're
not able to implement phred to do such things, what options are left?
Thank you so much,
Adhemar
On 07/29/2010 01:57 PM, Eric Aguiar wrote:
>
>
> -------- Original Message --------
> Subject: Re: [galaxy-dev] Help about zip upload (esd files)
> Date: Wed, 28 Jul 2010 12:09:06 -0400
> From:
> To: Eric Aguiar <eric(a)cebio.org>
> CC: List
>
>
>
> Eric Aguiar wrote:
> > Dear,
> >
> > I followed all steps in the galaxy tutorial, but I didn't have success.
> > I'm trying to create a datatype for megabase chromatograms (.esd) very
> > similar to the Ab1 ones.
> >
> > Here is my configurations.
> >
> >
> > *1 - Creating datatypes in datatypes_conf.xml*
> >
> > <datatype extension="zip" type="galaxy.datatypes.binary:Esd"
> > mimetype="application/zip" display_in_upload="true"/>
> > <datatype extension="esd" type="galaxy.datatypes.binary:Esd"
> > mimetype="application/octet-stream" display_in_upload="true"/>
> >
> > *2 - Defining types in lib/galaxy/datatypes/binary.py*
> >
> > class Esd( Binary ):
> > """Class describing an ab1 binary sequence file"""
> > file_ext = "esd"
> >
> > def set_peek( self, dataset, is_multi_byte=False ):
> > if not dataset.dataset.purged:
> > dataset.peek = "Binary chromatograms sequence file"
> > dataset.blurb = data.nice_size( dataset.get_size() )
> > else:
> > dataset.peek = 'file does not exist'
> > dataset.blurb = 'file purged from disk'
> > def display_peek( self, dataset ):
> > try:
> > return dataset.peek
> > except:
> > return "Binary esd sequence file (%s)" % ( data.nice_size(
> > dataset.get_size() ) )
> >
> >
> > class Zip( Binary ):
> > """Class describing a zip archive of binary sequence files"""
> > file_ext = "zip"
> >
> > def set_peek( self, dataset, is_multi_byte=False ):
> > if not dataset.dataset.purged:
> > zip_file = zipfile.ZipFile( dataset.file_name, "r" )
> > num_files = len( zip_file.namelist() )
> > dataset.peek = "Archive of %s binary sequence files" % (
> > str( num_files ) )
> > dataset.blurb = data.nice_size( dataset.get_size() )
> > else:
> > dataset.peek = 'file does not exist'
> > dataset.blurb = 'file purged from disk'
> > def display_peek( self, dataset ):
> > try:
> > return dataset.peek
> > except:
> > return "Binary sequence file archive (%s)" % (
> > data.nice_size( dataset.get_size() ) )
> > def get_mime( self ):
> > """Returns the mime type of the datatype"""
> > return 'application/zip'
> >
> >
> > When I'm going to send the file in zip format (.esd files compressed),it
> > shows me the following error:
> > *"An error occurred running this job: Invalid 'File Format' for archive
> > consisting of binary files - use 'Binseq.zip'"*
>
> Hi Eric,
>
> Have you updated to a recent revision of Galaxy? Binseq.zip and the
> code that prevents uploading zip files has been removed. Currently,
> though, a zip file can only contain one member.
>
> --nate
>
> >
> > I tried somethings, but I don't have success.
> >
> > Thank you,
> > --
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > galaxy-dev mailing list
> > galaxy-dev(a)lists.bx.psu.edu
> > http://lists.bx.psu.edu/listinfo/galaxy-dev
>
--
Adhemar Zerlotini Neto, PhD
FIOCRUZ - Fundação Oswaldo Cruz
CPqRR - Centro de Pesquisas René Rachou
http://www.cpqrr.fiocruz.br
http://www.cebio.org
+55 31 3337 3863
+55 31 3337 3649
10 years, 2 months
DRMMA and SGE problem
by George Chalkidis
Hi Galaxy-Dev-Team!
First of all, I would like to thank you for providing the community
with such a great framework!
I've downloaded and set up locally the latest packed release. When I
want to use Galaxy with the Sun Grid Engine, I run into problems. It
looks like Galaxy isn't able to fully control the Grid Engine and I
think it might be a version issue with DRMMA and our current SGE. Jobs
get submitted, appear in the queue and are executed, but their status
is not returned to Galaxy and therefore the job's status on the
website remains always "Job is waiting to run".
Here's a more detailed error-message:
------------------------------------------------------
[environment]
CentOS release 5.4 (Final)
SGE 6.2u5
Python 2.5.2
Jun 30th source tar.gz file
http://bitbucket.org/galaxy/galaxy-dist/get/tip.tar.gz
---------------------------------------------------------
[config]
$ cat universe_wsgi.ini |grep sge
# currently available are 'pbs' and 'sge'.
start_job_runners = sge
#default_cluster_job_runner = sge://default/mjobs.q/ cant lunch service
default_cluster_job_runner = sge:///mjobs.q/
$ grep -5 SGE_ROOT eggs.ini
[general]
repository = http://eggs.g2.bx.psu.edu/new
; these eggs must be scrambled for your local environment
no_auto = pbs_python DRMAA_python
SGE_ROOT = /home/geadmin/N1GE python scripts/scramble.py DRMAA_python
[eggs:platform]
bx_python = 0.5.0
Cheetah = 2.2.2
DRMAA_python = 0.2
---------------------------------------------------------
[Error Messages]
galaxy.jobs DEBUG 2010-07-16 17:30:24,794 job 10 dispatched
galaxy.jobs.runners.sge DEBUG 2010-07-16 17:30:27,480 (10) submitting
file /share1/home/icgc/galaxy-dist/database/pbs/galaxy_10.sh
galaxy.jobs.runners.sge DEBUG 2010-07-16 17:30:27,480 (10) command is:
perl /share1/home/icgc/galaxy-dist/tools/myTools/toolExample.pl
/share1/home/icgc/galaxy-dist/database/files/000/dataset_3.dat
/share1/home/icgc/galaxy-dist/database/files/000/dataset_10.dat
galaxy.jobs.runners.sge DEBUG 2010-07-16 17:30:27,522 (10) queued in
mjobs.q queue as 9440764
202.175.149.92 - - [16/Jul/2010:17:30:26 +0900] "POST
/root/history_item_updates HTTP/1.1" 200 -
"http://gw01.hgc.jp:8090/history" "Mozilla/5.0 (X11; U; Linux x86_64;
en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6"
galaxy.jobs.runners.sge ERROR 2010-07-16 17:30:29,524 (10/9440764)
Unable to check job status
Traceback (most recent call last):
File "/share1/home/icgc/galaxy-dist/lib/galaxy/jobs/runners/sge.py",
line 277, in check_watched_items
state = self.ds.getJobProgramStatus( job_id )
File "/usr/local/lib/python2.5/site-packages/DRMAA.py", line 395, in
getJobProgramStatus
return ps
TypeError: writelines() argument must be a sequence of strings
galaxy.jobs.runners.sge WARNING 2010-07-16 17:30:29,525 (10/9440764)
job will now be errored
galaxy.jobs.runners.sge ERROR 2010-07-16 17:30:29,526 Uncaught
exception failing job
Traceback (most recent call last):
File "/share1/home/icgc/galaxy-dist/lib/galaxy/jobs/runners/sge.py",
line 138, in run_next
self.fail_job( obj )
File "/share1/home/icgc/galaxy-dist/lib/galaxy/jobs/runners/sge.py",
line 338, in fail_job
self.stop_job( self.sa_session.query( self.app.model.Job ).get(
sge_job_state.job_wrapper.job_id ) )
AttributeError: 'SGEJobRunner' object has no attribute 'sa_session'
[when I use sample code]
#!/usr/bin/env python
import DRMAA
import time
import os
def main():
"""Submit a job, and check its progress.
Note, need file called sleeper.sh in home directory. An example:
echo 'Hello World $1'
sleep 30s
"""
s=DRMAA.Session()
s.init()
print 'Creating job template'
jt = s.createJobTemplate()
jt.remoteCommand = os.getcwd() + '/sleeper.sh'
jt.args = ['42','Simon says:']
jt.joinFiles=True
jt.outputPath=":"+DRMAA.JobTemplate.HOME_DIRECTORY+'/tmp/JOB_OUT'
jobid = s.runJob(jt)
print 'Your job has been submitted with id ' + jobid
# Who needs a case statement when you have dictionaries?
decodestatus = {
DRMAA.Session.UNDETERMINED: 'process status cannot be determined',
DRMAA.Session.QUEUED_ACTIVE: 'job is queued and active',
DRMAA.Session.SYSTEM_ON_HOLD: 'job is queued and in system hold',
DRMAA.Session.USER_ON_HOLD: 'job is queued and in user hold',
DRMAA.Session.USER_SYSTEM_ON_HOLD: 'job is queued and in user
and system hold',
DRMAA.Session.RUNNING: 'job is running',
DRMAA.Session.SYSTEM_SUSPENDED: 'job is system suspended',
DRMAA.Session.USER_SUSPENDED: 'job is user suspended',
DRMAA.Session.DONE: 'job finished normally',
DRMAA.Session.FAILED: 'job finished, but failed',
}
for ix in range(10):
print ix
print 'Checking ' + str(ix) + ' of 10 times'
status = s.getJobProgramStatus(jobid)
print decodestatus.get(status)
time.sleep(5)
print 'Cleaning up'
s.deleteJobTemplate(jt)
s.exit()
if __name__=='__main__':
main()
$ ./test
Creating job template
Your job has been submitted with id 9440771
0
Checking 0 of 10 times
Traceback (most recent call last):
File "./test", line 52, in <module>
main()
File "./test", line 43, in main
status = s.getJobProgramStatus(jobid)
File "/usr/local/lib/python2.5/site-packages/DRMAA.py", line 393, in
getJobProgramStatus
eno, ps, estr = cDRMAA.drmaa_job_ps(jobName)
ValueError: need more than 2 values to unpack
[debug]
print cDRMAA.drmaa_job_ps(jobName)
eno, ps, estr = cDRMAA.drmaa_job_ps(jobName)
=> ([0, 16], '')
Thank you for taking time! Hopefully, you can give me some hints on
how to resolve this issue.
Cheers,
George Chalkidis
10 years, 2 months
Textbox Size
by Dennis Gascoigne
I have tools that require a wide textbox area. It seems however that there
is a limit coded into Galaxy soewhere of about 60 characters. Can you advise
where it is so I can bump it up please? All i am trying to do is have a
textbox with area 20x170.
Cheers
Dennis
10 years, 2 months
Setup failed due to simplejson version
by simon andrews (BI)
I've just tried an initial installation of Galaxy (galaxy_dist) on a Fedora 13 machine. When I try to run the setup.sh script I get the error below:
Creating tool-data/shared/jars
Traceback (most recent call last):
File "./scripts/fetch_eggs.py", line 31, in <module>
c.resolve() # Only fetch eggs required by the config
File "/data/private/galaxy/galaxy_dist/lib/galaxy/eggs/__init__.py", line 284, in resolve
egg.resolve()
File "/data/private/galaxy/galaxy_dist/lib/galaxy/eggs/__init__.py", line 172, in resolve
r = pkg_resources.working_set.resolve( ( self.distribution.as_requirement(), ), env, self.fetch )
File "/data/private/galaxy/galaxy_dist/lib/pkg_resources.py", line 565, in res olve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: simplejson==1.5
Fedora 13 comes with simplejson v2.1.1. Is there really a strict requirement on v1.5 or should the test be >=1.5? Is there a way to parallel install different versions of this package if 1.5 is strictly required?
Thanks
Simon.
10 years, 2 months
LSF/DRMAA Support
by Nate Coraor
Hello,
Support for the LSF distributed resource manager (DRM) has recently been
committed to our development repository in changeset 4071:fd9514c5349f.
This support has actually been implemented through the use of the
python-drmaa module and none of the code is DRM-specific. This means
that any DRM which implements a DRMAA library should be able to be
supported now (including PBS Pro). For more information on how to
configure a DRMAA library to work with your DRM, please see:
http://www.drmaa.org/
The code is heavily based on the former SGE job runner, which will soon
be deprecated in favor of the new DRMAA job runner. Thanks to John
Brunelle for adapting and testing the original implementation. For
information on how to set up Galaxy to use the DRMAA job runner, please see:
http://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster
This changeset is not yet available in our stable distribution
repository, although it should be soon. In the meantime, you can grab
it from our development repository if you'd like:
http://bitbucket.org/galaxy/galaxy-central/
This code has had minimal testing and should not be used for a
production server. But please do test it out and report any bugs
(either here or in our issue tracker in Bitbucket).
Thanks!
--nate
10 years, 2 months
