- galaxy-dev - lists.galaxyproject.org

Workflow Error
by Sumedha Ganjoo 12 Jul '10

12 Jul '10

Hi, I am using a tool that interacts with Web services via Galaxy. Run on its own in the Galaxy GUI, it works fine. But when we run it in a workflow, with other tools it sometimes runs correctly , sometimes gives the following error: OperationalError: (OperationalError) database is locked u'UPDATE job SET update_time=?, command_line=? WHERE job.id = ?' ['2010-07-12 14:10:18.398960', 'python ${installationDirectory}/tools/${............ ..................................................... I was wondering if Galaxy has some timeouts implemented in their workflows because of which if the first step takes a while the second step of the workflow is executed simultaneously? Or any other explanation for such behavior. I would really appreciate a reply as soon as possible. Thanks. Regards, Sumedha -- Sumedha Ganjoo Graduate Assistant, Department Of Computer Science, University Of Georgia, Athens, GA , USA

2 1

TopHat and other tools with too many options
by Assaf Gordon 12 Jul '10

12 Jul '10

Hi all, I'm in the process of adapting TopHat to our needs, and there are just to many options... It's OK if you run it from the command line, but in Galaxy it looks like a big mess. Similar to Bowtie's tool situation, the "common" option are not specific enough (to our needs) and the "full options" mode is too hard to use (which will result in users not using it at all, or using it wrong). I'd like to request/propose a change in the way the GUI is rendered based on the XML tool. Mainly, to create logical parameter "groups": parameters which logically go together, and are related to one another. In the XML file, it could look like: <inputs> <group name="Introns" help="These settings control the intron sensitivity."> <param name="min_intron_length" type="integer" value="70" /> <param name="max_intron_length" type="integer" value="500000" /> </group> <group name="Quality" help="XXXXXX"> <param name="max_multihits" type="integer" value="40" label="Maximum number of alignments to be allowed" /> <param name="junction_filter" type="float" value="0.15" label="Minimum isoform fraction: /> </group> ... </inputs> And in the HTML output, the groups will be visually distinct, with some nice hide/expand javascript trick, see (fake) example here: http://cancan.cshl.edu/labmembers/gordon/files/galaxy_advanced_options.html IMHO, there are couple of advantages in this layout: 1. The "big-picture" of the available settings is immediately visitble (e.g. "introns", "quality", "segments" etc.). 2. Parameters are separated into logical groups, easier to understand what's being changed (as opposed to one very long cryptic list of parameters). 3. Advanced vs. Simple options are clearly marked 4. Since parameter groups can be hidden, when they are expanded they can contain a help paragraph - this is much easier for the user than scrolling up/down to see the help section below (also - the relevant help section now appears right next to the parameters). I guess this is not a trivial change, but without it it will get harder and harder to integrate complex tools. Comments are welcomed, -gordon

2 1

Cleanup hidden datasets cleanup script
by Dennis Gascoigne 11 Jul '10

11 Jul '10

Hi guys; We have taken advantage of your recent excellent additions to the workflow and have a lot of histories where intermediate datasets are hidden. It is becoming quite critical to us to catch these hidden history items in the cleanup scripts and delete them as they consume a lot of room and we cannot identify another way of removing them. The obvious place to remove hidden datasets appears to be with an option in the cleanup procedures -In previous discussions you have mentioned that this is something you propose doing and it would at first seem relatively quick. What is your timing on this, or has it already been done. If it is a while away, is there anything I should consider in writing/amedning a query to roll our own? Cheers Dennis

2 3

Re: [galaxy-dev] [galaxy-bugs] help about compress file
by Jeremy Goecks 09 Jul '10

09 Jul '10

Eric, The galaxy-dev mailing list (cc'd) is a good place to ask your question. This isn't my area of expertise, so hopefully someone else can chime in and help you out. J. On Jul 9, 2010, at 11:12 AM, Eric Aguiar wrote: > Jeremy, > > I followed all steps in the galaxy tutorial, but I didn't have success. > I'm trying to create a datatype for megabase chromatograms (.esd) very similar to the Ab1 ones. > > Here is my configurations. > > > 1 - Creating datatypes in datatypes_conf.xml > > <datatype extension="zip" type="galaxy.datatypes.binary:Esd" mimetype="application/zip" display_in_upload="true"/> > <datatype extension="esd" type="galaxy.datatypes.binary:Esd" mimetype="application/octet-stream" display_in_upload="true"/> > > 2 - Defining types in lib/galaxy/datatypes/binary.py > > class Esd( Binary ): > """Class describing an ab1 binary sequence file""" > file_ext = "esd" > > def set_peek( self, dataset, is_multi_byte=False ): > if not dataset.dataset.purged: > dataset.peek = "Binary chromatograms sequence file" > dataset.blurb = data.nice_size( dataset.get_size() ) > else: > dataset.peek = 'file does not exist' > dataset.blurb = 'file purged from disk' > def display_peek( self, dataset ): > try: > return dataset.peek > except: > return "Binary esd sequence file (%s)" % ( data.nice_size( dataset.get_size() ) ) > > > class Zip( Binary ): > """Class describing a zip archive of binary sequence files""" > file_ext = "zip" > > def set_peek( self, dataset, is_multi_byte=False ): > if not dataset.dataset.purged: > zip_file = zipfile.ZipFile( dataset.file_name, "r" ) > num_files = len( zip_file.namelist() ) > dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) > dataset.blurb = data.nice_size( dataset.get_size() ) > else: > dataset.peek = 'file does not exist' > dataset.blurb = 'file purged from disk' > def display_peek( self, dataset ): > try: > return dataset.peek > except: > return "Binary sequence file archive (%s)" % ( data.nice_size( dataset.get_size() ) ) > def get_mime( self ): > """Returns the mime type of the datatype""" > return 'application/zip' > > > When I'm going to send the file in zip format (.esd files compressed),it shows me the following error: > "An error occurred running this job: Invalid 'File Format' for archive consisting of binary files - use 'Binseq.zip'" > > I tried somethings, but I don't have success. > > Thank you, > > On 07/08/2010 05:50 PM, Jeremy Goecks wrote: >> >>> I would like to know about the use of compressed files (zip format) in get data app. I'm trying to send .esd files compressed but the program shows me the following error message: "The uploaded file contains inappropriate content". >> >> >> Hi Eric, >> >> Galaxy can accept zip files with a single compressed file but does not recognize .esd files. You'll need to convert the esd file into a format that Galaxy recognizes or run your own Galaxy instance and write your own datatype for Galaxy: >> >> http://bitbucket.org/galaxy/galaxy-central/wiki/AddingDatatypes >> >> Thanks, >> J. > > > -- > <eric_vcard.png>

1 0

Adding job
by Filip Balejko 08 Jul '10

08 Jul '10

Hi, I'm working on the project for Google Summer of Code: https://www.nescent.org/wg_phyloinformatics/Phyloinformatics_Summer_of_Code… Demo instance can be found here: http://137.110.191.252:8080/ (under the "Phylogenetics" section) As you can see, detailed results of the SLAC analysis are generated on demand. In this version computation is done when the request is handled (in datamonkey controller). It bypasses the job queue and I guess it might not be considered as the best approach. I was thinking about creating jobs for those on-demand computations. What do you think about this solution? I understand that to accomplish this, I have to create another tool and put my code there. Is it the only way to use the job queue? Is it possible to add a job which isn't displayed in the history box? best regards, Filip Balejko

1 0

Re: [galaxy-dev] Fwd: Enabling User Pages
by Jeremy Goecks 07 Jul '10

07 Jul '10

> Huh. I've updated to the latest galaxy-central, added enable_pages = > True to my universe_wsgi.ini (didn't see a sample in either this file > or the .sample) and restarted Galaxy. I can see the parameter check > in the base_panels template and can access the page list and editor by > manually using the URL, but the link isn't showing up for me under the > User menu. > > Did I miss something? That's weird. The Pages link works fine on my local instance, on our Test server, and on our main server. Are you using remote_user or other upstream config? This could be causing some trouble. The new universe_wsgi.ini.sample does seem to be missing documentation for enable_pages - I'll add it momentarily. Thanks, J.

3 3

SNP effect prediction
by Pablo Cingolani 06 Jul '10

06 Jul '10

I had to predict the effect of a few thousand SNP that we found in our re-sequencing project. I tried using ENSEMBL's tool (http://www.ensembl.org/Homo_sapiens/UserData/UploadVariations) but it was kind of painful (it only admits 700 SNPs, takes a long time, etc.). So I decided to write a java program (snpEff). I noticed that Galaxy does not have a tool for SNP prediction (see http://bitbucket.org/galaxy/galaxy-central/issue/156/synonymous-and-non-syn…) So I integrated snpEff it with Galaxy. Here is a link to a tar file which includes the jar files, databases, configuration files for Galaxy, etc. http://www.mcb.mcgill.ca/~pcingola/snpEff.tgz YOu'll find the databases for hg37 and mm37, but I'll be happy to create and update the tool for the genome of your choice. The program first loads the database for a genome (it takes 20 secs in my computer) and then calculates the effect for all SNPs (I used an Interval-Tree structure, so it's very fast). I hope you find it useful. Yours Pablo Cingolani P.S.: Needless to say, the code is open source (I'll post it on sourceforge as soon as possible). In the mean time you can download it here http://www.mcb.mcgill.ca/~pcingola/SnpEffect_eclipse_project.tgz P.S.2: I've attached an image of a demo run in my Galaxy server

2 2

Data Library Uploading Issue
by juan perin 06 Jul '10

06 Jul '10

In my latest pull from the galaxy update server, something has caused my instance to act up. I'm getting an error when trying to upload data into libraries. This is the text based error: URL: http://variome.chop.edu:8082/library_common/upload_library_dataset File '/home/perin/galaxy_dist2/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/perin/galaxy_dist2/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/home/perin/galaxy_dist2/lib/galaxy/web/controllers/library_common.py', line 978 in upload_library_dataset **kwd ) File '/home/perin/galaxy_dist2/lib/galaxy/web/controllers/library_common.py', line 1158 in upload_dataset message=util.sanitize_text( message ), File '/home/perin/galaxy_dist2/lib/galaxy/util/__init__.py', line 138 in sanitize_text for c in text: TypeError: 'NoneType' object is not iterable Strangely, the data gets queued up anyway, if you go back to the library, and you can even see the first few lines of the file in the preview, but it remains queued and never fully becomes available. Not sure what's going on, so was hoping for any suggestions. Thanks. Juan

2 1

Re: [galaxy-dev] [galaxy-user] Data Libraries Problems
by Greg Von Kuster 06 Jul '10

06 Jul '10

Galaxy works with Python 2.4 or higher ( but not Python 3 ). On Jul 2, 2010, at 3:15 PM, Juan Perin wrote: > No changes at all. I seem to get this with a new installation as well. Both using postgres and mysql. Is there a new python requirement perhaps? I may be running and older version. > > ~ juan > > On Jul 2, 2010, at 14:13, Greg Von Kuster <greg(a)bx.psu.edu> wrote: > >> Hello Juan, >> >> We're not able to reproduce this error. Have you made any changes to the code in your instance? >> >> Greg >> >> On Jul 2, 2010, at 12:16 PM, juan perin wrote: >> >>> In my latest pull from the galaxy update server, something has caused my >>> instance to act up. I'm getting an error when trying to upload data into >>> libraries. This is the text based error: >>> >>> >>> URL: http://variome.chop.edu:8082/library_common/upload_library_dataset >>> File >>> '/home/perin/galaxy_dist2/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', >>> line 364 in respond >>> app_iter = self.application(environ, detect_start_response) >>> File >>> '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', >>> line 98 in __call__ >>> environ, self.app) >>> File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', >>> line 539 in intercept_output >>> app_iter = application(environ, replacement_start_response) >>> File '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', >>> line 80 in __call__ >>> return self.application(environ, start_response) >>> File >>> '/home/perin/galaxy_dist2/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', >>> line 632 in __call__ >>> return self.application(environ, start_response) >>> File '/home/perin/galaxy_dist2/lib/galaxy/web/framework/base.py', line 145 >>> in __call__ >>> body = method( trans, **kwargs ) >>> File >>> '/home/perin/galaxy_dist2/lib/galaxy/web/controllers/library_common.py', >>> line 978 in upload_library_dataset >>> **kwd ) >>> File >>> '/home/perin/galaxy_dist2/lib/galaxy/web/controllers/library_common.py', >>> line 1158 in upload_dataset >>> message=util.sanitize_text( message ), >>> File '/home/perin/galaxy_dist2/lib/galaxy/util/__init__.py', line 138 in >>> sanitize_text >>> for c in text: >>> TypeError: 'NoneType' object is not iterable >>> >>> >>> >>> Strangely, the data gets queued up anyway, if you go back to the library, >>> and you can even see the first few lines of the file in the preview, but it >>> remains queued and never fully becomes available. Not sure what's going on, >>> so was hoping for any suggestions. >>> I just updated to Postgresql as well, but that didn't change the problem. >>> Thanks. >>> >>> Juan >>> _______________________________________________ >>> galaxy-user mailing list >>> galaxy-user(a)lists.bx.psu.edu >>> http://lists.bx.psu.edu/listinfo/galaxy-user >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

3 2

Enabling User Pages
by Jesse Erdmann 06 Jul '10

06 Jul '10

My Google/grep fu is failing me. How do I enable user pages in my local galaxy server? Thanks! -- Jesse Erdmann Bioinformatics Analyst Masonic Cancer Center University of Minnesota jerdmann(a)umn.edu 612-626-3123 jesse(a)jesseerdmann.com Twitter: http://twitter.com/jesseerdmann

3 2