July 2014 - galaxy-dev - lists.galaxyproject.org

Jobs stay queued
by julie dubois 28 Jul '14

28 Jul '14

Hi all, I've seen several posts for jobs which stay queued and never start with an installation of galaxy on a cluster. But our installation is just a local instance of galaxy in 1 PC and I have more than 100 jobs which stay queued and never start. How can I debug this problem (my version of galaxy is the latest 2june2014). And is it possible to run those jobs without restart galaxy because it would be very long to rerun each job manually after reloading galaxy. Thanks. Julie

3 7

tool xml substitutes special characters in text parameter
by Wolfgang Maier 28 Jul '14

28 Jul '14

Dear all, I noticed that with params of type "text" Galaxy seems to replace certain characters before passing them to the shell. As examples, it changes "@" to "__at__", "}" to "__cc__" and "\" to "X". Is this the standard behavior or am I doing something wrong ? And if it's standard, are there workarounds ? Best, Wolfgang

2 4

Menu label in toolshed
by Yec'han Laizet 28 Jul '14

28 Jul '14

Hi, I use labels as subsection separators in my too_conf.xml. Now, I use a local tooshed and I wonder if there is a way to declare such a labels for tools installed from my toolshed sot that the tool will be in the selected section with a label as subsection separator? Thanks. Yec'han ================================================ Dr. Yec'han LAIZET Ingenieur Bioinformatique Tel: +33 (0)5 57 12 27 75 _________________________________ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS ================================================

1 0

Error Setting Up Local Galaxy Server
by Kevin 26 Jul '14

26 Jul '14

To Whom It May Concern, First, my apologizes if this email is sent to the incorrect location for Galaxy suppose. I am only trying to find a solution to a problem I have setting up a local Galaxy instance to use for my project. Because my computer runs Windows 8 and cannot set up a local Galaxy instance, I am using Oracle VM Virtualbox to run an instance with Linux. I have already downloaded the files for Galaxy and the proper versions of Python. However, when ever I try to enter "$ sh run.sh" in terminal, the following error message appears: Traceback (most recent call last): File "./scripts/check_python.py", line 16, in check_python assert sys.version_info[:2] >= ( 2, 6 ) and sys.version_info[:2] <= ( 2, 7 ) AssertionError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./scripts/check_python.py", line 24, in <module> check_python() File "./scripts/check_python.py", line 18, in check_python print >>sys.stderr, msg TypeError: unsupported operand type(s) for >>: 'builtin_function_or_method' and '_io.TextIOWrapper' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./scripts/check_python.py", line 25, in <module> except StandardError: NameError: name 'StandardError' is not defined I do not know what to do with this error message, and require help getting around it so that I can start an instance on my computer. Any advice on how to deal with this problem would be much appreciated. Thank you for taking the time to read this email, Kevin Chang

2 1

P- Galaxy error
by lakshmi krishnakumaar 26 Jul '14

26 Jul '14

Dear sir I was trying to use the DDBJ pipeline for the high level analysis I was trying to use P-Galaxy but I am not able to upload the data from DDBJ pipeline -- With Regards Lakshmi.K M.S(By Research) Anna University Chennai- 44

1 0

sqlite galaxy datatype metadata
by Jim Johnson 25 Jul '14

25 Jul '14

Ira, Thanks for getting a sqlite datatype into galaxy. I was wanting to subclass a sqlite datatype for another application: cistrome CEAS I had made a sqlite datatype that captured a little metadata that I thought could be useful for the display peek, and perhaps for filtering. Is this worth considering? The peek for a CEAS DB would include table names, column names, and row count: SQLite Database GeneTable (chrom,name,strand,txStart,txEnd,cdsStart,cdsEnd,exonCount,exonStarts,exonEnds,name2) [24892] GenomeBGP (chrom,promoter,bipromoter,downstream,gene,rel_loc,rel_loc_cds,roi,chroms) [7] GenomeBGS (chrom,promoter,bipromoter,downstream,gene,rel_loc,rel_loc_cds,roi,Ns) [7] GenomePieP (chrom,promoter,downstream,gene,enhancer,chroms) [7] GenomePieS (chrom,promoter,downstream,gene,enhancer,total) [7] $ hg diff /Users/jj/gxt/gxt/lib/galaxy/datatypes/binary.py diff -r f002131cb905 lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py Fri Jul 25 12:01:34 2014 -0400 +++ b/lib/galaxy/datatypes/binary.py Fri Jul 25 12:13:05 2014 -0500 @@ -20,7 +20,7 @@ from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, TWOBIT_MAGIC_NUMBER_SWAP, TWOBIT_MAGIC_SIZE -from galaxy.datatypes.metadata import MetadataElement +from galaxy.datatypes.metadata import MetadataElement,ListParameter,DictParameter from galaxy.datatypes import metadata from galaxy.datatypes.sniff import * import dataproviders @@ -550,8 +550,36 @@ @dataproviders.decorators.has_dataproviders class SQlite ( Binary ): + """Class describing a Sqlite database """ + MetadataElement( name="tables", default=[], param=ListParameter, desc="Database Tables", readonly=True, visible=True, no_value=[] ) + MetadataElement( name="table_columns", default={}, param=DictParameter, desc="Database Table Columns", readonly=True, visible=True, no_value={} ) + MetadataElement( name="table_row_count", default={}, param=DictParameter, desc="Database Table Row Count", readonly=True, visible=True, no_value={} ) file_ext = "sqlite" + def init_meta( self, dataset, copy_from=None ): + Binary.init_meta( self, dataset, copy_from=copy_from ) + + def set_meta( self, dataset, overwrite = True, **kwd ): + try: + tables = [] + columns = dict() + rowcounts = dict() + conn = sqlite3.connect(dataset.file_name) + c = conn.cursor() + tables_query = "SELECT name,sql FROM sqlite_master WHERE type='table' ORDER BY name" + rslt = c.execute(tables_query).fetchall(); + for table,sql in rslt: + tables.append(table) + columns[table] = re.sub('^.*$(.*)$$','\\1',sql).split(',') + for table in tables: + row_query = "SELECT count(*) FROM %s" % table + rowcounts[table] = c.execute(row_query).fetchone()[0]; + dataset.metadata.tables = tables + dataset.metadata.table_columns = columns + dataset.metadata.table_row_count = rowcounts + except Exception, exc: + pass + # Connects and runs a query that should work on any real database # If the file is not sqlite, an exception will be thrown and the sniffer will return false def sniff( self, filename ): @@ -567,7 +595,14 @@ def set_peek( self, dataset, is_multi_byte=False ): if not dataset.dataset.purged: - dataset.peek = "SQLite Database" + lines = ['SQLite Database'] + if dataset.metadata.tables: + for table in dataset.metadata.tables: + try: + lines.append('%s (%s) [%s]' % (table,','.join(dataset.metadata.table_columns.get(table,[])),dataset.metadata.table_row_count[table])) + except: + continue + dataset.peek = '\n'.join(lines) dataset.blurb = data.nice_size( dataset.get_size() ) else: dataset.peek = 'file does not exist' Thanks, JJ -- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota

2 1

Send me your Visualizations
by sam guerler 25 Jul '14

25 Jul '14

Hello everyone, We are working on a central wiki page to create an overview for all custom visualizations. This includes visualizations using the visualization framework, but also visualizations created by custom tools. Please get back to me and provide information on any visualization which is not part of the Galaxy core package yet. You may send screenshots, slides, links, descriptions, tutorials etc. We will sort out and structure the information on the wiki page. Thanks, Sam

1 0

Run Jobs as Real User - How to Configure it for TORQUE
by Ping Luo 25 Jul '14

25 Jul '14

I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE? thank, Ping

3 4

Local deployments of galaxy
by Lapointe, David 25 Jul '14

25 Jul '14

Not exactly a dev question, but I was asked recently how many sites are running galaxy. The wiki page listing deployments seems not to reflect the actual number, since may are most likely private. Is there an estimate? -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu

3 2

handlers
by Shrum, Donald C 24 Jul '14

24 Jul '14

I could use a little bit of help in making some changes to our galaxy server. I'm in the process of setting up/testing a production galaxy server for our research computing center. Our server is setup with an apache proxy, ldap authentication and jobs will run as the logged in user. The server running galaxy has 24 cores. While most of the jobs submitted will go to either our HPC or the condor cluster there are some jobs that are small and should run on the galaxy server itself. I was planning to set up a single web handler and 23 job handlers. I don't expect the web server to get bogged down, especially since I have apache serving as a proxy. I expect a smaller (<100) number of users submitting many jobs. I just went with 23 handlers for no good reason other than the server has 24 cores. Perhaps there is a better way to discern the optimum number of job handlers. I'd like jobs submitted to galaxy to go either to our HPC, Condor, or one of the 23 local workers. Can galaxy effectively load balance itself in this way? Does the configuration below accomplish this? universe_wsgi.ini: [server:handler1] use = egg:Paste#http port = 8081 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 . . [server:handler23] job_conf.xml: <job_conf> <plugins workers="4"> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" /> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" /> </plugins> <handlers> <handler id="handler1"/> ... <handler id="handler23"/> </handlers> <destinations default="dynamic"> <destination id="local" runner="local" tags="head"/> <destination id="dynamic" runner="dynamic"> <param id="function">default</param> <param id="type">python</param> </destination> <destination id="moab" runner="drmaa" /> <destination id="local" runner="local" /> </destinations> In my destinations.py script I point tools to the appropriate destination: if tool_id.startswith('upload1'): return JobDestination(id="local", runner="local")

3 2