February 2011 - galaxy-dev - lists.galaxyproject.org

How can I execute one module with different input parameters in several nodes ?
by 조수안 07 Feb '11

07 Feb '11

Hi, How can I execute a module with different input parameters in several nodes ? For example, there is a module that calculates chromosome length, and a input file that contains 25 chromosome sequences. When I execute the module for the input file, I want to execute the module in 25 nodes with each chromosome sequence. In this situation, when I make a workflow, can I place the module once in the workflow ? not placing the module 25 times. Thanks Suan cho

2 1

Barcode Splitter - added param to add selected barcode splits as datasets to history
by Jim Johnson 07 Feb '11

07 Feb '11

One of our researchers asked how to move result files from "Barcode Splitter" html links into his history. I thought this should be able to be done directly using dynamic multiple outputs: ( https://bitbucket.org/galaxy/galaxy-central/wiki/ToolsMultipleOutput ). In my local copy of "Barcode Splitter" ( tools/fastx_toolkit/fastx_barcode_splitter.xml ), I've added a param to select barcodes for which to create new datasets in the history: <param name="barcodes" type="select" multiple="true" label="Select barcodes to add as new datasets to history"> <options from_dataset="BARCODE"> <column name="name" index="0"/> <column name="value" index="0"/> <filter type="unique_value" name="unq_bc" column="0" /> <filter type="add_value" name="unmatched" value="unmatched"/> </options> </param> There is a new script to handle the new param and create new datasets: tools/fastx_toolkit/fastx_barcode_splitter_galaxy_wrapper.py Would you be interested in those changes for the main codebase? Jim Johnson University of Minnesota Supercomputing Institute 117 Pleasant St. SE Minneapolis, MN 55455 Email: jj(a)umn.edu

1 0

One tool per file format, or one flexible tool?
by Peter Cock 07 Feb '11

07 Feb '11

In my earlier email about Barcode/adapter/primer trimming/filtering of sequences/reads I wrote: > ... > > Currently I have used three separate scripts, with three separate > XML files - one for each supported file type (FASTA, FASTQ, SFF). > They all have the same interface, so could be done as a single > XML wrapper. The only potential downside to that is that as written > the SFF script requires Biopython, while the FASTA and FASTQ > scripts currently use the Galaxy libraries instead. This external > dependency may be an issue if Galaxy were interested in > including this tool in the main distribution - or if I bundled this > as a single tool or tool-suite on the Tool Shed. > > Any thoughts? It occurred to me the same applies to three of my existing tools in the Tool Shed: * Filter FASTA by ID (uses Galaxy libraries) * Filter FASTQ by ID (uses Galaxy libraries) * Filter SFF by ID (uses Biopython) They could be replaced by a single meta-tool supporting all these file formats (and potentially more), say: * Filter sequences by ID On the one hand this modularity is useful if you don't want to install all the dependencies and only need some of the file formats, but they are conceptually doing the same task so it just clutters the Galaxy tools menu even more if they get three separate entries. Assuming I did go ahead and merge the tools, does the Tool Shed have any "obsoleted" or "deprecated" status, whereby people could still download it (e.g. if it is being used in a published workflow, not an issue here I don't think), but make it clear that another tool is recommended in its place? I could of course just update the description to say this in words. Peter

2 1

Migration Bug and Fix; Additional Questions
by John David Osborne 04 Feb '11

04 Feb '11

I ran into the error shown at the end of this email today. It looks like someone is trying to rename a sequence generator but using the wrong syntax for postogres (at least for my 8.1.2 version). Anyways, I worked around the problem by switching lib/galaxy/model/migrate/versions/0068_rename_sequencer_to_external_services.py to use "ATLER TABLE" instead of "ALTER SEQUENCE" and I migrated successfully. I'm a little confused why I was asked to migrate, as I wiped out my galaxy installation including my postgres database and ran it with a new (2.6.6) version of python. Also, I can't seem to kill the galaxy process with the usual control-c (used to be able to do this) but I have to explicitly kill it from another shell. I also can't get the stop-daemon thing to work from run.sh, I get "No PID file exists in paster.pid". Any thoughts on this? -John galaxy.model.migrate.check INFO 2011-02-03 17:15:19,544 The 'sequencer_type_id' column is renamed to 'external_service_type_id' in the renamed galaxy.model.migrate.check INFO 2011-02-03 17:15:19,544 table 'external_service'. Finally, adds a foreign key to the external_service table in the galaxy.model.migrate.check INFO 2011-02-03 17:15:19,544 sample_dataset table and populates it. galaxy.model.migrate.check INFO 2011-02-03 17:15:19,545 galaxy.model.migrate.check INFO 2011-02-03 17:15:19,545 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/home/galaxy/galaxy_dist/lib/galaxy/app.py", line 32, in __init__ create_or_verify_database( db_url, self.config.database_engine_options ) File "/home/galaxy/galaxy_dist/lib/galaxy/model/migrate/check.py", line 67, in create_or_verify_database migrate_to_current_version( engine, db_schema ) File "/home/galaxy/galaxy_dist/lib/galaxy/model/migrate/check.py", line 122, in migrate_to_current_version schema.runchange( ver, change, changeset.step ) File "/home/galaxy/galaxy_dist/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py", line 184, in runchange change.run(self.engine, step) File "/home/galaxy/galaxy_dist/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 101, in run func() File "lib/galaxy/model/migrate/versions/0068_rename_sequencer_to_external_services.py", line 106, in upgrade db_session.execute( cmd ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 755, in execute clause, params or {}) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) syntax error at or near "RENAME" LINE 1: ALTER SEQUENCE sequencer_id_seq RENAME TO external_service_i... ^ 'ALTER SEQUENCE sequencer_id_seq RENAME TO external_service_id_seq' {}

1 0

Installing reference genomes
by Geoff Jentry 03 Feb '11

03 Feb '11

Hello ... I've installed a Galaxy server on one of our local machines (X86_64 linux running CentOS 5). When I go to modules that have a "Select a reference genome:" option, the dropboxes are empty. I did find this thread: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-August/003138.html But it's still not making sense to me. To test things out, I've downloaded the chr*.fa.gz files for hg18 from UCSC. Based on the advice in this thread, I've used faToTwoBit on each of the .fa files. These files are all sitting in /home/galaxy/local/galaxy/genomes/hg18/ This poster says to add the location to tool-data/alignseq.loc, but this file's comments says that it uses axt files for alignment data and nib files for seqs - these are clearly neither. I added the following entry to my faseq.loc file: hg18 /home/galaxy/local/genomes/hg18 and to my twobit.loc: hg18 /home/galaxy/local/galaxy/genomes/hg18 But even after restarting the server hg18 isn't showing up in the reference genome. Clearly I'm doing something very wrong here, can anyone help me out here? Thanks -J

2 4

Tracking tool usage
by Peter Cock 03 Feb '11

03 Feb '11

Hi all, In brainstorming here about how to organise the tools menu in our local Galaxy, we considered an automatically populated list of the user's frequently or recently used tools. Unlike a favourites system, this doesn't require explicit action from the user. That brought us to another thought - does Galaxy track how often different tools are used? Even if that hasn't been done till now, it should be possible to trawl the database of all non-purged histories and tally up usage counts. That information would be helpful for many decisions - including optimising the listing for easy of use, or making sure the most used tools all have excellent documentation. If this could also count the run time as well as run count, it would be particularly helpful for identifying where most of the server time is spent, so which tools which would most benefit from optimisation or different hardware upgrades. Do you already do this internally for the main public Galaxy interface? Thanks, Peter

2 5

Fwd: [galaxy-user] listing attributes of data input
by Peter 03 Feb '11

03 Feb '11

I realise now this question was also posted to the dev list, which is probably a better place to discuss it. ---------- Forwarded message ---------- From: Peter <peter(a)maubp.freeserve.co.uk> Date: Thu, Feb 3, 2011 at 11:05 AM Subject: Re: [galaxy-user] listing attributes of data input To: Freddy de Bree <freddy.debree(a)wur.nl> Cc: galaxy-user(a)lists.bx.psu.edu On Thu, Feb 3, 2011 at 8:40 AM, Freddy de Bree <freddy.debree(a)wur.nl> wrote: > Dear all, > > I was wondering if there is a place within the Galaxy content (a config, an > xml file) > that gives me some handles on how to address attributes of data input. > > For example, I can get the 'file_name' of data input, by addressing this > attr > as 'input.file_name', and for example, the original name of the data file > can be addressed with 'input.name'. > > Anyone any clues where I can find this info? > > Freddy de Bree Some are given in examples on the main tool XML doc page, https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax Others I've noticed by looking at the provided XML wrappers, and/or email list questions. For example, .ext or .extension gives the Galaxy file type (e.g. fasta). Other than that, I guess you can always read the code - but I agree that a document describing this would be nice to have. Peter

2 1

listing attributes of data input
by Freddy de Bree 03 Feb '11

03 Feb '11

Dear all, I was wondering if there is a place within the Galaxy content (a config, an xml file) that gives me some handles on how to address attributes of data input. For example, I can get the 'file_name' of data input, by addressing this attr as 'input.file_name', and for example, the original name of the data file can be addressed with 'input.name'. Anyone any clues where I can find this info? Freddy de Bree

1 0

Problem upgrading DB from 72 to 73 (SQLite)
by Peter Cock 02 Feb '11

02 Feb '11

Should I file a bug report? $ sh manage_db.sh upgrade 72 -> 73... Migration script to add 'ldda_parent_id' column to the implicitly_converted_dataset_association table. Adding ldda_parent_id column to implicitly_converted_dataset_association table failed: SQLite does not support ALTER TABLE ADD FOREIGN KEY; see http://www.sqlite.org/lang_altertable.html done

4 7

Galaxy Admins - viewing user histories
by Peter 02 Feb '11

02 Feb '11

Hi all, I'm looking at the Galaxy Admin page, and I can see (and terminate) running jobs (via: Server, Manage jobs) which is useful. However, right now I would like to be able to check the status of a recently finished job (did it work or fail?). I can probably do this by (manually) querying the database, but it would be much nicer to be able to use the Admin webinterface to view user histories. Is this possible, and if not do you think it is it desirable? Thanks, Peter

2 1