February 2011 - galaxy-dev - lists.galaxyproject.org

Configuring a ProFTPd server for Galaxy
by Geoff Jentry 09 Feb '11

09 Feb '11

This is probably more appropriate for the proftpd list at this point, but my thought is that maybe someone has come across this while configuring a proftpd server for galaxy usage. I'm including my current output when I try to log on, a copy of my conf file, and output from the SQL log (in all three cases, with identifying info removed - users, hosts, IPs, etc). You can see from the SQL Log that it appears the log in is actually successful, but then the connection simply closes the connection. Any ideas as to what I've got configured incorrectly here? ####### Output [user@host ftp]$ ftp ftphost Connected to ftphost. 220 ProFTPD 1.3.3d Server (Galaxy FTP) [::ffff:IP] 500 AUTH not understood 500 AUTH not understood KERBEROS_V4 rejected as an authentication type Name (ftphost:user): galaxyUserEmail 331 Password required for galaxyUserEmail Password: 530 Login incorrect. Login failed. 421 Service not available, remote server has closed connection ftp> ################ proftpd.conf ServerName "Galaxy FTP" ServerType standalone DefaultServer on Port 21 Umask 077 DefaultRoot ~ CreateHome on dirmode 700 AllowOverwrite on AllowStoreRestart on SyslogFacility DAEMON SyslogLevel debug MaxInstances 30 User nobody Group nogroup ## SQL stuff <Global> ## mod_sql_passwd stuff SQLPasswordEngine on SQLPasswordEncoding hex SQLConnectInfo user@dbHost galaxy MYPASSWD SQLAuthTypes SHA1 SQLUserInfo custom:/LookupGalaxyUser SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email,password,'712','713','/home/galaxy/galaxy_dist/database/ftp/%U','/bin/bash'FROM galaxy_user WHERE email='%U'" SQLDefaultHomedir "/var/opt/local/proftpd" RequireValidShell off SQLAuthenticate users SQLLogFile /var/opt/local/proftpd/sqlLog.txt </Global> #### sqlLog.txt Feb 09 13:55:54 mod_sql/4.2.5[2938]: defaulting to 'postgres' backend Feb 09 13:55:54 mod_sql/4.2.5[2938]: backend module 'mod_sql_postgres/4.0.4' Feb 09 13:55:54 mod_sql/4.2.5[2938]: backend api 'mod_sql_api_v1' Feb 09 13:55:54 mod_sql/4.2.5[2938]: >>> sql_sess_init Feb 09 13:55:54 mod_sql/4.2.5[2938]: entering postgres cmd_defineconnection Feb 09 13:55:54 mod_sql/4.2.5[2938]: name: 'default' Feb 09 13:55:54 mod_sql/4.2.5[2938]: user: 'galaxy' Feb 09 13:55:54 mod_sql/4.2.5[2938]: host: 'dbHost' Feb 09 13:55:54 mod_sql/4.2.5[2938]: db: 'galaxy' Feb 09 13:55:54 mod_sql/4.2.5[2938]: port: '5432' Feb 09 13:55:54 mod_sql/4.2.5[2938]: ttl: '0' Feb 09 13:55:54 mod_sql/4.2.5[2938]: exiting postgres cmd_defineconnection Feb 09 13:55:54 mod_sql/4.2.5[2938]: mod_sql engine : on Feb 09 13:55:54 mod_sql/4.2.5[2938]: negative_cache : off Feb 09 13:55:54 mod_sql/4.2.5[2938]: authenticate : users Feb 09 13:55:54 mod_sql/4.2.5[2938]: usertable : users Feb 09 13:55:54 mod_sql/4.2.5[2938]: userid field : userid Feb 09 13:55:54 mod_sql/4.2.5[2938]: password field : passwd Feb 09 13:55:54 mod_sql/4.2.5[2938]: UID field : uid Feb 09 13:55:54 mod_sql/4.2.5[2938]: GID field : gid Feb 09 13:55:54 mod_sql/4.2.5[2938]: homedir field : homedir Feb 09 13:55:54 mod_sql/4.2.5[2938]: homedir(default) : '/var/opt/local/proftpd' Feb 09 13:55:54 mod_sql/4.2.5[2938]: shell field : shell Feb 09 13:55:54 mod_sql/4.2.5[2938]: SQLMinUserUID : 999 Feb 09 13:55:54 mod_sql/4.2.5[2938]: SQLMinUserGID : 999 Feb 09 13:55:54 mod_sql/4.2.5[2938]: <<< sql_sess_init Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> sql_pre_pass Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< sql_pre_pass Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: Postgres server version: 8.4.2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' opened Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 1 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache miss for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> sql_lookup Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> process_named_query 'LookupGalaxyUser' Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 1 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 1 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_select Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: query "SELECT email,password,'712','713','/home/galaxy/galaxy_dist/database/ftp/galaxyUserEmail','/bin/bash'FROM galaxy_user WHERE email='galaxyUserEmail'" Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 1 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_select Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< process_named_query 'LookupGalaxyUser' Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< sql_lookup Feb 09 13:55:59 mod_sql/4.2.5[2938]: user UID 712 below SQLMinUserUID 999, using SQLDefaultUID 65533 Feb 09 13:55:59 mod_sql/4.2.5[2938]: user GID 713 below SQLMinUserGID 999, using SQLDefaultGID 65533 Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache miss for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: user 'galaxyUserEmail' cached Feb 09 13:55:59 mod_sql/4.2.5[2938]: + pwd.pw_name : galaxyUserEmail Feb 09 13:55:59 mod_sql/4.2.5[2938]: + pwd.pw_uid : 65533 Feb 09 13:55:59 mod_sql/4.2.5[2938]: + pwd.pw_gid : 65533 Feb 09 13:55:59 mod_sql/4.2.5[2938]: + pwd.pw_dir : /home/galaxy/galaxy_dist/database/ftp/galaxyUserEmail Feb 09 13:55:59 mod_sql/4.2.5[2938]: + pwd.pw_shell : /bin/bash Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> cmd_auth Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 2 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_open Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 1 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_escapestring Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache hit for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> cmd_check Feb 09 13:55:59 mod_sql/4.2.5[2938]: checking password using SQLAuthType 'sha1' Feb 09 13:55:59 mod_sql/4.2.5[2938]: 'sha1' SQLAuthType handler reports success Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache hit for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< cmd_check Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< cmd_auth Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache hit for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: >>> cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: cache hit for user 'galaxyUserEmail' Feb 09 13:55:59 mod_sql/4.2.5[2938]: <<< cmd_getpwnam Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_exit Feb 09 13:55:59 mod_sql/4.2.5[2938]: entering postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' closed Feb 09 13:55:59 mod_sql/4.2.5[2938]: connection 'default' count is now 0 Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_close Feb 09 13:55:59 mod_sql/4.2.5[2938]: exiting postgres cmd_exit

1 1

Barcode/adapter/primer trimming/filtering of sequences/reads
by Peter Cock 09 Feb '11

09 Feb '11

Hi all, I'm currently working with some 454 data where the sample was amplified with selective primers, and therefore the reads need a little processing to remove the primer sequences before assembly or mapping (something that sff_extract cleverly spots and warns the user about when doing an SFF to FASTA/FASTQ conversion). The actual processing I want to do is very similar to spotting and removing barcodes or adapters - except that PRC primers are often degenerate, i.e. have an N in them representing the fact it is a pool of primers covering A, C, G and T at that point, and primers may come in pairs. Looking over the provided tools in Galaxy, the only relevant ones I saw are as follows: emboss_5/emboss_primersearch.xml - the text output does not look helpful for trimming my sequences - nothing else in Galaxy uses this format, does it? fastx_toolkit/fastx_barcode_splitter.xml - copes with 5' or 3' barcodes, but only handles fastqsolexa (discussed recently on the mailing list - I guess it could handle fastqsanger and fastqillumina as well), not FASTA or SFF. Also according to the FASTX docs for fastx_barcode_splitter.pl it require non-ambiguous barcodes (i.e. ACGT only), so using it with ambiguous primers won't work: http://hannonlab.cshl.edu/fastx_toolkit/commandline.html I did look on the tool shed and noticed Edward Kirton has done some wrappers for the "Suite of Newbler tools", but his sfffile wrapper does not (yet) include support for splitting SFF files using Roche's MID barcodes. Are there any other relevant tools I have overlooked? In the meantime I've started Galaxy wrappers for my own Python code to find and remove PCR primers, adapters, or barcodes (basically any short sequences). These can also be used to filter the reads (choose if non-matching reads are kept or not). However this isn't ideal for barcodes where you'd have to run the tool once for each barcode (or set of barcodes) to get them in a separate file. For specifying the barcode/primer/adapter sequences, are FASTA or tabular files more commonly used? Should I look at the Galaxy *.loc system to allow commonly used things the Roche MID barcodes to be predefined at the system level? I am currently taking a FASTA input file for the primers. A simple tabular file with ID and sequence in the first two columns would be easy to add - that is what fastx_toolkit/fastx_barcode_splitter.xml expects. EMBOSS primersearch wants a three column tabular file with ID, forward primer, reverse primer sequence. However, so far I am only looking at single primer analysis (our primers were pooled so I can remove the forward and reverse primers in two steps). Currently I have used three separate scripts, with three separate XML files - one for each supported file type (FASTA, FASTQ, SFF). They all have the same interface, so could be done as a single XML wrapper. The only potential downside to that is that as written the SFF script requires Biopython, while the FASTA and FASTQ scripts currently use the Galaxy libraries instead. This external dependency may be an issue if Galaxy were interested in including this tool in the main distribution - or if I bundled this as a single tool or tool-suite on the Tool Shed. Any thoughts? Regards, Peter

3 8

SLURM queue
by Oscar Eriksson 09 Feb '11

09 Feb '11

Hi, is there anyone here who has any experience running galaxy with the queuing system SLURM? I've tried to install SLURM and a DRMAA module (http://apps.man.poznan.pl/trac/slurm-drmaa) I use the line: $ export DRMAA_LIBRARY_PATH=/path/to/drmaa.so; python scripts/scramble.py DRMAA_python The script compiles the egg nicely. When I try to start galaxy, it seems to be looking for SGE_ROOT though. Any thoughts? Best regards, Oscar Eriksson Log file: galaxy.datatypes.registry DEBUG 2011-02-09 15:42:27,127 Loaded external metadata tool: __SET_METADATA__ galaxy.datatypes.registry DEBUG 2011-02-09 15:42:27,132 Loaded indexer: INDEXER_Coverage_0 galaxy.datatypes.registry DEBUG 2011-02-09 15:42:27,137 Loaded indexer: INDEXER_Coverage_0 galaxy.datatypes.registry DEBUG 2011-02-09 15:42:27,142 Loaded indexer: INDEXER_Interval_0 galaxy.jobs.runners.local INFO 2011-02-09 15:42:27,143 starting workers galaxy.jobs.runners.local DEBUG 2011-02-09 15:42:27,144 5 workers ready Traceback (most recent call last): File "/usr/ebiokit/galaxy/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/ebiokit/galaxy/lib/galaxy/app.py", line 72, in __init__ self.job_manager = jobs.JobManager( self ) File "/usr/ebiokit/galaxy/lib/galaxy/jobs/__init__.py", line 41, in __init__ self.dispatcher = DefaultJobDispatcher( app ) File "/usr/ebiokit/galaxy/lib/galaxy/jobs/__init__.py", line 749, in __init__ self.job_runners[runner_name] = runners.drmaa.DRMAAJobRunner( app ) File "/usr/ebiokit/galaxy/lib/galaxy/jobs/runners/drmaa.py", line 81, in __init__ self.ds.initialize() File "/usr/ebiokit/galaxy/eggs/drmaa-0.4b3-py2.6.egg/drmaa/__init__.py", line 274, in initialize _w.init(contactString) File "/usr/ebiokit/galaxy/eggs/drmaa-0.4b3-py2.6.egg/drmaa/wrappers.py", line 59, in init return _lib.drmaa_init(contact, error_buffer, sizeof(error_buffer)) File "/usr/ebiokit/galaxy/eggs/drmaa-0.4b3-py2.6.egg/drmaa/errors.py", line 90, in error_check raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value)) InternalException: code 1: Please set the environment variable SGE_ROOT.

2 1

Fwd: [galaxy-user] operating on, and getting at, large files in galaxy...
by Kelly Vincent 09 Feb '11

09 Feb '11

(I'm redirecting this to the galaxy-dev list, which is more appropriate since it's about a local installation.) Begin forwarded message: > From: Nick Schurch <N.Schurch(a)dundee.ac.uk> > Date: February 9, 2011 7:17:45 AM EST > To: galaxy-user(a)lists.bx.psu.edu > Subject: [galaxy-user] operating on, and getting at, large files in > galaxy... > > > Hi all, > > I've recently encountered a few problems when trying to use Galaxy > which are really driving me away from using it as a bioinformatics > platform for NGS. I was wonderinf if there are any simple solutions > that I've missed... > > Firstly, It seems that while there are a few solutions for getting > large files (a few GB) into a local install of galaxy without going > through HTTP, many tools that operate on these files produces > multiple, uncompressed large files which quickly eat up the disk > allocation. This is particularly significant in a workflow that has > multiple processing steps which each leave behind a large file. With > no way to compress or archive files produced by intermediate steps > in a workflow, and no desire to delete them since I may need to go > back to them and they can take hours to re-run, the only two > remaining options seem to be to save them and then delete them. > > And this brings me to the second problem. Getting large files out of > Galaxy. The only way to save large files from Galaxy (that I can > see) is the save icon, which downloads the file via http. This take > *ages* for a large file and also causes big headaches for my firefox > browser. I've taken a quick peek at the Galaxy file system to see if > I could just copy a file, but its almost completely indecipherable > if you want to find out what file in the file system corresponds to > a file saved from a tool. Is there some way to get the location of a > particular file on the galaxy file system, that I can just copy? > > -- > Cheers, > > Nick Schurch > > Data Analysis Group (The Barton Group), > School of Life Sciences, > University of Dundee, > Dow St, > Dundee, > DD1 5EH, > Scotland, > UK > > Tel: +44 1382 388707 > Fax: +44 1382 345 893 > > > > -- > Cheers, > > Nick Schurch > > Data Analysis Group (The Barton Group), > School of Life Sciences, > University of Dundee, > Dow St, > Dundee, > DD1 5EH, > Scotland, > UK > > Tel: +44 1382 388707 > Fax: +44 1382 345 893 > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

1 0

Galaxy cluster deplyment: PBS error 15031: Protocol (ASN.1)
by Sonali Amonkar 09 Feb '11

09 Feb '11

Hi, I am attempting to deploy a workflow in Galaxy located in PBS Torque cluster. The PBS manager delegates the workflow tool runs to the nodes and runs successfully. However, sometimes the tool run fails with the error on the GUI as "Failed due to error in cluster". The server log trace displays the error, galaxy.jobs.runners.pbs DEBUG 2011-02-03 05:17:09,880 (151) pbs_submit failed, PBS error 15031: Protocol (ASN.1) error galaxy.jobs DEBUG 2011-02-03 05:17:13,363 job 150 ended galaxy.jobs ERROR 2011-02-03 05:17:15,816 Unable to cleanup job 152 I searched for this error around a bit, but could not find a reason/solution for it. Can anyone point me to a possible cause of this error or if there is come configuration step I am missing here? Any help would be appreciated. Thank you for your time. Regards, Sonali Amonkar DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

1 0

Proposal to add a "format_source" attribute to the ToolConfig output data element
by Jim Johnson 08 Feb '11

08 Feb '11

I would like to propose adding a "format_source" attribute to the tool_config output data element. This would be a backward compatible change. It is analogous to the "metadata_source" attribute. Motivation: It is often convenient to have an output dataset inherit the format of an input dataset. The format="input" convention only works in the limited case when the 'last' input dataset is the single format one wants to reference. Using an explicit attribute "format_source" to pair an output dataset format to an input dataset format allows for multiple use and is not order dependent. While the "change_format" element can do also accomplish this, it requires more code. A "format_source" attribute is simpler and cleaner. Example: <inputs> <param name="fasta_in" type="data" format="fasta,align" optional="true" label="fasta - Fasta Sequences"/> <param name="qfile_in" type="data" format="qual,qual454,qualsolexa,qualillumina,qualsolid" optional="true" label="qfile - Fasta Quality"/> </inputs> <outputs>  <data format_source="fasta_in" metadata_source="fasta_in" name="fasta_out" label="${tool.name} on ${on_string}: pick.${fasta_in.datatype.file_ext}"> <filter>fasta_in != None</filter> </data>  <data format="qual" name="qfile_out" label="${tool.name} on ${on_string}: pick.${fasta_in.datatype.file_ext}"> <filter>qfile_in != None</filter> <change_format> <when input="${qfile_in.datatype.file_ext}" value="qual454" format="qual454" /> <when input="${qfile_in.datatype.file_ext}" value="qualsolexa" format="qualsolexa" /> <when input="${qfile_in.datatype.file_ext}" value="qualillumina" format="qualillumina" /> <when input="${qfile_in.datatype.file_ext}" value="qualsolid" format="qualsolid" /> </change_format> </data> <outputs> Code changes: $ hg diff lib/galaxy/tools/__init__.py diff -r 36c27bfbb7a8 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Tue Feb 08 15:21:19 2011 -0500 +++ b/lib/galaxy/tools/__init__.py Tue Feb 08 15:06:12 2011 -0600 @@ -249,10 +249,11 @@ (format, metadata_source, parent) """ - def __init__( self, name, format=None, metadata_source=None, + def __init__( self, name, format=None, format_source=None, metadata_source=None, parent=None, label=None, filters = None, actions = None ): self.name = name self.format = format + self.format_source = format_source self.metadata_source = metadata_source self.parent = parent self.label = label @@ -558,6 +559,7 @@ for data_elem in out_elem.findall("data"): output = ToolOutput( data_elem.get("name") ) output.format = data_elem.get("format", "data") + output.format_source = data_elem.get("format_source", "") output.change_format = data_elem.findall("change_format") output.metadata_source = data_elem.get("metadata_source", "") output.parent = data_elem.get("parent", None) $ hg diff lib/galaxy/tools/actions/__init__.py diff -r 36c27bfbb7a8 lib/galaxy/tools/actions/__init__.py --- a/lib/galaxy/tools/actions/__init__.py Tue Feb 08 15:21:19 2011 -0500 +++ b/lib/galaxy/tools/actions/__init__.py Tue Feb 08 15:39:21 2011 -0600 @@ -244,6 +244,11 @@ ext = output.format if ext == "input": ext = input_ext + if output.format_source != None and output.format_source in inp_data: + try: + ext = inp_data[output.format_source].ext + except Exception, e: + pass #process change_format tags if output.change_format: if params is None: And the hardest part would be the documentation change to: https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax

1 0

naming tool outputs
by Glen Beane 08 Feb '11

08 Feb '11

I have a tool that takes a pair of files and manipulates them and produces a new pair of files. One output file corresponds to the first input file, the second output file corresponds to the second input file. I would like to label them as such. Right now I get two datasets in the history that look like this: tool_name on data X and data Y I have tried setting the label of each output like this: label="my tool name on ${input1.name}" and label="my tool name on ${input2.name}" but this isn't exactly what I want. What are all of the various attributes of an input parameter that I can reference in an output label? I'd like to also use the id of the dataset in my history (the X and Y in my example above), but I can't seem to figure out how to reference it. -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153

1 0

galaxy-dev on Helix mailing list
by David Hoover 08 Feb '11

08 Feb '11

We've created a Galaxy development mailing list to go along with the Galaxy development site. This mailing list will be used for communicating changes and questions about developing for Galaxy on the Helix Systems. Not everyone is on it, only those who specifically requested for access. If you want to be on the galaxy-dev mailing list, just let me know. As you all are aware, Galaxy is up and running on Helix (http://galaxy.cit.nih.gov) Please log in and try it out. Most of all, please give us feedback about what works, what doesn't, and what could improve. Some of the tools are configured to run on the Biowulf cluster, most run locally. We also have a separate Galaxy development site (http://galaxy-dev.cit.nih.gov) This is mainly for developing new XML interfaces (so-called tools) for other command line applications already installed on Helix and Biowulf. We've added a few people to the galaxy-dev group, which is necessary to make any changes. The Galaxy development site is located in /data/galaxy/dev, and the XML files are in /data/galaxy/dev/tools. Because any changes to the XML files requires the Galaxy server be restarted, I've created a wrapper executable, init_galaxy. After making changes, on Biowulf type: rsh p1286 /data/galaxy/devlog/init_galaxy restart and this will restart the server on the Galaxy development host (p1286). Most importantly, the server logs are kept in /data/galaxy/devlog. These are key to debugging any changes made. The main configuration file (universe_wsgi.ini), the tool configuration file (tool_conf.xml), and the tool data configuration file (tool_data_table_conf.xml) are write accessible to those in the galaxy_dev group, as these will be necessary to change when new applications are added. The development server DOES NOT have access to the internet, so uploading files through HTTP and FTP won't work. Downloading data from UCSC or other external sites will also not work. However, you can copy local files from your /home and /data areas on Helix using the Import Helix file tool. There is a danger of stepping on each other's toes when making changes, so be sure to use this mailing list to let everyone know when you are making changes and restarting the server. Feel free to use this mailing list to ask questions and state what you did. Cheers, David Hoover Helix Systems Staff http://helix.nih.gov

1 0

Galaxy integration with LSF: seg fault
by Marina Gourtovaia 08 Feb '11

08 Feb '11

Hello I've set up Galaxy to use LSF. My first job has failed because Galaxy submitted it to the default queue, which was wrong in my case. However, Galaxy gracefully survived the failure, I was able to get the job number from the console output and figure out what went wrong. Next time I run the Galaxy with the LSB_DEFAULTQUEUE env variable set like this: LSB_DEFAULTQUEUE=test DRMAA_LIBRARY_PATH=/usr/local/lsf/7.0/linux2.6-glibc2.3-x86_64/lib/libdrmaa.so.1.0.4 PATH=/usr/bin:/software/solexa/bin:$PATH sh run.sh The job is submitted to the correct queue and at this point Galaxy fails with this error: run.sh: line 46: 6506 Segmentation fault python ./scripts/paster.py serve universe_wsgi.ini $@ The job successfully completes in its own time. When I try to run Galaxy again I get the following: galaxy.jobs DEBUG 2011-02-02 16:27:32,565 dispatching job 36 to drmaa runner galaxy.jobs INFO 2011-02-02 16:27:32,675 job 36 dispatched galaxy.jobs.runners.drmaa DEBUG 2011-02-02 16:27:33,192 (36) submitting file /nfs/users/nfs_m/mg8/mygalaxy/galaxy-dist/database/pbs/galaxy_36.sh galaxy.jobs.runners.drmaa DEBUG 2011-02-02 16:27:33,192 (36) command is: java -jar /nfs/users/nfs_m/mg8/mygalaxy/galaxy-dist/tool-data/shared/jars/SamToFastq.jar VALIDATION_STRINGENCY=SILENT QUIET=true INPUT=/lustre/scratch103/sanger/mg8/galaxy/datasets/000/dataset_16.dat FASTQ=/lustre/scratch103/sanger/mg8/galaxy/datasets/000/dataset_51.dat SECOND_END_FASTQ=/lustre/scratch103/sanger/mg8/galaxy/datasets/000/dataset_52.dat Job <855341> is submitted to queue <test>. run.sh: line 46: 6506 Segmentation fault python ./scripts/paster.py serve universe_wsgi.ini $@ ie looks like Galaxy is trying to pick up where it has left and fails again. I configured my job runners like this: start_job_runners = drmaa default_cluster_job_runner = drmaa:/// Any suggestions? Regards Marina -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

3 4

Populating tool select menu with database data?
by Leandro Hermida 08 Feb '11

08 Feb '11

Hi, Sorry I tried to search for this and couldn't find an example to this basic question.... where can I find an example on how to populate a tool select menu with database data? best, Leandro

4 6