- galaxy-dev - lists.galaxyproject.org

More on variable multiple output files: identification
by Al Simons 22 Oct '10

22 Oct '10

Thanks to the list's help, I have my multiple output files. The next problem is that the displayed descriptions all have the identical text as the "master" (first output) file. I need to know which is which. The data in each file that my tool is indistinguishable, so I can't label the data internally. I need to precisely track which resultant dataset corresponds to which of the program's output streams. I can't even dead-reckon, because the datasets appear to be randomly ordered with respect to either the file's creation time or its original name. It would be _really_ nice if the output name extracted from the file name could be included in (or just be) the dataset name displayed in the history. That would give the program control of the displayed name, which would be extremely helpful. I see a tantalizing clue at the bottom of the ToolsMultipleOutput page, to reference "__collected_datasets__['primary'][name]." in a post processing hook. However, I can't find any reference to post processing hooks in the wiki or on Google. I'd welcome a pointer to any documentation about post-processing. Is there a documentation section that I'm overlooking? Thanks for the support, -Al

3 5

integer or float parameters - regression on trunk?
by Peter 22 Oct '10

22 Oct '10

Hi all, I've found what I consider to be a regression from Daniel Blankenberg's change: > http://bitbucket.org/galaxy/galaxy-central/changeset/1f4d6b25a84e > > Fix for Integer/FloatToolParameter.get_html_field() when 'value' is provided > as an integer/float. Fixes an issue seen when saving workflows: If an integer > or float tool parameter is changed to a value of 0 or 0.0 and saved, the form > field would be redisplayed using the default tool value; and not the value that > is now saved in the database. If have a tool parameter like this: <param name="dummy" type="integer" value="" label="Test integer with empty default"/> Prior to Dan's change, the HTML would show with the value as specified (empty string). After Dan's change, the HTML page for the tool loads with "0" shown, which is not what I wanted (we can get that with value="0"). I think the old functionality is more useful - couple it with a validator and this forces the user to pick a value rather than always having a valid default. However, what I really want to include optional="true" value="" and allow the value to be an integer or the empty string. In this use-case the HTML should start with the given default shown (empty string as value). See also: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-October/003455.html Does that make sense? Regards, Peter

2 3

Disabling debug mode causes Runtime error
by Isabelle Phan 22 Oct '10

22 Oct '10

> http://gmod.827538.n3.nabble.com/Possible-Bug-tt969896.html are there plans to address this issue or is this not affecting others besides Michael and myself? This is currently a complete blocker for me: - downloading large files from Galaxy fails with an out of memory error - trying to fix that by setting debug = False causes galaxy to be unusable. The application starts up, but the History panel shows 'internal server error' and the app throws a RuntimeError: Traceback (most recent call last): File "/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 1053, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__ self.handle() File "/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 432, in handle BaseHTTPRequestHandler.handle(self) File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle self.handle_one_request() File "/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 427, in handle_one_request self.wsgi_execute() File "/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 290, in wsgi_execute self.wsgi_write_chunk(chunk) File "/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 125, in wsgi_write_chunk "Content returned before start_response called") RuntimeError: Content returned before start_response called I did an update of galaxy, migrated the database to version 58, and am still unable to switch off debugging. Galaxy is not running behind apache at present. Without the ability to download large-ish files, I am losing support to have Galaxy in production *sob*. Any help greatly appreciated. Isabelle

3 4

Multiple output, not known until tool run
by Al Simons 21 Oct '10

21 Oct '10

I'm integrating a tool which will have multiple output datasets, the number of which cannot be known until runtime. I'm able to create the files, but they don't appear in the history pane; as you can see in the copy of my xml below, I have set force_history_refresh="True". They are actually created and reside in the dataset directory alongside the primary output file. I've cloned the sample file from the wiki page: http://bitbucket.org/galaxy/galaxy-central/wiki/ToolsMultipleOutput, in the section Number of Output datasets cannot be determined until tool run. I've found similar queries using a search engine, but the responses refer the user to the sample xml on that page. Here is my adaptation of the sample xml: <tool id="multi_output_aks1" name="Multiple output" force_history_refresh="True"> <description>example</description> <command interpreter="/usr/bin/ruby">aks_multiple.rb $input1 $output1 $output1.id $__new_file_path__</command> <inputs> <param format="text" name="input1" type="data" label="My file"/> </inputs> <outputs> <data format="text" name="output1" label="Main output for ${input1}" /> </outputs> </tool> Did I miss something in my adaptation? Is the wiki sample wrong? Is Galaxy broken? (Downloaded yesterday--How does one tell the Galaxy version?) Thanks for any advice. -Al

2 2

Multi-match (--max) output file from Bowtie
by Al Simons 21 Oct '10

21 Oct '10

I'm a new Galaxy tool integrator. I'm integrating a tool that requires the secondary Bowtie output file of the reads it couldn't map due to there being too many matching locations (greater than the value specified by the -m option). This file is specified in Bowtie with the --max <filename> option. I don't see a way to request this additional output with the current Bowtie jacket and xml. Has anyone already modified the jacket to do this? I'm about to start delving into the Bowtie jacket and xml. Thanks for any pointers -Al

2 1

Multiple Output Issues
by SHAUN WEBB 21 Oct '10

21 Oct '10

Hi, I have been developing a few tools where the number of output files cannot be determined before the tool is run and I have been following the protocol on the wiki which works well. I do have a couple of issues: 1. Naming files - The returned history items are all returned with the same label, is there anyway to edit this? Perhaps I missed something simple or do I need to add a post-processing step somehow? 2. When I have several additional outputs or large output files these take longer to find their way in to the history and are often missed by the first automatic refresh (force_history_refresh=true). If I wait a few seconds and refresh again the extra history items appear. Would it be possible to check that all outputs are loaded before refreshing or to wait say 5s after the tools has completed running before refreshing? Any help with this is appreciated. Thanks Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Inquiry about Galaxy Log Information
by Xiandong Meng 21 Oct '10

21 Oct '10

Hello, I work for Berkeley Lab. Thanks for developing galaxy tools. I have installed the galaxy on our server and developed our own tools to run through the galaxy web interface. I was wondering if there are any ways to track the galaxy usage on certain tools, e.g., the number of successful submission for a particular tool. I am able to view the total number of datasets, but I am not sure how many jobs have been submitted to our system, since one job may generate multiple datasets. I don't know if there is a mechanism to monitor the system usage. In brief, I would like to check the how users use our tools, the number of job submissions and total CPU/wall time if possible. Thanks, Xiandong Meng Lawrence Berkeley National Laboratory

2 1

Re: [galaxy-dev] running galaxy jobs on pbs pro cluster
by Nate Coraor 21 Oct '10

21 Oct '10

Hi Laure, I just noticed this was posted on our user list, while it's probably more appropriate for the dev list. So I've moved the conversation there. Please consider subscribing, if you haven't already. The failure to find PBS is further up in the log, since this is a failure to find the TORQUE library, and you're trying to build against PBS Pro. However as of yesterday there's a new version (1.0.3) out that may solve this problem? You should probably try that and see if it works. --nate Laure QUINTRIC wrote: > > Hi Nate, > > For installing pbs_drmaa package we launch fisrt: > ./configure --prefix=/appli/drmaa/1.0.2 > --with-pbs=/appli/pbs/sles11/exec/10.4.0.101257 > > --with-pbs path we set is pbs pro root directory. > > In config.log file, the error log is the following : > > configure:12206: gcc -o conftest -pedantic -ansi -g -O2 -pthread > -D_REENTRANT -D_THREAD_SAFE -DNDEBUG -D_GNU_SOURCE > -I/appli/pbs/sles11/exec/10.4.0.101 > 257/include -L/appli/pbs/sles11/exec/10.4.0.101257/lib > -Wl,-R/appli/pbs/sles11/exec/10.4.0.101257/lib conftest.c -ltorque > -pthread >&5 > /usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: > cannot find -ltorque > collect2: ld returned 1 exit status > configure:12213: $? = 1 > configure: failed program was: > | /* confdefs.h. */ > | #define PACKAGE_NAME "FedStage DRMAA for PBS Pro" > | #define PACKAGE_TARNAME "pbs_drmaa" > | #define PACKAGE_VERSION "1.0.2" > | #define PACKAGE_STRING "FedStage DRMAA for PBS Pro 1.0.2" > | #define PACKAGE_BUGREPORT "mamonski(a)man.poznan.pl" > | #define PACKAGE "pbs_drmaa" > | #define VERSION "1.0.2" > | #define STDC_HEADERS 1 > | #define HAVE_SYS_TYPES_H 1 > | #define HAVE_SYS_STAT_H 1 > | #define HAVE_STDLIB_H 1 > | #define HAVE_STRING_H 1 > | #define HAVE_MEMORY_H 1 > | #define HAVE_STRINGS_H 1 > | #define HAVE_INTTYPES_H 1 > | #define HAVE_STDINT_H 1 > | #define HAVE_UNISTD_H 1 > | #define HAVE_DLFCN_H 1 > | #define LT_OBJDIR ".libs/" > | /* end confdefs.h. */ > | > | /* Override any GCC internal prototype to avoid an error. > | Use char because int might match the return type of a GCC > | builtin and then its argument prototype would still apply. */ > | #ifdef __cplusplus > | extern "C" > | #endif > | char pbse_to_txt (); > | int > | main () > | { > | return pbse_to_txt (); > | ; > | return 0; > | } > configure:12234: result: no > configure:12413: error: > PBS libraries/headers not found; > add --with-pbs to set PBS installation root. > > Headers and libraries are not found although there are located in > the path we set in --with-pbs option : > > service>108% ls /appli/pbs/sles11/exec/10.4.0.101257 > bin etc include lib man python sbin tcltk unsupported > > service>109% ls /appli/pbs/sles11/exec/10.4.0.101257/include/ > pbs_error.h pbs_ifl.h rm.h tm.h tm_.h > > service>110% ls /appli/pbs/sles11/exec/10.4.0.101257/lib > MPI ical init.d libattr.a liblog.a libnet.a libpbs.a > libsite.a pbs_sched.a pm python xpbs xpbsmon > > it seems we are not the only one having this issue with pbs pro. Do > you have any idea ? > > Thanks, > Laure > > >

2 2

Migrating data to a new installation
by Chris Zaleski 20 Oct '10

20 Oct '10

Greetings, I hope you can help with a question. I have a local installation of Galaxy we've been using for production runs with a certain project. In order to maintain consistency, we haven't done any upgrades in about a year. We recently installed a fresh up-to-date Galaxy on another server. Now that those production runs are almost complete, we'd like to migrate the data/files/histories from the old server to the new one. All the configuration between the old & new servers is exactly the same (same DB, same login criteria, same directory structures, etc). Also there is no data on the new server - it's perfectly fine to clobber it. I intended to try the following procedure: 1) Copy the file repository from old to new. 2) Dump & import the database from old to new. 3) Upgrade the database schema on the new server. However the upgrade procedure is failing with no status. As expected, the log file reports the following: "Exception: Your database has version '43' but this code expects version '57'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'." But the manage_db.sh script simply does nothing but run & exit. The database does not get upgraded & there's no output from the script. I tried the "verbose", "debug", and "force" flags, but nothing happens. Should this work? If so, do you have any suggestions how to get the DB upgraded? If my migration procedure is incorrect, how should I go about it? Thanks very much, Chris Zaleski

2 2

Re: [galaxy-dev] Dynamic output format
by Bossers, Alex 20 Oct '10

20 Oct '10

I think you can only use the metadata_source if the param outputFormat would be of type data and get an actual file. You also don't have INPUT defined. So for your situation I think a slight alteration in the options as written by Kelly should work; See the * marked lines that changed. There are more options to do it but the FILTER is most simple. Needless to say you also need the upper id and cmd part of the XML to get it working.... Alex <inputs> * <param name="input" type="data" format="fasta" label="input FastA" /> <param name="outputFormat" type="select" label="Select output format"> <option value="fasta" selected="true">Fasta</option> <option value="table">Table</option> </param> </inputs> <outputs> * <data name="output" format="fasta" > * <filter>outputFormat=="fasta"</filter> * </data> * <data name="output" format="fasta" > * <filter>outputFormat=="table" > * </data> </outputs> ________________________________________ Van: Zhe Chen [zhe(a)lanl.gov] Verzonden: dinsdag 19 oktober 2010 18:29 Aan: Bossers, Alex Onderwerp: RE: [galaxy-dev] Dynamic output format I try to change it to <inputs> <param name="outputFormat" type="select" label="Select output format"> <option value="fasta" selected="true">Fasta</option> <option value="table">Table</option> </param> </inputs> <outputs> <data name="output" format="input" metadata_source="outputFormat"/> </outputs> It gives me an error. > Yes, > A simple example is in the filter/sorter.xml and pasteWrapper tool_config. > Basically by the metadata_source in sorter: > > <outputs> > <data format="input" name="out_file1" metadata_source="input"/> > </outputs> > > A format example in the tools/filter/pasteWrapper.xml where you use > conditionals: > > <outputs> > <data format="input" name="out_file1" metadata_source="input1"> > <change_format> > <when input_dataset="input1" attribute="ext" value="bed" > format="interval"/> > </change_format> > </data> > </outputs> > > Hope this helped, > > Alex > > > -----Oorspronkelijk bericht----- > Van: galaxy-dev-bounces(a)lists.bx.psu.edu > [mailto:galaxy-dev-bounces@lists.bx.psu.edu] Namens Zhe Chen > Verzonden: dinsdag 19 oktober 2010 0:48 > Aan: galaxy-dev(a)bx.psu.edu > Onderwerp: [galaxy-dev] Dynamic output format > > Hi, > > Can the format of a tool's "output" be dynamic according to the user's > input? > > See example below, when user chose fasta, let the output format be fasta, > when user choose table, let the output format be tabular. > > <input> > <param name="outputFormat" type="select" label="Select output format"> > <option value="fasta" selected="true">Fasta</option> > <option value="table">Table</option> > <option value="mase">Mase/IG</option> > <option value="pretty">Pretty print</option> > <option value="outali">Output aligned</option> > <option value="clustal">Clustal</option> > <option value="phylipi">Phylip interleaved</option> > <option value="phylips">Phylip sequential</option> > <option value="msf">MSF</option> > <option value="rsf">RSF</option> > <option value="megai">MEGA interleaved</option> > <option value="megas">MEGA sequential</option> > <option value="slx">SLX</option> > <option value="gde">GDE</option> > <option value="GDEFlat">GDEflat</option> > <option value="stockholm">Stockholm</option> > <option value="nexusi">Nexus interleaved</option> > <option value="nexuss">Nexus sequential</option> > <option value="pir">PIR/NBRF</option> > <option value="gcg">GCG</option> > <option value="raw">Raw</option> > </param> > </input> > > <outputs> > <data name="output" format="tabular"/> > </outputs> > > > Thanks > _______________________________________________ > galaxy-dev mailing list > galaxy-dev(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-dev > > >

1 1