June 2011 - galaxy-dev - lists.galaxyproject.org

suggestions for the SAM-to-BAM tool
by Assaf Gordon 06 Jul '11

06 Jul '11

Hi, Couple of things that can be slightly improved in the SAM-to-BAM tool: 1. "Reference list" is not informative (it's the technical way to say: "list of chromosomes and their sizes based on a FASTA file"). Users do not generally know what "reference list" is. 2. The "Locally Cached" option is not informative (I had to look in the source code to understand what it means). What it should say is something like: "Get list of chromosomes/sizes based on the dataset's organism/database" (could be shorter, but should be friendly enough). 3. There's no option of having the chromosome list in the SAM file header. Some SAM files will contain the header (can even be done in the standard bowtie tool wrapper) - saves the need to specify where to get the "reference list" from. 4. Autodetection in the "set-metadata" step will go a long way here: if the SAM file already have a header, then no need to even ask about it. If it doesn't have a header but have a DBKEY, then we're still OK. If no DBKEY and no header, then complain or ask for a FASTA file from current history. (I realize the implementing this feature is hard and annoying, I don't imply that it's easy to do, just that it's needed). 5. Inside the python script (sam_to_bam.py) there's a comment that says: "for some reason the samtools view command gzips the resulting bam file without warning" . Not sure why one cares about that, but "samtools view -u" will output an uncompressed BAM file. 6. samtools support piping, so a lot of I/O (and some time) can be spared by piping the two commands together: samtools view -u -b -S "INPUT.SAM" | samtools sort - OUTPUT Instead of running two commands and generating a temporary unsorted BAM file. -gordon

2 1

rgWebLogo3 with Probabilities instead of Entropy
by Assaf Gordon 06 Jul '11

06 Jul '11

Hello Ross and Galaxy team, May I suggest this small patch, that enables WebLogo to plot either Entropy bits (the current default) or just nucleotides probabilities ? It uses the standard "-U" parameter of "weblogo3". -gordon

2 2

fasta integration in galaxy
by B M 05 Jul '11

05 Jul '11

Dear All, I have just finished integrating my blast data to my galaxy local instance, and now I would like to add my fasta files. Does anybody know how can I do that ? How should it look in the UI ? Thanks, best regards Ben

2 3

importing annotation with dataset
by Davide Cittaro 01 Jul '11

01 Jul '11

Hi, is it just me or when I try to import a dataset from a library to a history, annotation is not retained (whereas "info" is)? d /* Davide Cittaro, PhD Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

2 1

importing into history from library
by Glen Beane 01 Jul '11

01 Jul '11

We're running the latest galaxy-dist and noticed a problem importing files from libraries into our history. If we click the down arrow to the right of a dataset and choose "Import this dataset into selected histories" it works fine, however if our datasets are in folders within the library and we select multiple datasets and then do "Import selected datasets into histories" nothing happens. If I go to a library with datasets that are not in a folder then selecting multiple works. Has anyone else seen this? -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153

2 1

Using the tool after log in
by shashi shekhar 01 Jul '11

01 Jul '11

Hi All, I have local instance of galaxy . I want to make changes in such a way that after log in, user should able to submit any job or run any tool. Log in should be compulsory to run any tool including upload tool. Whenever someone clicks on tool without log in ,it should display message like "You must be logged in <http://192.168.62.51:8089/user/login?webapp=galaxy> to use Galaxy" . Reagrds shashi shekhar

3 2

public datasets/histories with external auth
by Shantanu Pavgi 01 Jul '11

01 Jul '11

Hi, We have a galaxy server setup using external shibboleth authentication. While we would like to have site behind authentication realm, there are instances when our galaxy datasets/histories need to be accessible publicly from other websites. We tried adding an exception to auth rule for /datasets path using Location directive in apache web server configuration, however galaxy server returned an error as: {{{ Access to Galaxy is denied Galaxy is configured to authenticate users via an external method (such as HTTP authentication in Apache), but a username was not provided by the upstream (proxy) server. This is generally due to a misconfiguration in the upstream server. }}} Is there any way to share public histories and datasets when galaxy is using external authentication mechanism? I have thought about setting up (fake) anonymous REMOTE_USER variable for /datasets path, but not sure whether this is correct approach. Also, would it require any galaxy code changes? Any thoughts? Thanks, Shantanu.

3 14

Re: [galaxy-dev] AttributeError: 'list' object has no attribute 'missing_meta' when doing multiple="true" input datasets
by Kip Bodi 01 Jul '11

01 Jul '11

Hi all, I'm having the same problem. I have a script that tries to take several history data sets as a single input: <command interpreter="perl">aggregate.pl --out $outfile $list_of_items_in_history</command> <inputs> <param format="txt" name="fitness_tables" type="data" multiple="true" label="Fitness score table, per site"/> </inputs> <outputs> <data format="tabular" name="outfile" /> </outputs> If I select only one history item it works, but if I press 'ctrl' to select multiple items, it fails: File '/storage/apps/galaxy_dist/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/usr/lib/python2.6/site-packages/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/usr/lib/python2.6/site-packages/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/usr/lib/python2.6/site-packages/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/usr/lib/python2.6/site-packages/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/storage/apps/galaxy_dist/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/storage/apps/galaxy_dist/lib/galaxy/web/controllers/tool_runner.py', line 68 in index template, vars = tool.handle_input( trans, params.__dict__ ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/__init__.py', line 979 in handle_input _, out_data = self.execute( trans, incoming=params, history=history ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/__init__.py', line 1271 in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/actions/__init__.py', line 360 in execute for name, value in tool.params_to_strings( incoming, trans.app ).iteritems(): File '/storage/apps/galaxy_dist/lib/galaxy/tools/__init__.py', line 1274 in params_to_strings return params_to_strings( self.inputs, params, app ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/parameters/__init__.py', line 79 in params_to_strings value = params[ key ].value_to_basic( value, app ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/parameters/basic.py', line 110 in value_to_basic return self.to_string( value, app ) File '/storage/apps/galaxy_dist/lib/galaxy/tools/parameters/basic.py', line 1457 in to_string return value.id AttributeError: 'list' object has no attribute 'id' Is using multiple history items as a single input not currently supported, or is there something else I'm doing wrong? Cheers, Kip On Thu, Jun 9, 2011 at 7:05 AM, Leandro Hermida <softdev(a)leandrohermida.com>wrote: > Hi again, > > Sorry to ping again about this, just very interested in knowing if > this is a missing feature in Galaxy or if I had configured something > wrong. > > Is it possible in Galaxy to have a tool with a multiple="true" input > dataset parameter? It doesn't seem so? > > best, > Leandro > > > On Tue, May 24, 2011 at 1:31 PM, Leandro Hermida > <softdev(a)leandrohermida.com> wrote: > > Hi Nate, > > > > Thanks for the ping, have fun at the conference! > > On Tue, May 24, 2011 at 11:22 AM, Nate Coraor <nate(a)bx.psu.edu> wrote: > >> > >> Hi Leandro, > >> > >> The Galaxy Team has recently been preparing for and traveling in advance > >> of the Galaxy Community Conference, which begins today. Unfortunately I > >> don't have an answer for you, but when we're all back at work next week, > >> someone should be able to respond. > >> > >> Thanks, > >> --nate > >> > >> Leandro Hermida wrote: > >> > Hi again, > >> > > >> > Very sorry to ask again, just gotten no repsonse, but is this a Galaxy > >> > bug > >> > or does anyone have a functioning tool that takes multiple datasets as > >> > an > >> > input parameter??? i.e. > >> > > >> > <param type="data" multiple="true" ... /> > >> > > >> > regards, > >> > Leandro > >> > > >> > On Thu, May 19, 2011 at 7:15 PM, Leandro Hermida > >> > <softdev(a)leandrohermida.com > >> > > wrote: > >> > > >> > > Hi again, > >> > > > >> > > I tried changing the format to txt and tabular which I have other > >> > > datasets > >> > > in my history and still the same error and stack trace in Galaxy. > >> > > > >> > > Is it possible at all to have a select multiple of datasets as an > >> > > input > >> > > parameter?? > >> > > > >> > > best, > >> > > Leandro > >> > > > >> > > > >> > > On Thu, May 19, 2011 at 6:59 PM, Leandro Hermida < > >> > > softdev(a)leandrohermida.com> wrote: > >> > > > >> > >> Hi Galaxy developers, > >> > >> > >> > >> Something seems maybe to be wrong with the format="html" type... I > >> > >> forgot > >> > >> to add before that my tool input param the format="html" attribute: > >> > >> > >> > >> <param type="data" multiple="true" format="html" name="input1" /> > >> > >> > >> > >> In another tool I have it outputs format="html" and this works and > >> > >> displays in Galaxy just fine. I would like to use multiple of these > >> > >> output > >> > >> datasets in my history as the input for this other tool but > something > >> > >> seems > >> > >> to be wrong if you try to do this? > >> > >> > >> > >> a bit lost, > >> > >> Leandro > >> > >> > >> > >> > >> > >> On Thu, May 19, 2011 at 3:30 PM, Leandro Hermida < > >> > >> softdev(a)leandrohermida.com> wrote: > >> > >> > >> > >>> Hi, > >> > >>> > >> > >>> I have a tool where the input is multi-select of datasets, e.g.: > >> > >>> > >> > >>> <param type="data" multiple="true" name="input1" /> > >> > >>> > >> > >>> I tested it to see what it would pass to my command and I get the > >> > >>> following debug page and error in Galaxy: > >> > >>> > >> > >>> AttributeError: 'list' object has no attribute 'missing_meta' > >> > >>> > >> > >>> The last part of the stack trace looks like: > >> > >>> > >> > >>> >> validator.validate( value, history ) > >> > >>> Module galaxy.tools.parameters.validation:185 in validate > >> > >>> > >> > >>> history <galaxy.model.History object at 0xb8ea190> > >> > >>> self <galaxy.tools.parameters.validation.MetadataValidator > >> > >>> object > >> > >>> at 0xb8e6250> > >> > >>> value [<galaxy.model.HistoryDatasetAssociation object at > 0xa255550>, > >> > >>> <galaxy.model.HistoryDataset ... Association object at 0xb8ea210>] > >> > >>> > >> > >>> >> if value and value.missing_meta( check = self.check, skip = > >> > >>> >> self.skip > >> > >>> ): > >> > >>> AttributeError: 'list' object has no attribute 'missing_meta' > >> > >>> > >> > >>> What am I doing wrong? > >> > >>> > >> > >>> regards, > >> > >>> Leandro > >> > >>> > >> > >>> > >> > >> > >> > > > >> > >> > ___________________________________________________________ > >> > Please keep all replies on the list by using "reply all" > >> > in your mail client. To manage your subscriptions to this > >> > and other Galaxy lists, please use the interface at: > >> > > >> > http://lists.bx.psu.edu/ > >> > > > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > -- Kip Lord Bodi Jr. Tufts University School of Medicine 136 Harrison Avenue, South Cove 502 Boston, MA 02118 617-636-3763 kip.bodi(a)tufts.edu www.tucf.org | genomics.med.tufts.edu

2 2

Re: [galaxy-dev] New galaxy tool shed
by Greg Von Kuster 30 Jun '11

30 Jun '11

Hello Marcel, On Jun 28, 2011, at 5:23 PM, Marcel Schumann wrote: > Hi Greg, > > Just two small points: > -- the new 'detailed description' cannot yet display linebreaks correctly. Looks a little bit confusing this way (for an example, just have a look a the description text I pasted into the field for CADDSuite) ;-) This issue has been fixed - thanks for pointing it out. > -- just to clarify: I actually meant that it would be very helpful if a version-number in the 'old' style (plain version number, not repository hash key) could be shown as well. This would make it easier for users to understand which release-version of the software is in the toolsuite. But otherwise, as a workaround, I could of course just add a version number to the toolsuite's short-description field ... Yes, attributes like tool_id, version, etc, fall into the category of repository metadata, a new feature which I am currently implementing. It should be available fairly soon. > > > Cheers, > Marcel > > > On 06/28/2011 10:05 PM, Greg Von Kuster wrote: >> Hello Marcel, >> >> Thanks for your message - see my inline comments below. >> >> On Jun 28, 2011, at 5:55 AM, Marcel Schumann wrote: >> >>> Hi Greg, hi Dave, >>> >>> I just uploaded a current version of CADDSuite for MacOS10.6 (in addition to the version for Linux) to the Galaxy tool shed. Thereby I noticed some problems and possible enhancements. I just wanted to let you know about them; I hope this is somewhat helpful ... >>> >>> Errors/problems: >>> -- If _not_ logged in, clicking on any tool or category will produce "Server error" ! >> >> >> This bug has been corrected - thanks for letting us know about it. >> >> >>> -- The new hg based tool shed does not display/support any tool(suite) version any more. I know that this was probably done on purpose, but I guess this will very confusing for users. If publicly available/published software has its own version number, it would be very nice if this could also be shown in the tool shed. >> >> >> Yes, versions are now the repository tip, which is displayed. >> >> >>> -- The new tool shed also does not show detailed description texts any more (in contrast to the previous tool shed). However, I guess these description text could be very important for potential users who search for tools for their tasks. Guessing what a tool/toolsuite does by just reading the one-line description currently supported by the new tool shed is of course not very easy ;-) >> >> >> I've added a Detailed description text field for repositories in the tool shed - thanks for requesting this. >> >> >>> >>> Ideas for enhancements: >>> -- It would be nice to have same kind of download counter >> >> >> I've added a field that displays the number of times a repository was downloaded using the pop-up menu options. Again, thanks for your request. >> >> >>> -- some way to enable users to submit feedback or comments would also be great >> >> >> There is already a Rate repository option on the Repository options pop-up menu. This option allows you to add a 1 - 5 star rating, along with your own Review comments. >> >> >>> -- last but not least, I think you should really advertise your tool shed on your homepages. On both getgalaxy.org and usegalaxy.org I never saw any link/description of the tool shed, but it really is a nice way to share tools :-) >> >> >> We'll look into this, thanks! >> >> >>> >>> >>> Cheers, >>> Marcel >>> >>> >>> -- >>> Marcel Schumann >>> >>> University of Tuebingen >>> Wilhelm Schickard Institute for Computer Science >>> Division for Applied Bioinformatics >>> Room C304, Sand 14, D-72076 Tuebingen >>> >>> phone: +49 (0)7071-29 70437 >>> fax: +49 (0)7071-29 5152 >>> email: schumann(a)informatik.uni-tuebingen.de >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > > > -- > Marcel Schumann > > University of Tuebingen > Wilhelm Schickard Institute for Computer Science > Division for Applied Bioinformatics > Room C304, Sand 14, D-72076 Tuebingen > > phone: +49 (0)7071-29 70437 > fax: +49 (0)7071-29 5152 > email: schumann(a)informatik.uni-tuebingen.de Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

database migration issue 64 ->65 - postgresql
by michael burrell (TSL) 30 Jun '11

30 Jun '11

Good Afternoon, Firstly am a massive fan of galaxy, really like the things that it can do, but I am a bit stuck updating our instance here and its starting to hold us back. I was hoping someone could offer me some assistance with a database migration (I am pulling my hair out). I am using postgresql and galaxy is updated to … galaxy@jic55119:~/software/galaxy-ceneral$ hg tip changeset: 5751:e132a1398caa tag: tip user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Jun 27 17:25:32 2011 -0400 summary: Additional security for loading visualizations. And I receive the following error which is blocking me, galaxy@jic55119:~/software/galaxy-ceneral$ sh manage_db.sh upgrade 64 -> 65... Migration script to add 'name' attribute to the JSON dict which describes a form definition field and the form values in the database. In the 'form_values' table, the 'content' column is now a JSON dict instead of a list. Traceback (most recent call last): File "./scripts/manage_db.py", line 63, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/software/galaxy-ceneral/eggs/sqlalchemy_migrate-0.5.4-py2.5.egg/migrate/versioning/shell.py", line 150, in main ret = command_func(**kwargs) File "/home/galaxy/software/galaxy-ceneral/eggs/sqlalchemy_migrate-0.5.4-py2.5.egg/migrate/versioning/api.py", line 221, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "/home/galaxy/software/galaxy-ceneral/eggs/sqlalchemy_migrate-0.5.4-py2.5.egg/migrate/versioning/api.py", line 349, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/software/galaxy-ceneral/eggs/sqlalchemy_migrate-0.5.4-py2.5.egg/migrate/versioning/schema.py", line 184, in runchange change.run(self.engine, step) File "/home/galaxy/software/galaxy-ceneral/eggs/sqlalchemy_migrate-0.5.4-py2.5.egg/migrate/versioning/script/py.py", line 101, in run func() File "lib/galaxy/model/migrate/versions/0065_add_name_to_form_fields_and_values.py", line 57, in upgrade db_session.execute( cmd ) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/orm/session.py", line 755, in execute clause, params or {}) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/galaxy/software/galaxy-ceneral/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) sqlalchemy.exc.ProgrammingError: (ProgrammingError) column "f" of relation "form_definition" does not exist LINE 1: UPDATE form_definition AS f SET f.fields='[{"layout": "", "n... ^ 'UPDATE form_definition AS f SET f.fields=\'[{"layout": "", "name": "field_0", "default": "yer nom", "required": "required", "label": "Sample Name", "visible": true, "helptext": "??", "type": "TextField"}, {"layout": "", "name": "field_1", "default": "", "required": "required", "label": "run_this?", "visible": true, "helptext": "??", "type": "CheckboxField"}]\' WHERE f.id=2' {} Thank you for all your assistance. Michael Burrell.

2 3