September 2012 - galaxy-dev - lists.galaxyproject.org

fetch sequences
by Unknown 05 Sep '12

05 Sep '12

Hi all, I want to fetch sequences from a file(in interval format) using custom genome build(using built-in index). How can I do that? I tried "fetch sequences" in the menu. But I could not find option for a custom genome. Thanks, Deeps

1 0

Programmatically deleting data sets from data libraries?
by Dan Sullivan 05 Sep '12

05 Sep '12

Hi, Galaxy Developers, I apologize for resurrecting another old thread (http://dev.list.galaxyproject.org/delete-data-library-via-API-td4553000.html) and for this long-winded email... First things first, I am trying to confirm what is suggested in the thread cited above (that data sets cannot be deleted from a Galaxy data library via the API). I am trying to confirm this because I have a vested interested in performing this operation to the extent that I've started investigating modifying the PostgreSQL tables directly if it can't be done via the API (more about this later). The gist of why I am trying to programmatically delete data from a data library is because I am trying to write/implement some custom python code that maintains consistency between a folder on the local filesystem and a corresponding Galaxy data library (i.e bi-directional synchronization). I need do do this in an automated fashion to account for the following two conditions; 1) The file on the filesystem gets deleted (i.e. the path that is referenced in the data library is no longer valid). 2) The MD5 of the file on the filesystem changes (i.e. the file was replaced or modified, and needs to be re-imported such that the correct metadata (i.e. file size) is reported via the Galaxy UI). Based on the limited amount of testing I have done, it doesn't appear to be possible to delete an actual data set from the data library via the API. Here is a test that leads me to believe that this is not possible; 1) I can delete a data library successfully without issue; Here is the output of me doing so: --(galaxy@crigalaxy)-(/group/galaxy/galaxy-dist/scripts/api)-- > ./delete.py 11f3cb91acb2ab1677f8265bxxxxxxxx http://localhost:8081/api/libraries/e85a3be143d5905b Response -------- {'synopsis': 'dansully', 'description': 'dansully', 'name': 'dansully'} --(galaxy@crigalaxy)-(/group/galaxy/galaxy-dist/scripts/api)-- 2) Whenever I try to delete an item in the data library, I get a 404, with the response "no action found for ..." --(galaxy@crigalaxy)-(/group/galaxy/galaxy-dist/scripts/api)-- > ./display.py 11f3cb91acb2ab1677f8265bxxxxxxxx http://localhost:8081/api/libraries/e85a3be143d5905b/contents/62e564808c536… Member Information ------------------ ldda_id: 62e564808c5368d4 misc_blurb: 2 lines name: whatever.txt data_type: txt file_name: /group/galaxy/galaxy-dist/database/files/008/dataset_8938.dat uploaded_by: dansully(a)uchicago.edu template_data: {} genome_build: ? model_class: LibraryDataset misc_info: uploaded txt file file_size: 329 metadata_data_lines: 2 message: id: 62e564808c5368d4 date_uploaded: 2012-08-29T15:48:38.335445 metadata_dbkey: ? --(galaxy@crigalaxy)-(/group/galaxy/galaxy-dist/scripts/api)-- > ./delete.py 11f3cb91acb2ab1677f8265bxxxxxxxx http://localhost:8081/api/libraries/e85a3be143d5905b/contents/62e564808c536… HTTP Error 404: Not Found 404 Not Found The resource could not be found. No action for /api/libraries/e85a3be143d5905b/contents/62e564808c5368d4 --(galaxy@crigalaxy)-(/group/galaxy/galaxy-dist/scripts/api)-- My hope of being able to actually use the API for a delete is that I am either not forming the URL string to correctly delete the data set, or the data{} dict implements a key that I am not aware of (it is my understanding that the Galaxy API is still under active development; so far I have not been able to locate any documentation that suggests any keys or attributes (other than the example code distributed)) that will make a delete operation for an individual data set feasible. Would it be possible for somebody with specific knowledge of the Galaxy API to comment on whether or not this functionality is implemented? All of this being said, if it is *not* possible to delete an individual data set from the Galaxy API, I am prepared to make a 'reasonable' effort to try and do this by modifying the galaxy back-end SQL tables directly. Based on the research I have done (I have enabled mod logging on the PostgreSQL database), here is what a delete operation from the Galaxy UI looks like in terms of database changes: 2012-08-29 08:44:20.361 CDT,"galaxy_","galaxy",7952,"127.0.0.1:46174",503e1a8d.1f10,41,"idle in transaction",2012-08-29 ent: UPDATE library_dataset SET update_time='2012-08-29T13:44:20.361070', deleted=true WHERE library_dataset.id = 6013",$ So, at this point (assuming that I cannot delete the data set from the Galaxy API), I'm trying to work backwards from information I know (i.e. file_name or ldda_id) to discern the library_dataset.id (I'm still digging through database query logs to try to determine how this is done (it appears that there are more than one query executed when a data library is rendered via the Galaxy UI). Determining the id of the dataset in question continues to be an ongoing challenge. Which leads me to one final last question. Could anybody tell me how the ID's (the ldda_id) that get returned by the Galaxy API are calculated? It is it some sort of a hash of composite or primary keys from the back-end tables? The reason why I am asking this is because I did a full dump of the database and searched (using grep) for the ldda_id (i.e. 62e564808c5368d4), and it didn't exist anywhere in the database (I was surprised by this). If anybody out there has programmatically deleted a data set from a Galaxy data library (via the API or other), or could shed some light on how to solve my problem, I'd love to hear from you. Thank-you so much for your time, and again, I apologize for my lengthly e-mail. Dan Sullivan

2 1

using data_ref to refer to column names?
by Dan Tenenbaum 04 Sep '12

04 Sep '12

Hi, I have some code that makes a plot and creates a legend from a given column name in the user's input data, which may have arbitrary column names. I'd like to create a dropdown list that has all the column names in it, but I don't know the names beforehand; I won't know them until the user uploads their data file. So all I can do is have a freeform text box where I ask the user to type a column name, which seems error-prone and ugly. I notice that the XY plotting tool has something sort of like this, where it can tell how many columns are in your dataset and lets you choose them, but it calls the columns "c1", "c2", etc. My datasets have a header line with column names and I want to show the actual column name, not reference them by column number. Is this possible? Is there a solution to this? I have a number of use cases like this, so it would be great if there was. Thanks, Dan

2 3

Getting syntax error on Join + environment_setup_file not read
by Peter Schmitt 04 Sep '12

04 Sep '12

New galaxy install. Running the tutorial Galaxy 101 and trying to do the join. Get the following error in the stderr of the job: Traceback (most recent call last): File "/opt/galaxy/galaxy-dist/tools/new_operations/gops_join.py", line 20, in ? from galaxy.tools.util.galaxyops import * File "/opt/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py", line 15, in ? from galaxy import util, jobs, model File "/opt/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 15, in ? from galaxy import util, model File "/opt/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/opt/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 146 finally: ^ SyntaxError: invalid syntax Traceback (most recent call last): File "./scripts/set_metadata.py", line 24, in ? import galaxy.model.mapping #need to load this before we unpickle, in order to setup properties assigned by the mappers File "/opt/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/opt/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 146 finally: ^ SyntaxError: invalid syntax Also I believe the line, environment_setup_file = scripts/galaxy.sh, is not being used any longer. When I put the line: export DRMAA_LIBRARY_PATH=/opt/drmaa/1.0.12/lib/libdrmaa.so.1.0.12, in there the server didn't see it. When I added that line to the top of run.sh, it does. -- * Pete Schmitt Technical Director: Discovery Cluster NH INBRE Grid Computational Genetics Lab Dartmouth College, HB 6223 L24 Berry/Baker Library Hanover, NH 03755 Phone: 603-646-8109 http://discovery.dartmouth.edu http://columbia.dartmouth.edu/grid http://www.epistasis.org *

2 1

password param type
by Marc Logghe 04 Sep '12

04 Sep '12

Hi, Kind of feature request: parameter of type 'password' so that an entered password is not shown in the html form nor in the logs. Or is there already a way to achieve this that I missed out ? Cheers, Marc ________________________________________ THIS E-MAIL MESSAGE IS INTENDED ONLY FOR THE USE OF THE INDIVIDUAL OR ENTITY TO WHICH IT IS ADDRESSED AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, CONFIDENTIAL AND EXEMPT FROM DISCLOSURE. If the reader of this E-mail message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately at ablynx(a)ablynx.com. Thank you for your co-operation. "NANOBODY" and "NANOCLONE" are registered trademarks of Ablynx N.V. ________________________________________

1 0

Filter options for type=select based on former selection
by Sascha Kastens 04 Sep '12

04 Sep '12

Hi all! I am facing a simple problem but can?t find a solution to solve it. Before I start describing my problem here is the code from my tool: <conditional name="sReference"> <param name="reference" type="select" label="Select a reference genome" help=""> <option value="mm9_64">mm9_64</option> <option value="hg19">hg19</option> <option value="custom">custom (from history, fasta)</option> </param> <when value="mm9_64"> <param name="ref" type="hidden" value="7596a73211022433" /> </when> <when value="hg19"> <param name="ref" type="hidden" value="f73725d4339c9140" /> </when> <when value="custom"> <param name="ref" type="hidden" value="b5d0b09143f99c68" /> <param name="cReference" format="fasta" type="data" label="Select the reference genome" /> </when> </conditional> <conditional name="sCuffcompare"> <param name="cuffcompare" type="boolean" truevalue="yes" falsevalue="no" label="Run Cuffcompare?" help="" /> <when value="yes"> <conditional name="sRefannotation"> <param name="refannotation" type="select" label="Will you select a reference annotation from your history or use a GATC supported one?" help="" > <option value="no_ref">No reference annotation</option> <option value="supported">Supported</option> <option value="history">History</option> </param> <when value="supported"> <param name="index" type="select" label="Select reference annotation" help=""> <options from_file="gatc_transcriptome_annotations.loc"> <column name="name" index="2" /> <column name="value" index="3" /> <filter type="sort_by" column="2" /> <filter type="param_value" ref="sReference.reference" column="0"/> <validator type="no_options" message="No annotations are available for the selected reference genome"/> </options> </param> </when> <when value="history"> <param name="cAnnotation" format="gtf" type="data" label="Select the reference annotation" /> </when> <when value="no_ref" /> </conditional> </when> <when value="no" /> </conditional> As you might see from the code above I am trying to filter the contents for <param name="index" type="select" label="Select reference annotation" help=""> based on the selected value in <param name="reference" type="select" label="Select a reference genome" help=""> (<filter type="param_value" ref="sReference.reference" column="0"/>). As soon as I try to load this configuration I get the error KeyError: ?sReference.reference?. So it seems, that the tool cannot access the key sReference.reference. I figured out that the tool can access only the "top" key from the conditional, i.e. I can use <filter type="param_value" ref="sReference" column="0"/> and no error appears while reloading the tools config. But this is not useful because sReference does not contain any value for filtering. Does anybody know how the tool can access variables/keys under a conditonal tag without being within the conditional tag itself? Thanks for any help/hints/advice! Cheers, Sascha

2 2

Tool shed empty
by Ivan Lovric 04 Sep '12

04 Sep '12

Hi there, I've been having some trouble with the Tool Shed recently. This image from the ToolShed wiki http://wiki.g2.bx.psu.edu/Tool%20Shed?action=AttachFile&do=get&target=user_… Suggests that there is a "Browse all repositories" link on the side panel in the non-admin interface, however the link seems to only appear on the admin interface. After some digging around, I've realized that the link really is formed only for the admin interface (see: templates/webapps/community/admin/index.mako). I guess it's been removed at some stage (however looking at the hg logs for this file didn't help me figure out when). But back to the real problem....clicking on "Browse by category" yields nothing - everything is empty. A bit about the configurations I tested... galaxy-central AND galaxy-dist - minimal configuration (setting database_file in community_wsgi.ini and unsetting it (same outcome)). One thing that has led me to believe that there is something quite wrong in my configuration is that there is no hgweb.config file created (apparently it should be created once the toolshed starts). Right now I'm looking around the hg/mercurial related code in the community/ dirs but I'm not getting very far...the HgController class is "loaded" but none of its methods ever get called (nor can I find out from the code where they're supposed to be called from). Has anyone else had this problem? I remember a few weeks ago on a clean galaxy-dist I had (lost now), I was able to see the repositories as per the user_register.png image above. Since then I've made new VM images and lost the original galaxy-dist so I can't go back and figure out what was different on that configuration...I'd very much appreciate any thoughts on this. Thanks! Ivan

1 1

8-bit bytestrings
by Gauthier Jean-Pierre 04 Sep '12

04 Sep '12

Hi, I am new to Galaxy and have not found a solution on dev-list to this 8-bit bytestrings error. That occurs on my local installation but not on Galaxy main server. Cheers Dataset generation errors *Dataset 22: Neighbor Joining Tree on data 20* The Galaxy framework encountered the following error while attempting to run the tool: Traceback (most recent call last): File "/data/APPLIS/GALAXY/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 123, in run_job job_wrapper.finish( stdout, stderr ) File "/data/APPLIS/GALAXY/galaxy-dist/lib/galaxy/jobs/__init__.py", line 573, in finish self.sa_session.flush() File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 773, in execute_save_steps self.execute_cyclical_dependencies(trans, task, False) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 789, in execute_cyclical_dependencies self.execute(trans, [t], isdelete) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/data/APPLIS/GALAXY/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE history_dataset_association SET update_time=?, info=?, blurb=?, peek=? WHERE history_dataset_association.id = ?' ['2012-09-04 06:23:08.928377', 'Single Alignment Analyses\nsh: /data/APPLIS/GALAXY/galaxy-dist/tool-data/HYPHY/HYPHY: Aucun fichier ou r\xc3\xa9pertoire de ce type\n', 'error', 'no peek', 55] -- Gauthier Jean-Pierre. Bio-informatique. INRA. UMR IGEPP. Domaine de la Motte. BP. 35327. 35653 LE RHEU CEDEX. tel : 33(0)2.23.48.51.68 fax : 33(0)2.23.48.51.50 mail : Jean-Pierre.Gauthier(a)rennes.inra.fr

1 0

Tool-integration: annovar
by Xinyi Chua 03 Sep '12

03 Sep '12

Hi all, I have a new project that requires setting up a local Galaxy instance for a client including integrating new tools and some reference data sources. I'm new to the Galaxy community so am also learning along the way. There's been a request to integrate the ANNOVAR tool for their system. I've searched the mailing-list archive and found this question was posed earlier in the year but there was no reply. I couldn't find any tool wrappers for it on Toolshed, but considering it is such a widely-used tool, I'm wondering whether anyone has any knowledge if this tool has been wrapped for Galaxy before? Many thanks, Xin-Yi _____________________________________ Xin-Yi Chua Senior Bioinformatician Queensland Facility for Advanced Bioinformatics Level 6, QBP University of Queensland, Q4072 T: +61 (0)7 3346 2125 M: +61 (0) 439 783 953 E: <mailto:xinyi.chua@qfab.org> xinyi.chua(a)qfab.org <http://www.qfab.org/> www.qfab.org

2 2

Submit commit to galaxy-central: Filter sam
by Joachim Jacob 03 Sep '12

03 Sep '12

Dear all, I have made a little change to Filter SAM tool, to allow keeping the header and comment line in a SAM file while filtering. The 'hg push' command did not work to get it on BitBucket (provided the username and password of bitbucket). Therefore, the 'hg export tip' can be found add http://dpaste.com/hold/795565/. Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

1 0