- galaxy-dev - lists.galaxyproject.org

Vagrant-Galaxy for development purposes
by Kandalaft, Iyad 01 Oct '14

01 Oct '14

Hi Everyone I wanted to share some code with the galaxy community and I am looking for opinions on the following approach. Overview: We want to create a standardized galaxy tool development environment to address specific problems we faced in the past while minimizing training and hand-holding efforts associated with galaxy development. Problems encountered: Getting start with Galaxy tool development is complicated as it requires an understanding of the whole system (not just Galaxy and the tool shed). Devs accidentally use the latest galaxy release to develop their tools instead of the in-house production server release. Dev's environment is tainted with system-wide installed software -> tool-dependencies can't be verified because the system installed binaries are on the path regardless. Dev's environment is substantially different from production server environment -> tool dependencies work in dev but not in prod. Several other issues that I can't quite remember Solution and outcomes: Use provisioning software such as vagrant to standardize the dev environment. · Reduce time and training required on getting galaxy/toolshed installed and started · Provides ability to easily blow away the dev environment and re-instantiate it · Provides a clean dev environment every time that isn't tainted with system-wide installations of software · Provides a more similar environment to the production server (assuming it runs the same OS) I hope that this is useful for other galaxy tool devs. I am open to collaboration to improve this initiative and I am looking for insight on some problems we are facing. The repo is on github: https://github.com/AAFC-MBB/vagrant-galaxy Forked from the Smithsonian/vagrant-galaxy repo Regards, Iyad Kandalaft Bioinformatics Application Developer, Science and Technology Branch Agriculture and Agri-Food Canada / Government of Canada Iyad.Kandalaft(a)Agr.gc.ca / Tel: 613-759-1228 / TTY: 613-773-2600 Développeur de logiciels bioinformatique, Direction Général de Science et Technology Agriculture et Agroalimentaire Canada / Gouvernement du Canada Iyad.Kandalaft(a)Agr.gc.ca / Tél. : 613-759-1228 / ATS: 613-773-2600

1 0

Login with a system user
by Daniel Ruiz Molina 01 Oct '14

01 Oct '14

Hello, I have configured Galaxy, but I would be very interested in configure it with PAM or /etc/passwd (/etc/shadow) instead of creating a user in the web interface. How can I configure Galaxy for authenticating with system files? Thanks.

3 2

Internal Server error when attemptng to use Admin Tab
by David Bernick 01 Oct '14

01 Oct '14

Good morning - I have acquired an Internal server error on our local galaxy install. It seems to be an unterminated string. How do I resolve this? thanks in advance - David Bernick UCSC Here is the relevant portion of the log (I hope). Error - <type 'exceptions.ValueError'>: Unterminated string starting at: line 1 column 65528 (char 65528) URL: http://campusrocks.cse.ucsc.edu:8080/admin/index File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 383 in decorator return func( self, trans, *args, **kwargs ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/base/controllers/admin.py', line 34 in index installing_repository_ids = suc.get_ids_of_tool_shed_repositories_being_installed( trans.app, as_string=True ) File '/projects/galaxy/bin/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 277 in get_ids_of_tool_shed_repositories_being_installed app.install_model.ToolShedRepository.status == loading_datatypes_status ) ): File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2348 in instances File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 2120 in _instance File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 1974 in populate_state File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', line 150 in fetch_col File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/types.py', line 658 in process File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/model/custom_types.py', line 52 in process_result_value return json_decoder.decode( str( _sniffnfix_pg9_hex(value) ) ) File '/usr/lib64/python2.6/json/decoder.py', line 319 in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File '/usr/lib64/python2.6/json/decoder.py', line 336 in raw_decode obj, end = self._scanner.iterscan(s, **kw).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 155 in JSONString return scanstring(match.string, match.end(), encoding, strict) ValueError: Unterminated string starting at: line 1 column 65528 (char 65528)

2 1

bowtie_indices.loc for 'bowtie_wrappers' tool
by Peter Briggs 01 Oct '14

01 Oct '14

Dear all Am I right in thinking that, although the current 'bowtie_wrappers' package from the toolshed creates the file .../galaxy-dist/tool-data/bowtie_indices.loc no data is ever actually loaded from this file (and so any entries added to it are ignored)? Empirically, it appears only entries in the .loc file under .../galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie_wrappers/9ca609a2a421/ are actually used to populate the relevant menu in the "bowtie for illumina" tool. Is there a reason why the redundant file is still created? Thanks, best wishes Peter Ps this is in galaxy-dist changeset 14355:007f6a80629a -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

1 0

October 2014 Galaxy Newsletter
by Dave Clements 01 Oct '14

01 Oct '14

Hello all, A lot is happening in in the Galaxy in October, and a lot of good stuff happened in September too. Take a look at the October 2014 Galaxy Newsletter <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10> for the complete story, but here are some highlights: - *Galaxy needs your input! <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#Galaxy_Needs_Your_Inpu…>* You may even get rewarded for your input ... - IRC channel to be logged <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#IRC_Channel_Policy_Cha…>, starting this month - Slides from ECCB'14 and upcoming events <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#Events> - 71 new papers <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#New_Papers>, including 6 featured papers - Who's Hiring? <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#Who.27s_Hiring> Lots of places, that's who - Two new public Galaxy Servers <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#New_Public_Servers> - New Deployment Catalog and Log Board entries <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#Community_Galaxy_Hubs> - 36 new ToolShed repos <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#ToolShed_Contribution> - And a smattering of other news <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#Other_News> too Happy October! Dave Clements and the Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Disk space issue - again
by Patel, Viren 30 Sep '14

30 Sep '14

Hello. I have a Galaxy user whose disk usage remains at 91% even after deleting and purging all histories and datasets (The initial disk usage was 99%). I have searched the mailing list (e.g. http://dev.list.galaxyproject.org/disk-space-usage-td4660164.html) and wiki posts and followed all the steps, including purging hidden data. I have the cleanup scripts running once a day via cron: 01 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -1 11 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -2 21 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -4 31 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -5 41 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -3 51 01 * * * cd ~/galaxy-dist && python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -r -d 60 -6 01 02 * * * find ~/tmp -mtime +60 -delete I have manually run the above commands with -d 1 to delete anything older than 1 day and I have restarted the Galaxy service. However the user¹s disk usage remains at 91%. I am at a loss of what to try next. Any suggestions or ideas of what the issue could be, or what I need to do additionally? I am running the April 2014 distribution of Galaxy. Thanks. Viren ________________________________ This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments).

2 1

2014 Galaxy Questionnaire
by Dave Clements 30 Sep '14

30 Sep '14

Hello all, The Galaxy Project is preparing for our next grant cycle and we are seeking your feedback and comments on on all things Galaxy. We have created two questionnaires, each focused on a different way to interact with Galaxy: 1. *Galaxy User Questionnaire <https://docs.google.com/forms/d/1NT3ma2g7rR-mWA_vgOg2Ia_6VTFiZ86um2JhX2SecC…>* 2. *Galaxy Admin, Tool Developer, and Galaxy Developer Questionnaire <https://docs.google.com/forms/d/1lszXJ2lDJ2dYCWKXBk1qT1jO7XdDDfxXmGRCUMZFX2…>* Please take a few minutes and fill out the surveys that apply to you. The questionnaires are structured so you can skip topics (whole pages!) that don't apply to you, and every question is optional. *And, to thank you for your time and effort, the Galaxy Project will increase your storage quota on usegalaxy.org <https://usegalaxy.org/> by 50GB. *That's a 20% increase. Let your voice be heard (and, get some storage)! The Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

datatype directory
by David Hoover 30 Sep '14

30 Sep '14

Why isn't there a datatype for a directory of files? This seems like such a simple thing. If an executable generates or expects a directory as its input or output, why must a fancy complicated composite datatype be created to handle this? David Hoover -------------------- David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov

2 1

Question about Terms and Conditions for public galaxy server under development
by audrey michel 29 Sep '14

29 Sep '14

Hi, We are developing a galaxy server for public usage and we would like to know if we should use the same terms and conditions as you provide for Galaxy main when we publicly launch our server: https://usegalaxy.org/static/terms.html If so, could you let us know if we can use the exact same text or if we need to replace "The Pennsylvania State University" with our University name? If we can't use the text in the link above, could you provide recommendations? Kind Regards, Audrey Michel

2 1

bwa and bwa-mem wrappers...
by Anton Nekrutenko 29 Sep '14

29 Sep '14

...have been committed to toolshed: https://toolshed.g2.bx.psu.edu/view/devteam/bwa_0_7_10 and github: https://github.com/nekrut/bwa-mem any feedback (especially in the form of github pull requests) is appreciated. One of the main changes is that these tools have no helper python scripts (all magic happens within the tool xml) and they produce BAM output. a. -- Anton Nekrutenko Professor of Biochemistry and Molecular Biology *http://nekrut.bx.psu.edu <http://nekrut.bx.psu.edu>* (814) 826-3051

1 0