April 14, 2013 Galaxy Distribution News Brief

Distribution Summary 2014_04_14

Highlights:

• Visualization framework updates and Trackster updates
• Tool Shed updates to repos, test frameworks, and documentation
• Over 100 genomes with updated content on our rsync server
• UI Dataset action controls expanded and overall unification of design
• API functionality additions and new job control/admin abilities
• Administrator, Configuration, Job control, and Egg features and updates
• 18 new community contributions added

Get Galaxy

Upgrades

• Review instructions for core build: Getting the Stable Distribution

• Review reset for Tool Shed repos: Resetting Metadata For Installed Repositories

Data

1. New and updated reference genomes added to Rsych server that mirrors contents of the public Main http://usegalaxy.org instance

   • New/updated genomes: 115
     • includes hg38
     • note: all available in tool "Extract Genomic DNA"
   • New bowtie indexes: 60
   • New bowtie2 indexes: 115
   • New BWA indexes: 54
   • New lastz indexes: 54
   • New sam indexes: 64
   • New picard (srma) indexes: 59
   • New liftOver mappings: 163

2. MAF hg19 100waymultiz conservation alignment sourced from UCSC

3. Replaced: The sam index for hg_g1k_b37 was updated to use "samtools faidx" with version 0.1.18 (original from GATK 1.4 bundle used 0.1.13). The older index lead to some issues with BAM<->SAM conversion using the newer SAMTools version.

Visualizations

1. Framework tuning https://trello.com/c/kdhJ1Dit

2. Trackster:

   • Sow complete read/feature across tile boundaries, in all tiles https://trello.com/c/Q03dC1QC

   • Better handling of deep coverage regions in read tracks https://trello.com/c/TxxzqWNx

Workflows

1. Prep for dataset collections: Refactor out common logic for running workflows across controllers https://trello.com/c/pdGNjllH

UI

1. New/Updated Dataset Action Controls:
   • Located at the top of the history pane
   • Ability to select: all, none, or check individual datasets
   • Actions: Hide, Unhide, Delete, Undelete, Permanently delete (Purge)

   • https://wiki.galaxyproject.org/Learn/Managing%20Datasets#Searching_Datasets

   • Additional actions pending review/implementation

   • https://trello.com/c/8gLGPzxV, https://trello.com/c/mq1m6X5M,

2. When multiple tool versions installed, default to the latest version https://trello.com/c/zRBZ9nvD

3. Better handling when one of many versions of a tool repo is uninstalled https://trello.com/c/BmdG2201

4. Unify the places where histories are shown and used: UI view and in core code https://trello.com/c/7cN0nmKH

5. Configure workflow menu, improve buttons and styling https://trello.com/c/xFzp3lt6

6. Handle disabled sessionStorage/localStorage more gracefully in the history pane https://trello.com/c/V2VCxzmx

7. Remove BaseView from base-mvc https://trello.com/c/xUV4qvIc

8. Do not serve non-HTML content that may contain JavaScript in such a way that web browsers execute it https://trello.com/c/G67ELiMI

9. There are now 2 interfaces to data libraries (the original and new Beta) and feedback is welcomed https://trello.com/c/d3muKvqr

10. Wiki page on how to compile the css. https://wiki.galaxyproject.org/Develop/CSS https://trello.com/c/gLI6hUjN

API

1. New functions:

   • Make histories importable and/or published via the API is now implimented (sharing with individual users a pending feature update) https://trello.com/c/Bcsadi1q

   • History export/import https://trello.com/c/qhu1KdWU

   • Ability to extract workflow from a history to workflows https://trello.com/c/ZTehdMZH

2. Improved history payload and returned error consistency https://trello.com/c/fWC7WgQf

3. Check ftype attribute (if defined) on test output datasets https://trello.com/c/hFFlM4gY

4. Published histories contents now fully accessible (no permissions issue) https://trello.com/c/KME3ZRRY

5. Correct exception formats used in history & HDA https://trello.com/c/bS9wY4Ri

Core

1. Admin: History view, allow importing purged datasets https://trello.com/c/tbhpQdMF

2. Admin: User list, show 'activated' and 'create_time' columns https://trello.com/c/C6MhEHWo

3. Admin: Setting user quota at zero is now permitted (no ZeroDivisionError) https://trello.com/c/38NoBtlL

4. Admin: Admin user now has permissions to copy datasets from any history to a library https://trello.com/c/Ed91qIUC

5. Admin: Improvements to 'Impersonate User' implementation https://trello.com/c/lny2me4u

6. Config: Rename tool-data/shared/ucsc/ucsc_build_sites.txt to ucsc_build_sites.txt.sample

7. Jobs: Prevent 'unicode' strings being sent to pbs library as hostname https://trello.com/c/NlFXYDta

8. Jobs: Fix Torque CLI runner to recognize complete ('C') state as 'ok' https://trello.com/c/vSO52k17

9. Jobs: Are now cancelled if a running history item is deleted (via API). https://trello.com/c/PDmUdtbw

10. LWR: Several updates including communication via message queue, more planned https://trello.com/c/6DcXqXNt

11. LWR: Introduce and extend along with ComputeEnvironment, to eliminate string based path rewriting https://trello.com/c/dL0poowc

12. Egg Update: Guppy 0.1.9+ https://trello.com/c/zeSnHEav

13. Egg Update: Boto 2.25 to facilitate launching into VPC https://trello.com/c/QXfJQhlS

14. Egg Addition: PyYAML 3.10 to be used initially with Cloudlaunch for parsing persistent_data yaml https://trello.com/c/JMZkqBtH

15. Lib: package_libpng_1_6_7 deprecated, replaced with package_libpng_1_6_8 https://trello.com/c/Wr5U8laV https://trello.com/c/65DOL7PL

Pull Requests Merged

Thanks to our Galaxy community contributors!

1. 309 - Job module for API (list and inspect jobs). Kyle Ellrott. https://trello.com/c/s2Gk6QlH

2. 312 - Remove unused imports and unused variables. Fix spacing. Nicola Soranzo. https://trello.com/c/iWaf1ak4

3. 316 - Fixing non-string parameter selection for job searches. Kyle Ellrott. https://trello.com/c/ZXZ9lEkN

4. 317 - Web API: safer and cleaner workflow parameter passing. Simone Leo. https://trello.com/c/JaC2EBM8

5. 318 - Adding deleted field to 'to_dict' and search selection of Workflows. Kyle Ellrott. https://trello.com/c/lv9q67nF

6. 315 - Allow jobs to be filtered by tool_id and history_id as well as by state. The tool_id is compared with the like operator. James Johnson https://trello.com/c/dvLvPUmK

7. 322 - Added "ignore lines starting with specific characters" to group tool. Clayton Turner. https://trello.com/c/j6TP6M7m

8. 323 - API methods to access Workflow Usage. Kyle Ellrott. https://trello.com/c/UetzHj64

9. 325 - Use placeholder instead of value in html5 - editor.mako. Trevor Wennblom. https://trello.com/c/PIZAXsOS

10. 326 - Specify third-party cookies must not be blocked - repository.py. Trevor Wennblom. https://trello.com/c/XsAUr6wl

11. 331 - Code housekeeping: standardize whitespace in various locations. Trevor Wennblom. https://trello.com/c/WtWdxP2n

12. 341 - Workflows API enhancements (pull request #337 corrected). Nicola Soranzo and Simone Leo. https://trello.com/c/zizGdqy1

13. 338 - Patch to expose the actual dataset id in the LDDA and HDA to_dict calls (in addition to the instance id). Kyle Ellrott https://trello.com/c/BrOscqTJ

14. 339 - Strip trailing slashes in tool IDs. Björn Grüning. https://trello.com/c/bpKNn66u

15. 334 - Optional Input Datasets Not Compatible with Parallelism Tag. Michael Li https://trello.com/c/WMgr8Xx0 https://trello.com/c/Muw6fd8h

16. 346 - Tool panel label bug correction. Nicola Soranzo. https://trello.com/c/8NpgYM3U

17. 353 - Add BCF datatype sniffing, so BCF files are not uncompressed during upload. Nicola Soranzo. https://trello.com/c/FVKZhlsU

18. 355 - Workflow API: show also workflow owner, so a user can check it before importing. Nicola Soranzo. https://trello.com/c/cMiaYcCI

Fixes

1. Fix issue with tool panel display when the tool_conf.xml has only a single entry https://trello.com/c/6nJIhOdJ

2. Fix issue when multiple versions of the same tool installed results in separate entries in the tool panel https://trello.com/c/SlNI79mR

3. Fix Published Page's JS error https://trello.com/c/yGw1DwwM

4. Fix error in Babel egg utf-8, OSX 10.7 https://trello.com/c/PUdh6SWw

5. Fix failures when using unicode in History name editable_text fields https://trello.com/c/MTdotrHR

6. Fix tool form image display for toolshed installed tools https://trello.com/c/5G6jlB4B

7. Fix to improve link format in user activation emails https://trello.com/c/JALX9DL3

8. Fix problem with rerun of tools having DataToolParameter set as optional https://trello.com/c/pzHcSGzK

9. Fix bug in Tool Panel when installing repositories https://trello.com/c/AiUefffC

Tool Shed

Documentation

1. Reorganization of the Tool Shed wiki is underway! See current changes https://wiki.galaxyproject.org/ToolShed and what is next up https://trello.com/c/Gg0jnll7

2. Data Manager documentation specific to toolshed http://wiki.galaxyproject.org/Tool%20Shed#The_contents_of_your_tool_shed_repository https://trello.com/c/OPOakK4w

Repositories

1. New Tool Shed categories https://trello.com/c/niCdaARH

   • Genome-Wide Association Study
   • Imaging
   • RNA

2. Eliminate the use of fabric as the installation process for repositories https://trello.com/c/zJoRROvv

3. Enhancements for updating a repository https://trello.com/c/c4Y9t7db

   • Will now trigger the install of new dependencies
   • Handles newly defined repository dependencies included in updates to installed repositories

4. Add support for the new repository type: Repository suite definition https://trello.com/c/sxXMmfLs

5. Defined a base set of utilities that can be expected to be on systems into which tool dependencies are being installed https://trello.com/c/gYlRdqwr, related to project-in-planning https://trello.com/c/7VTlX9rD

6. Dependency ez_setup.py added to keep package_pysam_0_7_7 from failing on tool shed https://trello.com/c/AkACDWix

7. Update Main Tool Shed Picard repo that contained invalid tools https://trello.com/c/mxxHWxLY

8. Update FreeBayes repo so that it specifies precompiled binaries in dependency definition https://trello.com/c/zYoVHKoV

9. Update EMBOSS repo so that it downloads a precompiled platform-specific binary https://trello.com/c/iW3Amhl5

Install and Test Framework

1. Tool shed install and test framework now works with API driven tool tests https://trello.com/c/vfr7b0VK

2. Test script to provide coverage for recently discovered scenario that produced a white ghost upon Galaxy install https://trello.com/c/JICmJueB

3. Test script to provide coverage for recently introduced repository administration feature https://trello.com/c/vjTrhIST

4. Test script to check toolshed dependency files for broken/invalid download URLs https://trello.com/c/i4GVgZwS

Tool Shed Fixes

1. Fix setup_r_environment broken in central repo https://trello.com/c/zyCzlgzo

2. Fix scenario where by uninstalling and reinstalling a repository that has been updated creates database entries for repositories that are persistently in the New state https://trello.com/c/iUDUM4Ut

3. Fixes in automated test framework https://trello.com/c/QWgsgll8

4. Fix bug in install and test framework related to samtools migration https://trello.com/c/sN2iLCCn

Project Updates

1. News

2. Events

3. Videos on Vimeo

4. April 2014 Galaxy Update

5. March 2014 Galaxy Update

About

GalaxyProject.org

The Galaxy Team is a part of the Center for Comparative Genomics and Bioinformatics at Penn State, and the Department of Biology at Johns Hopkinis University. 

Galaxy is supported in part by NSF, NHGRI, the Huck Institutes of the Life Sciences, and The Institute for CyberScience at Penn State, and Johns Hopkins University.

The public Main instance of Galaxy at http://usegalaxy.org utilizes infrastructure generously provided by the iPlant Collaborative at the Texas Advanced Computing Center, with support from the National Science Foundation.

Follow us on Twitter @galaxyproject or read our tweets at Galaxy on Twitter

Have a suggestion? Please see Using the Galaxy Issue Board, and be sure to up vote the issues important to YOU.

-- 
Jennifer Hillman-Jackson
http://galaxyproject.org