- galaxy-dev - lists.galaxyproject.org

workflow issues in local Galaxy....
by Nikhil Joshi 17 Sep '13

17 Sep '13

Hi all, So we use Galaxy to teach our bioinformatics courses and we have an install of Galaxy in the Amazon cloud that we have customized with various tools. However, we don't touch the Galaxy code base itself. Recently, we've been getting errors in creating and running workflows. We are getting "locked database" errors, sometimes the steps seem to run out of order, and sometimes the entire workflow just hangs. Also, the workflow editor seems to be having issues, like it allows connections between outputs and inputs of entirely different datatypes. Has anyone seen any issues like this? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

4 4

[RFC] Storing of tarballs and patches for tool_dependencies to enable reproducibility
by Bjoern Gruening 17 Sep '13

17 Sep '13

Hi, I want to start a discussion about the storage of tarballs to guarantee the availability to some degree. Currently, I store most of my tarballs in my github account, if I do not trust the official ftp/http server. Anyone has the same problems/concerns? Is there an official guideline/recommendation? James raised that topic in one thread ... and I really want to see that happen, to some extend: http://dev.list.galaxyproject.org/tool-dependencies-xml-format-tp4661410p46… I know it is an ambiguous task, but if Galaxy will be an reproducible system we need to think about that issue, discuss it and make a clear statement how far we want to go, what is feasible and what is not. Downstream it has many implications, already now for a few IUC members. For example it is hard to tell tool developers about reproducible tool_dependencies if no clear statement is ever made. A few problems that I encountered during tool development: - 'no stable links': tarballs on a 'lab'-website will change there links or delete old versions of tarballs - github: I'm really not sure if the 'raw' API I use for fetching single files or tarballs from my github account is stable and will remain. I also think I can not put in GB of tarballs in my github account, but currently its the best option I have - Sometimes you need to apply patches, these need to be stored somewhere. - If I store arbitrary tarballs in my github account and the installation routine in Galaxy, the user of my tools need a huge level of trust in my work. Moreover, the IUC can hardly control that (md5 checksums, next to each tarball?) In my opinion we need a central storage, where we can put our tarballs and so one. (mirrored ...) Some ideas: Two separated tool shed areas for one account: 1. version controlled 2. non-version controlled for tarballs and redirection files/rules, to redirect old links, maybe even redirect old repositories to new ones (assuming the history is the same and so on?) FTP Server with a few limitations, like file-size and authentication to make illegal file sharings harder. Ask the github guys if they are willing to support us? Build on top of Open Data initiatives, like the Open Data Portal in Swiss: http://www.bar.admin.ch/themen/01648/?lang=en Any comments, ideas? Cheers, Bjoern

4 6

Number of Threads Tophat2/Bowtie2
by Richard Kuo 17 Sep '13

17 Sep '13

Hello, It looks like the default wrapper that comes with Galaxy for Tophat2 and Bowtie2 does not include options for selecting how many threads/cores to use. Is there an easy way to get this feature in the wrappers or do I simply need to add it to the wrapper code? I Alternatively, does anyone know of tested/debugged wrappers for Tophat2/Bowtie2 that have this feature built in? Thank you, Richard

2 1

tool_dependencies.xml format
by James Taylor 17 Sep '13

17 Sep '13

All, I've been seeing some examples of tool_depedencies.xml come across of the list, and I'm wondering if there are ways that it can be simplified. When we were first defining these features, we talked about having high level recipes for certain types of installs. This could greatly simplify things. For example, can this: <tool_dependency> <package name="requests" version="1.2.3"> <install version="1.0"> <actions> <action type="download_by_url">http://pypi.python.org/packages/source/r/requests/re quests-1.2.3.tar.gz</action> <action type="make_directory">$INSTALL_DIR/lib/python</action> <action type="shell_command">export PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python && python setup.py install --home $INSTALL_DIR --install-scripts $INSTALL_DIR/bin</action> <action type="set_environment"> <environment_variable name="PYTHONPATH" action="append_to">$INSTALL_DIR/lib/python</environment_variable> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> </readme> </package> </tool_dependency> Be simplified to: <tool_dependency> <package name="requests" version="1.2.3"> <install recipe="python_package_setuptools" url="http://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz" /> </tool_dependency> The assumptions are: when version is not provided, it is 1.0 (we've always maintained compatibility in the past for config files so hopefully this never changes), when installing a python package the install directories and environment variables that need to be set are always the same. Similar recipes could be: autoconf: default to configure; make; make install, allow providing configuration options make_install: just make; make install; allow providing make options python_virtualenv ruby_rbenv r_package ... Basically, most of the times the steps to install a particular package are boilerplate, this would remove a ton of duplication in the recipe files. Also, a likely less popular proposal would be to go one step further, tool_dependencies.yaml: recipe: python_package_setuptools name: requests version: 1.2.3 url: http://pypi.python.org/packages/source/r/requests/requests-${version}.tar.gz -- jt

4 5

[GSoC2013] Update
by Saket Choudhary 17 Sep '13

17 Sep '13

I wrote a very short update here: http://galaxy-gsoc2013.blogspot.in/2013/09/penultimate-update.html Saket

1 0

CuffDiff read group tracking
by Matthew Paul 16 Sep '13

16 Sep '13

Hey Galaxy, I am trying to use CuffDIff with replicate analysis on a local instance. I saw that Jeremy Goecks had a solution to this problem: http://user.list.galaxyproject.org/Cuffdiff-changes-tt4655893.html#a4655901. Could I see/use this source code please? I can't really wait until the next update. Thanks, Matt Paul

2 1

Can the API add elements to a repeat parameter?
by Carlos Borroto 16 Sep '13

16 Sep '13

Hi, I'm interested on running workflows with tool definitions containing repeat inputs and arguments. I was wondering if there is a way to not have to set a fix number of elements ahead of time in the workflow. Thanks, Carlos

1 0

CILogon for Galaxy
by Yu (Marie) Ma 16 Sep '13

16 Sep '13

Hi, I have developed a thin front end client in PHP to enable CILogon authentication for Galaxy, and set up a dummy Galaxy instance demo at https://gw64.iu.xsede.org. Please let me know if this is of interest to anyone and I'd be happy share more details as well as the source. The choice of PHP instead of Python is because I was unable to find a good Python Oauth library that supports the RSA-SHA1 signature method required by CILogon's OAuth interface. Thanks Marie

1 0

Geek question re HTTP POST and Galaxy
by Ted Goldstein, Ph.D. 16 Sep '13

16 Sep '13

I've been knocking my head over this all afternoon and I was hoping to buy a vowel, use a lifeline, whatever. I'm trying to use the Ember JS framework (which is very nice) for some extensions. It relies heavily on a REST model using the same method with all of the HTTP forms including Action HTTP Verb URL Find GET /people/123 Find All GET /people Update PUT /people/123 (with the HTTP payload packed with the JSON content) Create POST /people Delete DELETE /people/123 How can I make an API function that responds to all five forms and what should be in buildapp.py? All of the code in Galaxy currently uses the kwd (keyword) parameter. But there are no keywords here. Confused, Ted

3 4

tools installed from a toolshed repository displayed in an arbitrary order
by Yulia.Arzhaeva＠csiro.au 16 Sep '13

16 Sep '13

Hello guys, After porting our Galaxy tools into a tool shed repository the tool sections got lost. I know there is a possibility to assign a repository to a section inside the tools panel when the repository is getting installed. However, we would still like to have sections inside the repository - that makes navigating the tools much neater. Also, the way shed_tool_conf.xml is populated with tools during repository installation seems arbitrary to me. Is there any way to control this, to have repository tools sorted/displayed in a certain order in Tools panel, in the very least, alphabetically? I've got many tools in my repository (and no, it is not possible to divide them into several repositories), and the way they are sorted or divided into sections makes navigating the tools much easier for a user. Thanks, Yulia

3 2