June 2014 - galaxy-dev - lists.galaxyproject.org

specifying default file by name in workflows
by Evan Bollig 19 Jun '14

19 Jun '14

I don't know all the subtleties of the galaxy workflow syntax. My goal is to specify the default input file names for a number of tools in a workflow. Is this possible, or am I limited to only providing the file type or extension? If possible, can you provide an example? Thanks, -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan(a)msi.umn.edu boll0107(a)umn.edu

2 2

Import of Capsules failing on local toolshed instance
by Will Holtz 18 Jun '14

18 Jun '14

I am now able to export capsules from the main/test toolsheds -- thanks Dave! When attempting to import these capsules into my local toolshed (latest_2014.06.02 for changeset fb68af9a775a) I receive the following error: URL: https://galaxy.lygos.com:99/repository/import_capsule File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/lib/galaxy/webapps/tool_shed/framework/middleware/remoteuser.py', line 74 in __call__ return self.app( environ, start_response ) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/webapps/tool_shed/controllers/repository.py', line 1992 in import_capsule import_util.check_status_and_reset_downloadable( trans, import_results_tups ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/import_util.py', line 34 in check_status_and_reset_downloadable tip_changeset_revision = repository.tip( trans.app ) AttributeError: 'NoneType' object has no attribute 'tip' I have seen the same behavior for capsules based on the following repositories from the test toolshed: package_biopython_1_62, package_vienna_rna_2_1, and package_bowtie_0_12_7. I am logged in as an admin user for the import process. thanks, -Will

2 4

Postgresql database time wrong...
by Neil.Burdett＠csiro.au 18 Jun '14

18 Jun '14

Hi I have quite a strange issue. I have a local install of Galaxy setup. When I type 'date' on my Ubuntu machine I get something like: Wed Jun 18 09:25:22 EST 2014 When i then execute a job and look in the database at the create_time i.e. # select create_time from job order by create_time; I get 2014-06-17 23:20:00.133828 So about 10 hours different. Is there some configuration I need to set as Brisbane is 10hrs ahead of GMT (coincidence?) Thanks Neil

2 3

How Galaxy prevents security attacks?
by Neil.Burdett＠csiro.au 18 Jun '14

18 Jun '14

Hi, we'd like to host a version of Galaxy, but our security team have identified some potential areas that could be subjected to attack. I was wondering if Galaxy had mechanism in place to avoid the following issues: Cross site scripting Description Cross site scripting (also referred to as XSS) is a vulnerability that allows an attacker to send malicious code (usually in the form of Javascript) to another user. Because a browser cannot know if the script should be trusted or not, it will execute the script in the user context allowing the attacker to access any cookies or session tokens retained by the browser Impact Malicious users may inject JavaScript, VBScript, ActiveX, HTML or Flash into a vulnerable application to fool a user in order to gather data from them. An attacker can steal the session cookie and take over the account, impersonating the user. It is also possible to modify the content of the page presented to the user Affected URLS and parameters: * /capaibl/visualization/list o update_time param * /capaibl/history/delete_current o purge * /capaibl/history/list o F-deleted o F-name * /capaibl/user/edit_address o Address_id o User_id * /capaibl/edit_info o Username o User_id * Capaibl/user/logout o Referrer * /capaibl/user/manage_user_info o Change_password_button o Cntrller o Current o Email o Id o Message o Status o Username o User_id * /capaibl/user/reset_password o Message * /caibl/user/set_default_permissions o cntrller Denial of Service Description of attack During security testing we used two different types of crawlers and both caused the system to crash. Without access to the backend it is hard to document the reason, however, an educated guess would be that there is limited memory available to sessions. When crawlers ask for multiple sessions, this could cause the webserver to allocate all its memory and crash. Attack scenario 1. Open burpsuite and start the proxy function 2. Open a browser and set burpsuite as the proxy 3. Navigate to the milxcloud homepage 4. Turn on the spider function within burpsuite The website will crash within seconds. There is an entry in universe_wsgi.ini # Galaxy encodes various internal values when these values will be output in # some format (for example, in a URL or cookie). You should set a key to be # used by the algorithm that encodes and decodes these values. It can be any # string. If left unchanged, anyone could construct a cookie that would grant # them access to others' sessions. #id_secret = USING THE DEFAULT IS NOT SECURE! Would this help address the XSS issue? Thanks for any information Neil

2 3

Gzipped input to functional tests with multiple="true"
by Peter Cock 18 Jun '14

18 Jun '14

Hi all, I think I've found a bug in the Galaxy test framework :( With most file inputs, a gzipped input file works fine (Galaxy's upload code handles unzipping it). However, with multiple="true" this seems to break (with the Twill backend, the API test framework is OK), e.g. <param name="filenames" type="data" format="fastq,mira" multiple="true" required="true" label="Read file(s)" help="Multiple files allowed, for example paired reads can be given as two files (MIRA looks at read names to identify pairs)." /> Fails: <param name="filenames" value="SRR639755_mito_pairs_sample.fastq.gz" ftype="fastqsanger" /> e.g. https://travis-ci.org/peterjc/pico_galaxy/builds/27426318 Excerpt from log: ====================================================================== FAIL: test_tool_000000 (functional.test_toolbox.TestForTool_mira_4_0_de_novo) MIRA v4.0 de novo assember ( mira_4_0_de_novo ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py", line 108, in test_tool self.do_it( td ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py", line 32, in do_it data_list = galaxy_interactor.run_tool( testdef, test_history ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py", line 449, in run_tool self.twill_test_case.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py", line 1789, in run_tool self.submit_form( **kwd ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py", line 1999, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'read_group_0|filenames' to value '['SRR639755_mito_pairs_sample.fastq.gz']' in form 'tool_form' threw exception: id=None name=None label='SRR639755_mito_pairs_sample.fastq.gz' control: <SelectControl(read_group_0|filenames=[80])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. This works, <param name="filenames" value="SRR639755_mito_pairs_sample.fastq" ftype="fastqsanger" /> e.g. https://travis-ci.org/peterjc/pico_galaxy/builds/27426336 See: https://github.com/peterjc/pico_galaxy/commit/e6967767535ca29debcdc19d7f050… Regards, Peter

1 1

What is the correct place under Galaxy for a database that's created by a tool?
by Melissa Cline 17 Jun '14

17 Jun '14

Hi folks, Hopefully this is a quick question. I'm working on a set of tools that will fire off a VM from within Galaxy and will then communicate with the VM. The VM will create a local database. The vision is that this won't be a shared database; in a shared Galaxy instance, each user will have his or her own database. What is the best place to create this database under the Galaxy file system? Thanks! Melissa

3 5

R bioconductor dependencies when creating toolshed installation
by Stef van Lieshout 17 Jun '14

17 Jun '14

Hi all, I'm running into some difficulties on how to setup the installation procedure for a galaxy tool which executes an R script and has certain dependencies (mainly bioconductor packages). R can deal with dependencies, packages can be installed with install.packages (has a "dependencies" argument) or biocLite() for bioconductor packages. Yet, now I want my tool to be available at toolsheds. To do this I see several options: 1) setting up tool_dependencies.xml with "R CMD INSTALL" for all packages. BUT: need to download all dependencies before install, and can older versions still be downloaded? Maybe need to upload them to toolshed too.. 2) setting up tool_dependencies.xml to call an installation script with Rscript (where I could use install.packages), BUT: Dependencies are taken care of. But how do I select specific (older) versions, because if I dont, installing at different time can give different version. 3) creating a repository for each package and have all of them as requirement in my galaxy tool. BUT: a lot of work for a lot of dependencies All have pros and cons, how do people deal with this? Stef

3 7

R version 3.1.0
by Stef van Lieshout 17 Jun '14

17 Jun '14

Any plans to create a R 3.1.0 repository? Thanks, Stef

1 0

Using Mesos to Enable distributed computing under Galaxy?
by Kyle Ellrott 17 Jun '14

17 Jun '14

I think one of the aspects where Galaxy is a bit soft is the ability to do distributed tasks. The current system of split/replicate/merge tasks based on file type is a bit limited and hard for tool developers to expand upon. Distributed computing is a non-trival thing to implement and I think it would be a better use of our time to use an already existing framework. And it would also mean one less API for tool writers to have to develop for. I was wondering if anybody has looked at Mesos ( http://mesos.apache.org/) You can see an overview of the Mesos architecture at https://github.com/apache/mesos/blob/master/docs/Mesos-Architecture.md The important thing about Mesos is that it provides an API for C/C++, Java/Scala and Python to write distributed frameworks. There are already implementations of frameworks for common parallel programming systems such as: - Hadoop (https://github.com/mesos/hadoop) - MPI ( https://github.com/apache/mesos/blob/master/docs/Running-torque-or-mpi-on-m… ) - Spark (http://spark-project.org) And you can find example Python framework at https://github.com/apache/mesos/tree/master/src/examples/python Integration with Galaxy would have three parts: 1) Add a system config variable to Galaxy called 'MESOS_URL' that is then passed to tool wrappers and allows them to contact the local mesos infrastructure (assuming the system has been configured) or pass a null if the system isn't available. 2) Write a tool runner that works as a mesos framework to executes single cpu jobs on the distributed system. 3) For instances where mesos is not available at a system wide level (say they only have access to an SGE based cluster), but the user wants to run distributed jobs, write a wrapper that can create a mesos cluster using the existing queueing system. For example, right now I run a Mesos system under the SGE queue system. I'm curious to see what other people think. Kyle

5 8

Can't startup the galaxy-central instance!
by xlwang 16 Jun '14

16 Jun '14

hi, I downloaded and installed the galaxy-central instance because I want to use toolshed. But I can't startup the instance, The following error occurred： Fetched http://eggs.galaxyproject.org/WebError/WebError-0.8a-py2.7.egg Fetched http://eggs.galaxyproject.org/twill/twill-0.9-py2.7.egg Fetch successful. Traceback (most recent call last): File "/export/galaxy-central/lib/galaxy/webapps/galaxy/buildapp.py", line 38, in app_factory from galaxy.app import UniverseApplication File "/export/galaxy-central/lib/galaxy/app.py", line 21, in <module> from galaxy.queue_worker import GalaxyQueueWorker File "/export/galaxy-central/lib/galaxy/queue_worker.py", line 10, in <module> import galaxy.queues File "/export/galaxy-central/lib/galaxy/queues.py", line 17, in <module> from kombu import Exchange, Queue File "/export/galaxy-central/eggs/kombu-3.0.13-py2.7.egg/kombu/__init__.py", line 67, in __getattr__ module = __import__(object_origins[name], None, None, [name]) File "/export/galaxy-central/eggs/kombu-3.0.13-py2.7.egg/kombu/entity.py", line 10, in <module> from .abstract import MaybeChannelBound File "/export/galaxy-central/eggs/kombu-3.0.13-py2.7.egg/kombu/abstract.py", line 12, in <module> from .connection import maybe_channel File "/export/galaxy-central/eggs/kombu-3.0.13-py2.7.egg/kombu/connection.py", line 24, in <module> from kombu import exceptions File "/export/galaxy-central/eggs/kombu-3.0.13-py2.7.egg/kombu/exceptions.py", line 12, in <module> from amqp import ChannelError, ConnectionError, ResourceError File "/export/galaxy-central/eggs/amqp-1.4.5-py2.7.egg/amqp/__init__.py", line 32, in <module> from .basic_message import Message # noqa File "/export/galaxy-central/eggs/amqp-1.4.5-py2.7.egg/amqp/basic_message.py", line 19, in <module> from .serialization import GenericContent File "/export/galaxy-central/eggs/amqp-1.4.5-py2.7.egg/amqp/serialization.py", line 33, in <module> from .five import int_types, long_t, string, string_t, items File "/export/galaxy-central/eggs/amqp-1.4.5-py2.7.egg/amqp/five.py", line 171, in <module> librt = ctypes.CDLL('librt.so.1', use_errno=True) TypeError: __init__() got an unexpected keyword argument 'use_errno' Why this happened?

1 0