July 2014 - galaxy-dev - lists.galaxyproject.org

Toolshed : tool_dependency xml, action problem
by Nicolas Lapalu 17 Jul '14

17 Jul '14

Hi all, I'm trying to package my tool wrapper to release it in the ToolShed. I need to download the associated binary file and change the permissions. I wrote a tool_dependency xml file with a downlod_binary and chmod actions. My tool_dependency_dir is well defined in my universe file. The download is ok, but I get an error during the "chmod" action. So I tried with your example, faToTwoBit (https://wiki.galaxyproject.org/DownloadingBinaries) and I get the same bug. Do you know the problem ? Do I need to configure someting else ? log : tool_shed.util.tool_dependency_util DEBUG 2014-07-07 11:28:34,346 Updating an existing record for version 0.0.1 of tool dependency faToTwoBit for revision 5ac8ed26842c of repository mytool by updating the status from Never installed to Installing. tool_shed.galaxy_install.tool_dependencies.recipe.step_handler DEBUG 2014-07-07 11:28:34,374 Attempting to download from http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/faToTwoBit to bin 127.0.0.1 - - [07/Jul/2014:11:28:36 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140610 Firefox/24.0 Iceweasel/24.6.0" tool_shed.galaxy_install.install_manager ERROR 2014-07-07 11:28:39,038 Error installing tool dependency faToTwoBit version 0.0.1. Traceback (most recent call last): File "/home/nlapalu/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict ) File "/home/nlapalu/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 79, in install_and_build_package current_dir = os.path.abspath( os.path.join( work_dir, dir ) ) File "/usr/lib/python2.7/posixpath.py", line 75, in join if b.startswith('/'): AttributeError: 'NoneType' object has no attribute 'startswith' Thanks for your help; nico

3 5

Fresh galaxy install fails to start
by Iry Witham 17 Jul '14

17 Jul '14

Hi Dev Team, I am attempting to get a new/clean installation running and am having difficulty. I did a clone from https://bitbucket.org/galaxy/galaxy-dist and made my needed configuration changes for my environment. I attempted to start it and needed to do the database upgrade via sh manage_db.sh upgrade. That went just fine. I tried another start and got the following error: migrate.versioning.repository DEBUG 2014-07-17 13:07:56,566 Config: {'db_settings': {'__name__': 'db_settings', 'required_dbs': '[]', 'version_table': 'migrate_tools', 'repository_id': 'GalaxyTools'}} tool_shed.galaxy_install.migrate.check DEBUG 2014-07-17 13:07:56,571 psycopg2 egg successfully loaded for postgres dialect Traceback (most recent call last): File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/app.py", line 46, in __init__ self._configure_models( check_migrate_databases=True, check_migrate_tools=check_migrate_tools, config_file=config_file ) File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/config.py", line 656, in _configure_models verify_tools( self, install_db_url, config_file, install_database_options ) File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/tool_shed/galaxy_install/migrate/check.py", line 150, in verify_tools raise Exception( msg ) Exception: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The list of files at the end of this message refers to tools that are configured to load into the tool panel for this Galaxy instance, but have been removed from the Galaxy distribution. These tools and their dependencies can be automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu. To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use). If you do this, be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should be removed from your file named ./tool_conf.xml. CRITICAL NOTE IF YOU PLAN TO INSTALL The location in which the tool repositories will be installed is the value of the 'tool_path' attribute in the <tool> tag of the file named ./migrated_tool_conf.xml (i.e., <toolbox tool_path="../shed_tools">). The default location setting is '../shed_tools', which may be problematic for some cluster environments, so make sure to change it before you execute the installation process if appropriate. The configured location must be outside of the Galaxy installation directory or it must be in a sub-directory protected by a properly configured .hgignore file if the directory is within the Galaxy installation directory hierarchy. This is because tool shed repositories will be installed using mercurial's clone feature, which creates .hg directories and associated mercurial repository files. Not having .hgignore properly configured could result in undesired behavior when modifying or updating your local Galaxy instance or the tool shed repositories if they are in directories that pose conflicts. See mercurial's .hgignore documentation at the following URL for details. http://mercurial.selenic.com/wiki/.hgignore The following tool dependencies can also optionally be installed (see the option flag in the command below). If you choose to install them (recommended), they will be installed within the location specified by the 'tool_dependency_dir' setting in your main Galaxy configuration file (e.g., uninverse_wsgi.ini). ------------------------------------ Tool Dependency ------------------------------------ Name: bx-python, Version: 0.7.1, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: R, Version: 2.11.0, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: rpy, Version: 1.0.3, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: numpy, Version: 1.7.1, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: yacca, Version: 1.0, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: kernlab, Version: 0.1-4, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: libxml2, Version: 2.9.1, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: libpng, Version: 1.6.7, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: fontconfig, Version: 2.11.1, Type: package ------------------------------------ ------------------------------------ Tool Dependency ------------------------------------ Name: blast+, Version: 2.2.26+, Type: package ------------------------------------ /hpcdata/galaxy-test/galaxy-setup/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/url.py:105: SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to 'postgresql'. The new URL format is postgresql[+driver]://<user>:<pass>@<host>/<dbname> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv sh ./scripts/migrate_tools/0011_tools.sh ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies' option to the above command like this: vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv sh ./scripts/migrate_tools/0011_tools.sh install_dependencies ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Tool dependencies can be installed after the repositories have been installed as well. After the installation process finishes, you can start your Galaxy server. As part of this installation process, entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these tools will continue to be loaded into your tool panel. Because of this, existing entries for these tools have been removed from your file named ./tool_conf.xml. fasta_to_tabular.xml tabular_to_fasta.xml annotation_profiler.xml pca.xml cca.xml kpca.xml kcca.xml fasta_compute_length.xml fasta_filter_by_length.xml fasta_concatenate_by_species.xml short_reads_figure_score.xml short_reads_trim_seq.xml megablast_wrapper.xml megablast_xml_parser.xml <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Removing PID file manager.pid I ignored that since it is related to a config file I copied from my previous instance. I retried another start and I get the following: migrate.versioning.repository DEBUG 2014-07-17 13:09:59,145 Config: {'db_settings': {'__name__': 'db_settings', 'required_dbs': '[]', 'version_table': 'migrate_tools', 'repository_id': 'GalaxyTools'}} tool_shed.galaxy_install.migrate.check DEBUG 2014-07-17 13:09:59,161 psycopg2 egg successfully loaded for postgres dialect tool_shed.galaxy_install.migrate.check INFO 2014-07-17 13:09:59,215 At migrate_tools version 11 galaxy.model.orm DEBUG 2014-07-17 13:09:59,224 psycopg2 egg successfully loaded for postgres dialect galaxy.config INFO 2014-07-17 13:09:59,251 Install database targetting Galaxy's database configuration. sqlalchemy.pool.QueuePool WARNING 2014-07-17 13:10:01,869 Error closing cursor: current transaction is aborted, commands ignored until end of transaction block Traceback (most recent call last): File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/app.py", line 52, in __init__ self._configure_datatypes_registry( self.installed_repository_manager ) File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/config.py", line 605, in _configure_datatypes_registry installed_repository_manager.load_proprietary_datatypes() File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/tool_shed/galaxy_install/installed_repository_manager.py", line 278, in load_proprietary_datatypes .order_by( self.install_model.ToolShedRepository.table.c.id ): File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2227, in __iter__ return self._execute_and_instances(context) File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2242, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) ProgrammingError: (ProgrammingError) column tool_shed_repository.status does not exist LINE 1: ...ist_to_shed AS tool_shed_repository_dist_to_shed, tool_shed_... ^ 'SELECT tool_shed_repository.id AS tool_shed_repository_id, tool_shed_repository.create_time AS tool_shed_repository_create_time, tool_shed_repository.update_time AS tool_shed_repository_update_time, tool_shed_repository.tool_shed AS tool_shed_repository_tool_shed, tool_shed_repository.name AS tool_shed_repository_name, tool_shed_repository.description AS tool_shed_repository_description, tool_shed_repository.owner AS tool_shed_repository_owner, tool_shed_repository.installed_changeset_revision AS tool_shed_repository_installed_changeset_revision, tool_shed_repository.changeset_revision AS tool_shed_repository_changeset_revision, tool_shed_repository.ctx_rev AS tool_shed_repository_ctx_rev, tool_shed_repository.metadata AS tool_shed_repository_metadata, tool_shed_repository.includes_datatypes AS tool_shed_repository_includes_datatypes, tool_shed_repository.tool_shed_status AS tool_shed_repository_tool_shed_status, tool_shed_repository.deleted AS tool_shed_repository_deleted, tool_shed_repository.uninstalled AS tool_shed_repository_uninstalled, tool_shed_repository.dist_to_shed AS tool_shed_repository_dist_to_shed, tool_shed_repository.status AS tool_shed_repository_status, tool_shed_repository.error_message AS tool_shed_repository_error_message \nFROM tool_shed_repository \nWHERE tool_shed_repository.includes_datatypes = %(includes_datatypes_1)s AND tool_shed_repository.deleted = %(deleted_1)s ORDER BY tool_shed_repository.id' {'includes_datatypes_1': True, 'deleted_1': False} Removing PID file manager.pid Can someone point me to a resolution? Thanks, __________________________________ Iry T. Witham Scientific Applications Administrator Scientific Computing Group Computational Sciences Dept. The Jackson Laboratory 600 Main Street Bar Harbor, ME 04609 Phone: 207-288-6744 email: iry.witham(a)jax.org [cid:3CA173D8-424E-483A-8F9D-9E6C007C18DE] The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

1 0

Multiple output tools in Workflow
by Calogero Zarbo 17 Jul '14

17 Jul '14

Hello to everybody I'm developing my own tool that need to switch the numbers of output files according to a parameter selected by the user from a list in the <inputs> tag. How can I do such thing? Here is the XML code: <inputs> <param name="input_dataset" label="Input dataset" type="data" format="showelab-dataset,fbk-svm-dataset"/> <conditional name="format_condition"> <param name="format_options" label="Choose the type of dataset you want to split" type="select"> <option value="fbk">".fbk-svm-dataset" Format</option> <option value="showelab" selected="True">".showelab-dataset" Format</option> </param> <when value="fbk"> <param name="input_fbk_dataset_labels" label="Input dataset labels" type="data" format="fbk-labels"/> </when> </conditional> </inputs> <outputs> <data format="showelab-dataset" name="trainingDataset" label="Training Dataset extracted from ${input_dataset.name}"> <filter>format_options == "showelab"</filter> </data> <data format="showelab-dataset" name="validationDataset" label="Validation Dataset extracted from ${input_dataset.name}"> <filter>format_options == "showelab"</filter> </data> <data format="fbk-svm-dataset" name="trainingDataset" label="Training Dataset extracted from ${input_dataset.name}"> <filter>format_options == "fbk"</filter> </data> <data format="fbk-labels" name="trainingLabels" label="Training Dataset Labels extracted from ${input_fbk_dataset_labels.name}"> <filter>format_options == "fbk"</filter> </data> <data format="fbk-svm-dataset" name="validationDataset" label="Validation Dataset extracted from ${input_dataset.name}"> <filter>format_options == "fbk"</filter> </data> <data format="fbk-labels" name="validationLabels" label="Validation Dataset Labels extracted from ${input_fbk_dataset_labels.name}"> <filter>format_options == "fbk"</filter> </data> </outputs> Basically I would like to have in the Workflow Canvas that the outputs displayed in the interface will change according to the format_options select parameter. Thanks in advance. -- Calogero Zarbo FBK - Fondazione Bruno Kessler Center for Information Technology Mpba - Predictive Models for Biomedicine & Environment Via Sommarive, 18- 38123 Povo (Trento) IT e-mail: zarbo(a)fbk.eu <https://webmail.fbk.eu/owa/redir.aspx?C=648442797b254191ab144726f9a535e7&UR…> phone: +39 0461 314 674

2 5

Workflow with many inputs
by Alexandre Defelicibus 17 Jul '14

17 Jul '14

Hi all, I have a tool that receives multiple inputs, and it is working fine if I run it stand alone. However, I want to create a workflow where this tool receive as an input the output of other tools. I can connect the output from the other tools in the input of this tool, but the tool receives only one input. Is there a way to set multiple inputs to a tool in a workflow? I've found several similar questions like mine, but I haven't found an answer. I appreciate your help. Best regards, -- Alexandre Defelicibus Mestrando em Bioengenharia Programa de Pós-Graduação em Bioengenharia Universidade de São Paulo - USP

2 2

Download URL needs user to be logged in first
by Neil.Burdett＠csiro.au 17 Jul '14

17 Jul '14

Hi, galaxy provides a download_url to download zipped up files. However, the user must first log into Galaxy to commence the download. Is there a way to turn this feature off, so that users can click on the link and download the data without first login Thanks Neil

3 3

Updated Freebayes Wrapper - Bugs
by Lance Parsons 16 Jul '14

16 Jul '14

Thanks for the major update of the Freebayes wrapper, excellent! I've run into two issues, however. 1) When using "set allelic scope" I get the following error: Fatal error: Exit code 1 () freebayes: unrecognized option `--min-repeat-length' did you mean --min-repeat-size ? 2) When using a vcf file as input I get the following error: Fatal error: Exit code 1 () open: No such file or directory [bgzf_check_bgzf] failed to open the file: input_variant_vcf.vcf.gz [tabix++] was bgzip used to compress this file? input_variant_vcf.vcf.gz I'd be happy to help resolve these. Is there a bitbucket repo somewhere to submit pull requests? -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

3 2

Galaxy error
by Beck, Emily A 16 Jul '14

16 Jul '14

user username emilybeck quota_percent 19 total_disk_usage 52086193970 nice_total_disk_usage 48.5 GB email emily-beck(a)uiowa.edu is_admin false tags_used model_class User id 03f0554e2314759e source HDACollection(f313d1d65c4ee57e,451) xhr readyState 4 responseText {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0} responseJSON err_msg Uncaught exception in exposed API method: err_code 0 status 500 statusText Internal Server Error responseHeaders Server nginx/1.4.7 Date Wed, 16 Jul 2014 15:19:52 GMT Content-Type application/json Transfer-Encoding chunked Connection keep-alive Cache-Control max-age=0,no-cache,no-store options data parse true emulateHTTP false emulateJSON false Emily Beck PhD Candidate Llopart Lab 469 Biology Building Iowa City, IA 52242 Lab: (319)335-3430

1 0

Uploading Multiple files
by Neil.Burdett＠csiro.au 16 Jul '14

16 Jul '14

Hi, It was suggested recently that Galaxy would soon be providing the ability to upload multiple files rather than single files. Does anyone know when this might be available in Galaxy? I've downloaded the most recent version of Galaxy but as yet it is not present in the "stable" version Thanks for any information Neil

2 1

Re: [galaxy-dev] Toolshed roadmap
by Greg Von Kuster 15 Jul '14

15 Jul '14

Hello Eric, Please use Biostar or the Galaxy dev mail list rather than individual email addresses in order to ensure faster responses. Pleasse see my responses below. On Jul 14, 2014, at 10:20 AM, Eric Rasche <rasche.eric(a)yandex.ru> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Howdy Greg, > > Since there isn't really a public/published roadmap and you're the > toolshed guy, I'm emailing to see if I can find out what's in the > toolshed's future. The tool shed Trello board at https://trello.com/b/vHqpKpZF/galaxy-tool-shed is the roadmap for the Tool Shed. Just like the Galaxy Development Trello board, you should be able to add new cards to the Inbox, and vote on existing cards to raise their priority. Please let us know if you are not able to do these things. > > Specifically, I'm wondering if there will be API calls for the following > things: > > - - creating repositories I've added the following Trello card for the above feature. However, it is currently possible to create new repositories via the API by importing a repository capsule (i.e., ~/api/repositories/import_capsule). https://trello.com/c/cUUWeBCi/234-enhance-the-api-to-enable-creation-of-new… > - - obtaining a list of categories The above was introduced in the June 2, 2014 release, so, for example, the following will work. https://testtoolshed.g2.bx.psu.edu/api/categories > - - obtaining category information within the /api/repositories call I've added the following Trello card for this feature. https://trello.com/c/n7P0GE1P/235-enhance-the-api-repositories-feature > > Cheers, > Eric > > - -- > Eric Rasche > Programmer II > Center for Phage Technology > Texas A&M University > College Station, TX 77843 > 404-692-2048 > esr(a)tamu.edu > rasche.eric(a)yandex.ru > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.22 (GNU/Linux) > > iQIcBAEBAgAGBQJTw+dEAAoJEMqDXdrsMcpVKJ0P/0cJVIeFZ4NXmR0IPgMsQBMK > j6yw4rk69iUBgS0phk19kHaOi4kQtsrlwtpled13MEWBL2MBcHSFSHGn8RsvH1pW > JzbbQiFN/AOycTaDnV91veG3JuJ2MHAA2IAzx+fyjqlZV5ekZfK1q3xYB0E7EKyl > iojoqP5ewa8FC29LB0TnAswmEA384S6sPmqUicRmL5m0QR6hBBCf44nfN/Zvl2iw > ZqdKovO1EZBJ4vEHhZnN4rAdvFnqhghBOB/mjoKVGMIsJvu3I66PxhV+VhFdoLKJ > yaSWWAd3gwp0gP8NDEunDRkF1rM0eJgVou9eUtPPQkxun5S7c7bACMYdhmpyCcVu > wgNE2UnP0x1EFDx8q77WmT78nMp7khxWU8Q/3YPoGTr4r3DfSYrvMqD81uoIvkBJ > MqB89mJS2VqHi88idzIlyIYP3CE7mDEjDcHpLonY+uFxc3LNQEOcP0pvvPpNTkmw > Oix60X+GSACH0s8lqlInzEhu1OqbW0nokAayvDmzYjmttrVOFqQdhUFTYxq7El5M > yXE31fX+VTHnIbx/ZPSvl1TqbldrTFs/J4KnvNbf4PuseLQDRmsnrFJ7lSii2xSc > imqQiwyt7lCRHCO5+w+jSaMUPQ3SrbGxsbAckqK+HkCllJ87v7oC1CBLugOP5425 > 2E7ad35BvZ1bGmumAw9q > =ZXxZ > -----END PGP SIGNATURE-----

2 2

Tool dependency environments with functional tests
by Aaron Petkau 14 Jul '14

14 Jul '14

Hello everyone, I've been trying to get some functional tests developed for my tools and I'm trying to figure out how to properly run them. I'm following the guide at https://wiki.galaxyproject.org/TestingInstalledTools but when I run the functional tests against tools installed from my tool shed I get some strange error messages like: Can't open perl script "//find-repeats.pl": No such file or directory It can't find the script because in the tool XML file it is being run using an environment variable like <command interpreter="perl"> /\$VCF2PSEUDO/find-repeats.pl -l $length -p $pid $fasta > $out The environment variable is set up in one of the dependency packages for this tool. If I set this environment variable manually "export VCF2PSEUDO=/path/to/dir" and then run the tests with: export GALAXY_TOOL_DEPENDENCY_DIR=tool_dependencies sh run_functional_tests.sh -installed Then all my tests properly work. So, I'm wondering does the run_functional_tests.sh script setup the environment for dependency packages for each tool? Or is there something I'm missing? Thanks and sorry for the longing email. I'm still new to setting up and using functional tests in Galaxy, Aaron

1 1