- galaxy-dev - lists.galaxyproject.org

toolshed error on cloudman
by Karen Miga 12 Dec '13

12 Dec '13

Hello, I am currently unable to install a number of tools from toolshed to our cloud instance. Unfortunately, this pops up for quite a few tools (I am currently trying to install cuffmerge, but I see this again and again). Any help would be appreciated! I am seeing the following error: URL: http://ec2-54-204-115-185.compute-1.amazonaws.com/admin_toolshed/prepare_fo… File '/mnt/galaxy/galaxy-app/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/galaxy/galaxy-app/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/galaxy-app/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/galaxy-app/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/galaxy/galaxy-app/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/galaxy/galaxy-app/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/galaxy/galaxy-app/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 896 in prepare_for_install common_install_util.get_dependencies_for_repository( trans, tool_shed_url, repo_info_dict, includes_tool_dependencies ) File '/mnt/galaxy/galaxy-app/lib/tool_shed/util/common_install_util.py', line 84 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install( trans, repo_info_tuple ) File '/mnt/galaxy/galaxy-app/lib/tool_shed/util/common_install_util.py', line 190 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple( rd_tup ) File '/mnt/galaxy/galaxy-app/lib/tool_shed/util/shed_util_common.py', line 1106 in parse_repository_dependency_tuple prior_installation_required = str( prior_installation_required ) UnboundLocalError: local variable 'prior_installation_required' referenced before assignment -- Karen H. Miga, PhD Postdoctoral Scholar University of California, Santa Cruz (831)459-5232

2 1

cloudman console error
by Karen Miga 12 Dec '13

12 Dec '13

Hello, I am unable to reach my cloudman console. I was trying to reboot the master. The instance is running on AWS console with no error, I just have no way of access the admin console to terminate or figure out the error through cloud launch. Further, I am now unable to launch a new instance. When attempting to start a new instance I am unable to attach my existing volume-id. Error message below: - 19:14:14 - Error processing filesystems in existing cluster configuration: 'Volume' object has no attribute 'volume_id' Any help would be extremely appreciated. Karen

1 0

a galaxy of issues; Job output not returned from cluster,drmaa getting wrong LSF job id and job.o files not being found and moving files occurring before upload completes.
by Hazard, E. Starr 12 Dec '13

12 Dec '13

This is a stand alone instance of Galaxy built Nov 28 a small research cluster using LSF and built on RHEL6.2. galaxy.tools.actions.upload_common INFO 2013-12-05 14:33:01,106 tool upload1 created job id 10 Working directory for job is: /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 galaxy.jobs.handler DEBUG 2013-12-05 14:33:01,290 (10) Dispatching to drmaa runner galaxy.jobs DEBUG 2013-12-05 14:33:01,441 (10) Persisting job destination (destination id: drmaa) galaxy.jobs.handler INFO 2013-12-05 14:33:01,471 (10) Job dispatched galaxy.tools.deps DEBUG 2013-12-05 14:33:01,693 Building dependency shell command for dependency 'samtools' galaxy.tools.deps WARNING 2013-12-05 14:33:01,693 Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners.drmaa DEBUG 2013-12-05 14:33:02,352 (10) submitting file /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10/ galaxy_10.sh galaxy.jobs.runners.drmaa DEBUG 2013-12-05 14:33:02,352 (10) command is: pythonŠ. galaxy.jobs DEBUG 2013-12-05 14:33:02,379 (10) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 galaxy 50982 galaxy.jobs.runners.drmaa DEBUG 2013-12-05 14:33:02,528 (10) submitting with credentials: galaxy [uid: 50981] galaxy.jobs.runners.drmaa DEBUG 2013-12-05 14:33:02,574 (10) Job script for external submission is: /depot/shared/app/Galaxy/galaxy-dist/database/lsf/10.jt_json galaxy.jobs.runners.drmaa INFO 2013-12-05 14:33:02,772 (10) queued as Job <28526> is submitted to default queue <medium_priority>. 28526 galaxy.jobs DEBUG 2013-12-05 14:33:02,837 (10) Persisting job destination (destination id: drmaa) galaxy.jobs.runners.drmaa INFO 2013-12-05 14:33:03,237 (10/Job <28526> is submitted to default queue <medium_priority>. 28526) job left DRM queue with following message: code 18: invalid LSF job id: Job <28526> is submitted to default queue <medium_priority>. 28526 galaxy.jobs DEBUG 2013-12-05 14:33:03,303 (10) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 galaxy 50982 128.23.163.166 - - [05/Dec/2013:14:33:05 -0400] "GET /api/histories/50a7a2e81473b416/contents HTTP/1.1" 200 - "http://hpcc3.musc.edu:8089/root" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0" galaxy.jobs.runners ERROR 2013-12-05 14:33:08,661 (10/Job <28526> is submitted to default queue <medium_priority>. 28526) Job output not returned from cluster: [Errno 2] No such file or directory: '/depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 /galaxy_10.o' galaxy.jobs DEBUG 2013-12-05 14:33:08,701 (10) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 galaxy 50982 galaxy.jobs DEBUG 2013-12-05 14:33:09,049 finish(): Moved /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10/ galaxy_dataset_16.dat to /depot/shared/app/Galaxy/galaxy-dist/database/files/000/dataset_16.dat galaxy.jobs DEBUG 2013-12-05 14:33:09,049 finish(): Moved /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10/ galaxy_dataset_15.dat to /depot/shared/app/Galaxy/galaxy-dist/database/files/000/dataset_15.dat galaxy.jobs DEBUG 2013-12-05 14:33:09,105 setting dataset state to ERROR galaxy.jobs DEBUG 2013-12-05 14:33:09,126 setting dataset state to ERROR galaxy.jobs DEBUG 2013-12-05 14:33:09,214 job 10 ended "Job output not returned from cluster² appears in History, BUT in fact files are being written to directories indicated in my universe_wsgi.ini I am having no success getting a solution to this error "job left DRM queue with following message: code 18: invalid LSF job id" This file "No such file or directory: '/depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10 /galaxy_10.o¹ ³ exists AFTER the job finishes. SO at 2013-12-05 14:33:03 the file had not been written but does appear later. This operation is completing before the file can be uploaded and so an empty file is moved Moved /depot/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/10/ galaxy_dataset_15.dat to /depot/shared/app/Galaxy/galaxy-dist/database/files/000/dataset_15.dat >From my universe_wsgi.ini: file_path = database/files new_file_path = database/job_working_directory job_working_directory = database/job_working_directory cluster_files_directory = database/job_working_directory Any troubleshooting suggestions appreciated. Starr

2 1

problems converting large bcf files to vcf
by Berner, Thomas 12 Dec '13

12 Dec '13

Hey guys, we have a problem in our local galaxy instance (10392:1ae95b3aa98d release_2013.08.12) using bcftools (Version: 0.1.19-44428cd) from the tool shed to convert our bcf files into vcf. With bcf files less than ~ 3 GB everything works fine, but above that (we have a 13gb bcf file), the job gets stuck and is not terminating, neither successful nor with an error. Can somebody confirm this behavior? Piping the Mpileup output directly into the bcftools using > Samtools mpileup -Duf "REFERENZ" "BAM_FILES" | bcftools view -cvg - > OUTPUT at the command line works fine. Trying to integrate this into the Mpileup wrapper failed so far. Can somebody give us a hint how to solve this problem? thanks in advance, Thomas --- Thomas Berner Julius Kühn-Institut (JKI) - Federal Research Centre for Cultivated Plants - Erwin Baur-Straße 27 06484 Quedlinburg - Germany - Phone: ++49 ( 0 ) 3946 47 562 EMail: thomas.berner(a)jki.bund.de<mailto:thomas.berner@jki.bund.de>

1 0

Again: Variable inputs in a workflow
by Preussner, Jens 12 Dec '13

12 Dec '13

Dear galaxy-dev's, about a year ago you discussed on variable number of inputs into a workflow (see http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-November/012012.html and http://thread.gmane.org/gmane.science.biology.galaxy.devel/4502/focus=4502, for example). We're interested in having something simple like a number of fastq files that have to undergo automated quality control and trimming. A workflow would describe steps that need to be done per file (like fastqc, quality trimmer and so on), and in the end, a tabular data summarizes statistics on all files. Of course, we could prebuild workflows for 2,3,4,5,..n input files and join the output, but it would be much cooler to have it variable. Since it's possible to start many instances of a workflow (i.e. one instance per file), this would be a good starting point. But how would one combine outputs of those instances? Is anyone out there having experience with such setups? Which files are involved in starting many instances of a workflow? Any other ideas or suggestions on how to go from here? Thanks a lot for any input! Best, Jens Max-Planck-Institute for Heart and Lung Research Bioinformatics Service - FGI Ludwigstraße 43 61231 Bad Nauheim Phone. +49 6032 705 1765 Mail. jens.preussner(a)mpi-bn.mpg.de<mailto:jens.preussner@mpi-bn.mpg.de>

3 2

Re: [galaxy-dev] Galaxy wrappers for CLC Assembly Cell (CLCbio)
by Eric Kuyt 11 Dec '13

11 Dec '13

Hi Peter, After installing the clc testing galaxy wrapper I noticed there are still some static paths in the wrapper. I'm in favor of expecting binaries in the system path, but maybe that's a matter of taste. Greets, Eric Kuijt On 30 October 2013 17:03, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > Hello all, > > This is just to announce I am working on a wrapper for > "CLC Assembly Cell" which is the CLCbio commercial > command line assembly tool suite. > http://www.clcbio.com/products/clc-assembly-cell/ > > Our institute bought a licence primarily for use on plant > genomes where other assemblers at the time required > too much RAM to complete. This assembler is both fast > and low memory, which can be very useful. > > Wrapper development here: > https://github.com/peterjc/pico_galaxy/tree/master/tools/clc_assembly_cell > > Prototype releases will be on the Test Tool Shed (soon): > http://testtoolshed.g2.bx.psu.edu/view/peterjc/clc_assembly_cell > > Stable Tool Shed releases will be here (later): > http://toolshed.g2.bx.psu.edu/view/peterjc/clc_assembly_cell > > I would be interested to hear from anyone else with access to > a licensed copy of the tool interested in using it from Galaxy. > e.g. Is it reasonable to assume the tools are on the $PATH, > or is using a specific environment variable more helpful? > > Regards, > > Peter > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > >

3 5

funny bug with latest galaxy release when using IE
by Ambrose Andongabo (RRes-Roth) 11 Dec '13

11 Dec '13

Dear All, We get a very annoying behaviour using the latest release(04 November) of galaxy on Internet explorer. We tested it on I.E version 9 and version 10 and we get the same behaviour as you can see from the screenshot below [cid:image001.png@01CEF5CD.2BEF7FE0] I hope there is somebody who knows why this is happening Thanks Ambrose -- This message has been scanned for viruses and dangerous content by MailScanner, and we believe but do not warrant that this e-mail and any attachments thereto do not contain any viruses. However, you are fully responsible for performing any virus scanning.

2 1

inconsistent BAM-to-SAM behavior
by Jennifer Jackson 11 Dec '13

11 Dec '13

Hi Mike, I am moving this over to the galaxy-dev(a)bx.psu.edu mailing list as it concerns a local install. This is the best way to make contact with others working with the same type of Galaxy. https://wiki.galaxyproject.org/MailingLists A few things to provide more context that will help with troubleshooting: 1 - Are you running the most current version of Galaxy? https://wiki.galaxyproject.org/DevNewsBriefs (Nov 4, 2013) 2 - The local has some of the initial production server changes made? initial install: http://getgalaxy.org next steps are under: https://wiki.galaxyproject.org/Admin in particular: https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer 3 - You installed SAMTools from the Tool Shed? The config is correct? And are running the latest version? https://wiki.galaxyproject.org/Tool%20Shed#Installing.2C_maintaining_and_un… https://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies https://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies 4 - When you run the same dataset(s) through on the public Main Galaxy instance at http://usegalaxy.org, is it successful or does it fail? I wasn't sure if you meant that the same file will fail and then pass, or if some fail and some pass. If you could clarify that, it would help. It could mean the difference between a processing problem (pointing to a config or other issue) and a simple memory problem (where allocating more could help). Please keep all replies on the mailing list with a cc. Thanks! Jen Galaxy team On 12/10/13 12:07 PM, Michael Mason wrote: > Hi, > > I have run some top hap output through the samtools BAM-to-SAM > wrapper. It creates the SAM file about 50% of the time. The rest of > the time it makes an empty file. In this case if I just rerun it often > works. I was wondering if anyone else has seen this behavior. I am > using a cloud server and think I might need to change some settings. > This causes my workflow to fail for many libraries so I also > interested in any workarounds anyone has figured out. > Thanks in advance for any help. > Cheers > Mike -- Jennifer Hillman-Jackson http://galaxyproject.org

2 2

How to get an API key during execution?
by Neil.Burdett＠csiro.au 10 Dec '13

10 Dec '13

Hi, I know that if I put : ${ __app__.model.User.get( $__user_id__ ).api_keys[0].key } in my tools xml file I am able to get the current api key. The xml calls my python script with this value, which i then use to execute workflows via the API. However, if this is value empty/blank is there a way I can generate an api key it from within my python script. i.e. is there a method I can call which would generate this for me? Thanks Neil

2 1

The latest galaxy-dist run.sh failed
by zong-pei.han＠imm.ox.ac.uk 10 Dec '13

10 Dec '13

Hi, We've just installed the latest galaxy-dist: hg clone https://bitbucket.org/galaxy/galaxy-dist/ cd galaxy-dist hg -v update stable The server runs: CentOS release 6.5, python 2.6.8, and mysql-5.1.71-1.el6.x86_64 "sh run.sh" has failed: ...... migrate.versioning.repository DEBUG 2013-12-10 09:35:51,145 Repository lib/galaxy/model/migrate loaded successfully migrate.versioning.repository DEBUG 2013-12-10 09:35:51,145 Config: {'db_settings': {'__name__': 'db_settings', 'required_dbs': '[]', 'version_table': 'migrate_version', 'repository_id': 'Galaxy'}} galaxy.model.migrate.check DEBUG 2013-12-10 09:35:51,152 MySQL_python egg successfully loaded for mysql dialect galaxy.model.migrate.check INFO 2013-12-10 09:35:51,309 No database, initializing Traceback (most recent call last): File "/wwwdata/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/wwwdata/galaxy/lib/galaxy/app.py", line 54, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options, app=self ) File "/wwwdata/galaxy/lib/galaxy/model/migrate/check.py", line 64, in create_or_verify_database db_schema = schema.ControlledSchema( engine, migrate_repository ) File "/wwwdata/galaxy/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/schema.py", line 32, in __init__ self.load() File "/wwwdata/galaxy/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/schema.py", line 44, in load self.table = Table(tname, self.meta, autoload=True) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py", line 318, in __new__ table._init(name, metadata, *args, **kw) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py", line 381, in _init self._autoload(metadata, autoload_with, include_columns) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py", line 408, in _autoload self, include_columns, exclude_columns File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2425, in run_callable return conn.run_callable(callable_, *args, **kwargs) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1968, in run_callable return callable_(self, *args, **kwargs) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py", line 260, in reflecttable return insp.reflecttable(table, include_columns, exclude_columns) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py", line 350, in reflecttable tbl_opts = self.get_table_options(table_name, schema, **table.kwargs) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py", line 178, in get_table_options **kw) File "<string>", line 1, in <lambda> File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py", line 47, in cache ret = fn(self, con, *args, **kw) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 2064, in get_table_options parsed_state = self._parsed_state_or_create(connection, table_name, schema, **kw) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 2163, in _parsed_state_or_create info_cache=kw.get('info_cache', None) File "<string>", line 1, in <lambda> File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py", line 47, in cache ret = fn(self, con, *args, **kw) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 2188, in _setup_parser full_name=full_name) File "/wwwdata/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 2275, in _show_create_table raise exc.NoSuchTableError(full_name) DatabaseNotControlledError: `migrate_version` Your help is much appreciated. Zong-Pei Han

1 1