September 2013 - galaxy-dev - lists.galaxyproject.org

Bug: Galaxy Tool Shed dependency communication via HTTP GET method
by Richard Burhans 27 Sep '13

27 Sep '13

When installing Galaxy Tool Shed repositories into a local Galaxy instance, tool shed dependecies are exchanged as an encoded python dictionary via the HTTP GET method. If a tool shed repository contains enough dependencies, the length of this encoded dictionary can become extremely long. In my case, it became long enough for my Apache server to choke on it: [Wed Sep 11 15:34:53 2013] [error] [client XXX.XXX.XXX.XXX] request failed: URI too long (longer than 8190) The HTTP standard does not place a limit on the length of a URI, but the de facto limit is 2000 characters. See the following stackoverflow answer: http://stackoverflow.com/a/417184 I would recommend either: 1) communicating toolshed dependencies via an HTTP POST 2) redesigning tool shed <-> instance communications As a stopgap measure, I've modified lib/tool_shed/util/encoding_util.py to use a more compact encoding scheme. I'm attaching a copy in case someone else runs into the same problem. -rico

2 4

Slice bam
by jean_jasinski＠agilent.com 27 Sep '13

27 Sep '13

I wanted to remove all the non-standard reads (reads from non-standard chromosomes) from a bam file. I used Slice bam and the bed file contained all of the chromosome lengths for chr1-22, x, and y. But all I get back in the bam file are the unaligned chromosomes. What am I doing wrong? Jean Jasinski

1 0

Partial test information on (Test) Tool Shed
by Peter Cock 27 Sep '13

27 Sep '13

Hi all, This is a slight change to my usual theme of completely missing nightly test results ;) Looking at http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/70e7dcbf6573 (Main Tool Shed) or http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/aecdffbc08fc (Test Tool Shed) today, illustrated with a little ASCII art, we have: Automated tool test results | Tool test results |-- Automated test environment |-- Tools missing tests or test data These sections can be expanded for details about the test environment or missing tests. What is missing is the section for the successful tool tests - is this deliberate, or a bug? Most of the NBCI BLAST+ tools have tests which should be passing, and this positive information is being hidden from the Tool Shed user :( Regards, Peter

1 0

pass user groups to dynamic job runner
by Vandeweyer Geert 27 Sep '13

27 Sep '13

Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok => run ; fail => give message. Or can I hide tools from the menu based on usergroup? Best, Geert

3 5

dead link
by Kevin Viken 26 Sep '13

26 Sep '13

The link for datasets is dead: http://screencast.g2.bx.psu.edu/usinggalaxy_datasets/flow.html thanks, Kevin

1 0

Access to history_id from tool config XML
by Tim Dreszer 26 Sep '13

26 Sep '13

It would be useful to have access $__history_id__ in the same way as one accesses $__user_id__. I am running a local instance of galaxy and have modified the following file to achieve this: lib/galaxy/jobs/__init__.py At line ~694: incoming['__user_name__'] = user_name + if job.history and job.history.id: + incoming['__history_id__'] = job.history.id + else: + incoming['__history_id__'] = 'unknown' I have tested this change and it appears to give me exactly what I want. My question: does this change appear correct and can it be incorporated into the main galaxy code-base? Thanks, -tim

2 4

Limit number of Jobs to Trinity
by Carlos Canchaya 26 Sep '13

26 Sep '13

Hi guys, I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis? Bests, Carlos -- Carlos Canchaya

4 5

Re: [galaxy-dev] pgcleanup problem
by Eric Kuyt 26 Sep '13

26 Sep '13

Hi Dannon, I am also using postgres 9.1.9 (from Ubuntu precise repository) > $/usr/lib/postgresql/9.1/bin/postgres --version > -> postgres (PostgreSQL) 9.1.9 > hg summary > -> parent: 10411:c42567f43aa7 tip ./scripts/cleanup_datasets/pgcleanup.py -d -o1 -s 'delete_datasets' > __load_config INFO 2013-09-26 10:02:10,368 Reading config from > /opt/galaxy_testing/universe_wsgi.ini > __connect_db INFO 2013-09-26 10:02:10,371 Connecting to database with URL: > #############@#### > Traceback (most recent call last): > File "./scripts/cleanup_datasets/pgcleanup.py", line 774, in <module> > cleanup = Cleanup() > File "./scripts/cleanup_datasets/pgcleanup.py", line 55, in __init__ > self.__connect_db() > File "./scripts/cleanup_datasets/pgcleanup.py", line 115, in __connect_db > self.conn = psycopg2.connect(**args) > TypeError: 'username' is an invalid keyword argument for this function and if I add *args['user'] = args['username'] > **del args['username']* @ line 112 in pgcleanup.py the script seems to work. ./scripts/cleanup_datasets/pgcleanup.py -d -o1 -s 'delete_datasets' > delete_datasets INFO 2013-09-26 10:04:10,430 Marking deleted all Datasets > whose associations are all marked as deleted/purged that are older than the > specified number of days. > _create_event DEBUG 2013-09-26 10:04:10,457 SQL is: > INSERT INTO cleanup_event > (create_time, message) > VALUES (NOW(), 'delete_datasets') > RETURNING id; > > _create_event INFO 2013-09-26 10:04:10,457 Executing SQL > _create_event INFO 2013-09-26 10:04:10,458 Database status: INSERT 0 1 > _update DEBUG 2013-09-26 10:04:10,458 SQL is: > WITH deleted_dataset_ids > AS ( UPDATE dataset > SET deleted = true, > update_time = NOW() > WHERE NOT deleted > AND NOT EXISTS (SELECT true > FROM > library_dataset_dataset_association > WHERE (NOT deleted > OR update_time > >= (NOW() - interval '1 days')) > AND dataset.id = > dataset_id) > AND NOT EXISTS (SELECT true > FROM > history_dataset_association > WHERE (NOT purged > OR update_time > >= (NOW() - interval '1 days')) > AND dataset.id = > dataset_id) > RETURNING id), > dataset_events > AS (INSERT INTO cleanup_event_dataset_association > (create_time, cleanup_event_id, > dataset_id) > SELECT NOW(), 2, id > FROM deleted_dataset_ids) > SELECT id > FROM deleted_dataset_ids > ORDER BY id; > > _update INFO 2013-09-26 10:04:10,458 Executing SQL > _update INFO 2013-09-26 10:04:10,461 Database status: SELECT 0 > _flush INFO 2013-09-26 10:04:10,475 All changes committed > _open_logfile DEBUG 2013-09-26 10:04:10,475 Opening log file: > /opt/galaxy_testing/scripts/cleanup_datasets/delete_datasets.log > _close_logfile DEBUG 2013-09-26 10:04:10,476 Closing log file: > /opt/galaxy_testing/scripts/cleanup_datasets/delete_datasets.log > _run INFO 2013-09-26 10:04:10,476 Finished delete_datasets On 25 September 2013 17:45, Dannon Baker <dannon.baker(a)gmail.com> wrote: > Eric, > > What version of posgresql are you using? The script has a comment that > indicates you need 9.1+ (and I'm using 9.1.9), and it works out of the box > for me. > > -Dannon > > > On Fri, Sep 20, 2013 at 7:35 AM, Eric Kuyt <eric.kuijt(a)wur.nl<mailto: > eric.kuijt(a)wur.nl>> wrote: > Hi All, > > I'm trying to do some cleanup in my test environment (galaxy-dist) > > and pgcleanup.py ends with > > Traceback (most recent call last): > File "./scripts/cleanup_datasets/pgcleanup.py", line 773, in <module> > cleanup = Cleanup() > File "./scripts/cleanup_datasets/pgcleanup.py", line 55, in __init__ > self.__connect_db() > File "./scripts/cleanup_datasets/pgcleanup.py", line 114, in __connect_db > self.conn = psycopg2.connect(**args) > TypeError: 'username' is an invalid keyword argument for this function > > Unless I add > > args['user'] = args['username'] > del args['username'] > > > Is this a bug or a config error? > > Eric > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > > -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

1 0

Need help setting up galaxy with swift object store.
by Varun Mittal 25 Sep '13

25 Sep '13

Hello everyone, I have configured swift object store at my infrastructure. Next i want to configure galaxy to be able to use swift object store. I am having trouble with configuration. Here's what the error is: galaxy.objectstore DEBUG 2013-09-26 08:07:05,306 Getting a connection object for 'swift' object store Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/home/galaxy/galaxy-dist/lib/galaxy/app.py", line 57, in __init__ self.object_store = build_object_store_from_config(self.config, fsmon=True) File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 1031, in build_object_store_from_config return S3ObjectStore(config=config) File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 387, in __init__ self.bucket = self._get_bucket(self.config.os_bucket_name) File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 476, in _get_bucket bucket = self.s3_conn.get_bucket(bucket_name) File "/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py", line 391, in get_bucket bucket.get_all_keys(headers, maxkeys=0) File "/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/bucket.py", line 360, in get_all_keys '', headers, **params) File "/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/bucket.py", line 317, in _get_all query_args=s) File "/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py", line 460, in make_request auth_path = self.calling_format.build_auth_path(bucket, key) File "/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py", line 92, in build_auth_path path = '/' + bucket TypeError: cannot concatenate 'str' and 'NoneType' objects Config in universe_wsgi.ini is: # Object store mode (valid options are: disk, s3, swift, distributed, hierarchical) object_store = swift os_access_key = 'RocksCluster:galaxy-swift-user' os_secret_key = password' #os_bucket_name = <name of an existing object store bucket or container> # If using 'swift' object store, you must specify the following connection properties os_host = 192.168.100.104 os_port = 8888 os_is_secure = False os_conn_path = /v2.0/ # Reduced redundancy can be used only with the 's3' object store #os_use_reduced_redundancy = False # Size (in GB) that the cache used by object store should be limited to. # If the value is not specified, the cache size will be limited only by the # file system size. The file system location of the cache is considered the # configuration of the ``file_path`` directive defined above. #object_store_cache_size = 100 -- Varun Mittal varunmittal91(a)gmail.com

1 0

Visualisation of VCF files in trackster
by Ulf Schaefer 25 Sep '13

25 Sep '13

Dear all My attempts to visualise a vcf file in trackster fail. I can visualise the corresponding SAM/BAM files fine. I can visualise the same file on Galaxy main too, but not in my local install. I receive the following error message: "Trackster Error Input error: found interval with block-counts not matching starts/sizes on line. sort: write failed: standard output: Broken pipe sort: write error needLargeMem: trying to allocate 0 bytes (limit: 100000000000)" The vcf format is in version 4.1. I am creating it from the sorted BAM file with samtools mpileup in the most basic way. What am I missing? Thanks a lot for your help Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

2 3