- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Large files in blend4j
by Eric Kuyt 10 Sep '13

10 Sep '13

Hi John, Some news on the debugging yet? I managed to deploy blend4j in my local maven installation, so now I can test the client.setChunkedEncodingSize(CHUNKED_ENCODING_SIZE); line on my setting. Unfortunately with ChunkedEncoding enabled, it seems that the file is not (recognized by || sent to) galaxy. this is the errormessage: galaxy.web.framework ERROR 2013-09-10 16:06:15,688 Uncaught exception in exposed API method: Traceback (most recent call last): File "/opt/galaxy_testing/lib/galaxy/web/framework/__init__.py", line 189, in decorator rval = func( self, trans, *args, **kwargs) TypeError: create() takes exactly 3 arguments (2 given) 10.79.24.114 - - [10/Sep/2013:16:06:15 +0200] "POST /galaxy_testing/api/tools?key=1234567890abcdef1234567890abcdef1 HTTP/1.1" 500 - "-" "Java/1.7.0_25" Do you know this problem, or could there also be something wrong with my config? Thanks for the fast reply! On 9 September 2013 19:51, John Chilton <chilton(a)msi.umn.edu> wrote: > Glad to hear blend4j is working well, annoyed to here it is not > streaming files out by default. > > Based on your ideas I started some work here, but it is not right. I > spent the last half an hour looking around Galaxy trying to figure out > why history creation via the API was broken, not realizing it was > because I called setChunkedEncodingSize on the client. > > https://github.com/jmchilton/blend4j/commit/22608de210f805fc192c86a836ea55c… > > I have created an issue in github for this: > > https://github.com/jmchilton/blend4j/issues/5 > > Feel free to try to hack on this some more, it might actually take > some non-trivial refactoring to get > client.setChunkedEncodingSize(CHUNKED_ENCODING_SIZE); to be set > conditionally somehow. Right now all requests are sharing the same > underlying client object. > > I will try to work on this more as I have time. > > -John > > On Mon, Sep 9, 2013 at 10:58 AM, Eric Kuyt <eric.kuijt(a)wur.nl> wrote: >> Hi all, >> >> I'm using blend4j for some time now and I'm quite happy about it, but >> now I run into some OutOfMemory issues. >> When I try to upload large files to galaxy (I know http might be not >> the best option) some byte array is getting to large. A possible >> solution would be file chunking in jersey, is this possible in blend4j >> and how could I implement this feature? >> >> Thanks, >> >> Eric >> > > -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

1 0

Resolving requirement type "binary" and "python-module"
by Hans-philipp Brachvogel 10 Sep '13

10 Sep '13

Hi all, we run galaxy on a cluster and real user job sumission (via DRMAA). The problem is getting the Galaxy standard tools to work with environment-module. We use environment-modules to fit the environment to the needs of a job and want to do the same for galaxy jobs. I highly expect the need to handle different versions of the same program on our galaxy instance. It is no problem to configure indivdual evironments for new tools, as we can simply set the necessary <requirement type="package" version="xyz"> entries in the tool.xml and set a 'module load xyz' in the corresponding env.sh files. The challenge is to get the galaxy standard tools to work. Several of them do not define a requirement-package tag. Instead one finds the types "binary" or "python-module". But these are not handeled by Managed Tool Dependencies (env.sh). Especially tools that require python-module=rpy, should also have a package=R, or: if python-module=Gnuplot I would also need a package='gnuplot' entry to trigger the corresponding module load command. I also found binary=gnuplot, or binary=R... these also need to be loaded on our system. The only way I can think of solving this, is by editing these tool.xmls by hand or do you guys think there is a more general solution? Are there by chance any methods to resolve binaries and python-modules? (could not find anything about it) Greetings, Phil

3 4

Deleted Accounts
by Ricardo Lebrón Aguilar 10 Sep '13

10 Sep '13

Dear Administrator, My apologies for using more than one account in Galaxy Main Server. I am a master student. I present my thesis this month. Please, re-enable my accounts for a week to give me time to download the data I need. I have no time to redo all my experiments. I'm sorry for the inconvenience. I would not do it again. Please consider my request. Thank you very much. Regards, Ricardo Lebrón.

1 0

Statistical Genomics Post-doctoral Position in Makova Lab at Penn State
by Dave Clements 09 Sep '13

09 Sep '13

Hello all, There's a post-doc opening in the Makova Lab at Penn State. See below. Dave C Are you interested in genomics and do you have skills in Bioinformatics, Computational Biology and Statistics? The Makova lab in the Department of Biology at The Pennsylvania State University is looking to hire a postdoctoral researcher for an NSF-funded project examining regional variation in mutation rates (see our recent publication in PNAS: Kuruppumullage Don, Ananda, Chiaromonte, Makova 2013)<http://www.pnas.org//content/early/2013/08/16/1221792110.abstract>. With new sequencing technologies, multiple human genomes and their detailed annotations (e.g., ENCODE) are suddenly accessible to us. This gives us a terrific opportunity to explore previously inaccessible evolutionary processes (e.g., mutation) and other biological associations (location of genes in certain mutation states, as identified by HMMs). Such knowledge is vital in a clinical setting where disease mutations need to be evaluated. Our resources and links with medical researchers at Hershey Medical School and computational biologists from the Galaxy team<http://galaxyproject.org/>put us in a great position to address these questions. Candidates should have experience in bioinformatics, working knowledge of statistics and should have a broad understanding of molecular biology and genetics. Familiarity with next-generation sequencing data analysis is desirable. A PhD is required. You will be joining an established dynamic group. We are part of the Center for Medical Genomics<http://www.huck.psu.edu/center/medical-genomics>and of the Center for Comparative Genomics and Bioinformatics <http://www.bx.psu.edu/>. Penn State is a vibrant scientific community with particular strengths in genomics, bioinformatics and molecular evolution. Our location, in State College, Pennsylvania, is known for excellent schools and numerous opportunities for outdoor activities. The starting date is flexible, with an earlier date preferred. This position is funded for one year from date of hire, with good possibility of refunding. Interested applicants should send a pdf with a CV, a statement of research interests, and contact information for three referees to Kateryna Makova <kmakova(a)bx.psu.edu>, indicating postdoc in the subject line. Employment will require successful completion of background check(s) in accordance with University policies. Penn State is committed to affirmative action, equal opportunity, and the diversity of its workforce. -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0

questions on Galaxy installation under Apache
by Jingchao Zhang 09 Sep '13

09 Sep '13

Hello everyone, I installed both Galaxy and Apache in a Red Hat linux machine and they work fine individually. If I open http://localhost then a welcome page comes out showing the httpd is working. In galaxy I changed host=0.0.0.0 and "sh run.sh", then the http://localhost<http://localhost/>:8080 will direct me to the Galaxy page. Now I want to put Galaxy under Apache and here is what I did: 1. First I comment the "host=0.0.0.0" line to stop the build in web server in Galaxy and then "sh run.sh" to make Galaxy running. 2. Then I put these lines RewriteEngine On RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] into my "httpd.conf" file. Then I restart the httpd service. Also, I made changes to the "universe_wsgi.ini" file as instructed and restart Galaxy: [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy After I do these, either http://localhost<http://localhost/>:8080 nor http://localhost<http://localhost/>/galaxy works. I find two threads talked about this problem before: http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td4654499.… http://dev.list.galaxyproject.org/Running-Galaxy-behind-apache2-td4624545.h… but my Red Hat httpd does not has this "/etc/apache2/sites-available/default" directory and only has "/etc/httpd/conf/httpd.conf". Could someone please tell me where should I put those lines in the "httpd.conf" file? I have tries various places and none of them works. Any help would be deeply appreciated. Thanks, Jim

2 1

Large files in blend4j
by Eric Kuyt 09 Sep '13

09 Sep '13

Hi all, I'm using blend4j for some time now and I'm quite happy about it, but now I run into some OutOfMemory issues. When I try to upload large files to galaxy (I know http might be not the best option) some byte array is getting to large. A possible solution would be file chunking in jersey, is this possible in blend4j and how could I implement this feature? Thanks, Eric

2 1

View Tool Wrapper Code without Installing
by Richard Kuo 09 Sep '13

09 Sep '13

Hello, Is there a way to see the code for the tool wrappers in the tool shed without downloading them? I seem to remember that you used to be able to see the xml and other code before but now it looks like you just have to install directly from the tool shed. Thank you, Richard

2 1

API question: is it possible to start exporting histories
by Joachim Jacob | VIB | 09 Sep '13

09 Sep '13

Hi all, Is it currently possible to export histories via the API? I have one user who want to download and store her data. Total sizes of all her histories combined is about ~700GB. I think this is a typical API job: to list all her histories, start 'exporting to file' all of them, and finally download all exported histories to a disk. But looking at the current API code, I doubt whether this can be done. Hence my question. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team

2 2

"from galaxy.datatypes.data import Data" import error
by Björn Grüning 07 Sep '13

07 Sep '13

Hi, can anyone confirm that circular import error? Thanks, Bjoern --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-6-4ce0a1a2e4d4> in <module>() ----> 1 from galaxy.datatypes.data import Text, Data /home/bag/projects/galaxy/galaxy-central/lib/galaxy/datatypes/data.py in <module>() 1 import logging ----> 2 import metadata 3 import mimetypes 4 import os 5 import shutil /home/bag/projects/galaxy/galaxy-central/lib/galaxy/datatypes/metadata.py in <module>() 19 from os.path import abspath 20 ---> 21 import galaxy.model 22 from galaxy.util import listify, stringify_dictionary_keys, string_as_bool 23 from galaxy.util.odict import odict /home/bag/projects/galaxy/galaxy-central/lib/galaxy/model/__init__.py in <module>() 21 22 import galaxy.datatypes ---> 23 import galaxy.datatypes.registry 24 import galaxy.security.passwords 25 from galaxy.datatypes.metadata import MetadataCollection /home/bag/projects/galaxy/galaxy-central/lib/galaxy/datatypes/registry.py in <module>() 3 """ 4 import os, sys, tempfile, threading, logging, imp ----> 5 import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, assembly, ngsindex, graph 6 import galaxy.util 7 from galaxy.util.odict import odict /home/bag/projects/galaxy/galaxy-central/lib/galaxy/datatypes/tabular.py in <module>() 10 from cgi import escape 11 from galaxy import util ---> 12 from galaxy.datatypes import data 13 from galaxy.datatypes import metadata 14 from galaxy.datatypes.checkers import is_gzip ImportError: cannot import name data

1 0

Tool-shed: Migration scripts vs direct install
by Shantanu Pavgi (Campus) 06 Sep '13

06 Sep '13

I was wondering if we can move dist installed tools to tool-shed version by installing them manually using the admin web interface. Do migration scripts provide any functionality that we would lose if we use admin web interface (dist_to_shed column in database and it's significance)? -- Thanks, Shantanu

1 0