April 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy 'environment' variables
by Joachim Jacob 11 Apr '12

11 Apr '12

Hello all, Where can I find a list of 'environment' variables in Galaxy, such as: $GALAXY_HOME ${GALAXY_DATA_INDEX_DIR} More specifically, I am looking for the temporary directory setting of a user's Galaxy (as set in the universe_wsgi.ini). Kind regards, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

1 0

solution + request -> RE: Error viewing BAM files in IGV
by Lukasse, Pieter 11 Apr '12

11 Apr '12

I had similar error stack trace and discovered that it does not occur in IE. It seems to be some problem in the combination Chrome + the simple web-server that comes by default with Galaxy. See for example the link http://stackoverflow.com/questions/8088941/broken-pipe-error-in-django-nonr… for similar error in another platform. @Galaxy developers: can you please figure this out? Thanks and regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl -----Original Message----- From: galaxy-dev-bounces(a)lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Nate Coraor Sent: woensdag 1 februari 2012 16:07 To: jj(a)umn.edu Cc: galaxy-dev(a)lists.bx.psu.edu Subject: Re: [galaxy-dev] Error viewing BAM files in IGV On Feb 1, 2012, at 10:05 AM, Jim Johnson wrote: > Nate, > > John Chilton got that fixed for us. As you suggest, it involved a fix in our proxy configuration. > > JJ Great, thanks! --nate > > On 2/1/12 9:01 AM, Nate Coraor wrote: >> On Dec 19, 2011, at 12:24 PM, Jim Johnson wrote: >> >>> Hi, >>> >>> I'm seeing the same behavior. Galaxy is returning a web page rather than the requested .bai index file for the BAM file. >>> In class WebApplication ( lib/galaxy/web/framework/base.py ) in __call__( self, environ, start_response ) line 133 >>> # Setup the transaction >>> trans = self.transaction_factory( environ ) Gets routed to >>> the root controller and returns the galaxy server main page html. >>> >>> JJ >> Hi JJ, >> >> What URL is causing this? I figured this was a proxy prefix problem, but I tried setting it up here and the URL to the bai is still correct: >> >> >> http://example.org/<proxy_prefix>/display_application/<encoded_datase >> t_id>/igv_bam/local_default/None/data/galaxy_<encoded_dataset_id>.bam >> .bai >> >> --nate >> >>> >>>> On Dec 14, 2011, at 4:22 PM, Alexander Graf wrote: >>>> >>>> >>>>> Hi nate, >>>>> I have tried it with several BAM files, resulting in the same error. >>>>> If I download the BAM and bai files from Galaxy and load it into IGV manually, everything works like charm. >>>>> Up to now I could not figure out why it is not working. >>>>> Could I have better success switching to the nginx-server? >>>>> >>>> Hi Alex, >>>> >>>> It should work with Apache as well. I don't have an environment set up here to test, but could you take a look at the Apache access and error logs to determine whether the file is being found and read properly? It's possible that the request is returning something other than a 200 code and the file data. >>>> >>>> --nate >>>> >>>> >>>>> Alex >>>>> >>>>> Am 12.12.2011 um 16:11 schrieb Nate Coraor: >>>>> >>>>> >>>>>> On Dec 9, 2011, at 6:45 AM, Alexander Graf wrote: >>>>>> >>>>>> >>>>>>> Hello, >>>>>>> I have recently updated our Galaxy dist and I'm running into problems viewing BAM files in IGV (v2.0.22), saying: Invalid GZIP header. >>>>>>> >>>>>> Hi Alex, >>>>>> >>>>>> Your config below looks okay at first glance. Can you verify that the file in question is a valid BAM? Or is this happening with all BAMs? >>>>>> >>>>>> --nate >>>>>> >>>>>> >>>>>>> I have configured the Apache as explained in the Wiki using this http.conf: >>>>>>> >>>>>>> >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> ---- >>>>>>> <VirtualHost *:80> >>>>>>> ServerName 127.0.0.1 >>>>>>> RewriteEngine on >>>>>>> >>>>>>> RewriteRule ^ >>>>>>> /galaxy$ /galaxy/ >>>>>>> [R] >>>>>>> RewriteRule ^ >>>>>>> /galaxy/static/style/(.*) >>>>>>> /opt/galaxy/static/june_2007_style/blue/ >>>>>>> $1 [L] >>>>>>> RewriteRule ^ >>>>>>> /galaxy/static/scripts/(.*) /opt/galaxy/static/scripts/packed/ >>>>>>> $1 [L] >>>>>>> RewriteRule ^ >>>>>>> /galaxy/static/(.*) /opt/galaxy/static/ >>>>>>> $1 [L] >>>>>>> RewriteRule ^/galaxy/favicon.ico /opt/galaxy/static/favicon.ico >>>>>>> [L] RewriteRule ^/galaxy/robots.txt >>>>>>> /opt/galaxy/static/robots.txt [L] RewriteRule ^/galaxy(.*) >>>>>>> http://localhost:8081$1 [P] >>>>>>> >>>>>>> <Proxy >>>>>>> http://localhost:8081 >>>>>>> Order deny,allow >>>>>>> Allow from all >>>>>>> </Proxy> >>>>>>> >>>>>>> <Location "/galaxy"> >>>>>>> # Define the authentication method >>>>>>> XSendFile on >>>>>>> XSendFilePath / >>>>>>> # Compress all uncompressed content.http.conf >>>>>>> SetOutputFilter DEFLATE >>>>>>> SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary >>>>>>> SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip >>>>>>> dont-vary </Location> <Directory "/galaxy/static"> >>>>>>> ExpiresActive On >>>>>>> ExpiresDefault "access plus 6 hours" >>>>>>> </Directory> >>>>>>> </VirtualHost> >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> -------- >>>>>>> >>>>>>> >>>>>>> By having this parts changed in universe_wsgi.ini: >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> -------- >>>>>>> [server:main] >>>>>>> use = egg:Paste#http >>>>>>> port = 8081 >>>>>>> host = 0.0.0.0 >>>>>>> use_threadpool = True >>>>>>> >>>>>>> [filter:gzip] >>>>>>> use = egg:Paste#gzip >>>>>>> >>>>>>> [filter:proxy-prefix] >>>>>>> use = egg:PasteDeploy#prefix >>>>>>> prefix = /galaxy >>>>>>> >>>>>>> [app:main] >>>>>>> paste.app_factory = galaxy.web.buildapp:app_factory filter-with >>>>>>> = proxy-prefix cookie_path = /galaxy apache_xsendfile = True >>>>>>> upstream_gzip = False >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> -------- >>>>>>> >>>>>>> The resulting Galaxy error log is: >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> ------------------------- galaxy.web.framework DEBUG 2011-12-09 >>>>>>> 12:32:08,825 Error: this request returned None from get_history(): >>>>>>> http://10.153.182.203/galaxy/root >>>>>>> >>>>>>> 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "GET /galaxy/root?app_action=data&user_id=c9a3f3a19e75965d&app_name=igv_bam&link_name=local_default&action_param=galaxy_9b0f702d0207cd78.bam.bai&dataset_id=9b0f702d0207cd78 HTTP/1.1" 200 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" >>>>>>> 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "HEAD /galaxy/display_application/9b0f702d0207cd78/igv_bam/local_default/c9a3f3a19e75965d/data/galaxy_9b0f702d0207cd78.bam HTTP/1.1" 302 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" >>>>>>> galaxy.web.framework DEBUG 2011-12-09 12:32:08,915 Error: this request returned None from get_history(): >>>>>>> http://10.153.182.203/galaxy/root >>>>>>> >>>>>>> 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "HEAD /galaxy/root?app_action=data&user_id=c9a3f3a19e75965d&app_name=igv_bam&link_name=local_default&action_param=galaxy_9b0f702d0207cd78.bam&dataset_id=9b0f702d0207cd78 HTTP/1.1" 200 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" >>>>>>> ---------------------------------------- >>>>>>> Exception happened during processing of request from >>>>>>> ('127.0.0.1', 52683) Traceback (most recent call last): >>>>>>> File "/opt/galaxy/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 1053, in process_request_in_thread >>>>>>> self.finish_request(request, client_address) File >>>>>>> "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request >>>>>>> self.RequestHandlerClass(request, client_address, self) File >>>>>>> "/usr/lib/python2.6/SocketServer.py", line 618, in __init__ >>>>>>> self.finish() >>>>>>> File "/usr/lib/python2.6/SocketServer.py", line 661, in finish >>>>>>> self.wfile.flush() >>>>>>> File "/usr/lib/python2.6/socket.py", line 297, in flush >>>>>>> self._sock.sendall(buffer(data, write_offset, buffer_size)) >>>>>>> error: [Errno 32] Broken pipe >>>>>>> ---------------------------------------- >>>>>>> >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> ---------------------------------------------------------------- >>>>>>> --------------------------- >>>>>>> >>>>>>> >>>>>>> Thanks in advance for your help >>>>>>> >>>>>>> Alex >>>>>>> >>>>>>> ___________________________________________________________ >>>>>>> Please keep all replies on the list by using "reply all" >>>>>>> in your mail client. To manage your subscriptions to this and >>>>>>> other Galaxy lists, please use the interface at: >>>>>>> >>>>>>> >>>>>>> http://lists.bx.psu.edu/ >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this and other >>> Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

1 0

Nginx upload configuration question
by Bartlett G Ailey 11 Apr '12

11 Apr '12

Dear All, When I turn on Nginx upload (as below) I get and error: Not Found The resource cannot be found No route for /data/database/tmp/upload_store The nginx upload settings for universe.ini are: # nginx can also handle file uploads (user-to-Galaxy) via nginx_upload_module. # Configuration for this is complex and explained in detail in the # documentation linked above. The upload store is a temporary directory in # which files uploaded by the upload module will be placed. nginx_upload_store = True # This value overrides the action set on the file upload form, e.g. the web # path where the nginx_upload_module has been configured to intercept upload # requests. nginx_upload_path = /data/database/tmp/upload_store The nginx.conf settings are: location /_upload { upload_store /data/database/tmp/upload_store; upload_pass_form_field ""; upload_set_form_field "__${upload_field_name}__is_composite" "true"; upload_set_form_field "__${upload_field_name}__keys" "name path"; upload_set_form_field "${upload_field_name}_name" "$upload_file_name"; upload_set_form_field "${upload_field_name}_path" "$upload_tmp_path"; upload_pass_args on; upload_pass /_upload_done; } location /_upload_done { set $dst /tool_runner/index; if ($args ~ nginx_redir=([^&]+)) { set $dst $1; } rewrite "" $dst; } } If I comment out the 2 universe.ini settings above then the upload works correctly. I would appreciate any suggestions, Yours Bart.

2 2

Is there any way to restrict the access of a local Galaxy instance
by JIE CHEN 11 Apr '12

11 Apr '12

Dear all, I installed a local Galaxy instance for my company. Different from the regular setting, we want to restrict the access of our local Galaxy instance only to the people in our company, which means it is not open to public. After new configuration, to use the service provided by our Galaxy instance, a user must log in first. And to register a new user account, the user must use a valid email with a valid domain. Any one have any idea how can i do that? Or any direction i should go into to configure it? Any advise is welcome. Thank you! Tyler

2 1

Directories in $output.files_path
by Jose Navas 10 Apr '12

10 Apr '12

Hi Galaxy developers, I'm adding a tool to my Galaxy instance which generates some directories in the directory $output.files_path, but when I look in database/files/000/dataset_X_files/ directory, the directories has been copied as an empty plain text files. There is any option in universe_wsgi.ini which I have to modify? Thanks, Jose

1 0

Re: [galaxy-dev] Problem uploading files from filesystem paths
by Robert Chase 10 Apr '12

10 Apr '12

Hello, I set the library_import_dir setting in my universe_wsgi.ini file so that I could upload a directory of files into galaxy. I'm getting the following error screen when I attempt the upload from /tmp. The permissions are that galaxy owns the files in /tmp. I get the same error message when I try to upload the files from my computer. -rw-r--r-- 1 galaxy stapusr 656960285 Apr 10 14:36 117681r.bam Uploaded by:rerpc@channing.harvard.edu Date uploaded:2012-04-10 File size:626.5 Mb Data type:auto Build:? Miscellaneous information:uploaded bam fileTraceback (most recent call last): File "/share/shared/galaxy-central-hbc/tools/data_source/upload.py", line 410, in __main__() File "/share/shared/galaxy-central-hbc/tools/data_source/upload.py", line 399, in __main__ Job Standard Error Traceback (most recent call last): File "/share/shared/galaxy-central-hbc/tools/data_source/upload.py", line 410, in __main__() File "/share/shared/galaxy-central-hbc/tools/data_source/upload.py", line 399, in __main__ add_file( dataset, registry, json_file, output_path ) File "/share/shared/galaxy-central-hbc/tools/data_source/upload.py", line 338, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/share/shared/galaxy-central-hbc/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/share/shared/galaxy-central-hbc/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/local/python27/lib/python2.7/subprocess.py", line 672, in __init__ errread, errwrite) File "/usr/local/python27/lib/python2.7/subprocess.py", line 1202, in _execute_child raise child_exception OSError: [Errno 13] Permission denied error Database/Build:? Number of data lines:None Disk file:/share/shared/galaxy-central-hbc/database/files/000/dataset_40.dat

1 0

Traceback on original execution
by Daniel Patrick Sullivan 10 Apr '12

10 Apr '12

Hi, Galaxy, Developers, My name is Dan Sullivan and I work at the University of Chicago in the newly-formed Center for Research Informatics. This is my first post to this list, and I am a very new Galaxy user, so please be patient with me :-) I have a question regarding an error I am reciecveing when I try to start up galaxy for the first time. I think it might be related to the older version of python I am running (2.4.3-46.el5). I am running this on an RHEL 5 box and the output of "hg parent" command shows 6799 as the chageset number. When I attempt to start galaxy for the first time, I get the following traceback: Traceback (most recent call last): File "./scripts/paster.py", line 34, in ? command.run() File "/opt/local/galaxy-dist/eggs/PasteScript-1.7.3-py2.4.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/opt/local/galaxy-dist/eggs/PasteScript-1.7.3-py2.4.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/opt/local/galaxy-dist/eggs/PasteScript-1.7.3-py2.4.egg/paste/script/command.py", line 218, in run result = self.command() File "/opt/local/galaxy-dist/eggs/PasteScript-1.7.3-py2.4.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/opt/local/galaxy-dist/eggs/PasteScript-1.7.3-py2.4.egg/paste/script/serve.py", line 311, in loadapp return loadapp( File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 224, in loadobj global_conf=global_conf) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 248, in loadcontext global_conf=global_conf) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 278, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 413, in get_context section) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 458, in _context_from_explicit value = import_string(found_expr) File "/opt/local/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.4.egg/paste/deploy/loadwsgi.py", line 18, in import_string return pkg_resources.EntryPoint.parse("x="+s).load(False) File "/opt/local/galaxy-dist/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/opt/local/galaxy-dist/lib/galaxy/web/buildapp.py", line 19, in ? from galaxy import config, jobs, util, tools File "/opt/local/galaxy-dist/lib/galaxy/jobs/__init__.py", line 4, in ? from galaxy import util, model File "/opt/local/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/opt/local/galaxy-dist/lib/galaxy/datatypes/registry.py", line 146 finally: ^ SyntaxError: invalid syntax [dansully@pegasus tmp]$ Is this somthing that somebody could possibly commment on? Should I just try to install a more recent version of python? Thank-you so much for your help and guidance. Dan Sullivan

2 1

is there a simple way to access an arbitrary dataset's absolute path?
by W. Augustine Dunn III 10 Apr '12

10 Apr '12

Hello all: There are times when I want to explore data produced inside my local galaxy site without using galaxy tools. For example, I may want to run a custom script or even just a grep type function from the command line on a couple CuffDiff output files to get an idea of whats inside. This requires a way to get the absolute path to that dataset as it lives in galaxy's internal structure. The only way that I have found to do this is to place this dataset in the shared_data/data_libraries and then view this dataset's info as admin. This is not practical for the types of things I want to do (simple one-off old-school command line investigations on arbitrary files). Is there a way for me to add this info to the standard "view info" page format of datasets so that I can do this sort of thing easily? Mind you, I am not planning to alter the data in galaxy, just look at it and sometimes dump a summary table to a folder outside of galaxy's domain. Thanks, Gus -- In science, "fact" can only mean "confirmed to such a degree that it would be perverse to withhold provisional assent." I suppose that apples might start to rise tomorrow, but the possibility does not merit equal time in physics classrooms. *-Stephen Jay Gould*

2 2

Re: [galaxy-dev] [galaxy-user] permissions on folder created by Galaxy
by Jennifer Jackson 10 Apr '12

10 Apr '12

Hello, I am going to forward your question over to the galaxy-dev(a)bx.psu.edu mailing list so that it can get more visibility in the development community. That list is probably where you will want to post regarding similar topics in the future: http://wiki.g2.bx.psu.edu/Support#Mailing_Lists To get things started, I am wondering if you are asking about datasets in histories or data in libraries? Parts of your question seem to refer to both, or I have misunderstood (my apologies!). If about data in libraries, then you can get around this by not actually moving the data into the Galaxy directory structure at all, and instead just link in data from some other location on the filesystem (where you can control the permissions). See the "Copy data into Galaxy? No" option described on this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files Other data/lib/security related links, mostly admin UI: http://wiki.g2.bx.psu.edu/Learn/Security Features http://wiki.g2.bx.psu.edu/Admin/Data Libraries/Libraries http://wiki.g2.bx.psu.edu/Admin/Data Libraries/Library Security Hopefully this helps or some more discussion on the galaxy-dev list about your question will find a solution, Jen Galaxy team On 4/6/12 2:42 AM, Guilhem Sempéré wrote: > Hi, > > I would like folders created by Galaxy under the location defined by the > "file_path" parameter (which defaults to database/files) to be created > with permissions 775 instead of 755. Is this possible? > > The reason for me asking this question is that I have developed an > external tool (running under Tomcat) for feeding Galaxy libraries. And > my tomcat user needs to be allowed to write to the mentioned folders. > > Any ideas? > > G. > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

2 1

Using quota won't decrease
by Makis Ladoukakis 10 Apr '12

10 Apr '12

Dear Galaxy users, I run a local instance of Galaxy on my university's server and I stumbled upon an issue concerning using quota on the upper right corner of the interface. It says I'm using a certain amount of GBs even though I have deleted all raw data from the /galaxy-dist/database/files directory. Has anyone encountered this sort of problem before? Thank you in advance, Makis Ladoukakis

1 0