galaxy-dev November 2014

galaxy-dev@lists.galaxyproject.org

61 participants
91 discussions

Nginx Config
by Nicholas Kline 01 Dec '14

01 Dec '14

Hi, I am trying to setup a local production Galaxy running on Ubuntu, Nginx, PostgreSQL, and ProFTD. When visiting http://localhost, "502 Bad Gateway" is returned. Visiting https://localhost returns "unable to connect". I was expecting to see the Galaxy website instead. My Nginx configuration file is included below. Thank you for your help. ########################### # /etc/nginx/nginx.conf ########################### user www-data; worker_processes 4; pid /run/nginx.pid; events { worker_connections 768; # multi_accept on; } http { ## # Basic Settings ## sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; gzip_http_version 1.1; gzip_vary on; gzip_comp_level 4; gzip_proxied any; gzip_types text/plain text/css application/x-javascript text/xml application/xml text/javascript application/json; gzip_buffers 16 8k; gzip_disable "MSIE [1-6].(?!.*SV1)"; ## # nginx-naxsi config ## # Uncomment it if you installed nginx-naxsi ## #include /etc/nginx/naxsi_core.rules; ## # nginx-passenger config ## # Uncomment it if you installed nginx-passenger ## #passenger_root /usr; #passenger_ruby /usr/bin/ruby; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; #include /etc/nginx/sites-enabled/*; ## # Galaxy ## upstream galaxy_app { # As defined in galaxy config file universe_wsgi.ini server localhost:8080; server localhost:8081; server localhost:8082; server localhost:8083; server localhost:8084; server localhost:8085; server localhost:8086; server localhost:8087; server localhost:8088; server localhost:8089; } server { # Maximum upload size that can be handled by POST requests through nginx. # Disabled because using FTP instead of web interface. #client_max_body_size 10G; location / { proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # If using SSL, makes Galaxy aware of what type of URL it should generate for external sites like Biomart. proxy_set_header X-URL-SCHEME https; } # nginx is more efficient at serving static content. # expires 24h is for caching purposes. location /static { alias /home/galaxy/galaxy-dist/static; expires 24h; } location /static/style { alias /home/galaxy/galaxy-dist/static/june_2007_style/blue; expires 24h; } location /static/scripts { alias /home/galaxy/galaxy-dist/static/scripts/packed; } location /favicon.ico { alias /home/galaxy/galaxy-dist/static/favicon.ico; } location /robots.txt { alias /home/galaxy/galaxy-dist/static/robots.txt; } # Use nginx to send files (downloads) instead of Galaxy. location /_x_accel_redirect/ { internal; alias /; } } } ########################### # /var/log/nginx/error.log ########################### 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8089/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8081/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8082/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8083/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8084/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8085/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8086/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8087/", host: "localhost" 2014/11/25 15:43:21 [error] 56433#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8088/", host: "localhost"

3 2

Creation of a backup/migrate script for data/users
by Rémy Dernat 28 Nov '14

28 Nov '14

Hi, I would like to create a code to backup and migrate(*) galaxy datas, based on the api and/or with the script I did here: https://github.com/remyd1/galaxy_debug_params/blob/master/source/Templates/… To do that, I would use the galaxy model, to retrieve everything in the old one and recreate everything in the new one. However, I would like to be able to do that without my web server running. That is to say, a "cold" migration to avoid any current task to write additionnal things into the database. Obviously, that is a problem for the api because this one use the web server. Could you give some advice to do that ? Should I recreate a code from 0 ? Could I use the code in the api ? Is there a way to create a code that could use the api differently (recreate a copy of the api without the web server) ? Kind regards, Remy (*) this is important; that means you could import data from an old galaxy to a recent one (or the opposite); a simple backup of files and a dump of the db is not enough.

2 2

datatype dependencies
by Eric Rasche 28 Nov '14

28 Nov '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Let's pretend for a second that I'm rather lazy (oh...wait), and I have ZERO interest in writing datatype parsers to sniff and validate whether or not a specific file is a specific datatype. I'm a sysadmin and bioinformatician, and I've worked with dozens of libraries that exist to parse file formats, and they all die in flames when I feed them bad data. Would it be possible to somehow define requirements for datatypes? I don't want to take on the burden of code I write saying "yes, I've sniffed+validated this and it is absolutely a genbank file". That's a lot of responsibility, especially if people have malformed genbank files and their tools fail as a result. I would like to do this with BioPython and turf the validation to another library that exists to parse genbank files, that will raise and exception if they're invalid. > def sniff(self, filename): > from Bio import SeqIO > try: > self.records = list(SeqIO.parse( filename, "genbank" )) > return True > except: > self.records = None > return False > > def validate(self, dataset): > from Bio import SeqIO > errors = list() > try: > self.records = list(SeqIO.parse( dataset.file_name, "genbank" )) > except Exception, e: > errors.append(e) > return errors > > def set_meta(self, dataset, **kwd): > if self.records is not None: > dataset.metadata.number_of_sequences = len(self.records) so much easier! And I can shift the burden of validation and sniffing to upstream, rather than any failures being my fault and requiring maintenance of a complex sniffer. Cheers, Eric - -- Eric Rasche Programmer II Center for Phage Technology Texas A&M University College Station, TX 77843 404-692-2048 esr(a)tamu.edu rasche.eric(a)yandex.ru -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJTyBmyAAoJEMqDXdrsMcpVQa0P/jj0edAKM6QsodhRWHglR92W tej1tJjtPgtJ15wsFzq6wVfhbL5J39ytsWjjtk//jhVNXh4FEE/OFZe6Nx9uTFKP ybazyTrLSCrxsST+w+Rx8Q9vfzShr87vjP+fC1k5i2EZOgogPOcQml1ouOHHjC6z pArrwPOvL3ZxWJG7oEcZjUjrPD8+ffhfQ/x096YYIMw7Hg74d50ARwtawJRoslZD JnYWa+aUOcsvC3QMrLKkDm4qBaTHa5x7x7P07Lcx7X65iMPDcuMZNtImiLztNscF QwbbdJdcs8oeSRRnmKgAllRAKf4dMeiyaSI+muVzNlpvLlSMZBNawD0bO1OXmIQH vAaV0eU+rYmDJSGo330o+RydvlDJENTXOkDt0TxmvfYAPtg2TlJCiWUdL7V1LqqF n8J5Z7Cu/sqRGSr5ww6KY27QHq6TU1WZDsVZiyEWJeKg3HGzp0MUmzMdr7iSZawK gnZxv6qg3+FlSqA30niyAuxEq588vS8uEFjjOfhnNLsUM7FAuFANF5z9bPOhG2qM Xjc3/NY7NsERd9nsIwfRuz0DWni8upvZ39vfeRZ3OAW9NwjRzqXrQiQp08XHa934 z4EBnpcWc9rNSV/3APF/imecBTOoiKtZfzIfILLtOPGE407Bmd8cE8hWyW7ipvrT QU6DIimj3eoMn+elXDfX =M+s5 -----END PGP SIGNATURE-----

4 7

galaxy fails to start when setuptools>=6.1
by Evan Bollig PhD 26 Nov '14

26 Nov '14

I tested all versions of setuptools available in PyPI from 0.6c11 up to 7.0. I found that galaxy fails to fetch eggs with the following error when setuptools > 6.0.2 (i.e., 6.1 and 7.0). ##### * Creating directory in /mnt for galaxy data * Starting Galaxy Galaxy Some eggs are out of date, attempting to fetch... Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 46, in <module> c.resolve() # Only fetch eggs required by the config File "/opt/galaxy/web/lib/galaxy/eggs/__init__.py", line 347, in resolve egg.resolve() File "/opt/galaxy/web/lib/galaxy/eggs/__init__.py", line 192, in resolve if e.args[1].key != e.args[0].key: IndexError: tuple index out of range Fetch failed. ##### Cheers, -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan(a)msi.umn.edu boll0107(a)umn.edu

2 1

galaxy with torque
by Fernandez Edgar 26 Nov '14

26 Nov '14

Hello all, My name is Edgar Fernandez. I'm a sys. admin. at University of Montreal. I've contacted you a while back about installing galaxy and I've successfully done it on a redhat 6 server. I see myself in a situation where I need to utilise all my redhat servers (who are identical to the server running the galaxy website). I've also successfully installed a server torque with compute notes and clients nodes. What are the last step to make the link between galaxy and torque? Also, once that connection is made, how will galaxy keep track of the jobs sent? I mean who will it know this job that just finished is for this user and not another ? Also, my torque installation is so that my server running galaxy is a submit node and a client node. I hope this is not a problem. Please help! Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication * Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218

2 2

Re: [galaxy-dev] samtools: command not found in bowtie2
by Eric Kuyt 26 Nov '14

26 Nov '14

Could you try going to Admin -> Manage Installed tool shed repositories uninstall -> reinstall package_samtools_0_1_18 On 26 November 2014 at 16:55, Anna Terry <iseeglass(a)googlemail.com> wrote: > yway, I am also getting the problem with samtools filter sam or bam -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

3 5

Re: [galaxy-dev] samtools: command not found in bowtie2
by Anna Terry 26 Nov '14

26 Nov '14

So I actually still have a problem, (I guess the setting was still there somewhere from the bash_profile) Anyway, I am also getting the problem with samtools filter sam or bam Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 251, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found I have tried also what you said, linking the samtools binary to default, but still the same problem. Any other ideas? On 26 November 2014 at 14:01, Anna Terry <iseeglass(a)googlemail.com> wrote: > Great, I've done it and it works. > > galaxy-dist/tools/samtools/default > cd galaxy-dist/tools/samtools/default/ > cp -rs ~/galaxy-dist/tools/samtools/0.1.19/* . > > Thanks Bjoern and Eric > > > On 26 November 2014 at 13:54, Eric Kuyt <eric.kuijt(a)wur.nl> wrote: > >> Bjoerns comment was the right one, samtools isn't automatically installed >> as a dependency somehow. >> >> so the best option is to install samtools >> https://toolshed.g2.bx.psu.edu/view/devteam/package_samtools_0_1_19 >> >> and link the installed binary to ~/tool_deps/samtools/0.19/iuc/2828727744/ >> to ~/tool_deps/samtools/default >> >> Eric >> >> >> On 26 November 2014 at 14:44, Anna Terry <iseeglass(a)googlemail.com> >> wrote: >> >>> Here is the contents of the log file: >>> >>> Successfully downloaded from url: >>> http://depot.galaxyproject.org/package/linux/x86_64/bowtie2/bowtie2-2.1.0-l… >>> >>> >>> On 26 November 2014 at 13:26, Eric Kuyt <eric.kuijt(a)wur.nl<mailto: >>> eric.kuijt(a)wur.nl>> wrote: >>> Hi Anna, >>> >>> Setting your own path variables shouldn't be necessary for toolshed >>> tools, so probably something is wrong with the installation. could you post >>> your >>> >>> >>> tooldependencies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/INSTALLATION.log >>> >>> file? >>> >>> On 26 November 2014 at 13:49, Anna Terry <iseeglass(a)googlemail.com >>> <mailto:iseeglass@googlemail.com>> wrote: >>> Hi, >>> I installed bowtie2 from the toolshed, but get the following error when >>> I run it: >>> >>> >>> Traceback (most recent call last): >>> File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line >>> 129, in queue_job >>> job_wrapper.finish( stdout, stderr, exit_code ) >>> File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, >>> in finish >>> dataset.datatype.set_meta( dataset, overwrite=False ) # call >>> datatype.set_meta directly for the initial set_meta call during dataset >>> creation >>> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line >>> 251, in set_meta >>> raise Exception, "Error Setting BAM Metadata: %s" % stderr >>> Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not >>> found >>> >>> I'm not sure why this won't work? >>> samtools is a dependency for the bowtie2 tool and is installed, but >>> cannot seem to find it. >>> There must me something slightly wrong on my server as I can't see loads >>> of errors like this on this mailing list. >>> >>> I have added the path from the samtools env file to .bash_profile for >>> galaxy and this allows this tool to run, but is not the correct solution, >>> as doesn't work when versions change. >>> >>> Could you let me know where I'm going wrong >>> Thanks >>> Anna >>> >>> >>> >>> -- >>> Central Veterinary Institute of Wageningen UR (CVI) >>> Department of Infection Biology >>> PO box 65, 8200 AB Lelystad, NL >>> Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad >>> >>> Tel: +31-(0)320-293391 >>> Fax: +31-(0)320-238153 >>> E-mail: eric.kuijt(a)wur.nl<mailto:eric.kuijt@wur.nl> >>> Web: http://www.cvi.wur.nl >>> >>> >> >> >> -- >> Central Veterinary Institute of Wageningen UR (CVI) >> Department of Infection Biology >> PO box 65, 8200 AB Lelystad, NL >> Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad >> >> Tel: +31-(0)320-293391 >> Fax: +31-(0)320-238153 >> E-mail: eric.kuijt(a)wur.nl >> Web: http://www.cvi.wur.nl >> > >

1 1

Egg build failed for bx_python 0.7.2
by Rafael Guerrero-Preston 26 Nov '14

26 Nov '14

I have sensitive clinical data and would like to run Galaxy on a local production server. While creating/acquiring the required Python eggs to start the server for the first time using the sh run.sh command I got the following error message. How should I proceed? gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -arch ppc -arch i386 -g -O2 -DNDEBUG -g -O3 -Isrc/kent -Isrc -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lib/bx/bitset.c -o build/temp.macosx-10.3-fat-2.7/lib/bx/bitset.o unable to execute gcc-4.0: No such file or directory error: command 'gcc-4.0' failed with exit status 1 Traceback (most recent call last): File "scripts/scramble.py", line 50, in <module> egg.scramble() File "/Users/rafaelguerrero-preston/galaxy-dist/lib/galaxy/eggs/scramble.py", line 57, in scramble self.run_scramble_script() File "/Users/rafaelguerrero-preston/galaxy-dist/lib/galaxy/eggs/scramble.py", line 206, in run_scramble_script raise ScrambleFailure( self, "%s(): Egg build failed for %s %s" % ( sys._getframe().f_code.co_name, self.name, self.version ) ) galaxy.eggs.scramble.ScrambleFailure: run_scramble_script(): Egg build failed for bx_python 0.7.2

2 1

importing a large number of samples via a sample sheet
by Ryan G 26 Nov '14

26 Nov '14

Bumping this question....no response in over a week... Hi all - I've recently come back to Galaxy to see how its progressed over the last few years. The tools shed is completely new to me and will take some exploring on my part. My immediate question is if there is a way to import a large number of samples associated with a project via a sample sheet (txt) file? I have a text file with 3 columns: sample name, location of fastq files. I haven't seen a way to import this and haven't seen anything in the docs either. I suspect the answer is no, but I'm hoping the answer is actually yes....?

2 4

samtools: command not found in bowtie2
by Anna Terry 26 Nov '14

26 Nov '14

Hi, I installed bowtie2 from the toolshed, but get the following error when I run it: Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 251, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found I'm not sure why this won't work? samtools is a dependency for the bowtie2 tool and is installed, but cannot seem to find it. There must me something slightly wrong on my server as I can't see loads of errors like this on this mailing list. I have added the path from the samtools env file to .bash_profile for galaxy and this allows this tool to run, but is not the correct solution, as doesn't work when versions change. Could you let me know where I'm going wrong Thanks Anna

2 1

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev November 2014