- galaxy-dev - lists.galaxyproject.org

Login issue with a nginx proxy
by Alexandre Loywick 16 Oct '14

16 Oct '14

Hello galaxians, I'm currently trying to set up a production galaxy and make it available to the public but I face a weird issue, probably because of my nginx proxy configuration. Galaxy is on a kvm virtual machine running on a host using nginx as proxy. to make it a bit clearer: 172.30.24.35 (client) ---> http://172.30.24.63/machine1 (Host & nginx proxy) --- > http://192.168.122.254:8080/ (virtual machine) I can connect to galaxy but when logging in, the virtual machine redirects me on 192.168.122.254:8080, which is unreachable from the 172.30.24.0 network. If I log in from the Host, it's all fine. in the universe_wsgi.ini, i've set: # Define the proxy-prefix filter. [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /machine1 ... # [filter:proxy-prefix] section above. filter-with = proxy-prefix as described in the documentation. My virtual machine nginx conf file: user galaxy; worker_processes 2; #error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' # '$status $body_bytes_sent "$http_referer" ' # '"$http_user_agent" "$http_x_forwarded_for"'; #access_log logs/access.log main; sendfile on; #tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 65; gzip on; gzip_http_version 1.1; gzip_vary on; gzip_comp_level 4; gzip_proxied any; gzip_types text/plain text/css application/x-javascript text/xml application/xml text/javascript application/json; gzip_buffers 16 8k; gzip_disable "MSIE [1-6].(?!.*SV1)"; upstream galaxy_app{ server localhost:8080; } server { client_max_body_size 10G; listen 80; server_name localhost; #charset koi8-r; #access_log logs/host.access.log main; location /machine1 { proxy_pass http://galaxy_app; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } location /machine1/static { alias /home/galaxy/galaxy-dist/static; } location /machine1/static/style { alias /home/galaxy/galaxy-dist/static/june_2007_style/blue; } location /machine1/static/scripts { alias /home/galaxy/galaxy-dist/static/scripts/packed; } location /machine1/favicon.ico { alias /home/galaxy/galaxy-dist/static/favicon.ico; } location /machine1/robots.txt { alias /home/galaxy/galaxy-dist/static/robots.txt; } location /_x_accel_redirect/ { internal; alias /machine1; } #error_page 404 /404.html; #redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } } And my host configuration : server { listen 80 default_server; listen [::]:80 default_server ipv6only=on; root /usr/share/nginx/html; index index.html index.htm; # Make site accessible from http://localhost/ server_name localhost; location / { try_files $uri $uri/ =404; } location /machine1 { proxy_pass http://192.168.122.254:8080; } }

1 0

snpEff and java issue
by Iry Witham 15 Oct '14

15 Oct '14

Hi Team, I had manually installed the latest version of snpEff based on the recommendation of Pablo and after modifying the XML files I had a working version of snpEff that produced a vcf. However, this morning I reran my workflow and it is failing again, but this time I am getting the following error: Fatal error: Exit code 1 (Error) Exception in thread "main" java.lang.UnsupportedClassVersionError: ca/mcgill/mcb/pcingola/snpEffect/commandLine/SnpEff : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:643) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:277) at java.net.URLClassLoader.access$000(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:212) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:323) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:268) Could not find the main class: ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff. Program will exit. Nothing has been changed since I had success. Regards, Iry The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

1 0

snpeff tool for Galaxy extra_files_path
by Jim Johnson 15 Oct '14

15 Oct '14

Andrew, Thanks for investigating this. I changed the subject and sent to the galaxy dev list. I've had a number of tools quit working recently. Particularly tools that inspect the extra_files_path when setting metadata, Defuse, Rsem, SnpEff. I think there was a change in the galaxy framework: The extra_files_path when referenced from an input or output in the cheetah template sections of the tool config xml will be relative to the job working directly rather than the files location. I've just changed a few of my tools on my server yesterday from: <param_name>.extra_files_path to: <param_name>.dataset.extra_files_path and they now work again. Dan or John, is that the right way to handle this? Thanks, JJ On 10/13/14, 9:29 PM, Andrew Lonie wrote: > Hi Jim. I am probably going about this the wrong way, but I am not > clear on how to report tool errors (if in fact this is a tool error!) > > I've been trialling your snpeff wrapper from the test toolshed and > getting a consistent error with the SnpEff Download and SnpEff sub > tools (the SnpSift dbNSFP works fine). The problem seems to be with an > attribute declaration and manifests during database download as: > > Traceback (most recent call last): > File "/mnt/galaxy/galaxy-app/lib/galaxy/jobs/runners/__init__.py", > line 564, in finish_job > job_state.job_wrapper.finish( stdout, stderr, exit_code ) > File "/mnt/galaxy/galaxy-app/lib/galaxy/jobs/__init__.py", line > 1107, in finish > dataset.datatype.set_meta( dataset, overwrite=False ) # call > datatype.set_meta directly for the initial set_meta call during > dataset creation > File "/mnt/galaxy/shed_tools/testtoolshed.g2.bx.psu.edu/repos/iuc/snpeff/1938721334b3/snpeff/lib/galaxy/datatypes/snpeff.py", > line 21, in set_meta > data_dir = dataset.files_path > AttributeError: 'HistoryDatasetAssociation' object has no attribute 'files_path' > > > We fiddled around with the wrapper, eventually replacing > 'dataset.files_path' with 'dataset.extra_files_path' in snpeff.py, > which fixed the download bug, but then SnpEff subtool itself threw a > similar error when I tried to use that database from the history. > > I chased up a bit more but cannot understand the various posts on > files_path vs extra_files_path > > I've shared a history with both of these errors here: > http://130.56.251.62/galaxy/u/alonie/h/unnamed-history > > Maybe this is a problem with our Galaxy image? > > Any help appreciated! > > Andrew > > > > > A/Prof Andrew Lonie > University of Melbourne -- James E. Johnson Minnesota Supercomputing Institute University of Minnesota

3 7

Set output dbkey from parameter value
by Nikos Sidiropoulos 15 Oct '14

15 Oct '14

Hi all, I'm trying to set the dbkey of an output file from the value (text) of a parameter. The parameter I want to use is genome. <conditional name="bedgraph"> <param name="check" type="select" label="Produce BedGraph output" help="Can be displayed directly on UCSC browser. One file per normalisation method." > <option value="no" selected="True">No</option> <option value="yes">Yes</option> </param> <when value="yes"> <param name="bed_file" type="data" format="bed" label="Transcripts ins BED format" help="12 column BED file containing trancript definitions." /> <param name="genome" type="text" label="Genome Build" help="E.g. hg19" /> <param name="track_name" type="text" label="Track Name" size="20" value="Track Name" /> </when> <when value="no" /> </conditional> and this is how I've set the output: <data format="bedgraph" name="bedgraph_slograt" label="${tool.name} on ${on_string}: Smoot Log2ratio (bedGraph)" from_work_dir="output_dir/slograt.bedgraph"> <filter> bedgraph['check'] == 'yes' and slograt['check'] == 'yes' </filter> <action type="metadata" name="dbkey"> <option type="from_param" name="bedgraph.genome" param_attribute="value" /> </action> </data> When I run the tool the dbkey isn't set to the output file. Does anyone know a workaround? Bests, Nikos

2 4

UCSC Main table browser
by Shrum, Donald C 15 Oct '14

15 Oct '14

When I click get output in the UCSC main table browser in firefox nothing happens. I can fix this by going to about:config in firefox and disabling security.mixed_content.block_active_content I am running galaxy behind an apache proxy using https as we have users authenticate with their hpc accounts. Is there a fix for this aside from making the change to firefox? Thanks, Don FSU Research Computing Center

2 1

Assertion errors when starting Galaxy
by Ricardo Perez 14 Oct '14

14 Oct '14

Hello, When starting Galaxy after updating it we get the assertion errors that follow the following format: galaxy.datatypes.registry ERROR 2014-10-14 15:51:40,004 Error loading display application (./display_applications/ucsc/bam.xml): Filename and id attributes required for dynamic_links Traceback (most recent call last): File "/home/galaxy/galaxy-dist.old/lib/galaxy/datatypes/registry.py", line 576, in load_display_applications display_app = DisplayApplication.from_file( config_path, self ) File "/home/galaxy/galaxy-dist.old/lib/galaxy/datatypes/display_applications/application.py", line 170, in from_file return cls.from_elem( parse_xml( filename ).getroot(), datatypes_registry ) File "/home/galaxy/galaxy-dist.old/lib/galaxy/datatypes/display_applications/application.py", line 183, in from_elem for link in DynamicDisplayApplicationBuilder( dynamic_links, rval, datatypes_registry.build_sites ): File "/home/galaxy/galaxy-dist.old/lib/galaxy/datatypes/display_applications/application.py", line 91, in __init__ assert filename is not None, 'Filename and id attributes required for dynamic_links' AssertionError: Filename and id attributes required for dynamic_links Any one else getting the same thing after updating their Galaxy instance? Thank You, --Ricardo

1 0

Galaxy database corrupted
by Thuy Linh Chu 14 Oct '14

14 Oct '14

Recently I tried to upload some annotation/reference files into Galaxy. After couple failed tries using the upload method, I was able to do it via the using the full file path. However, after these tries/retries, I believe Galaxy database got confused and was corrupted. Existing workflows with existing reference were used when datasets that previously were processed without any problems, now have funky result set names and parameters to tool wrappers reference funky paths. Here’s an example of the error I’m encountering. The highlighted part references the dbSNP file: python /home/svcbioinfo/galaxy/galaxy-reproduce/tools/gatkV2/gatk_wrapper.py --max_jvm_heap "16g" --max_jvm_heap_fraction "1" --stdout "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24218.dat" -d "-I" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24216.dat" "bam" "gatk_input_first" -d "" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/_metadata_files/001/metadata_1491.dat" "bam_index" "gatk_input_first" -p 'java -jar "/home/svcbioinfo/galaxy/galaxy-reproduce/tool-data/shared/jars/gatk/GenomeAnalysisTKv2.jar" -T "RealignerTargetCreator" --num_threads 4 -o "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24217.dat" -et "NO_ET" -K "/home/svcbioinfo/galaxy/galaxy-reproduce/tool-data/shared/jars/genomichealth.com.key" -R "/mnt/ngs/analysis/refData/processed/bwa/hg19_decoy_rcrs8001sp.fasta" ' -d "-known:dbsnp,%(file_type)s" "/mnt/ngs/analysis/tchu/Workflow_RWC/SEQUENCER99/140201_SNTEST_0021_TESTFC21/analysis_31/primary/fastq/99_21_3_ACAGTG_R2.fastq" "fastq" "input_dbsnp_0" -d "-known:indels,%(file_type)s" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/016/dataset_16596.dat" "vcf" "input_indels_0" -p '--pedigreeValidationType "STRICT"' -d "--intervals" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/022/dataset_22821.dat" "bed" "input_intervals_0" -p '--interval_padding "400"' -p '--interval_set_rule "UNION"' -p '--downsampling_type "NONE"' -p ' --validation_strictness "SILENT" --interval_merging "ALL" ' -p ' --unsafe "LENIENT_VCF_PROCESSING" The highlighted part should be the dbSNP file which typically has a Galaxy based name and type of “vcf” such as below: python /home/svcbioinfo/galaxy/galaxy-reproduce/tools/gatkV2/gatk_wrapper.py --max_jvm_heap "16g" --max_jvm_heap_fraction "1" --stdout "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24314.dat" -d "-I" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24312.dat" "bam" "gatk_input_first" -d "" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/_metadata_files/001/metadata_1492.dat" "bam_index" "gatk_input_first" -p 'java -jar "/home/svcbioinfo/galaxy/galaxy-reproduce/tool-data/shared/jars/gatk/GenomeAnalysisTKv2.jar" -T "RealignerTargetCreator" --num_threads 4 -o "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/024/dataset_24313.dat" -et "NO_ET" -K "/home/svcbioinfo/galaxy/galaxy-reproduce/tool-data/shared/jars/genomichealth.com.key" -R "/mnt/ngs/analysis/refData/processed/bwa/hg19_decoy_rcrs.fasta" ' -d "-known:dbsnp,%(file_type)s" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/016/dataset_16598.dat" "vcf" "input_dbsnp_0" -d "-known:indels,%(file_type)s" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/016/dataset_16596.dat" "vcf" "input_indels_0" -p '--pedigreeValidationType "STRICT"' -d "--intervals" "/mnt/ngs/analysis/svcbioinfo/DATA_reproduce/files/016/dataset_16617.dat" "bed" "input_intervals_0" -p '--interval_padding "400"' -p '--interval_set_rule "UNION"' -p '--downsampling_type "NONE"' -p ' --validation_strictness "SILENT" --interval_merging "ALL" ' -p ' --unsafe "LENIENT_VCF_PROCESSING" I wonder if there's a way to recover from this or I'll have to reinstall the database. Thanks for your help. Thuy-Linh Chu

1 0

CloudMan with HTCondor and Slurm & mounting pathes
by Abdularahman Azab 14 Oct '14

14 Oct '14

Dear all, I've installed a galaxy cloud cluster using usergalaxy.org/cloudlaunch It runs fine, but I need to know how to let the cluster run on THCondor or slurm instead of SGE. I can see that Condor is already installed on the galaxy-master node since the condor_status command is there. But how can I switch the default galaxy job-runner between SGE, HTCondor, and slurm Also regarding the mounting option of a e.g. s3 bucket on the master node. It works fine and it can be accessed from the command line. But how to let galaxy see this mounted path for e.g, reading input files and storing resulted history elements in it? Thank you, Yours sincerely, Abdulrahman Azab Head engineer, ELIXIR.NO / The Genomic HyperBrowser team Department of Informatics, University of Oslo, Boks 1072 Blindern, NO-0316 OSLO, Norway Email: azab(a)ifi.uio.no</owa/abdulara(a)ifi.uio.no>, Cell-phone: +47 46797339 ---- Senior Lecturer in Computer Engineering Faculty of Engineering, University of Mansoura, 35516-Mansoura, Egypt Email: abdulrahman.azab(a)mans.edu.eg</owa/abdulrahman.azab(a)mans.edu.eg>

2 1

error running API with example_watch_folder.py
by Neil.Burdett＠csiro.au 14 Oct '14

14 Oct '14

Hi, when I use the example_watch_folder.py to check for files in an input directory and process them if they are present. The workflow runs to completion successfully, but the "example_watch_folder.py" task crashes with the following errors, thus it can only process one file at a time, as the example_watch_folder.py has to be constantly restarted. python example_watch_folder.py d2fcd3feb4c6318c496d55fa8869b67c http://barium-rbh/new/api/ /home/galaxy/cte_input /home/galaxy/cte_output "API Imports" f597429621d6eb2b in_folder [/home/galaxy/cte_input] fullpath [/home/galaxy/cte_input/141_S_0851_MRI_T1_Screening.nii.gz] submit library_id [f2db41e1fa331b3e] libset [[{u'url': u'/new/api/libraries/f2db41e1fa331b3e/contents/82b264d8c3d11790', u'id': u'82b264d8c3d11790', u'name': u'141_S_0851_MRI_T1_Screening.nii.gz'}]] waits for several minutes .... Traceback (most recent call last): File "example_watch_folder.py", line 97, in <module> main(api_key, api_url, in_folder, out_folder, data_library, workflow ) File "example_watch_folder.py", line 77, in main res = submit( api_key, api_url + 'workflows', wf_data, return_formatted=False) File "common.py", line 117, in submit r = post( api_key, url, data ) File "common.py", line 51, in post return json.loads( urllib2.urlopen( req ).read() ) File "/usr/lib/python2.7/socket.py", line 351, in read data = self._sock.recv(rbufsize) File "/usr/lib/python2.7/httplib.py", line 541, in read return self._read_chunked(amt) File "/usr/lib/python2.7/httplib.py", line 586, in _read_chunked raise IncompleteRead(''.join(value)) httplib.IncompleteRead: IncompleteRead(176 bytes read) Due to the crash there is no outputs shown on the screen. Any ideas? Thanks Neil p.s. there is nothing in the log file.

2 2

failure to export workflows
by Wolfgang Maier 13 Oct '14

13 Oct '14

Dear all, our local Galaxy does not seem to be able to export workflows anymore, instead fails with, for example: Not Found The resource could not be found. No route for /u/w/myworkflow/json-download We are running latest_2014.10.06 . Has anyone else experienced this ? Thanks, Wolfgang

3 6