February 2012 - galaxy-dev - lists.galaxyproject.org

Stalled upload jobs under "Admin", "Manage jobs"
by Peter Cock 30 Apr '12

30 Apr '12

Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users. e.g. Job ID User Last Update Tool State Command Line Job Runner PID/Cluster ID 2352 xxxx 21 hours ago upload1 upload None None None ... 2339 yyyy 19 hours ago upload1 upload None None None The job numbers are consecutive (2339 to 2352) and reflect a problem for a couple of hours yesterday morning. I believe this was due to the underlying file system being unmounted (without restarting Galaxy), and at the time restarting Galaxy fixed uploading files. Test jobs since then have completed normally - but these zombie jobs remain. Using the "Stop jobs" option does not clear these dead upload jobs. Restarting the Galaxy server does not clear them either. This is our production server and was running galaxy-dist, changeset 5743:720455407d1c - which I have now updated to the current release, 6621:26920e20157f - which makes no difference to these stalled jobs. Does anyone have any insight into what might be wrong, and how to get rid of these zombie tasks? Thanks, Peter

3 5

Error on local galaxy using SAM-to-BAM tool on a cluster
by Carlos Borroto 12 Apr '12

12 Apr '12

Hi, I'm running into this error: "Error sorting alignments from (/tmp/5800600.1.all.q/tmpXOc5mD/tmpAZCzt_), " When using SAM-to-BAM tool on a locally install Galaxy using a SGE cluster. I'm using the last version of galaxy-dist. I'm guessing I have a problem with the configuration for the tmp folder. I have this on "universe_wsgi.ini": # Temporary files are stored in this directory. new_file_path = /home/cborroto/galaxy_dist/database/tmp But I don't see this directory being used and from the error looks like /tmp in the node is used. I wonder if this is the problem, as I don't know if there is enough space in the local /tmp directory at the nodes? I ran the same tool in a subset of the same SAM file and it ran fine. Also, I see this in the description of the tool: "This tool uses the SAMTools toolkit to produce an indexed BAM file based on a sorted input SAM file." But what I actually need is to sort a SAM file output from bwa, I haven't found any other way than to converting it to BAM. Looking at "sam_to_bam.py" I see the BAM file will also be sorted. Would it be wrong to feed an unsorted SAM file into this tool? Finally, just to be sure there is nothing wrong with the initial SAM file, I ran "samtools view ..." and "samtools sort ..." on this file manually outside of Galaxy and it ran fine. Thanks in advance, Carlos

4 5

file download returns empty file or webpage fails with ERR_CONNECTION_CLOSED
by Derrick LIN 03 Apr '12

03 Apr '12

Hi guys, I have setup a test galaxy and configured Apache as proxy. It's been running fine when I did all testing on my machine with Chrome. Recently, I let some of my colleagues to try it out, then they experience some file download issues. On one machine, both FireFox and IE8 downloaded empty files (zero byte). But on another Linux machine, FireFox downloaded a complete file successfully. On two MacBook machines, Chrome encountered webpage error (only the right hand side) as soon as download button was pressed. Saying: *This webpage is not available* * * *error 100 (net::ERR_CONNECTION_CLOSED): The server unexpectedly closed the connection.* I have seems someone posted the exact same issue before but no solution has been given. I enabled Apache xsend file: <Location "/"> XSendFile on XSendFileAllowAbove on</Location> And configured galaxy to be aware of this in universe_wsgi.ini apache_xsendfile = True. Any help will be appreciated. Thanks Derrick

2 1

functional test failures
by Anne Pajon 27 Mar '12

27 Mar '12

Hello, I am trying to have all functional tests on our production server (galaxy-dist 6621:26920e20157f) run successfully but I do still have failures that I do not know how to solve. Theses failures are related to unexpected differences probably due to versions of software that have been upgraded recently. ( Extract genomic DNA 1 ) > Test-1 ( Extract genomic DNA 1 ) > Test-2 ( Extract genomic DNA 1 ) > Test-3 ( Extract genomic DNA 1 ) > Test-4 ( Extract genomic DNA 1 ) > Test-5 ( GeneBed_Maf_Fasta2 ) > Test-1 ( Interval2Maf1 ) > Test-1 ( Interval2Maf1 ) > Test-3 ( Interval_Maf_Merged_Fasta2 ) > Test-1 ( Interval_Maf_Merged_Fasta2 ) > Test-2 ( LinearRegression1 ) > Test-1 ( XY_Plot_1 ) > Test-1 ( cca1 ) > Test-1 ( histogram_rpy ) > Test-1 ( kpca1 ) > Test-1 ( pca1 ) > Test-1 ( pca1 ) > Test-2 ( pca1 ) > Test-3 ( plot_for_lda_output1 ) > Test-1 ( qual_stats_boxplot ) > Test-1 AssertionError: History item 2 different than expected, difference (using diff) ( gops_join_1 ) > Test-4 TwillAssertionError: code is 500 != 200 I do also have failures due to missing data but I am struggling to find out where to download them. ( gd_select_snps ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error 'bighorn' does not appear in location file '/opt/local/home/webapp/galaxy/tool-data/gd.chrlens.loc' ( Annotation_Profiler_0 ) > Test-1 ( Annotation_Profiler_0 ) > Test-2 AssertionError: Configuration error: Table directory is missing (/opt/local/home/webapp/galaxy/tool-data/annotation_profiler/hg18) ( Interval2Maf_pairwise1 ) > Test-1 AssertionError: Attempting to set field 'mafType' to value '['PAIRWISE_hg17_fr1']' in form 'tool_form' threw exception: cannot find value/label "PAIRWISE_hg17_fr1" in list control ( maf_stats1 ) > Test-1 ( maf_stats1 ) > Test-2 AssertionError: Attempting to set field 'maf_source_type|mafType' to value '['8_WAY_MULTIZ_hg17']' in form 'tool_form' threw exception: cannot find value/label "8_WAY_MULTIZ_hg17" in list control ( phastOdds_for_intervals ) > Test-1 AssertionError: Attempting to set field 'score_file' to value '['/galaxy/data/phastOdds_precomputed/encode_SEP-2005_\ tba.v2_phastOdds']' in form 'tool_form' threw exception: cannot find value/label "/galaxy/data/phas..2_phastOdds" in li\ st control ( aggregate_scores_in_intervals2 ) > Test-1 ( aggregate_scores_in_intervals2 ) > Test-2 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error ( gencode_partition1 ) > Test-1 Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error Error loading partitioning dataset. ( lda_analy1 ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error The last failure is related to genome diversity and the fact that it calls python2.5. I've changed it to python for testing but the test still fails. ( gd_extract_flanking_dna ) > Test-1 ( gd_extract_primers ) > Test-1 ( gd_select_restriction_enzymes ) > Test-1 AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error python2.5: command not found > if changed to python: ERROR: 'NoneType' object has no attribute 'rfind' Is there a way to fix these failures? Any help would be very much appreciated. Thanks, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

2 3

refresh_on_change is broken?
by Leandro Hermida 23 Mar '12

23 Mar '12

Hello, Seems like the refresh_on_change functionality is broken in the latest galaxy-dist. If you apply it to an input parameter it doesn't seem to refresh the page when you change the selection of that parameter. Does anyone else also have the problem? regards, Leandro

3 9

Importing data from password protected UCSC Genome Browser
by Federico De Masi 22 Mar '12

22 Mar '12

Dear colleagues, I have been struggling with this issue for quite a while now, and I don't seem to find a solution (neither in situ nor in webo). here it is: Because of the confidentiality of our data, I have setup both a Galaxy server and a UCSC Genome Browser locally on my server. Both are run through Apache, using the same .htpasswd authentication method and file (ie: same usernmae/password combo for each). Everything works great except that I do not seem to be able to send/receive data from Galaxy to UCSC and vice versa. When I try to "get data" from UCSC main (my local instance), I get the followin error: "Tool execution generated the following error message: The remote data source application may be off line, please try again later. Error: EOF when reading a line The tool produced the following additional output: Enter username for SG at 127.0.0.1:" where SG is the AuthName in vhost.conf and 127.0.0.1 is UCSC's IP. Whenever I try to to send data to UCSC ("Display at UCSC main" in the history boxes), I get this guy: "Expected 200 http://127.0.0.3/display_application/48916fac0de9a85d/ucsc_bigwig/main/b9ba…: 401 Authorization Required" Where 127.0.0.3 is galaxy's IP. I have: - hardcoded username:passowrd on UCSC's "send to galaxy" routine but to no avail - hardcode username:passowrd on Galaxy's definiton of UCSC's "main" URL, but to no avail - I have removed UCSC authentication, and everything works fine - I have kept UCSC authentication and let Galaxy "open", doesn't work - I remove authentication to both servers, and it all works cool. Somehow, Galaxy doesn't like to talk to a protected UCSC browser. I'll be happy to post my vhost.conf files if needed. Can anyone help, please? Thanks a lot, Fred -- Federico De Masi, PhD, Assistant Professor The Technical University of Denmark - DTU Center for Biological Sequence Analysis - CBS Kemitorvet 208/002 DK-2800 KGS. LYNGBY, DENMARK Telephone: (+45) 45 25 24 21 Fax: (+45) 45 93 15 85 http://rg.cbs.dtu.dk

3 2

Can't clone workflows
by Liisa Koski 21 Mar '12

21 Mar '12

Hello, I have lost the ability to clone workflows in my local installation of Galaxy (the latest galaxy-dist). When I try...it just hangs...and hangs... Any help would be much appreciated. Thanks, Liisa

2 2

Error Setting BAM Metadata
by Liisa Koski 16 Mar '12

16 Mar '12

Hello, I am trying to upload BAM files (by pasting a URL) to my history(or DataLibrary) and get the following error. These are bam files which I had previously uploaded with no problems. Traceback (most recent call last): File "/doolittle/Galaxy/galaxy_dist/lib/galaxy/jobs/runners/local.py", line 126, in run_job job_wrapper.finish( stdout, stderr ) File "/doolittle/Galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 618, in finish dataset.set_meta( overwrite = False ) File "/doolittle/Galaxy/galaxy_dist/lib/galaxy/model/__init__.py", line 874, in set_meta return self.datatype.set_meta( self, **kwd ) File "/doolittle/Galaxy/galaxy_dist/lib/galaxy/datatypes/binary.py", line 179, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: [bam_header_read] EOF marker is absent. The input is probably truncated. [bam_header_read] invalid BAM binary header (this is not a BAM file) I ran bamtools on the unix command line to see if there was anything wrong with the file(s) but nothing. I tried uploading different bam files from other projects and get the same error. I did do an update to the latest release yesterday...if that helps? Thanks in advance, Liisa

2 3

NFS, Cluster and working directories questions
by Louise-Amélie Schmitt 14 Mar '12

14 Mar '12

Hello, We're currently trying to switch to a big cluster but we have a lot of doubts and questions, especially since the I/O is a serious issue for our NFS. - We saw the outputs_to_working_directory option in the .ini file, but it only concerns output files, is there a way to make a local copy of all the input files in the job working directory? (including indices) - Can we set the job working directory as an absolute path so it's local to the nodes, like /tmp ? - If it's possible, will the job working directories created for each job be cleaned properly at the end? Thanks, L-A

2 3

bug with Galaxy form validation and dynamic_options population of drop-down menus where option values are integers
by Leandro Hermida 13 Mar '12

13 Mar '12

Hello, If you have a form with an input like this: <param name="input1" type="select" dynamic_options="get_options()" /> And the options dynamically generated have integers as values, e.g. options tuples like this: [ "One", 1, 1 ], [ "Two", 2, 0 ], [ "Three", 3, 0 ] When you try to submit the galaxy form you always get the validation error "An invalid option was selected, please verify". regards, Leandro

2 3