May 2012 - galaxy-dev - lists.galaxyproject.org

Puzzling test failure with data_column parameter
by Peter Cock 24 Apr '13

24 Apr '13

Hi all, I'm working on a FASTA filter script, as per this email: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html Current code here: http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/ I've added a basic test, but it won't run (see output below). I think the interesting bit of the error output is this exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control, coming from function set_form_control_value in the twill.utils module. This is something to do with the columns parameter, a multiselect option referencing the columns in this tabular file. Removing the validator makes no difference. However, if I change the columns parameter to a plain text parameter, the test passes: $ hg diff diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml --- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000 +++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000 @@ -4,9 +4,7 @@ <inputs> <param name="input_fasta" type="data" format="fasta" label="FASTA file to filter on the identifiers"/> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing FASTA identifiers"/> - <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing FASTA identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> - <validator type="no_options" message="Pick at least one column"/> - </param> + <param name="columns" type="text" label="Column(s) containing FASTA identifiers" /> </inputs> <outputs> <data name="output_pos" format="fasta" label="With matched ID" /> I've tried searching the provided wrappers for similar examples (column multi-select from a tabular file). I found filters/uniq.xml (works but uses a bed file rather than a simple tabular file) and stats/cor.xml (needs R and rpy which I don't have installed yet) which have tests, and finally plotting/bar_chart.xml which has no unit tests. Would someone familiar with the internals of the Galaxy tests and how they set tool parameters be able to try reproducing this for me? The branch has all the unit test files required, and there are no new dependencies needed. Thank you, Peter -- Here is the output (on Linux - other tests tried pass): $ ./run_functional_tests.sh -id fasta_filter_by_id ... nose.plugins.manager DEBUG 2010-11-23 13:59:32,958 DefaultPluginManager load plugin sqlalchemy = sqlalchemy.test.noseplugin:NoseSQLAlchemy nose.plugins.manager DEBUG 2010-11-23 13:59:32,961 DefaultPluginManager load plugin nosetestdiff = nosetestdiff.plugin:NoseTestDiff nose.plugins.manager DEBUG 2010-11-23 13:59:32,962 DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ... galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:34,905 tool upload1 created job id 1 galaxy.jobs DEBUG 2010-11-23 13:59:42,709 dispatching job 1 to local runner galaxy.jobs INFO 2010-11-23 13:59:42,899 job 1 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:43,213 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:44,033 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs DEBUG 2010-11-23 13:59:44,414 job 1 ended galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:49,927 tool upload1 created job id 2 galaxy.jobs DEBUG 2010-11-23 13:59:50,067 dispatching job 2 to local runner galaxy.jobs INFO 2010-11-23 13:59:50,702 job 2 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,059 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,874 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs DEBUG 2010-11-23 13:59:53,420 job 2 ended base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase DEBUG 2010-11-23 13:59:54,488 ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 FAIL ====================================================================== FAIL: Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 160, in test_tool self.do_it( td ) File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 101, in do_it self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes ) File "/home/pjcock/repositories/galaxy-central/test/base/twilltestcase.py", line 656, in verify_dataset_correctness raise AssertionError( errmsg ) AssertionError: History item 1 different than expected, difference (using diff): --- local_file +++ history_data @@ -1,2 +1,61 @@ +>sp|Q9BS26|ERP44_HUMAN Endoplasmic reticulum resident protein 44 OS=Homo sapiens GN=ERP44 PE=1 SV=1 +MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRF +SQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMK +REYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFER +VANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDK +CVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHAD +CDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREF +HHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL +>sp|Q9NSY1|BMP2K_HUMAN BMP-2-inducible protein kinase OS=Homo sapiens GN=BMP2K PE=1 SV=2 +MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEG +GFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSIS +DNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRD +LKVENILLNDGGNYVLCDFGSATNKFLNPQKDGVNVVEEEIKKYTTLSYRAPEMINLYGG +KPITTKADIWALGCLLYKLCFFTLPFGESQVAICDGNFTIPDNSRYSRNIHCLIRFMLEP +DPEHRPDIFQVSYFAFKFAKKDCPVSNINNSSIPSALPEPMTASEAAARKSQIKARITDT +IGPTETSIAPRQRPKANSATTATPSVLTIQSSATPVKVLAPGEFGNHRPKGALRPGNGPE +ILLGQGPPQQPPQQHRVLQQLQQGDWRLQQLHLQHRHPHQQQQQQQQQQQQQQQQQQQQQ +QQQQQQHHHHHHHHLLQDAYMQQYQHATQQQQMLQQQFLMHSVYQPQPSASQYPTMMPQY +QQAFFQQQMLAQHQPSQQQASPEYLTSPQEFSPALVSYTSSLPAQVGTIMDSSYSANRSV +ADKEAIANFTNQKNISNPPDMSGWNPFGEDNFSKLTEEELLDREFDLLRSNRLEERASSD +KNVDSLSAPHNHPPEDPFGSVPFISHSGSPEKKAEHSSINQENGTANPIKNGKTSPASKD +QRTGKKTSVQGQVQKGNDESESDFESDPPSPKSSEEEEQDDEEVLQGEQGDFNDDDTEPE +NLGHRPLLMDSEDEEEEEKHSSDSDYEQAKAKYSDMSSVYRDRSGSGPTQDLNTILLTSA +QLSSDVAVETPKQEFDVFGAVPFFAVRAQQPQQEKNEKNLPQHRFPAAGLEQEEFDVFTK +APFSKKVNVQECHAVGPEAHTIPGYPKSVDVFGSTPFQPFLTSTSKSESNEDLFGLVPFD +EITGSQQQKVKQRSLQKLSSRQRRTKQDMSKSNGKRHHGTPTSTKKTLKPTYRTPERARR +HKKVGRRDSQSSNEFLTISDSKENISVALTDGKDRGNVLQPEESLLDPFGAKPFHSPDLS +WHPPHQGLSDIRADHNTVLPGRPRQNSLHGSFHSADVLKMDDFGAVPFTELVVQSITPHQ +SQQSQPVELDPFGAAPFPSKQ +>sp|P06213|INSR_HUMAN Insulin receptor OS=Homo sapiens GN=INSR PE=1 SV=4 +MATGGRRGAAAAPLLVAVAALLLGAAGHLYPGEVCPGMDIRNNLTRLHELENCSVIEGHL +QILLMFKTRPEDFRDLSFPKLIMITDYLLLFRVYGLESLKDLFPNLTVIRGSRLFFNYAL +VIFEMVHLKELGLYNLMNITRGSVRIEKNNELCYLATIDWSRILDSVEDNYIVLNKDDNE +ECGDICPGTAKGKTNCPATVINGQFVERCWTHSHCQKVCPTICKSHGCTAEGLCCHSECL +GNCSQPDDPTKCVACRNFYLDGRCVETCPPPYYHFQDWRCVNFSFCQDLHHKCKNSRRQG +CHQYVIHNNKCIPECPSGYTMNSSNLLCTPCLGPCPKVCHLLEGEKTIDSVTSAQELRGC +TVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETL -------------------- >> begin captured stdout << --------------------- Uploaded file: four_human_proteins.fasta , ftype: fasta , extra: {'ftype': 'fasta', 'value': 'four_human_proteins.fasta', 'children': []} Uploaded file: blastp_four_human_vs_rhodopsin.tabular , ftype: tabular , extra: {'ftype': 'tabular', 'value': 'blastp_four_human_vs_rhodopsin.tabular', 'children': []} form 'tool_form' contains the following controls ( note the values ) control 0: <HiddenControl(tool_id=fasta_filter_by_id) (readonly)> control 1: <HiddenControl(tool_state=800255c1626362313139396662383639346664316133316565303361646336323163663764636566333538623a376232323566356637303631363736353566356632323361323033303263323032323639366537303735373435663636363137333734363132323361323032323331323232633230323236333666366337353664366537333232336132303232366537353663366332323263323032323639366537303735373435663734363136323735366336313732323233613230323233323232376471002e) (readonly)> control 2: <SelectControl(input_fasta=[*1])> control 3: <SelectControl(input_tabular=[*2])> control 4: <SelectControl(columns=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])> control 5: <SubmitControl(runtool_btn=Execute) (readonly)> page_inputs (0) {'input_fasta': ['four_human_proteins.fasta'], 'columns': ['1'], 'input_tabular': ['blastp_four_human_vs_rhodopsin.tabular']} --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 1 galaxy.jobs: DEBUG: dispatching job 1 to local runner galaxy.jobs: INFO: job 1 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs: DEBUG: job 1 ended galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 2 galaxy.jobs: DEBUG: dispatching job 2 to local runner galaxy.jobs: INFO: job 2 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs: DEBUG: job 2 ended base.twilltestcase: DEBUG: In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase: DEBUG: ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 21.504s FAILED (failures=1) functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down embedded web server functional_tests.py INFO 2010-11-23 13:59:54,497 Embedded web server stopped functional_tests.py INFO 2010-11-23 13:59:54,497 Shutting down app galaxy.jobs INFO 2010-11-23 13:59:54,497 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,498 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:54,499 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,499 job stopper stopped functional_tests.py INFO 2010-11-23 13:59:54,499 Embedded Universe application stopped functional_tests.py INFO 2010-11-23 13:59:54,500 Cleaning up temporary files in /tmp/tmpu4uu_o galaxy.jobs INFO 2010-11-23 13:59:55,410 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,411 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:55,411 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,412 job stopper stopped 'run_functional_tests.sh help' for help Note: galaxy test framework uses tool_conf.xml.sample, not tool_conf.xml

2 3

resubmit a job if the node fails
by zhengqiu cai 10 Apr '13

10 Apr '13

Hi, Can Galaxy resubmit a job if the node where the job is running fails? I know sge can do that by using qsub -r. It should be very useful if Galaxy can do that. Thank you, Cai

3 2

Tabular file metadata - columns names
by Peter 22 Feb '13

22 Feb '13

Hi all, I'd like to know more about Galaxy's column metadata for tabular files. In the workflow editor under "Edit Step Actions" you can pick "Assign Columns", and then give column numbers for five predefined cases: Chrom, Start, End, Strand, Name. Do these "named columns" get shown anywhere in the Galaxy UI? For example, in a column select parameter widget? Is it possible to assign these columns in a tool's wrapper XML file? >From http://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax I'm aware of the metadata_source attribute to *copy* the meta data from the input file, but that isn't always relevant. Can I somehow specify that my tool has tabular output where column 1 is "Name"? Is it possible to introduce additional column types? e.g. "evalue" or "Description". Thanks, Peter

3 7

run_functional_tests.sh -sid option
by Peter 11 Feb '13

11 Feb '13

Hi, I'm trying to use run_functional_tests.sh to run all the tests for a section (group of tools). I've read the output from: ./run_functional_tests.sh help For example, from the tools_conf.xml.sample we have <section name="ENCODE Tools" id="EncodeTools"> <tool file="encode/gencode_partition.xml" /> <tool file="encode/random_intervals.xml" /> </section> And looking at these tools' XML files, <tool id="gencode_partition1" name="Gencode Partition"> and: <tool id="random_intervals1" name="Random Intervals"> I'd like to run the functional tests for the ENCODE tools, Using the switch for an individual tool id (-id) works, ./run_functional_tests.sh -id gencode_partition1 ... Ran 1 test in 22.302s ... and so does this (well, it say the tool doesn't have any tests which is true in this example): ./run_functional_tests.sh -id random_intervals1 ... Ran 1 test in 0.027s ... However, I also tried using the section id switch (-sid), ./run_functional_tests.sh -sid EncodeTools ... Ran 0 tests in 0.000s ... I also tried using the section name, /run_functional_tests.sh -sid "ENCODE Tools" Is this (-sid not working) a known issue, or am I using it wrong? Thanks, Peter

4 5

Visualise Galaxy workflows using graphviz
by Peter van Heusden 21 Jan '13

21 Jan '13

Hi there Because my Galaxy workflows tend to sprawl outside the viewable space in my browser, I've written a script that, when given a workflow JSON file as input, writes out a graphviz dot format graph of the workflow. By default the graph treats datasets as nodes and analyses as edges (unlike the Galaxy workflow editor, but quite like the way the history is presented), but you can choose to have analyses as nodes (and thus datasets as edges) too. In case anyone else finds this useful, the code can be downloaded from my bitbucket: https://bitbucket.org/pvanheus/galaxy/src/f29453f3d9d8/contrib/workflow_to_… Usage is something like: ./workflow_to_dot.py Galaxy-Workflow.ga |dot -Tsvg >workflow.svg and then you can view your workflow.svg with eog or another viewer. Peter

3 3

Problems with Internet Explorer 9?
by Peter Cock 19 Dec '12

19 Dec '12

Hello all, Are there (or were there) any known problems with Galaxy under Microsoft Internet Explorer 9? One of our biologists is now having problems with our local Galaxy server, probably after updating his browser. Specifically new history entries do not seem to automatically update from queued to running to done, and also the "peep" functionality to expand/collapse a history entry had stopped working. This was on a large history of about 100 entries. We installed Firefox 10, logged into Galaxy, and it was fine with this same history. A quick test on the public Galaxy with a single history entry seemed to work in both browsers, so perhaps this is something you've already fixed in the latest Galaxy - or perhaps it depends on the size of the history. Any ideas? Has anyone else seen something similar? Thanks, Peter

2 1

producing HTML output with images
by Nikhil Joshi 19 Dec '12

19 Dec '12

Hi all, I am having trouble producing HTML output with images. In the past I have been able to produce HTML files with no images and it seemed to work fine. However, now I am writing a script that produces diagnostic images and I want to display all of the images on one page using HTML. I am using the "files_path" variable to create the plots in the working directory, and then I am using the "extra_files_path" variable to access the final plot from the html file. I look at the resulting HTML files and it points to the proper plot full path, and the plot DOES exist... but when I click to view the output, the plot doesn't render. It just shows an empty box with a "broken image" icon, however, the text does render. I copy the plot file, and I can view it fine by itself. I copy the plot and the html and I can view the page just fine offline, but galaxy doesn't want to render the image for some reason. What am I doing wrong? - Nik. -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center Davis, CA

3 4

Auto delete dataset after workflow run?
by Praveen Raj Somarajan 03 Dec '12

03 Dec '12

Hello All, Is there a way to set "auto delete" some datafiles at the end of workflow run? I'm asking this because it is sometimes useful to delete some outputs (say intermediate files, .logs, etc), for example "auto delete" .sam file after executing sam-to-bam tool, which would free-up lot of space. This is especially useful when work with large number of samples. Look forward to any comments/suggestions. Best, Raj. ________________________________ This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions that are unlawful. This e-mail may contain viruses. Ocimum Biosolutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system. OCIMUMBIO SOLUTIONS (P) LTD

3 4

Hiding a tool via tool_conf.xml
by Peter Cock 30 Oct '12

30 Oct '12

Hi all, I'm aware that in an individual tool's XML wrapper you can add hidden="true" to the <tool> element to hide the tool from the listing shown in Galaxy's left hand panel (but the tool is still loaded and can be called by old workflows etc). Can this be done via the tool_conf.xml file as well? This would seem useful from a system configuration point of view where it seems wrong to edit the tool wrappers themselves. In fact, it might also be nice to add hidden="true" support to <section> as well as <tool> in tool_conf.xml - although I'm not so sure how useful that would be. Peter

1 2

Re: [galaxy-dev] Sample tracking data transfer hangs "in queue" forever
by TerAvest, Emily 27 Jul '12

27 Jul '12

Hi Leandro, I am also experiencing the same problem with the latest version of galaxy. I just attempted to connect our sequencer for the first time yesterday. I do not have an older version of galaxy to test and compare to see if it works in earlier versions. I am able to transfer data from the sequence to the import directory, however it is not moved to the data library. My data_transfer.log also has the same error. "The server could not comply with the request since it is either malformed or otherwise incorrect." Does anyone have a solution for this? Thanks Emily ------------------------------ Message: 17 Date: Wed, 28 Mar 2012 13:15:47 +0200 From: Leandro Hermida <softdev(a)leandrohermida.com> To: Luobin Yang <yangluob(a)isu.edu> Cc: Galaxy Dev <galaxy-dev(a)bx.psu.edu> Subject: Re: [galaxy-dev] Sample tracking data transfer hangs "in queue" forever Message-ID: <CAOhzmPJ67udRQBhfChgyxNrXsm3QcY1c7jna1kR0aJBNKEKJxA(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Dear Galaxy Dev, Ok I have gotten further found out what was going on, in my data_transfer.log I was getting HTTP 404 Not Authorized when the data transfer was trying to access Galaxy API URLs http://galaxyserver/api/... This is because we are using external user authentication as documented in http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy and this puts all of Galaxy behind this authentication. Since the Galaxy API uses API keys to essentially authenticate you have to change you <Location /> ... </Location> container to not match URLs starting with /api. To do this you have to change it to <LocationMatch "^/(?!api)"> .... </LocationMatch>. best, leandro On Mon, Mar 26, 2012 at 6:10 PM, Luobin Yang <yangluob(a)isu.edu> wrote: > I've got the same problem and when I looked at the data_transfer.log, I saw > the following message: > > 2012-03-01 15:12:27,338 - datatx_13870 - > (u'9c17d84742cd2acb63d88b5bd41d968f', > u'http://xxx.xxx.xxx.xxx/api/samples/2d9035b3fc152403', > {'sample_dataset_ids': ['a799d38679e985db', '33b43b4e7093c91f'], > 'error_msg': '', 'update_type': 'sample_dataset_transfer_status', > 'new_status': 'Adding to data library'}) > 2012-03-01 15:12:27,342 - datatx_13870 - Error. <!DOCTYPE HTML PUBLIC > "-//IETF//DTD HTML 2.0//EN"> > <html><head> > <title>405 Method Not Allowed</title> > </head><body> > <h1>Method Not Allowed</h1> > <p>The requested method PUT is not allowed for the URL > /api/samples/2d9035b3fc152403.</p> > <hr> > > Using Galaxy instead of Apache as the web server changed the sample status > from "in queue" to complete but didn't add the downloaded files to the data > library. > > Luobin > > On Mon, Mar 26, 2012 at 9:54 AM, Leandro Hermida > <softdev(a)leandrohermida.com> wrote: >> >> Dear Galaxy Dev, >> >> I've set up the Galaxy sample tracking system data transfer >> functionality exactly as specified here >> https://main.g2.bx.psu.edu/u/rkchak/p/data-transfer, but when I >> attempt to transfer datasets it puts them into the queue and then >> never seems to transfer anything. >> >> The galaxy_listener.log shows: >> >> 2012-03-26 17:16:24,515 - GalaxyAMQP - GALAXY LISTENER PID: 8738 - >> {'config_file': 'universe_wsgi.ini', 'http_server_section': >> 'server:main'} >> 2012-03-26 17:16:24,518 - GalaxyAMQP - {'exchange': 'galaxy_exchange', >> 'rabbitmqctl_path': >> >> '/gpfshpc/software/pmrdcdb/galaxy/rabbitmq/rabbitmq_server-2.8.1/sbin/rabbitmqctl', >> 'userid': 'galaxy_dev', 'routing_key': 'bar_code_scanner', 'queue': >> 'galaxy_queue', 'host': '127.0.0.1', 'virtual_host': >> 'galaxy_dev_messaging_engine', 'password': 'mypassword', 'port': >> '5672'} >> 2012-03-26 17:16:24,565 - GalaxyAMQP - Connected to rabbitmq server - >> 127.0.0.1:5672 >> 2012-03-26 17:31:23,393 - GalaxyAMQP - MESSAGE RECVD: data_transfer >> 2012-03-26 17:31:23,393 - GalaxyAMQP - DATA TRANSFER >> 2012-03-26 17:31:23,401 - GalaxyAMQP - Started process (25406): python >> >> "/gpfshpc/software/pmrdcdb/galaxy/galaxy_dist_dev/scripts/galaxy_messaging/server/data_transfer.py" >> " <data_transfer> >> <galaxy_host>rd-galaxydev.app.pmi</galaxy_host> >> <api_key>69bbd1a9b18eda2a888e9963820d8ef0</api_key> >> <data_host>chp723</data_host> >> <data_user>galaxy</data_user> >> <data_password>mypassword</data_password> >> <request_id>12</request_id> >> <sample_id>12</sample_id> >> <library_id>15</library_id> >> <folder_id>52</folder_id> ? ? ? ? ? ? ? ? ? ?<dataset> >> ?<dataset_id>10</dataset_id> >> <name>ttt_2.fastq.gz</name> >> <file>/gpfshpc/data/pmrdcdb/RNA-Seq/datasets/ttt-2.fastq.gz</file> >> ? ? ? ? ?</dataset><dataset> >> <dataset_id>9</dataset_id> >> <name>ttt_1.fastq.gz</name> >> <file>/gpfshpc/data/pmrdcdb/RNA-Seq/datasets/ttt-1.fastq.gz</file> >> ? ? ? ? ?</dataset> ? ? ? ? ? ? ? ?</data_transfer>" >> "universe_wsgi.ini" >> >> The FASTQ files are for testing and aren't very big (7.6 MB), what >> could be going wrong? >> >> regards, >> Leandro >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. ?To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> ?http://lists.bx.psu.edu/ > > ************ This email message and any electronic files transmitted with it relating to the official business of Codexis, Inc. and its affiliates are proprietary, strictly confidential, and may be legally privileged. This email message is intended solely for the use of the individual(s) to whom it is addressed. If you believe you have received this email in error, please delete this email from your system and notify the sender immediately. If you are not the intended recipient, you should not disclose, distribute, use or copy this email or take any action in reliance on its contents.

4 9