- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
by Peter Cock 25 Apr '13

25 Apr '13

On Wed, Apr 24, 2013 at 10:12 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > > ..., I've still finding some other cases where despite having > tests defined, nothing shows up on the Tool Shed test results page. > > Here's an example where I see that two tools (promoter2 and > wolf_psort) have no tests, but the results of all the other tools > in this suite are not mentioned (some expected to fail/be skipped > due to missing dependencies): > > http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp > Revision 13:dc958c2a963a > > Equivalently: > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp > Revision: 14:0d8b1d20ce9c > > -------------------------------------------------------------------------------- > > Here's another example where based on what happens locally, > I am expecting a failure: > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename > Revision: 1:6ce8634e2035 > > This is failing for me locally, but again I suspect this is due to an > issue in the framework - much like the blastxml_to_top_descr > test is failing due to the datatypes not being updated, here there > seems to be something similar going wrong regarding the > second uploaded input file (which is used for to provide the > potential column parameters). See this thread: > > http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003867.html > http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014261.html > > Regards, > > Peter Hi Dave, Yet another case, http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus Revision: 8:1f546099212f and, http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus Revision: 5:d42346e675c4 These list five or six invalid tools which do not define any unit tests, yet there is no mention of the tools present which do have tests (and if they passed or not). Even if ncbi_blastdbcmd_wrapper doesn't (yet) have a test, I still want to know if the ncbi_blastp_wrapper tests pass ;) Are this further symptoms of whatever was solved here? https://trello.com/card/toolshed-automated-functional-test-bug/506338ce32ae… Or does would a new Trello issue be best? Thanks, Peter

2 4

GalaxyAdmins Group: Future Directions?
by Dave Clements 25 Apr '13

25 Apr '13

Hello all, The GalaxyAdmins group is coming up on it's one year anniversary (coinciding with GCC2013) and this is a good opportunity to discuss what the future of the group should be. Some starting topics for discussion are on the GalaxyAdmins Future Directions page (http://wiki.galaxyproject.org/Community/GalaxyAdmins/Future) These include * What should the group's goals and activities be? * What type of leadership structure should the group have, and how should it be selected? The discussion, however, is wide open to any topic relevant to the group. If you have any opinions or suggestions please reply to the group. Anyone with an interest in the group is encouraged to post. Once the discussion settles, I will summarize the discussion on the wiki page and suggest an action plan for making those things happen. Thanks, Dave C. -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

5 9

Merging files: Error merging filesglobal name 'shutil' is not defined
by Peter Cock 25 Apr '13

25 Apr '13

Hi all, While testing a BLAST search using plain text output (the 'txt' datatype in Galaxy) or subclasses (like 'blastxml') using galaxy-dist with task splitting enabled, I found I new bug when the files are merged: Error merging files global name 'shutil' is not defined This is a side effect of this commit due to a missing import (perhaps the imports changed since Bjoern wrote the patch): https://bitbucket.org/galaxy/galaxy-central/commits/510b063792de89332428ed1… In addition to the missing import in data.py, there is a potential similar issue with binary.py but this is masked by an evil import * line ;) My suggested patch for the import issue below, however right now even though the merge is failing, Galaxy failed to treat the job as a failure - it was shown as a green successful job with "Error merging files" as stdout, and for stderr: "global name 'shutil' is not defined", but return code 0. So there is a deeper problem with the merge error not marking the job as failed. Also there is a minor bug in that the stdout and stderr are combined for the peep text without any white space - presumably most of the time the stdout will have a trailing new line, but not here - thus "filesglobal" [sic], not say "files global" or a newline. There's a second patch below for that minor issue too (although it seems my new lines get turned into spaces by the time they are shown in the history peep text). Regards, Peter -- $ hg diff diff -r 8b9ca63f9128 lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/datatypes/binary.py Thu Apr 25 11:32:51 2013 +0100 @@ -12,7 +12,7 @@ from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, TWOBIT_MAGIC_NUMBER_SWAP, TWOBIT_MAGIC_SIZE from urllib import urlencode, quote_plus import zipfile, gzip -import os, subprocess, tempfile +import os, subprocess, tempfile, shutil import struct log = logging.getLogger(__name__) diff -r 8b9ca63f9128 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/datatypes/data.py Thu Apr 25 11:32:51 2013 +0100 @@ -3,6 +3,7 @@ import mimetypes import os import sys +import shutil import tempfile import zipfile from cgi import escape $ hg diff diff -r 8b9ca63f9128 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/jobs/__init__.py Thu Apr 25 11:43:30 2013 +0100 @@ -924,7 +924,13 @@ for dataset in dataset_assoc.dataset.dataset.history_associations + dataset_assoc.dataset.dataset.library_associations: #need to update all associated output hdas, i.e. history was shared with job running dataset.blurb = 'done' dataset.peek = 'no peek' - dataset.info = ( dataset.info or '' ) + context['stdout'] + context['stderr'] + dataset.info = (dataset.info or '') + if context['stdout'].strip(): + #Ensure white space between entries + dataset.info = dataset.info.rstrip() + "\n" + context['stdout'].strip() + if context['stderr'].strip(): + #Ensure white space between entries + dataset.info = dataset.info.rstrip() + "\n" + context['stderr'].strip() dataset.tool_version = self.version_string dataset.set_size() if 'uuid' in context:

2 2

traceback - local error?
by Langhorst, Brad 25 Apr '13

25 Apr '13

172.17.130.203 - - [19/Apr/2013:10:31:20 -0400] "GET /api/datasets/dd61915e24aa3fd0?hda_ldda=hda&data_type=converted_datasets_state&chrom=chr1 HTTP/1.1" 200 - "http://galaxy.neb.com/visualization/trackster?dataset_id=b8edc02014d74add&h…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.65 Safari/537.31" galaxy.webapps.galaxy.api.datasets ERROR 2013-04-19 10:31:21,775 Error in dataset API at listing contents: unpack requires a string argument of length 4: unpack requires a string argument of length 4 Traceback (most recent call last): File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 41, in show is_true( kwd.get( 'retry', False ) ) ) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 1037, in has_data f, bbi = self._get_dataset() File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 1170, in _get_dataset return f, BigWigFile(file=f) File "bigwig_file.pyx", line 127, in bx.bbi.bigwig_file.BigWigFile.__init__ (lib/bx/bbi/bigwig_file.c:3011) File "bbi_file.pyx", line 119, in bx.bbi.bbi_file.BBIFile.__init__ (lib/bx/bbi/bbi_file.c:3508) File "bbi_file.pyx", line 129, in bx.bbi.bbi_file.BBIFile.open (lib/bx/bbi/bbi_file.c:3660) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg/bx/misc/binary_file.py", line 30, in __init__ if struct.unpack( ">I", bytes )[0] == magic: error: unpack requires a string argument of length 4 Is this a bug or have I done something unexpected? I'm trying to use the trackbrowser to visualize some bam files that I aligned to a genome that I had to install manually (Zea_mays_B73_RefGen_v2), maybe i've screwed something up. I was initially able to see some reads, but after reverting to the sample datatypes file and restarting galaxy i can't see anything anymore. I also added a .2bit file to twobit.locs Zea_mays_AGPv2 /mnt/ngswork/galaxy/galaxy-dist/tool-data/genome/Zea_mays_AGPv2/seq/Zea_mays_AGPv2.2bit maybe that's related... I've tried copying the bams to a new history - and they seem to be reindexing as a result, but I still see no reads. Any ideas? Brad -- Brad Langhorst langhorst(a)neb.com<mailto:langhorst@neb.com>

2 1

Blast Errors
by Ganote, Carrie L 25 Apr '13

25 Apr '13

Hi list, I installed the latest toolshed NCBI Blast+ into a new Galaxy and I'm trying to pinpoint these errors:

1 0

Puzzling test failure with data_column parameter
by Peter Cock 24 Apr '13

24 Apr '13

Hi all, I'm working on a FASTA filter script, as per this email: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html Current code here: http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/ I've added a basic test, but it won't run (see output below). I think the interesting bit of the error output is this exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control, coming from function set_form_control_value in the twill.utils module. This is something to do with the columns parameter, a multiselect option referencing the columns in this tabular file. Removing the validator makes no difference. However, if I change the columns parameter to a plain text parameter, the test passes: $ hg diff diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml --- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000 +++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000 @@ -4,9 +4,7 @@ <inputs> <param name="input_fasta" type="data" format="fasta" label="FASTA file to filter on the identifiers"/> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing FASTA identifiers"/> - <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing FASTA identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> - <validator type="no_options" message="Pick at least one column"/> - </param> + <param name="columns" type="text" label="Column(s) containing FASTA identifiers" /> </inputs> <outputs> <data name="output_pos" format="fasta" label="With matched ID" /> I've tried searching the provided wrappers for similar examples (column multi-select from a tabular file). I found filters/uniq.xml (works but uses a bed file rather than a simple tabular file) and stats/cor.xml (needs R and rpy which I don't have installed yet) which have tests, and finally plotting/bar_chart.xml which has no unit tests. Would someone familiar with the internals of the Galaxy tests and how they set tool parameters be able to try reproducing this for me? The branch has all the unit test files required, and there are no new dependencies needed. Thank you, Peter -- Here is the output (on Linux - other tests tried pass): $ ./run_functional_tests.sh -id fasta_filter_by_id ... nose.plugins.manager DEBUG 2010-11-23 13:59:32,958 DefaultPluginManager load plugin sqlalchemy = sqlalchemy.test.noseplugin:NoseSQLAlchemy nose.plugins.manager DEBUG 2010-11-23 13:59:32,961 DefaultPluginManager load plugin nosetestdiff = nosetestdiff.plugin:NoseTestDiff nose.plugins.manager DEBUG 2010-11-23 13:59:32,962 DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ... galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:34,905 tool upload1 created job id 1 galaxy.jobs DEBUG 2010-11-23 13:59:42,709 dispatching job 1 to local runner galaxy.jobs INFO 2010-11-23 13:59:42,899 job 1 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:43,213 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:44,033 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs DEBUG 2010-11-23 13:59:44,414 job 1 ended galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:49,927 tool upload1 created job id 2 galaxy.jobs DEBUG 2010-11-23 13:59:50,067 dispatching job 2 to local runner galaxy.jobs INFO 2010-11-23 13:59:50,702 job 2 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,059 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,874 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs DEBUG 2010-11-23 13:59:53,420 job 2 ended base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase DEBUG 2010-11-23 13:59:54,488 ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 FAIL ====================================================================== FAIL: Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 160, in test_tool self.do_it( td ) File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 101, in do_it self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes ) File "/home/pjcock/repositories/galaxy-central/test/base/twilltestcase.py", line 656, in verify_dataset_correctness raise AssertionError( errmsg ) AssertionError: History item 1 different than expected, difference (using diff): --- local_file +++ history_data @@ -1,2 +1,61 @@ +>sp|Q9BS26|ERP44_HUMAN Endoplasmic reticulum resident protein 44 OS=Homo sapiens GN=ERP44 PE=1 SV=1 +MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRF +SQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMK +REYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFER +VANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDK +CVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHAD +CDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREF +HHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL +>sp|Q9NSY1|BMP2K_HUMAN BMP-2-inducible protein kinase OS=Homo sapiens GN=BMP2K PE=1 SV=2 +MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEG +GFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSIS +DNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRD +LKVENILLNDGGNYVLCDFGSATNKFLNPQKDGVNVVEEEIKKYTTLSYRAPEMINLYGG +KPITTKADIWALGCLLYKLCFFTLPFGESQVAICDGNFTIPDNSRYSRNIHCLIRFMLEP +DPEHRPDIFQVSYFAFKFAKKDCPVSNINNSSIPSALPEPMTASEAAARKSQIKARITDT +IGPTETSIAPRQRPKANSATTATPSVLTIQSSATPVKVLAPGEFGNHRPKGALRPGNGPE +ILLGQGPPQQPPQQHRVLQQLQQGDWRLQQLHLQHRHPHQQQQQQQQQQQQQQQQQQQQQ +QQQQQQHHHHHHHHLLQDAYMQQYQHATQQQQMLQQQFLMHSVYQPQPSASQYPTMMPQY +QQAFFQQQMLAQHQPSQQQASPEYLTSPQEFSPALVSYTSSLPAQVGTIMDSSYSANRSV +ADKEAIANFTNQKNISNPPDMSGWNPFGEDNFSKLTEEELLDREFDLLRSNRLEERASSD +KNVDSLSAPHNHPPEDPFGSVPFISHSGSPEKKAEHSSINQENGTANPIKNGKTSPASKD +QRTGKKTSVQGQVQKGNDESESDFESDPPSPKSSEEEEQDDEEVLQGEQGDFNDDDTEPE +NLGHRPLLMDSEDEEEEEKHSSDSDYEQAKAKYSDMSSVYRDRSGSGPTQDLNTILLTSA +QLSSDVAVETPKQEFDVFGAVPFFAVRAQQPQQEKNEKNLPQHRFPAAGLEQEEFDVFTK +APFSKKVNVQECHAVGPEAHTIPGYPKSVDVFGSTPFQPFLTSTSKSESNEDLFGLVPFD +EITGSQQQKVKQRSLQKLSSRQRRTKQDMSKSNGKRHHGTPTSTKKTLKPTYRTPERARR +HKKVGRRDSQSSNEFLTISDSKENISVALTDGKDRGNVLQPEESLLDPFGAKPFHSPDLS +WHPPHQGLSDIRADHNTVLPGRPRQNSLHGSFHSADVLKMDDFGAVPFTELVVQSITPHQ +SQQSQPVELDPFGAAPFPSKQ +>sp|P06213|INSR_HUMAN Insulin receptor OS=Homo sapiens GN=INSR PE=1 SV=4 +MATGGRRGAAAAPLLVAVAALLLGAAGHLYPGEVCPGMDIRNNLTRLHELENCSVIEGHL +QILLMFKTRPEDFRDLSFPKLIMITDYLLLFRVYGLESLKDLFPNLTVIRGSRLFFNYAL +VIFEMVHLKELGLYNLMNITRGSVRIEKNNELCYLATIDWSRILDSVEDNYIVLNKDDNE +ECGDICPGTAKGKTNCPATVINGQFVERCWTHSHCQKVCPTICKSHGCTAEGLCCHSECL +GNCSQPDDPTKCVACRNFYLDGRCVETCPPPYYHFQDWRCVNFSFCQDLHHKCKNSRRQG +CHQYVIHNNKCIPECPSGYTMNSSNLLCTPCLGPCPKVCHLLEGEKTIDSVTSAQELRGC +TVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETL -------------------- >> begin captured stdout << --------------------- Uploaded file: four_human_proteins.fasta , ftype: fasta , extra: {'ftype': 'fasta', 'value': 'four_human_proteins.fasta', 'children': []} Uploaded file: blastp_four_human_vs_rhodopsin.tabular , ftype: tabular , extra: {'ftype': 'tabular', 'value': 'blastp_four_human_vs_rhodopsin.tabular', 'children': []} form 'tool_form' contains the following controls ( note the values ) control 0: <HiddenControl(tool_id=fasta_filter_by_id) (readonly)> control 1: <HiddenControl(tool_state=800255c1626362313139396662383639346664316133316565303361646336323163663764636566333538623a376232323566356637303631363736353566356632323361323033303263323032323639366537303735373435663636363137333734363132323361323032323331323232633230323236333666366337353664366537333232336132303232366537353663366332323263323032323639366537303735373435663734363136323735366336313732323233613230323233323232376471002e) (readonly)> control 2: <SelectControl(input_fasta=[*1])> control 3: <SelectControl(input_tabular=[*2])> control 4: <SelectControl(columns=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])> control 5: <SubmitControl(runtool_btn=Execute) (readonly)> page_inputs (0) {'input_fasta': ['four_human_proteins.fasta'], 'columns': ['1'], 'input_tabular': ['blastp_four_human_vs_rhodopsin.tabular']} --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 1 galaxy.jobs: DEBUG: dispatching job 1 to local runner galaxy.jobs: INFO: job 1 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs: DEBUG: job 1 ended galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 2 galaxy.jobs: DEBUG: dispatching job 2 to local runner galaxy.jobs: INFO: job 2 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs: DEBUG: job 2 ended base.twilltestcase: DEBUG: In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase: DEBUG: ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 21.504s FAILED (failures=1) functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down embedded web server functional_tests.py INFO 2010-11-23 13:59:54,497 Embedded web server stopped functional_tests.py INFO 2010-11-23 13:59:54,497 Shutting down app galaxy.jobs INFO 2010-11-23 13:59:54,497 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,498 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:54,499 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,499 job stopper stopped functional_tests.py INFO 2010-11-23 13:59:54,499 Embedded Universe application stopped functional_tests.py INFO 2010-11-23 13:59:54,500 Cleaning up temporary files in /tmp/tmpu4uu_o galaxy.jobs INFO 2010-11-23 13:59:55,410 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,411 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:55,411 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,412 job stopper stopped 'run_functional_tests.sh help' for help Note: galaxy test framework uses tool_conf.xml.sample, not tool_conf.xml

2 3

Output a file stored within a server directory to the current history in galaxy
by Venice Margarette Juanillas 24 Apr '13

24 Apr '13

Hi all, I'm Venice and I'm working on a local galaxy instance here at IRRI. This question might sound really naive but I really can't work around it. I'm creating an xml for a tool where the user will select a list of file names from the drop-down as input. I stored these files in a directory within galaxy-dist. Based on the user input, Galaxy must output the user-selected file. I created this xml: <tool id="matrix2flapjack" name="NT Matrix to Flapjack"> <description>file conversion</description> <command interpreter="perl">matrix2flapjack.pl $inputFile $outputFile $snp_map</command> <inputs> <param format="tabular" name="inputFile" type="data" label="Nucleotide Matrix file"/> <param name="snp_map" type="select" label="SNP map file"> <option value="/home/galaxy/galaxy-dist/SNP_maps/11861_snp_map.txt">11861</option> <option value="/home/galaxy/galaxy-dist/SNP_maps/11862_snp_map.txt">11862</option> <option value="/home/galaxy/galaxy-dist/SNP_maps/13033_snp_map.txt">13033</option> </param> </inputs> <outputs> <data format="tabular" name="outputFile" label="Flapjack Format of ${inputFile.name} "> </data> <data format="tabular" name="snp_map" label="SNP Map"/> </outputs> <help> This tool converts a SNP NT matrix file to Flapjack file format. </help> </tool> However, it doesn't output the file. Instead, it prompts, "Either the dataset does not exist or you don't have permission to access it." Thanks in advance for your reply. Thanks and Best regards, Venice

1 0

Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
by Adam Brenner 23 Apr '13

23 Apr '13

I have the following job_conf.xml file setup for our cluster: http://pastebin.com/raw.php?i=0uNhcVJk When running jobs from my admin account via Galaxy it runs under the same user as galaxy is, for testing, this as root. I need for it to run under the real user. The email address / galaxy account I am using to test with does match with a user on the cluster -- user(a)example.com where user is a valid user account with correct permissions to run jobs on our cluster and queue. I followed the information here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_J… Any ideas as to why galaxy_external* is not being called? The paster.log file does not show anything useful regarding calling external scripts. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

1 2

Problem using FTP upload
by Jillian Rowe 23 Apr '13

23 Apr '13

Hello, I would like to use the FTP upload option without actually having an FTP service setup. Instead users will scp to the directory. ftp_upload_dir = import/users ftp_upload_site = notnothing if a create a import/users/username with username as my login I still get the message: Your FTP upload directory contains no files. This Galaxy server allows you to upload files via FTP. To upload some files, log in to the FTP server at localhost using your Galaxy credentials (email address and password). According to this email tot he mailing list http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td393419… I thought I could do a setup such as I am trying now, but now I am not so sure. Any ideas? Best, Jillian

2 5

ToolShed test complaining core bundled test file is missing
by Peter Cock 23 Apr '13

23 Apr '13

Hi Greg/Dave, I've noticed an oddity with another unit test failure on the main Tool Shed, http://toolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss Revision: 1:922d69bd5258 Tool id: get_orfs_or_cdss version: 0.0.2 Reason test is invalid One or more test files are missing for tool get_orfs_or_cdss: Ssuis.fasta The tool tests should be failing as they are omitting some test files, which I'd not included in the uploaded tar ball (I'll fix this shortly): test-data/get_orf_input.fasta test-data/get_orf_input.Suis_ORF.nuc.fasta test-data/get_orf_input.Suis_ORF.prot.fasta test-data/get_orf_input.t11_nuc_out.fasta test-data/get_orf_input.t11_open_nuc_out.fasta test-data/get_orf_input.t11_open_prot_out.fasta test-data/get_orf_input.t11_prot_out.fasta test-data/get_orf_input.t1_nuc_out.fasta test-data/get_orf_input.t1_prot_out.fasta However, I believe the ToolShed warning is misleading as test-data/Ssuis.fasta is already present as part of the Galaxy core install. This is why I have used it (and other bundled files like it) to minimise the download size of the tool itself. Thanks, Peter

2 2