- galaxy-dev - lists.galaxyproject.org

sharing tools?
by Edward Kirton 07 Apr '10

07 Apr '10

Hi, I've wondering what's the best way to share tools with other galaxy site sysops? I've got tool config xml files and associated datatype config files for many popular programs, like newbler, velvet, vmatch, mummer, lucy, uclust, cd-hit, more. Plus I've written several small but useful tools/utilities. I'd like to support Galaxy by making them available to others (and am interested in getting tools other have created as well). Ed Kirton US DOE JGI

2 1

funny bug: a tool with a variable named "type" causes exception in workflows
by Assaf Gordon 06 Apr '10

06 Apr '10

Hi, I have a tool which have a variable named "type" (see example XML file below). This tool works fine when running it manually, but when adding this tool to a workflow *and* changing the default options from "typeA" to "typeB" - an exception is thrown when trying to save the workflow, and the "saving workflow" progress bar hangs forever. XML: ========================== <tool id="dummy_tool" name="Test tool"> <description>with a variable named 'type'</description> <command>echo $type > '$output'</command> <inputs> <param name="type" type="select" label="Type variable"> <option value="typeA">Type A</option> <option value="typeB">Type B</option> <option value="typeC">Type C</option> </param> </inputs> <outputs> <data format="txt" name="output" /> </outputs> </tool> ========================== The exception: ===== URL: http://tango.cshl.edu/galaxy/workflow/editor_form_post File '/home/gordon/projects/galaxy_devel/eggs/WebError-0.8a-py2.5.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/gordon/projects/galaxy_devel/eggs/Paste-1.6-py2.5.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/gordon/projects/galaxy_devel/eggs/Paste-1.6-py2.5.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/gordon/projects/galaxy_devel/eggs/Paste-1.6-py2.5.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/gordon/projects/galaxy_devel/lib/galaxy/web/framework/middleware/remoteuser.py', line 103 in __call__ return self.app( environ, start_response ) File '/home/gordon/projects/galaxy_devel/eggs/Paste-1.6-py2.5.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/gordon/projects/galaxy_devel/lib/galaxy/web/framework/base.py', line 125 in __call__ body = method( trans, **kwargs ) File '/home/gordon/projects/galaxy_devel/lib/galaxy/web/framework/__init__.py', line 62 in decorator return simplejson.dumps( func( self, trans, *args, **kwargs ) ) File '/home/gordon/projects/galaxy_devel/lib/galaxy/web/controllers/workflow.py', line 519 in editor_form_post 'tool_state': incoming.pop("tool_state") File '/home/gordon/projects/galaxy_devel/lib/galaxy/workflow/modules.py', line 286 in from_dict assert type in self.module_types AssertionError: ===== It seems like somehow the tool variable "type" overwrote the value of the python variable 'type' in that module (see attached screenshot). Maybe there is such a thing as 'too much' of a dynamic language ;) The obvious workaround is not to name my XML variables with a name that might conflict with galaxy's internals. -gordon P.S. My galaxy version is based on 3545:2590120aed68 .

1 0

problem getting Galaxy to run on web
by nsfox＠uwaterloo.ca 06 Apr '10

06 Apr '10

Hi, I have a Galaxy Instance that is up and running. I started running it locally on my computer. There I can access it using my web browser (firefox) but of course I can't use the Galaxy instance from a different computer. So I then moved a copy of the Galaxy instance to a web server. Galaxy runs on the web server but I cannot access the Galaxy instance through my web browser on my computer. However I can use the Galaxy instance by starting firefox from the command line on the web server. Access through the firewall on the web server was granted but is there another firewall or something I may need to change to allow web access? Any help is greatly appreciated. Thanks, Natalie

2 1

Wiki Addition: Site Map and Site Index
by Kelly Vincent 06 Apr '10

06 Apr '10

Everyone, Bitbucket (our code and wiki host) does not offer any way to search the wiki, so there are now two new pages to fill the void: SiteMap and SiteIndex. These are both generated automatically. The idea is that the pages should be generated by a script that is scheduled to run each night, but I'm still getting that going--in the meantime, I'll be running the script manually. SiteMap is a simple listing of all the pages found under the base wiki directory. You can get a pretty good idea what the page is about just by the name of it, which is all that's included in this list. You can see SiteMap at http://bitbucket.org/galaxy/galaxy-central/wiki/SiteMap. SiteIndex is more complicated. It is a traditional index, which means that (almost) every word that appears in the wiki is included with a list of the pages on which it appears following it. A stoplist was used to remove useless or meaningless words, but there is still some junk that has made it past this filter. The index involves multiple pages, divided up alphabetically. It's available at http://bitbucket.org/galaxy/galaxy-central/wiki/SiteIndex . Regards, Kelly

3 3

change_format and conditonal vars
by Edward Kirton 05 Apr '10

05 Apr '10

hi, how do i reference conditional variables from within the change_format tag? i couldn't find any examples that did this. an example tool config xml file follows. thanks! ed kirton us doe jgi <tool id="fasta_get_seqs" name="Get Sequences"> <description>Create a Fasta/Qual file of selected sequences</description> <command interpreter="perl">fasta_get_seqs.pl $intype.infile $input_table $id_column $ignorecase $cosorted $output #for $i in $filters "${i.condition} ${i.column} ${i.value}" #end for </command> <inputs> <conditional name='intype'> <param name='sel' type='select' label='Input file type'> <option value='Fasta'>Fasta</option> <option value='Qual454'>Qual454</option> </param> <when value='Fasta'> <param name="infile" type="data" format="fasta" label="Fasta file"/> </when> <when value='Qual454'> <param name="infile" type="data" format="qual454" label="Qual454 file"/> </when> </conditional> <param name="input_table" type="data" format="tabular" label="File containing target IDs"/> <param name='id_column' type='data_column' data_ref='input_table' label='Column containing sequence IDs'/> <param name='ignorecase' type='boolean' truevalue='1' falsevalue='0' label='Ignore case?' /> <param name="cosorted" type="boolean" checked="true" truevalue='1' falsevalue='0' label='Sequences appear in same order in both files'/> <repeat name='filters' title='Filter'> <param name='condition' type='select' label='Condition'> <option value='s_eq'>text equals</option> <option value='s_ne'>text not equals</option> <option value='s_contains'>text contains</option> <option value='s_startswith'>text starts with</option> <option value='s_endswith'>text ends with</option> <option value='n_eq'>number ==</option> <option value='n_ne'>number !=</option> <option value='n_gt'>number ></option> <option value='n_lt'>number <</option> </param> <param name='column' type='data_column' data_ref='input_table' label='On column'/> <param name='value' type='text' label='Value'/> </repeat> </inputs> <outputs> <data name="output" format="fasta"> <change_format> <when input="intype.sel" value="Qual454" format="qual454" /> </change_format> </data> </outputs> <help> **What it does** Produces a Fasta/Qual file of selected sequences. If the files are cosorted, a much faster algorithm is used. If it fails, it assumes the files were not cosorted and automatically tries again using the slower algorithm which does not require cosorted files. One or more filters may be included to evaluate text (equal, not equal, contains, starts with) or numeric data (==, !=, >, <). </help> </tool>

2 1

"Never-starting" job
by Assaf Gordon 05 Apr '10

05 Apr '10

Hello, Somewhat related to the recent "never-ending" job email, We have a situation where a new dataset is created (status=new), but due to an exception, the job never runs and the dataset remains new (gray) forever. One way to create such an exception is to enter unicode characters which confuses some module down the line. Example: Enter the following expression in the "Filter and Sort" -> "Filter" tool: c1 == “chr12“ If my email program didn't mangle things up, you'll notice that these are not double-quotes (ASCII 34) - they are Unicode Character 'LEFT DOUBLE QUOTATION MARK' (U+201C) which are common if you copy&paste double-quotes from/to Word or Internet-Explorer. Executing this filter tool, the following exception is returned (I verified it on your 'test' server): =================== URL: http://test.g2.bx.psu.edu/tool_runner/index File '/galaxy/home/g2test/galaxy_test/eggs/Paste-1.6-py2.5.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/galaxy/home/g2test/galaxy_test/eggs/Paste-1.6-py2.5.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/galaxy/home/g2test/galaxy_test/eggs/Paste-1.6-py2.5.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/web/framework/base.py', line 125 in __call__ body = method( trans, **kwargs ) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/web/controllers/tool_runner.py', line 59 in index template, vars = tool.handle_input( trans, params.__dict__ ) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/__init__.py', line 831 in handle_input _, out_data = self.execute( trans, incoming=params ) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/__init__.py', line 1103 in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid ) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/actions/__init__.py', line 309 in execute for name, value in tool.params_to_strings( incoming, trans.app ).iteritems(): File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/__init__.py', line 1106 in params_to_strings return params_to_strings( self.inputs, params, app ) File '/var/opt/galaxy/test/lib/galaxy/tools/parameters/__init__.py', line 79 in params_to_strings File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/parameters/basic.py', line 103 in value_to_basic return self.to_string( value, app ) File '/galaxy/home/g2test/galaxy_test/lib/galaxy/tools/parameters/basic.py', line 94 in to_string return str( value ) UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 4: ordinal not in range(128) ===================== A new dataset is created (in the 'dataset' table), but no job is created for it. It will stay gray forever. This also confuses users (who usually ignore the exception and just click "refresh") - they will wait for this job forever, and even use it as input for further jobs - which will never run. Many thanks to Delphine Fagegaltier for reporting and helping reproduce this bug. Regards, -gordon

1 0

'change_format' doesn't work with workflows
by Assaf Gordon 02 Apr '10

02 Apr '10

Hi, Tools with "change_format" in the <outputs> tag aren't working well in workflows: I have a tool which uses the "change_format" option to output different types of files. The XML definition is similar to this: --------- <outputs> <data format="fasta" name="output" metadata_source="input" label="$input.tag (FASTA)" > <change_format> <when input="wanttype" value="coordinates" format="bed" label="$input.tag (BED)" /> </change_format> </data> </outputs> ---------- The tool allows the user to select the output file type: FASTA or BED (the variable "wanttype"). This works fine when running the tool manually, but the tool can't be used in a workflow: the output type is always "FASTA" (the "change_format" is ignored) and so the tool can not be connected to other tools which need "bed" input. -gordon

1 0

"HTTP 403 Forbidden" when using custom GenomeBrowser server + external authentication
by Assaf Gordon 02 Apr '10

02 Apr '10

Hello, I've noticed a small problem when using a custom GenomeBrowser server and having external authentication enabled - Galaxy returns HTTP 403, because of hard-coded values of allowed hosts. The issue was introduced in 1648:682c179a2f84 . My setup might not be very common, so it's not an urgent issue. My setup is: 1. external authentication is enabled in "universe_wsgi.ini": remote_user_maildomain = cshl.edu 2. I use only a local Genome Browser mirror in "universe_wsgi.ini", like so: ucsc_display_sites = cshl I do no have the "main" and "test" links, because my server is on an internal network. The result: Galaxy shows "display at UCSC 'cshl'" link, clicking on that links redirects to my GenomeBrowser server, but when the GenomeBrowser server connects back to my Galaxy server, it gets HTTP 403. Possible cause(s): 1. In "lib/galaxy/web/framework/middleware/remoteuser.py", the list of allowed GenomeBrowser hosts is hard-coded. Adding the my server's name to the list is a workaround. 2. Still in "remoteuser.py", there's a hard-coded check that only enables userless login if the names "main" or "test" appear in the "ucsc_display_sites" INI configuration. As mentioned, I don't have either "main" or "test", so that check fails, and "self.allow_ucsc_main" remains "False". Adding the name of my site to the "if" is a workaround. The attached patch is a hard-coded workaround (just for my server) - use it as an example if you're in a similar situation. I guess a more modular fix is complicated, taking into account all possible scenarios (and the new "bx_display_sites" which is not even used in "remoteuser.py"). -gordon

1 0

Re: [galaxy-dev] [galaxy-bugs] help me please m in great trouble
by Kelly Vincent 02 Apr '10

02 Apr '10

Hi, If I understand your setup correctly, you have taken the original clustalw, ncbiblast, etc. perl code and put it directly into your new directory. And then you created a new Galaxy XML file which calls the appropriate Perl script in that same directory. This is totally fine in theory, but you need to make sure that the inputs and outputs are being handled correctly. I think your problem may be that you never give the the Perl script the name of the output files you want the tool to produce in Galaxy. You also need to give it the input file name. The way Galaxy handles inputs and outputs is by name--when you specify an output variable in the XML, Galaxy creates a file on the file system, which is associated with the output variable name in the XML. It is generally incorrect behavior if your tool generates a file that it names itself, without it being one of the output files already created by Galaxy. So if the Perl script does not accept inputs or outputs as arguments, you should modify it to do so. You also need to make certain that the command you have near the top of the XML file has all of required arguments for the Perl script (either hard-coded or based on parameters in the XML file), in addition to the named inputs and outputs. If this doesn't help you get it working, send us one of your XML files and the Perl file associated with it and we can take a look. Regards, Kelly On 4/2/10 10:42 AM, aeman zahra wrote: > I am a student of Bs Bioinformatics. I have been using the open source > galaxy platform to integrate tool and make a work flow as my Final > semester project. My supervisor happens to be ill and thus is unable > to facilitate me in any respect. I here by take this opportunity to > request that If any of you could help me fixing my problem. I would > extremely grateful. > PROBLEM > My project problem is that , first of all i am using perl code tools > for galaxy because galaxy dose not support installed tools from > command prompt. so m using perl code tools for example clustalw, > wublast, muscle and many more. sir these perl code tools are running > correctly on command prompt and give me desire output, but when i put > them it to the galaxy by making there xml file it gives an error that > error occour while running this job.. and sir actually galaxy took > tool's correct output as an error. i dnt know how to resolve it i > tried alot but couldnt resolve it.. > sir breifly m telling you that how m embedding perl code tools into > galaxy: > 1.go to ebi site, there are some perl code tools like clustalw, > muscle,ncbiblast etc > 2.i took them and download them > 3. go to command prompt and try to run them by giving commands like > perl ncbiblast.pl -S aemanzahra(a)comsats.edu.pk > <mailto:aemanzahra@comsats.edu.pk> -D uniprot -P blastp > 4.when it runs correctly i put them and embed them into the galaxy. > now in galaxy > 1. insatlled galaxy > 2. go in to tools directory > 3. make a new directory of new tool ito tools directory e.g clustalw tool > 4.go into clustalw tool and made two files in it. clustalw.pl in which > perl code of clustalw tool is present. > and secondly clustalw.xml is present of this tool that i made. > 5.then run galaxy by sh run.sh command > 6.galaxy web page opens > 7. and i clicked on my added tool clustalw there and give it an input > file of multiple sequences. > 8. it runs. but after running it gives me an error that "error occour > while running this job" but it took the output files of this tool as > an error. > "The tools run and three output files are created but the Galaxy > takes them as errors." > WHAT PROBLEM IS THERE? > > waitin for your reply > regards aeman > > > _______________________________________________ > galaxy-bugs mailing list > galaxy-bugs(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-bugs >

1 0

please help me please
by aeman zahra 02 Apr '10

02 Apr '10

I am a student of Bs Bioinformatics. I have been using the open source galaxy platform to integrate tool and make a work flow as my Final semester project. My supervisor happens to be ill and thus is unable to facilitate me in any respect. I here by take this opportunity to request that If any of you could help me fixing my problem. I would extremely grateful. The PROBLEM http://bitbucket.org/galaxy/galaxy-central/wiki/ExternalDisplayApplications… The URL has an explanation concerning BAM and BAI files. the explanation on the URL are a little vague to me so if any one of you could kindly further elaborate. I am having problems in displaying the results of my added tools on the interface. The tools run and three output files are created but the Galaxy takes them as errors. If you could identify or sent a protocol which could help me fixing this problem. Regards Aeman..

2 1