February 2012 - galaxy-dev - lists.galaxyproject.org

Syntax issue for adding new tool
by Jeffrey Long 07 Feb '12

07 Feb '12

Hi all, I am having trouble with what looked to me at first to be a "simple" syntactical issue with writing the XML wrapper for a new Galaxy tool. I had thought this would be easy, but perhaps I am missing something incredibly simple, as I am having no success at all. The relevant parts look something like this: <command interpreter="sh"> mycommand.sh -i $input #if $param_file.use_params == "true"# -p $params #end if </command> <inputs> <param format="fasta" name="input" type="data" label="Source File"/> <conditional name="param_file"> <param name="use_params" type="select" label="Use custom parameter file?"> <option value="true">Yes</option> <option value="false" selected="true">No</option> </param> <when value="true"> <param format="text" name="params" type="data" label="Parameter File" value=""/> </when> </conditional> </inputs> It's very simple, I just want the line "-p $params" to appear in the command line when the parameter "use_params" parameter is set to true. However, when building the command line statement to run, I get this error: ---------------------- Traceback (most recent call last): File "/Users/jrl909/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 59, in run_job job_wrapper.prepare() File "/Users/jrl909/Galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 424, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/Users/jrl909/Galaxy/galaxy-dist/lib/galaxy/tools/__init__.py", line 1827, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/Users/jrl909/Galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/Users/jrl909/Galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-macosx-10.6-intel-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1328561167_73_97196.py", line 84, in respond NotFound: cannot find 'use_params' while searching for 'param_file.use_params' ------------------- As far as I can tell, the syntax I've used here is the same as that found in the example file suggested by the Galaxy documentation: interval2maf.xml (what I see as the "relevant" sections being reproduced below): --------------- <command interpreter="python"> #if $maf_source_type.maf_source == "user" #interval2maf.py --dbkey=${input1.dbkey} --chromCol=${input1.metadata.chromCol} --startCol=${input1.metadata.startCol} --endCol=${input1.metadata.endCol} --strandCol=${input1.metadata.strandCol} --mafFile=$maf_source_type.mafFile --mafIndex=$maf_source_type.mafFile.metadata.maf_index --interval_file=$input1 --output_file=$out_file1 --mafIndexFile=${GALAXY_DATA_INDEX_DIR}/maf_index.loc --species=$maf_source_type.species #else #interval2maf.py --dbkey=${input1.dbkey} --chromCol=${input1.metadata.chromCol} --startCol=${input1.metadata.startCol} --endCol=${input1.metadata.endCol} --strandCol=${input1.metadata.strandCol} --mafType=$maf_source_type.mafType --interval_file=$input1 --output_file=$out_file1 --mafIndexFile=${GALAXY_DATA_INDEX_DIR}/maf_index.loc --species=$maf_source_type.species #end if# --split_blocks_by_species=$split_blocks_by_species_selector.split_blocks_by_species #if $split_blocks_by_species_selector.split_blocks_by_species == "split_blocks_by_species"# --remove_all_gap_columns=$split_blocks_by_species_selector.remove_all_gap_columns #end if </command> <inputs> ... blahblahblah ... <conditional name="split_blocks_by_species_selector"> <param name="split_blocks_by_species" type="select" label="Split blocks by species" help="Not usually applicable. See help below for more information."> <option value="split_blocks_by_species">Split by species</option> <option value="dont_split_blocks_by_species" selected="true">Do not split</option> </param> <when value="dont_split_blocks_by_species">  </when> <when value="split_blocks_by_species"> <param name="remove_all_gap_columns" type="select" label="Collapse empty alignment columns"> <option value="remove_all_gap_columns" selected="true">Collapse empty columns</option> <option value="do_not_remove_all_gap_columns">Do not collapse</ option> </param> </when> </conditional> </inputs> ------------------- As you can see, the field 'split_blocks_by_species' is referenced in an #if-block in the command specification the same way my 'use_params' field is. I would like very much to know the apparently crucial difference that I'm missing! Thanks, -Jeff

2 1

Re: [galaxy-dev] running functional tests on a galaxy server
by Steve.Mcmahon＠csiro.au 06 Feb '12

06 Feb '12

Hi Chris and Nate, I'm also interested in being able to run functional tests against either our production server or a test server with an identical setup to make sure that not only the tools are working but their interaction with various compute resources are also working etc. If we get this going we would like to run this automatically and regularly so that administrators and users will be able to see whether particular tools are working or not. Regards. Steve McMahon Solutions architect & senior systems administrator ASC Cluster Services Information Management & Technology (IM&T) CSIRO Phone: +61-2-62142968 | Mobile: +61-4-00779318 steve.mcmahon(a)csiro.au<mailto:steve.mcmahon@csiro.au> | www.csiro.au<http://www.csiro.au/> PO Box 225, DICKSON ACT 2602 1 Wilf Crane Crescent, Yarralumla ACT 2600

1 0

Re: [galaxy-dev] Postgresql To Galaxy
by Nate Coraor 06 Feb '12

06 Feb '12

On Feb 6, 2012, at 10:56 AM, Rehan Saleem wrote: > Hi Nate, > well i don’t want to use galaxy default database , i want to set my own tools for example, i have created a function totaloverlapcount and its work perfectly fine in my separate postgresql database called ova , now i want to add a tool in galaxy with a name Total Overlap Count , as we know whenever we want to add a tool we configure it in tool_config file. > Now if a user click on Total Overlap Count then my postgresql function totaloverlapcount should be called , and that function will perform query on my own created database(ova) and then return the result into galaxy interface Hi Rehan, Please use 'reply all' to keep replies on the mailing list. Since your tool already works with a separate postgres database outside of Galaxy, you should be able to add the tool to Galaxy without modification. I presume the tool has some means of knowing what host, user and port to connect to, and the setup of this information is the same as it would be when running it on the command line (via a config file, environment, hardcoded in the tool...?). --nate > > From: Nate Coraor <nate(a)bx.psu.edu> > To: Rehan Saleem <pk_rehan(a)yahoo.com> > Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> > Sent: Monday, February 6, 2012 8:07 PM > Subject: Re: [galaxy-dev] Postgresql To Galaxy > > On Feb 6, 2012, at 6:37 AM, Rehan Saleem wrote: > > > Hi , > > i have created postgreSQL database where i have created all functions and views which performs annotations and some other functions , how can i show my postgresql database table to galaxyas output . i have connected my database from universe_wsgi.ini to my linux host where i am running galaxy and my database is on windows host, i need the python script which take values from table and show them back to galaxy. > > THANKS > > Hi Rehan, > > I am a little unsure what you're trying to do. Do you want to display contents of this database in some sort of display within Galaxy, use them as tool inputs, or outputs? > > I would not suggest using this custom database as Galaxy's database (set in database_connection) if it contains separate data from the Galaxy application. Galaxy should live in its own database away from any other tables you use for other purposes. > > --nate > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > > >

1 0

Problems with custom data types and sniffers in shed tools
by Ira Cooke 06 Feb '12

06 Feb '12

Hi, We're developing a suite of galaxy tools for doing proteomics. As part of that we're developing a display application ( https://bitbucket.org/Andrew_Brock/proteomics-visualise ) for viewing pepXML and protXML files and we rely very heavily on custom datatypes. We've got this working in a fork of galaxy-dist ( https://bitbucket.org/iracooke/galaxy-proteomics ) but would love to be able to get rid of this fork and integrate all our customisations into a shed tool. I initially tried following the instructions here http://wiki.g2.bx.psu.edu/Tool%20Shed#Including_proprietary_data_types_that… and was able to successfully get my custom datatypes to load. Unfortunately though I could not get my sniffers to work. I think there are two reasons for this; The first problem is that it looks like the sniffers are never loaded. I get errors like this when the tool is loaded; galaxy.datatypes.registry DEBUG 2012-02-06 10:26:51,317 Loading datatypes from /Users/iracooke/Sources/shed_tools_galaxy_central/toolshed.g2.bx.psu.edu/repos/jjohnson/gmap/93911bac43da/gmap/tool-data/datatypes_conf.xml galaxy.datatypes.registry WARNING 2012-02-06 10:26:51,318 Error appending sniffer for datatype galaxy.datatypes.gmap:IntervalAnnotation to sniff_order: No module named gmap galaxy.datatypes.registry WARNING 2012-02-06 10:26:51,319 Error appending sniffer for datatype galaxy.datatypes.gmap:SpliceSiteAnnotation to sniff_order: No module named gmap galaxy.datatypes.registry WARNING 2012-02-06 10:26:51,319 Error appending sniffer for datatype galaxy.datatypes.gmap:IntronAnnotation to sniff_order: No module named gmap galaxy.datatypes.registry WARNING 2012-02-06 10:26:51,319 Error appending sniffer for datatype galaxy.datatypes.gmap:SNPAnnotation to sniff_order: No module named gmap These errors aren't just restricted to my tool ... as you can see from the above they also occur in the gmap example installed from the main galaxy toolshed. I tried hacking the code in registration.py to get this to work ... it looks like the section where sniffers are loaded does not use the "imported_modules" variable. I was able to get this error message to go away, but I still don't get proper sniffer behaviour (eg when I click "Auto detect" when editing a dataset the dataset is set to a generic type). Another issue is that I would like the sniffers loaded from my shed_tool to be used by the "Upload" tool .. but if I look at the source for upload.py and upload.xml I see -- Upload.xml <command interpreter="python"> upload.py $GALAXY_ROOT_DIR $GALAXY_DATATYPES_CONF_FILE $paramfile --Upload.py def __main__(): .... registry = Registry() registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] ) which looks like the upload tool is not configured to use the custom datatypes available in shed tools. Would it be possible to make the following changes; (a) Add custom sniffers to the sniff order when shed tools are loaded. Importantly since custom datatypes are usually quite specific I would suggest that these are loaded at the top of the sniff order? Or alternatively if a sniffer is for a datatype that descends from a superclass it should have priority over the parent class (since by definition it is more specific). (b) Change the upload tool so that it respects custom sniffers in shed tools. I guess that our case is a bit unusual in that we are trying to co-opt galaxy ( a genomics tool) to do proteomics ... so I understand that these changes might not be a priority. Nevertheless, if this could be done it would be fantastic for us as we could abandon our fork and have all our functionality included in a shed tool. Regards Ira

2 2

xml tool wrapper <conditional>
by Holger Klein 06 Feb '12

06 Feb '12

Dear all, I'm working on a tool wrapper which for a sequence scoring tool. It's supposed to score sequences either using a library installed to galaxy (tool-data/models.loc) or datasets from the user history. I tried to implement this behavior using the <conditional> / <when value> mechanism, simplified code follows below. Using locally installed models (from models.loc) fails with "NotFound: cannot find 'models'", although in the details view of the failed tool run "model database" points to the right file. Using the model file from the history works. Defining _only_ locally installed models from models.loc also works (removing the <conditional> stuff and leaving only the part inside <when value='local'> </when>). The commandline might look a bit strange but is correct. Can anybody spot what is going wrong here? Regards, Holger -- <command> calcModels --scoreFasta -- --fa $fasta_in --bgFa $fasta_background --models $models > $output_table </command> <inputs> <param format="fasta" name="fasta_in" type="data" label="Input Fasta File" /> <param format="fasta" name="fasta_background" type="data" label="Background Fasta File" /> <conditional name="ModelSource"> <param name="models" type="select" label="model source" help="History or installed models?" value="local"> <option value="local">Locally installed models</option> <option value="history">Models from your history</option> </param> <when value="local"> <param name="models" type="select" label="model database"> <options from_file="models.loc"> <column name="name" index="1"/> <column name="value" index="2"/> </options> </param> </when> <when value="history"> <param name="models" type="data" format="tabular" label="model database" /> </when> </conditional> </inputs> -- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

2 2

Writing a database interative tool
by Matloob Khushi 06 Feb '12

06 Feb '12

Hi Galaxy Developers I aim to write a tool that will interact with Postgres database; here is what I wrote to test database connectivity. This script resides in /galaxy-dist/tools/test_tool/ and gets invoked from the front end. However script generate following error: No module named psycopg2. Here is my code: #!/usr/bin/env python import os.path import os from galaxy import eggs from galaxy.model import * from sqlalchemy import * engine = create_engine('postgresql://postgres:@192.168.0.6:5432/galaxy') connection = engine.connect() result_set = connection.execute( "select id from galaxy_user" ) for row in result_set: print row[ 'id' ] connection.close() Traceback (most recent call last): File "/galaxy-dist/tools/overlapanalysis/db_test.py", line 10, in ? engine = create_engine('postgresql://postgres:@192.168.0.6:5432/galaxy') File "/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/__init__.py", line 223, in create_engine return strategy.create(*args, **kwargs) File "/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/strategies.py", line 62, in create dbapi = dialect_cls.dbapi(**dbapi_args) File "/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/databases/postgres.py", line 368, in dbapi import psycopg2 as psycopg ImportError: No module named psycopg2 Thanks for any help. Regards MK

2 1

Postgresql To Galaxy
by Rehan Saleem 06 Feb '12

06 Feb '12

Hi , i have created postgreSQL database where i have created all functions and views which performs annotations and some other functions , how can i show my postgresql database table to galaxyas output . i have connected my database from universe_wsgi.ini to my linux host where i am running galaxy and my database is on windows host, i need the python script which take values from table and show them back to galaxy. THANKS

2 1

Tool syntax to embed external web page doesn't quite work as desired anymore
by Leandro Hermida 06 Feb '12

06 Feb '12

Hi everyone, It used to be in the previous or so galaxy-dist that if you wanted to have a tool which displayed a web page from another web app in the center Galaxy frame you simply did: <tool name="View External Web Page" id="view_external_page" version="1.0.0"> <description>from External App</description> <inputs action="http://my/url/path" check_values="false" method="get" /> </tool> But this doesn't work anymore, instead you get a form with an "Execute" button which you then have to click on in order to go to the external web page you wanted to embed. How do I get the old desired behavior? regards, Leandro

1 0

Running Galaxy on Rocks Cluster
by Derrick LIN 05 Feb '12

05 Feb '12

Hi guys, Is there anyone has actually setup galaxy on a Rocks based cluster? I know that galaxy should be hosted on a submission host so that it can submit jobs to the cluster. Rocks provides preconfigured DRM such as SGE, so I start thinking if galaxy host should be deployed and managed by Rocks (e.g. as a Login Appliance in Rocks' term). Sorry if this is not a pure galaxy issue, but I guess people in here will be more appropriate to answer this. Regards, Derrick

3 3

Feature requests - viewing data and annotations
by SHAUN WEBB 05 Feb '12

05 Feb '12

Restricting datasets to show only the first megabyte is a great idea when dealing with large data files but sometimes I want to see the end of a file to check it is complete or to check the progress of a running job. At the moment I use the helper script to find the file on the server and then use tail. Would it be possible to add a button in the dataset view (next to "show all" and "save") that allows you to see the end of the file? Also, I have noticed a few discussions regarding tracking datasets through a history and sensible naming of history items. In my experience the best way to keep track of a history, especially one you haven't visited for a while, is to keep good notes. The tags and annotations for individual history items is good but I don't think the space provided for a history annotation is practical either in size or format. Ideally I would like to have a large area to add notes while still being able to view datasets. Perhaps turning on annotation could split the main pane in to 2 areas with adjustable size with an annotation area on the bottom? Maybe there would be a way to link Galaxy pages directly to histories? I'm not sure what the best way to implement this would be but I'd like to see an improvement. Thanks to all the Galaxy team Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1