Hi Peter,

i wrote the tool_dependencies.xml

<?xml version="1.0"?> <tool_dependency> <package name="jvarkit" version="1.0.0"> <install version="1.0.0"> <actions> <action type="set_environment"> <environment_variable name="JVARKITDIR" action="set_to">$INSTALL_DIR</environment_variable> </action> </actions> </install> <readme></readme> </package> </tool_dependency>

and updated vcffilterjs.xml

<?xml version="1.0"?> <tool hidden="false" id="com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS" version="1.0.0" name="vcffilterjs"> <requirements> <requirement type="package" version="1.0.0">jvarkit</requirement> <requirement type="binary">java</requirement> </requirements>

<command><![CDATA[java -cp \$JVARKITDIR/commons-jexl-2.1.1.jar:\$JVARKITDIR/commons-logging-1.1.1.jar:\$JVARKITDIR/htsjdk-1.129.jar:\$JVARKITDIR/snappy-java-1.0.3-rc3.jar:\$JVARKITDIR/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} ]]></command> <inputs> <param format="vcf" name="input" type="data" label="VCF input"/> <param name="expression" type="text" label="javascript expression"/> </inputs> <outputs> <data format="vcf" name="output"/> </outputs> <stdio> <exit_code range="1:"/> <exit_code range=":-1"/> </stdio> <help></help> </tool>

I still get an error:

(...)

:24,289 (31) command is: java -cp $JVARKITDIR/commons-jexl-2.1.1.jar:$JVARKITDIR/commons-logging-1.1.1.jar:$JVARKITDIR/htsjdk-1.129.jar:$JVARKITDIR/snappy-java-1.0.3-rc3.jar:$JVARKITDIR/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e 'variant!=nullX' -o /home/lindenb/tmp/GALAXY/galaxy-galaxy-dist-32e4527bab2d/database/files/000/dataset_27.dat /home/lindenb/tmp/GALAXY/galaxy-galaxy-dist-32e4527bab2d/database/files/000/dataset_1.dat; return_code=$?; c (...) galaxy.jobs DEBUG 2015-02-26 13:31:24,333 (31) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2015-02-26 13:31:25,689 execution finished: /home/lindenb/tmp/GALAXY/galaxy-galaxy-dist-32e4527bab2d/database/job_working_directory/000/31/galaxy_31.sh galaxy.jobs.output_checker INFO 2015-02-26 13:31:25,717 Job 31: Fatal error: Exit code 1 () galaxy.jobs DEBUG 2015-02-26 13:31:25,784 setting dataset state to ERROR

The error displayed in galaxy is just an empty VCF header (that why I asked about the error log)

Thank you for your help

PS: By the way, my argument was -e 'variant!=null;' , why do I see '-e 'variant!=nullX' in the log ?

Pierre

On Thu, Feb 26, 2015 at 10:59 AM, Pierre Lindenbaum pierre.lindenbaum@univ-nantes.fr wrote:

...

Hi Galaxy,

I'm trying to put one of my java tool in galaxy.

I've copied the required files in my tool directory under /tools/jvarkit (I can see it galaxy)

 $ ls
 commons-jexl-2.1.1.jar
 commons-logging-1.1.1.jar
 htsjdk-1.129.jar
 snappy-java-1.0.3-rc3.jar
 vcffilterjs.jar
 vcffilterjs.xml

Here is the content of vcffilterjs.xml

   <?xml version="1.0"?>
   <tool hidden="false"

id="com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS" version="1.0.0" name="vcffilterjs"> <requirements> <requirement type="binary">java</requirement> </requirements> <command><![CDATA[java -cp $INSTALL_DIR/commons-jexl-2.1.1.jar:$INSTALL_DIR/commons-logging-1.1.1.jar:$INSTALL_DIR/htsjdk-1.129.jar:$INSTALL_DIR/snappy-java-1.0.3-rc3.jar:$INSTALL_DIR/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} 2> /tmp/error ]]></command> <inputs> <param format="vcf" name="input" type="data" label="VCF input"/> <param name="expression" type="text" label="javascript expression"/> </inputs> <outputs> <data format="vcf" name="output"/> </outputs> <stdio> <exit_code range="1:"/> <exit_code range=":-1"/> </stdio> <help>(...)</help> </tool>

When I run this tool, the log says:

    NotFound: cannot find 'INSTALL_DIR'

If I replace $INSTALL_DIR with $INSTALL_DIR ,

Yes, you need to escape the dollar sign.

...

the error on stderr is :

 Error: Could not find or load main class

com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS

How can I make this work ?

Try inserting something like this into the command as a diagnostic:

echo "$INSTALL_DIR" &amp;&amp;

Where do you define $INSTALL_DIR? Is this environment variable done at system level, or are you trying to do this within a Galaxy defined package.

If the later, you need a <requirements type="package" ...> tag, which cross references a package defined within a tool_dependencies.xml file. See for example:

https://github.com/peterjc/pico_galaxy/tree/master/tools/clinod

...

Bonus question: is there a way to catch the error file to display it in galaxy when a process fails ?

Which error file? Do you mean stderr? That is captured and visible via the "eye" icon when the dataset is expanded in the history pane.

Peter

You'll need to white list the semi-colon which Galaxy has replaced as

a security precaution (user free text can be exploited). See <sanitizer> here: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax Peter

Thanks: adding :

<sanitizer> <valid initial="string.printable"> <remove value="&apos;"/> </valid> </sanitizer>

fixed the probem.

Could you confirm if the environment variable is being set now?

Hum the variable is not set , I've changed <command> to

<command><![CDATA[echo "JVARKITDIR=\$JVARKITDIR expression=${expression} " > /tmp/error7.txt && java -cp \$JVARKITDIR/commons-jexl-2.1.1.jar:\$JVARKITDIR/commons-logging-1.1.1.jar:\$JVARKITDIR/htsjdk-1.129.jar:\$JVARKITDIR/snappy-java-1.0.3-rc3.jar:\$JVARKITDIR/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} 2>> /tmp/error7.txt ]]>

here is the content of tmp/error7.txt

$ cat /tmp/error7.txt JVARKITDIR= expression=variant!=null; Error: Could not find or load main class com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS

P.

On 02/26/2015 02:51 PM, John Chilton wrote:

Just to make sure - since it hasn't been explicitly stated - is this tool being installed from the tool shed? The environment variable setup in tool_dependencies.xml will only be available for tool shed installs.

no, my future plan is to put it in the tool shed but for now, I'm just trying to make it run in my local instance of galaxy.

I'm sorry if that wasn't clear.

I've just downloaded the latest version of galaxy via https://bitbucket.org/galaxy/galaxy-dist/get/tip.zip ( galaxy-galaxy-dist-02ddea42faad.zip ) and re-tried.

I tried to print the variable $__tool_directory__ with the tool.xml below

but it's not defined

$ cat /tmp/error7.txt dir= . Error: Could not find or load main class com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS

the xml:

<?xml version="1.0"?> <tool hidden="false" id="com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS" version="1.0.0" name="vcffilterjs"> <requirements> <requirement type="binary">java</requirement> </requirements>

<command><![CDATA[echo "dir=\$__tool_directory__ ." > /tmp/error7.txt && java -cp \$__tool_directory__/commons-jexl-2.1.1.jar:\$__tool_directory__/commons-logging-1.1.1.jar:\$__tool_directory__/htsjdk-1.129.jar:\$__tool_directory__/snappy-java-1.0.3-rc3.jar:\$__tool_directory__/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} 2>> /tmp/error7.txt ]]></command> <inputs> <param format="vcf" name="input" type="data" label="VCF input"/> <param name="expression" type="text" label="javascript expression"> <sanitizer> <valid initial="string.printable"> <remove value="&apos;"/> </valid> </sanitizer> </param> </inputs> <outputs> <data format="vcf" name="output"/> </outputs> <stdio> <exit_code range="1:"/> <exit_code range=":-1"/> </stdio> <help></help> </tool>

On Thu, Feb 26, 2015 at 2:43 PM, John Chilton jmchilton@gmail.com wrote:

Also not need to escape $__tool_directory__ (so use $__tool_directory__ instead of $__tool_directory__).

-John

If this was unclear, $VAR in the <command> XML fragment is taken to be a Python variable available within the Cheetah parser. Typically this would be one of your usr-set parameters.

Escaping the dollar as $VAR in the XML fragment means it is ignored by the Cheetah parser and becomes $VAR in the command line string, meaning an environment variable.

Using bash syntax you can insert a default value, e.g. many tools use something like -t "${GALAXY_SLOTS:-8}" which would default to 8 threads even in older Galaxy instances where the $GALAXY_SLOTS environment variable wasn't setup.

Peter

Well - this version does feature Sam's complete rewrite of the front end for tools - so things will be pretty different.

Oddly though your tool loaded for me - can you open up your JavaScript console and let me know if there are any errors?

-John

On Thu, Feb 26, 2015 at 10:14 AM, Pierre Lindenbaum pierre.lindenbaum@univ-nantes.fr wrote:

On 02/26/2015 03:43 PM, John Chilton wrote:

...

Sorry I wasn't clear about that - it hasn't reached galaxy-dist yet - you can download the latest snapshot of 15.03 release (I believe it is running on usegalaxy.org as of yesterday) via this link instead https://bitbucket.org/galaxy/galaxy-central/get/release_15.03.zip.

Also not need to escape $__tool_directory__ (so use $__tool_directory__ instead of $__tool_directory__).

-John

thank you for your help John & Peter

hum... is there anything new with this version about tool.xml ?. I can see my tool in the right menu but the main page is blank (no input fields)

127.0.0.1 - - [26/Feb/2015:16:09:07 +0200] "GET

/tool_runner?tool_id=com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [26/Feb/2015:16:09:17 +0200] "GET

/tool_runner?tool_id=com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [26/Feb/2015:16:09:27 +0200] "GET

/tool_runner?tool_id=com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [26/Feb/2015:16:09:28 +0200] "GET

/tool_runner?tool_id=com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0"

$ xmllint --format tools/jvarkit2/vcffilterjs.xml

<?xml version="1.0"?>

<tool hidden="false" id="com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS" version="1.0.0" name="vcffilterjs">

<requirements> <requirement type="binary">java</requirement> </requirements> <command><![CDATA[echo "dir=$__tool_directory__ ." > /tmp/error7.txt && java -cp $__tool_directory__/commons-jexl-2.1.1.jar:$__tool_directory__/commons-logging-1.1.1.jar:$__tool_directory__/htsjdk-1.129.jar:$__tool_directory__/snappy-java-1.0.3-rc3.jar:$__tool_directory__/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} 2>> /tmp/error7.txt ]]></command> <inputs> <param format="vcf" name="input" type="data" label="VCF input"/> <param name="expression" type="text" label="javascript expression"> <sanitizer> <valid initial="string.printable"> <remove value="'"/> </valid> </sanitizer> </param> </inputs> <outputs> <data format="vcf" name="output"/> </outputs> <stdio> <exit_code range="1:"/> <exit_code range=":-1"/> </stdio> <help>Help</help> </tool>

On Thu, Feb 26, 2015 at 1:35 PM, Pierre Lindenbaum pierre.lindenbaum@univ-nantes.fr wrote:

...

You'll need to white list the semi-colon which Galaxy has replaced as a security precaution (user free text can be exploited). See <sanitizer> here: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax Peter

Thanks: adding :

   <sanitizer>
          <valid initial="string.printable">
                 <remove value="&apos;"/>
            </valid>
     </sanitizer>

fixed the probem.

Good.

...

Could you confirm if the environment variable is being set now?

Hum the variable is not set , ...

How was the tool installed into your Galaxy? Was it via a (local) Tool Shed which would have processed the tool_dependencies.xml file?

Or, have you manually added your tool XML to Galaxy by hand (which is what I still do when developing a new tool)? In this case you'd have to manually set the environment variable. (I delay messing about with dependencies until after the tool works locally, and then test them via the Test Tool Shed.)

Peter