Hi John and all,
We are running an old version of Galaxy v15.10 and use Pulsar to stage certain jobs to our hpc cluster. This works great and we love it. However, we have noticed that for certain tools which require metadata to be staged, although the metadata file is actually staged, the Galaxy pulsar runner fails to rewrite the file path for the metadata correctly. Here is an example command from the latest Freebayes wrappers:
ln -s -f '/cluster/galaxy/pulsar/files/staging/32728/inputs/dataset_54494.dat' 'b_0.bam' && ln -s -f '/Volumes/ngs/database/files/_metadata_files/002/metadata_2754.dat' 'b_0.bam.bai' && ln -s -f '/cluster/galaxy/pulsar/files/staging/32728/inputs/dataset_54496.dat' 'b_1.bam' && ln -s -f '/Volumes/ngs/database/files/_metadata_files/002/metadata_2755.dat' 'b_1.bam.bai' && samtools view -H b_0.bam | grep "^@SQ" | cut -f 2- | awk '{ gsub("^SN:","",$1); gsub("^LN:","",$2); print $1"\t0\t"$2; }' >> regions_all.bed && samtools view -H b_1.bam | grep "^@SQ" | cut -f 2- | awk '{ gsub("^SN:","",$1); gsub("^LN:","",$2); print $1"\t0\t"$2; }' >> regions_all.bed && sort -u regions_all.bed > regions_uniq.bed && mkdir vcf_output && mkdir failed_alleles && mkdir trace && for i in `cat regions_uniq.bed | awk '{print $1":"$2".."$3}'`; do echo " freebayes --region '$i' --bam 'b_0.bam' --bam 'b_1.bam' --fasta-reference '/cluster/galaxy/indexes/danRer7/sam_index/danRer7.fa' --vcf './vcf_output/part_$i.vcf' --standard-filters --min-coverage '3' "; done > freebayes_commands.sh && cat freebayes_commands.sh | parallel --no-notice -j ${GALAXY_SLOTS:-1} && grep "^#" "./vcf_output/part_$i.vcf" > header.txt && for i in `cat regions_uniq.bed | awk '{print $1":"$2".."$3}'`; do cat "./vcf_output/part_$i.vcf" | grep -v "^#" || true ; done | sort -k1,1 -k2,2n -k5,5 -u | cat header.txt - > '/cluster/galaxy/pulsar/files/staging/32728/outputs/dataset_54897.dat’
As you can see, the file path for the BAM file is correctly re-written but not the file path for the bam.bai index, which still has a file path local to Galaxy.
Has anyone come across this problem before and is there a fix? We are very hesitant to upgrade our Galaxy server as everything else is working perfectly for us right now.
Thanks, Richard