Hi Leon, Can you see if there is any info reported reported by running the MAF consistency checking script at scripts/tools/maf/check_loc_file.py path/to/maf_index.loc? You'll need to make sure that required dependencies (e.g. galaxy_root/lib and galaxy_root/eggs) are included in your PYTHONPATH. As far as cutting down a MAF to only contain the set of desired species and depending upon your needs, you can use the tools included with Galaxy, or alternatively the maf_thread_for_species.py script in bx-python. Thanks for using Galaxy, Dan On Sep 11, 2013, at 2:10 PM, Leon Mei wrote:
Hi Dan,
Here are the output: == hmei@ubuntu:~$ ls -lah /mnt/galaxyIndices/alignments/maf/ total 236G drwxrwsr-x 2 postgres postgres 4.0K Sep 10 00:08 . drwxrwsr-x 4 postgres postgres 4.0K Sep 9 16:27 .. -rw-rw-r-- 1 postgres postgres 13G Oct 31 2009 chr10.maf -rw-rw-r-- 1 postgres postgres 189M Sep 9 17:29 chr10.maf.index -rw-rw-r-- 1 postgres postgres 189M Sep 8 21:35 chr11_gl000202_random.maf -rw-rw-r-- 1 postgres postgres 12G Oct 31 2009 chr11.maf -rw-rw-r-- 1 postgres postgres 184M Sep 9 18:07 chr11.maf.index -rw-rw-r-- 1 postgres postgres 12G Oct 31 2009 chr12.maf -rw-rw-r-- 1 postgres postgres 186M Sep 9 18:31 chr12.maf.index -rw-rw-r-- 1 postgres postgres 8.7G Oct 31 2009 chr13.maf -rw-rw-r-- 1 postgres postgres 135M Sep 9 18:49 chr13.maf.index -rw-rw-r-- 1 postgres postgres 8.3G Oct 31 2009 chr14.maf -rw-rw-r-- 1 postgres postgres 125M Sep 9 19:05 chr14.maf.index -rw-rw-r-- 1 postgres postgres 7.3G Oct 31 2009 chr15.maf -rw-rw-r-- 1 postgres postgres 112M Sep 9 19:19 chr15.maf.index -rw-rw-r-- 1 postgres postgres 7.1G Oct 31 2009 chr16.maf -rw-rw-r-- 1 postgres postgres 108M Sep 9 19:32 chr16.maf.index -rw-rw-r-- 1 postgres postgres 39M Oct 31 2009 chr17_ctg5_hap1.maf -rw-rw-r-- 1 postgres postgres 456K Sep 9 19:45 chr17_ctg5_hap1.maf.index -rw-rw-r-- 1 postgres postgres 753K Oct 31 2009 chr17_gl000203_random.maf -rw-rw-r-- 1 postgres postgres 45K Sep 9 19:45 chr17_gl000203_random.maf.index -rw-rw-r-- 1 postgres postgres 5.1M Oct 31 2009 chr17_gl000204_random.maf -rw-rw-r-- 1 postgres postgres 101K Sep 9 19:45 chr17_gl000204_random.maf.index -rw-rw-r-- 1 postgres postgres 1.6M Oct 31 2009 chr17_gl000205_random.maf -rw-rw-r-- 1 postgres postgres 33K Sep 9 19:45 chr17_gl000205_random.maf.index -rw-rw-r-- 1 postgres postgres 968K Oct 31 2009 chr17_gl000206_random.maf -rw-rw-r-- 1 postgres postgres 24K Sep 9 19:45 chr17_gl000206_random.maf.index -rw-rw-r-- 1 postgres postgres 7.3G Oct 31 2009 chr17.maf -rw-rw-r-- 1 postgres postgres 112M Sep 9 19:45 chr17.maf.index -rw-rw-r-- 1 postgres postgres 265K Oct 31 2009 chr18_gl000207_random.maf -rw-rw-r-- 1 postgres postgres 3.5K Sep 9 19:59 chr18_gl000207_random.maf.index -rw-rw-r-- 1 postgres postgres 7.0G Oct 31 2009 chr18.maf -rw-rw-r-- 1 postgres postgres 106M Sep 9 19:59 chr18.maf.index -rw-rw-r-- 1 postgres postgres 4.2K Oct 31 2009 chr19_gl000208_random.maf -rw-rw-r-- 1 postgres postgres 193 Sep 9 20:05 chr19_gl000208_random.maf.index -rw-rw-r-- 1 postgres postgres 2.6M Oct 31 2009 chr19_gl000209_random.maf -rw-rw-r-- 1 postgres postgres 54K Sep 9 20:05 chr19_gl000209_random.maf.index -rw-rw-r-- 1 postgres postgres 3.5G Oct 31 2009 chr19.maf -rw-rw-r-- 1 postgres postgres 48M Sep 9 20:05 chr19.maf.index -rw-rw-r-- 1 postgres postgres 4.1M Oct 31 2009 chr1_gl000191_random.maf -rw-rw-r-- 1 postgres postgres 108K Sep 9 20:05 chr1_gl000191_random.maf.index -rw-rw-r-- 1 postgres postgres 22M Oct 31 2009 chr1_gl000192_random.maf -rw-rw-r-- 1 postgres postgres 376K Sep 9 20:06 chr1_gl000192_random.maf.index -rw-rw-r-- 1 postgres postgres 21G Oct 31 2009 chr1.maf -rw-rw-r-- 1 postgres postgres 330M Sep 9 17:13 chr1.maf.index -rw-rw-r-- 1 postgres postgres 5.6G Oct 31 2009 chr20.maf -rw-rw-r-- 1 postgres postgres 83M Sep 9 20:59 chr20.maf.index -rw-rw-r-- 1 postgres postgres 342K Oct 31 2009 chr21_gl000210_random.maf -rw-rw-r-- 1 postgres postgres 31K Sep 9 21:05 chr21_gl000210_random.maf.index -rw-rw-r-- 1 postgres postgres 2.9G Oct 31 2009 chr21.maf -rw-rw-r-- 1 postgres postgres 38M Sep 9 21:05 chr21.maf.index -rw-rw-r-- 1 postgres postgres 2.8G Oct 31 2009 chr22.maf -rw-rw-r-- 1 postgres postgres 37M Sep 9 21:10 chr22.maf.index -rw-rw-r-- 1 postgres postgres 23G Oct 31 2009 chr2.maf -rw-rw-r-- 1 postgres postgres 355M Sep 9 20:49 chr2.maf.index -rw-rw-r-- 1 postgres postgres 19G Oct 31 2009 chr3.maf -rw-rw-r-- 1 postgres postgres 291M Sep 9 21:45 chr3.maf.index -rw-rw-r-- 1 postgres postgres 9.1M Oct 31 2009 chr4_ctg9_hap1.maf -rw-rw-r-- 1 postgres postgres 150K Sep 9 22:16 chr4_ctg9_hap1.maf.index -rw-rw-r-- 1 postgres postgres 4.8M Oct 31 2009 chr4_gl000193_random.maf -rw-rw-r-- 1 postgres postgres 101K Sep 9 22:16 chr4_gl000193_random.maf.index -rw-rw-r-- 1 postgres postgres 2.2M Oct 31 2009 chr4_gl000194_random.maf -rw-rw-r-- 1 postgres postgres 112K Sep 9 22:16 chr4_gl000194_random.maf.index -rw-rw-r-- 1 postgres postgres 17G Oct 31 2009 chr4.maf -rw-rw-r-- 1 postgres postgres 257M Sep 9 22:16 chr4.maf.index -rw-rw-r-- 1 postgres postgres 17G Oct 31 2009 chr5.maf -rw-rw-r-- 1 postgres postgres 255M Sep 9 22:47 chr5.maf.index -rw-rw-r-- 1 postgres postgres 44M Oct 31 2009 chr6_apd_hap1.maf -rw-rw-r-- 1 postgres postgres 496K Sep 9 23:18 chr6_apd_hap1.maf.index -rw-rw-r-- 1 postgres postgres 96M Oct 31 2009 chr6_cox_hap2.maf -rw-rw-r-- 1 postgres postgres 1.1M Sep 9 23:18 chr6_cox_hap2.maf.index -rw-rw-r-- 1 postgres postgres 86M Oct 31 2009 chr6_dbb_hap3.maf -rw-rw-r-- 1 postgres postgres 926K Sep 9 23:18 chr6_dbb_hap3.maf.index -rw-rw-r-- 1 postgres postgres 16G Oct 31 2009 chr6.maf -rw-rw-r-- 1 postgres postgres 249M Sep 9 23:17 chr6.maf.index -rw-rw-r-- 1 postgres postgres 78M Oct 31 2009 chr6_mann_hap4.maf -rw-rw-r-- 1 postgres postgres 848K Sep 9 23:18 chr6_mann_hap4.maf.index -rw-rw-r-- 1 postgres postgres 80M Oct 31 2009 chr6_mcf_hap5.maf -rw-rw-r-- 1 postgres postgres 854K Sep 9 23:18 chr6_mcf_hap5.maf.index -rw-rw-r-- 1 postgres postgres 87M Oct 31 2009 chr6_qbl_hap6.maf -rw-rw-r-- 1 postgres postgres 949K Sep 9 23:18 chr6_qbl_hap6.maf.index -rw-rw-r-- 1 postgres postgres 80M Oct 31 2009 chr6_ssto_hap7.maf -rw-rw-r-- 1 postgres postgres 871K Sep 9 23:19 chr6_ssto_hap7.maf.index -rw-rw-r-- 1 postgres postgres 2.4M Oct 31 2009 chr7_gl000195_random.maf -rw-rw-r-- 1 postgres postgres 56K Sep 9 23:44 chr7_gl000195_random.maf.index -rw-rw-r-- 1 postgres postgres 14G Oct 31 2009 chr7.maf -rw-rw-r-- 1 postgres postgres 214M Sep 9 23:44 chr7.maf.index -rw-rw-r-- 1 postgres postgres 551K Oct 31 2009 chr8_gl000196_random.maf -rw-rw-r-- 1 postgres postgres 50K Sep 10 00:08 chr8_gl000196_random.maf.index -rw-rw-r-- 1 postgres postgres 1.4M Oct 31 2009 chr8_gl000197_random.maf -rw-rw-r-- 1 postgres postgres 56K Sep 10 00:08 chr8_gl000197_random.maf.index -rw-rw-r-- 1 postgres postgres 13G Oct 31 2009 chr8.maf -rw-rw-r-- 1 postgres postgres 201M Sep 10 00:08 chr8.maf.index ==
And thanks for the suggestion of adding an extra dir level, I will do that.
At the moment, I only need primates, but I couldn't find a way to get subset of mafs (http://hgdownload.cse.ucsc.edu/goldenPath/hg19/), so I downloaded the 46 way files and index primates first. If you knows a trick to get a selected subset, I would love to hear it.
Thanks!
Leon
On Wed, Sep 11, 2013 at 6:38 PM, Daniel Blankenberg <dan@bx.psu.edu> wrote:
Hi Leon,
What is the output of "ls -lah /mnt/galaxyIndices/alignments/maf/"?
Also, you may what to use additional sub-directory structure for your file locations in case you want to add more alignments later, e.g., /mnt/galaxyIndices/hg19/maf/10_way_multiz/ or similiar
Also, note that although you are only indexing (for extraction) for the specified listed species, the alignments will contain all of the species present in the original file (you list 46way multiZ as the source).
Thanks for using Galaxy,
Dan
On Sep 10, 2013, at 6:44 PM, Leon Mei wrote:
Dear colleagues,
I am having some trouble installing MAF datasets at our server. When I run "Extract MAF blocks given a set of genomic intervals", I got this error message:
"Fatal Error: The MAF source specified (10_WAY_MULTIZ_hg19) appears to be invalid."
Below is what I did by following the wiki page: http://wiki.galaxyproject.org/Admin/Datatypes/Add%20MAFs
1, download all mafs for hg19 from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/multiz46way/maf/
2, index them using "maf_build_index.py --species=hg19,panTro2, gorGor1,ponAbe2,papHam1,calJac1,tarSyr1,micMur1,otoGar1,rheMac2 $file". Our user requested primates only.
3, Here is my maf_index.loc file: == 10-way multiZ (hg19) 10_WAY_MULTIZ_hg19 hg19,panTro2,gorGor1,ponAbe2,papHam1,calJac1,tarSyr1,micMur1,otoGar1,rheMac2 hg19,panTro2,gorGor1,ponAbe2,papHam1,calJac1,tarSyr1,micMur1,otoGar1,rheMac2 /mnt/galaxyIndices/alignments/maf/chr1.maf,/mnt/galaxyIndices/alignments/m af/chr10.maf,/mnt/galaxyIndices/alignments/maf/chr11.maf,/mnt/galaxyIndices/alignments/maf/chr11_gl000202_random.maf,/mnt/galaxyIndices/ali gnments/maf/chr12.maf,/mnt/galaxyIndices/alignments/maf/chr13.maf,/mnt/galaxyIndices/alignments/maf/chr14.maf,/mnt/galaxyIndices/alignments /maf/chr15.maf,/mnt/galaxyIndices/alignments/maf/chr16.maf,/mnt/galaxyIndices/alignments/maf/chr17.maf,/mnt/galaxyIndices/alignments/maf/ch r17_ctg5_hap1.maf,/mnt/galaxyIndices/alignments/maf/chr17_gl000203_random.maf,/mnt/galaxyIndices/alignments/maf/chr17_gl000204_random.maf,/ mnt/galaxyIndices/alignments/maf/chr17_gl000205_random.maf,/mnt/galaxyIndices/alignments/maf/chr17_gl000206_random.maf,/mnt/galaxyIndices/a lignments/maf/chr18.maf,/mnt/galaxyIndices/alignments/maf/chr18_gl000207_random.maf,/mnt/galaxyIndices/alignments/maf/chr19.maf,/mnt/galaxy Indices/alignments/maf/chr19_gl000208_random.maf,/mnt/galaxyIndices/alignments/maf/chr19_gl000209_random.maf,/mnt/galaxyIndices/alignments/ maf/chr1_gl000191_random.maf,/mnt/galaxyIndices/alignments/maf/chr1_gl000192_random.maf,/mnt/galaxyIndices/alignments/maf/chr2.maf,/mnt/gal axyIndices/alignments/maf/chr20.maf,/mnt/galaxyIndices/alignments/maf/chr21.maf,/mnt/galaxyIndices/alignments/maf/chr21_gl000210_random.maf ,/mnt/galaxyIndices/alignments/maf/chr22.maf,/mnt/galaxyIndices/alignments/maf/chr3.maf,/mnt/galaxyIndices/alignments/maf/chr4.maf,/mnt/gal axyIndices/alignments/maf/chr4_ctg9_hap1.maf,/mnt/galaxyIndices/alignments/maf/chr4_gl000193_random.maf,/mnt/galaxyIndices/alignments/maf/c hr4_gl000194_random.maf,/mnt/galaxyIndices/alignments/maf/chr5.maf,/mnt/galaxyIndices/alignments/maf/chr6.maf,/mnt/galaxyIndices/alignments /maf/chr6_apd_hap1.maf,/mnt/galaxyIndices/alignments/maf/chr6_cox_hap2.maf,/mnt/galaxyIndices/alignments/maf/chr6_dbb_hap3.maf,/mnt/galaxyI ndices/alignments/maf/chr6_mann_hap4.maf,/mnt/galaxyIndices/alignments/maf/chr6_mcf_hap5.maf,/mnt/galaxyIndices/alignments/maf/chr6_qbl_hap 6.maf,/mnt/galaxyIndices/alignments/maf/chr6_ssto_hap7.maf,/mnt/galaxyIndices/alignments/maf/chr7.maf,/mnt/galaxyIndices/alignments/maf/chr 7_gl000195_random.maf,/mnt/galaxyIndices/alignments/maf/chr8.maf,/mnt/galaxyIndices/alignments/maf/chr8_gl000196_random.maf,/mnt/galaxyIndi ces/alignments/maf/chr8_gl000197_random.maf ==
Any hint on which step is wrong or missing is highly appreciate!
Thanks, Leon
-- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - http://ngs.nbic.nl Skype: leon_mei Mobile: +31 6 41709231 ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - http://ngs.nbic.nl Skype: leon_mei Mobile: +31 6 41709231