| 1 |
## For the Impatient |
| 2 |
|
| 3 |
```sh |
| 4 |
# Download bwakit (or from <http://sourceforge.net/projects/bio-bwa/files/bwakit/> manually) |
| 5 |
wget -O- http://sourceforge.net/projects/bio-bwa/files/bwakit/bwakit-0.7.12_x64-linux.tar.bz2/download \ |
| 6 |
| gzip -dc | tar xf - |
| 7 |
# Generate the GRCh38+ALT+decoy+HLA and create the BWA index |
| 8 |
bwa.kit/run-gen-ref hs38DH # download GRCh38 and write hs38DH.fa |
| 9 |
bwa.kit/bwa index hs38DH.fa # create BWA index |
| 10 |
# mapping |
| 11 |
bwa.kit/run-bwamem -o out -H hs38DH.fa read1.fq read2.fq | sh # skip "|sh" to show command lines |
| 12 |
``` |
| 13 |
|
| 14 |
This generates `out.aln.bam` as the final alignment, `out.hla.top` for best HLA |
| 15 |
genotypes on each gene and `out.hla.all` for other possible HLA genotypes. |
| 16 |
Please check out [bwa/bwakit/README.md][kithelp] for details. |
| 17 |
|
| 18 |
## Background |
| 19 |
|
| 20 |
GRCh38 consists of several components: chromosomal assembly, unlocalized contigs |
| 21 |
(chromosome known but location unknown), unplaced contigs (chromosome unknown) |
| 22 |
and ALT contigs (long clustered variations). The combination of the first three |
| 23 |
components is called the *primary assembly*. It is recommended to use the |
| 24 |
complete primary assembly for all analyses. Using ALT contigs in read mapping is |
| 25 |
tricky. |
| 26 |
|
| 27 |
GRCh38 ALT contigs are totaled 109Mb in length, spanning 60Mbp of the primary |
| 28 |
assembly. However, sequences that are highly diverged from the primary assembly |
| 29 |
only contribute a few million bp. Most subsequences of ALT contigs are nearly |
| 30 |
identical to the primary assembly. If we align sequence reads to GRCh38+ALT |
| 31 |
blindly, we will get many additional reads with zero mapping quality and miss |
| 32 |
variants on them. It is crucial to make mappers aware of ALTs. |
| 33 |
|
| 34 |
BWA-MEM is ALT-aware. It essentially computes mapping quality across the |
| 35 |
non-redundant content of the primary assembly plus the ALT contigs and is free |
| 36 |
of the problem above. |
| 37 |
|
| 38 |
## Methods |
| 39 |
|
| 40 |
### Sequence alignment |
| 41 |
|
| 42 |
As of now, ALT mapping is done in two separate steps: BWA-MEM mapping and |
| 43 |
postprocessing. The `bwa.kit/run-bwamem` script performs the two steps when ALT |
| 44 |
contigs are present. The following picture shows an example about how BWA-MEM |
| 45 |
infers mapping quality and reports alignment after step 2: |
| 46 |
|
| 47 |
 |
| 48 |
|
| 49 |
#### Step 1: BWA-MEM mapping |
| 50 |
|
| 51 |
At this step, BWA-MEM reads the ALT contig names from "*idxbase*.alt", ignoring |
| 52 |
the ALT-to-ref alignment, and labels a potential hit as *ALT* or *non-ALT*, |
| 53 |
depending on whether the hit lands on an ALT contig or not. BWA-MEM then reports |
| 54 |
alignments and assigns mapQ following these two rules: |
| 55 |
|
| 56 |
1. The mapQ of a non-ALT hit is computed across non-ALT hits only. The mapQ of |
| 57 |
an ALT hit is computed across all hits. |
| 58 |
|
| 59 |
2. If there are no non-ALT hits, the best ALT hit is outputted as the primary |
| 60 |
alignment. If there are both ALT and non-ALT hits, non-ALT hits will be |
| 61 |
primary and ALT hits be supplementary (SAM flag 0x800). |
| 62 |
|
| 63 |
In theory, non-ALT alignments from step 1 should be identical to alignments |
| 64 |
against the reference genome with ALT contigs. In practice, the two types of |
| 65 |
alignments may differ in rare cases due to seeding heuristics. When an ALT hit |
| 66 |
is significantly better than non-ALT hits, BWA-MEM may miss seeds on the |
| 67 |
non-ALT hits. |
| 68 |
|
| 69 |
If we don't care about ALT hits, we may skip postprocessing (step 2). |
| 70 |
Nonetheless, postprocessing is recommended as it improves mapQ and gives more |
| 71 |
information about ALT hits. |
| 72 |
|
| 73 |
#### Step 2: Postprocessing |
| 74 |
|
| 75 |
Postprocessing is done with a separate script `bwa-postalt.js`. It reads all |
| 76 |
potential hits reported in the XA tag, lifts ALT hits to the chromosomal |
| 77 |
positions using the ALT-to-ref alignment, groups them based on overlaps between |
| 78 |
their lifted positions, and then re-estimates mapQ across the best scoring hit |
| 79 |
in each group. Being aware of the ALT-to-ref alignment, this script can greatly |
| 80 |
improve mapQ of ALT hits and occasionally improve mapQ of non-ALT hits. It also |
| 81 |
writes each hit overlapping the reported hit into a separate SAM line. This |
| 82 |
enables variant calling on each ALT contig independent of others. |
| 83 |
|
| 84 |
### On the completeness of GRCh38+ALT |
| 85 |
|
| 86 |
While GRCh38 is much more complete than GRCh37, it is still missing some true |
| 87 |
human sequences. To make sure every piece of sequence in the reference assembly |
| 88 |
is correct, the [Genome Reference Consortium][grc] (GRC) require each ALT contig |
| 89 |
to have enough support from multiple sources before considering to add it to the |
| 90 |
reference assembly. This careful and sophisticated procedure has left out some |
| 91 |
sequences, one of which is [this example][novel], a 10kb contig assembled from |
| 92 |
CHM1 short reads and present also in NA12878. You can try [BLAT][blat] or |
| 93 |
[BLAST][blast] to see where it maps. |
| 94 |
|
| 95 |
For a more complete reference genome, we compiled a new set of decoy sequences |
| 96 |
from GenBank clones and the de novo assembly of 254 public [SGDP][sgdp] samples. |
| 97 |
The sequences are included in `hs38DH-extra.fa` from the [BWA binary |
| 98 |
package][res]. |
| 99 |
|
| 100 |
In addition to decoy, we also put multiple alleles of HLA genes in |
| 101 |
`hs38DH-extra.fa`. These genomic sequences were acquired from [IMGT/HLA][hladb], |
| 102 |
version 3.18.0 and are used to collect reads sequenced from these genes. |
| 103 |
|
| 104 |
### HLA typing |
| 105 |
|
| 106 |
HLA genes are known to be associated with many autoimmune diseases, infectious |
| 107 |
diseases and drug responses. They are among the most important genes but are |
| 108 |
rarely studied by WGS projects due to the high sequence divergence between |
| 109 |
HLA genes and the reference genome in these regions. |
| 110 |
|
| 111 |
By including the HLA gene regions in the reference assembly as ALT contigs, we |
| 112 |
are able to effectively identify reads coming from these genes. We also provide |
| 113 |
a pipeline, which is included in the [BWA binary package][res], to type the |
| 114 |
several classic HLA genes. The pipeline is conceptually simple. It de novo |
| 115 |
assembles sequence reads mapped to each gene, aligns exon sequences of each |
| 116 |
allele to the assembled contigs and then finds the pairs of alleles that best |
| 117 |
explain the contigs. In practice, however, the completeness of IMGT/HLA and |
| 118 |
copy-number changes related to these genes are not so straightforward to |
| 119 |
resolve. HLA typing may not always be successful. Users may also consider to use |
| 120 |
other programs for typing such as [Warren et al (2012)][hla4], [Liu et al |
| 121 |
(2013)][hla2], [Bai et al (2014)][hla3] and [Dilthey et al (2014)][hla1], though |
| 122 |
most of them are distributed under restrictive licenses. |
| 123 |
|
| 124 |
## Preliminary Evaluation |
| 125 |
|
| 126 |
To check whether GRCh38 is better than GRCh37, we mapped the CHM1 and NA12878 |
| 127 |
unitigs to GRCh37 primary (hs37), GRCh38 primary (hs38) and GRCh38+ALT+decoy |
| 128 |
(hs38DH), and called small variants from the alignment. CHM1 is haploid. |
| 129 |
Ideally, heterozygous calls are false positives (FP). NA12878 is diploid. The |
| 130 |
true positive (TP) heterozygous calls from NA12878 are approximately equal |
| 131 |
to the difference between NA12878 and CHM1 heterozygous calls. A better assembly |
| 132 |
should yield higher TP and lower FP. The following table shows the numbers for |
| 133 |
these assemblies: |
| 134 |
|
| 135 |
|Assembly|hs37 |hs38 |hs38DH|CHM1_1.1| huref| |
| 136 |
|:------:|------:|------:|------:|------:|------:| |
| 137 |
|FP | 255706| 168068| 142516|307172 | 575634| |
| 138 |
|TP |2142260|2163113|2150844|2167235|2137053| |
| 139 |
|
| 140 |
With this measurement, hs38 is clearly better than hs37. Genome hs38DH reduces |
| 141 |
FP by ~25k but also reduces TP by ~12k. We manually inspected variants called |
| 142 |
from hs38 only and found the majority of them are associated with excessive read |
| 143 |
depth, clustered variants or weak alignment. We believe most hs38-only calls are |
| 144 |
problematic. In addition, if we compare two NA12878 replicates from HiSeq X10 |
| 145 |
with nearly identical library construction, the difference is ~140k, an order |
| 146 |
of magnitude higher than the difference between hs38 and hs38DH. ALT contigs, |
| 147 |
decoy and HLA genes in hs38DH improve variant calling and enable the analyses of |
| 148 |
ALT contigs and HLA typing at little cost. |
| 149 |
|
| 150 |
## Problems and Future Development |
| 151 |
|
| 152 |
There are some uncertainties about ALT mappings - we are not sure whether they |
| 153 |
help biological discovery and don't know the best way to analyze them. Without |
| 154 |
clear demand from downstream analyses, it is very difficult to design the |
| 155 |
optimal mapping strategy. The current BWA-MEM method is just a start. If it |
| 156 |
turns out to be useful in research, we will probably rewrite bwa-postalt.js in C |
| 157 |
for performance; if not, we may make changes. It is also possible that we might |
| 158 |
make breakthrough on the representation of multiple genomes, in which case, we |
| 159 |
can even get rid of ALT contigs for good. |
| 160 |
|
| 161 |
|
| 162 |
|
| 163 |
[res]: https://sourceforge.net/projects/bio-bwa/files/bwakit |
| 164 |
[sb]: https://github.com/GregoryFaust/samblaster |
| 165 |
[grc]: http://www.ncbi.nlm.nih.gov/projects/genome/assembly/grc/ |
| 166 |
[novel]: https://gist.github.com/lh3/9935148b71f04ba1a8cc |
| 167 |
[blat]: https://genome.ucsc.edu/cgi-bin/hgBlat |
| 168 |
[blast]: http://blast.st-va.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&PAGE_TYPE=BlastSearch&LINK_LOC=blasthome |
| 169 |
[sgdp]: http://www.simonsfoundation.org/life-sciences/simons-genome-diversity-project/ |
| 170 |
[hladb]: http://www.ebi.ac.uk/ipd/imgt/hla/ |
| 171 |
[grcdef]: http://www.ncbi.nlm.nih.gov/projects/genome/assembly/grc/info/definitions.shtml |
| 172 |
[hla1]: http://biorxiv.org/content/early/2014/07/08/006973 |
| 173 |
[hlalink]: http://www.hladiseaseassociations.com |
| 174 |
[hlatools]: https://www.biostars.org/p/93245/ |
| 175 |
[hla2]: http://nar.oxfordjournals.org/content/41/14/e142.full.pdf+html |
| 176 |
[hla3]: http://www.biomedcentral.com/1471-2164/15/325 |
| 177 |
[hla4]: http://genomemedicine.com/content/4/12/95 |
| 178 |
[kithelp]: https://github.com/lh3/bwa/tree/master/bwakit |