Debian Codegraph @

doc/pages/cis_permutation.html

raw
1 <html>
2 <head>
3 <meta http-equiv="content-type" content="text/html; charset=iso-8859-1"/>
4 <meta http-equiv="Cache-Control" content="max-age=3600"/>
5 <meta name="description" content="description"/>
6 <meta name="keywords" content="keywords"/>
7 <meta name="author" content="author"/>
8 <link rel="stylesheet" type="text/css" href="../style/default.css" media="screen"/>
9 <title>FastQTL</title>
10 <script language="Javascript" src="../script/print_last_modif_date.js" ></script>
11 </head>
12
13 <body>
14 <div class="content">
15
16 <div class="item" id="cis_perm1">
17 <h1>Default permutation pass</h1>
18 <p>
19 To perform a permutation based analysis of the example data set using a total of 1,000 permutations, use:
20 </p>
21
22 <code>
23 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 <b>--permute 1000</b> --out permutations.default.txt.gz
24 </code>
25
26 <p>
27 This produces this output on the screen when everything works correctly:
28 </p>
29
30 <img WIDTH=40% src="../img/fastqtl_screenshot_permutation.jpg"></img>
31
32 <br>
33 <p>
34 You can increase the number of permutations to 10,000 using:
35 </p>
36
37 <code>
38 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 <b>--permute 10000</b> --out permutations.10K.txt.gz
39 </code>
40
41 <p>
42 You can also perform an adaptive permutation pass on the example data set (i.e. with a number of permutations that varies in function of the significance).
43 To run between 100 and 100,000 permutations, use:
44 </p>
45
46 <code>
47 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 <b>--permute 100 100000</b> --out permutations.adaptive.txt.gz
48 </code>
49
50 </div>
51
52 <div class="item" id="cis_map3">
53 <h1>Association testing parameters</h1>
54 <p>
55 Associations between genotype dosages and phenotype quantifications are measured with linear regressions (<a href="http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient">here</a>), similar to the R/lm function. This model assumes that phenotypes are normally distributed.
56 If your phenotype quantifications are not normally distributed, you can force them to match normal distributions N(0, 1) by using:
57 </p>
58
59 <code>
60 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 --permute 1000 --out permutations.quantile.txt.gz <b>--normal</b>
61 </code>
62
63 <p>
64 To change the cis-window size (i.e. the maximal distance spanned by phenotype-variant pairs to be considered for testing) from default value 1e6 bp to 2e6 bp, use:
65 </p>
66
67 <code>
68 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 --permute 1000 --out permutations.window2Mb.txt.gz <b>--window 2e6</b>
69 </code>
70
71 <p>
72 To change the seed of the random number generator, which is particularly useful to replicate an analysis, use:
73 </p>
74
75 <code>
76 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 --permute 1000 --out permutations.seed.txt.gz <b>--seed 123456789</b>
77 </code>
78
79 <p>
80 To add covariates in the linear regressions used for association testing, use:
81 </p>
82
83 <code>
84 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 --permute 1000 --out permutations.txt.gz <b>--cov covariates.txt.gz</b>
85 </code>
86
87 <p>
88 The file <b>covariates.txt.gz</b> needs to be formatted as described <b>here</b>.
89 </p>
90 </div>
91
92 <div class="item" id="cis_map4">
93 <h1>Excluding/Including data</h1>
94 <p>
95 To exclude samples, variants, phenotypes or covariates from the analysis, you can use one of these options:
96 <ol>
97 <li>To exclude samples: <i>--exclude-samples file.exc</i></li>
98 <li>To exclude variants: <i>--exclude-sites file.exc</i></li>
99 <li>To exclude phenotypes: <i>--exclude-phenotypes file.exc</i></li>
100 <li>To exclude caovariates: <i>--exclude-covariates file.exc</i></li>
101 </ol>
102 </p>
103
104 <p>
105 For instance, if you want to ignore 3 samples when analyzing the example data set, first create a text file containing the IDs of the samples to be excluded, called here <i>file.exc</i>:
106 </p>
107
108 <code>
109 UNR1<br>
110 UNR2<br>
111 UNR3<br>
112 </code>
113
114 <p>
115 Then, run the following command line:
116 </p>
117
118 <code>
119 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --region 22:17000000-18000000 --permute 1000 --out permutations.sub.txt.gz <b>--exclude-samples file.exc</b>
120 </code>
121
122 <p>
123 Similarly to these 4 options for data exclusion, you can also specify the set of samples, variants, phenotypes and covariates to be included in the analysis using the options: <i>--include-samples, --include-sites, --include-phenotypes and --include-covariates</i>, respectively.
124 </p>
125 </div>
126
127 <div class="item" id="cis_map5">
128 <h1>Parallelization</h1>
129 <p>
130 As a first way to facilitate parallelization on compute cluster, we developed an option to run the analysis for just a chunk of molecular phenotypes.
131 The region of interest is specified with the standard <b>chr:start-end</b> format.
132 Then, FastQTL extracts all phenotypes in this region, then all genotypes given the specified cis-window size and finally perform the analysis for this data subset.
133 For instance, to run QTL mapping only for molecular phenotypes on chr22 with coordinates between 18Mb and 20Mb, use:
134 </p>
135
136 <code>
137 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz <b>--region chr22:18000000-20000000</b> --permute 1000 --out permutations.18M20M.txt.gz
138 </code>
139
140 <note>
141 This option requires both the genotype and phenotype files to be indexed with TABIX!
142 </note>
143
144 <p>
145 This strategy is quite similar to what is commonly used for genotype imputation, where only small chunks of data are imputed one at a time in seperate jobs.
146 However in practice, it is usually quite complicated to properly split the genome into a given number of chunks with correct coordinates.
147 To facilitate this, we embedded all coordinates into a chunk-based system such that you only need to specify the chunk index you want to run.
148 Then, splitting the genome into chunks, extraction of data, and analysis are automatically performed.
149 For instance, to run analysis on chunk number 25 when splitting the example data set (i.e. genome) into 30 chunks, just run:
150 </p>
151
152 <code>
153 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.chunk25.txt.gz <b>--chunk 25 30</b>
154 </code>
155
156 <p>
157 If you want to submit the whole analysis into 30 jobs on your compute cluster, just run:
158 </p>
159
160 <code>
161 for j in <b>$(seq 1 30)</b>; do<br>
162 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;echo "fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.txt.gz <b>--chunk $j 30</b>" | qsub<br>
163 done
164 </code>
165
166 <p>
167 Here <b>qsub</b> needs to be changed according to the job submission system used (bsub, psub, etc...).
168 </p>
169
170 <note>
171 In this simple example, we only split the data into 30 chunks.
172 However, a realistic whole genome analysis would require to split the data in 200, 500 or even 1,000 chunks to fully use the capabilities of a modern compute cluster.
173 </note>
174
175 <p>
176 Finally, we also developed a slightly different parallelization option that, this time, allows to generate all required command lines and write into a file.
177 Let take the same example as before, that is splitting the analysis into 10 jobs:
178 </p>
179
180 <code>
181 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations <b>--commands 10 commands.10.txt</b>
182 </code>
183
184 <p>
185 Now if you look at the file <b>commands.10.txt</b>, you'll get this:
186 </p>
187
188 <code>
189 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:17517460-20748406 --region 22:17517460-20748406<br>
190 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:36424473-39052635 --region 22:36424473-39052635<br>
191 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:24407642-30163001 --region 22:24407642-30163001<br>
192 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:42017123-45704850 --region 22:42017123-45704850<br>
193 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:20792146-22307210 --region 22:20792146-22307210<br>
194 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:39052641-39915701 --region 22:39052641-39915701<br>
195 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:30163352-36044443 --region 22:30163352-36044443<br>
196 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:45809500-51222092 --region 22:45809500-51222092<br>
197 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:22337213-24322661 --region 22:22337213-24322661<br>
198 fastQTL --vcf genotypes.vcf.gz --bed phenotypes.bed.gz --permute 1000 --out permutations.22:39928860-42017101 --region 22:39928860-42017101<br>
199 </code>
200
201 <p>
202 Where, region coordinates are automatically determined given the total number of chunks provided.
203 Then, you can submit all these commands on a cluster using for example:
204 </p>
205
206 <code>
207 while read c; do<br>
208 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;echo $c | qsub<br>
209 done <b>| commands.10.txt</b>
210 </code>
211 </div>
212
213 <div class="item" id="cis_map8">
214 <h1>Output file format of a <b>permutation</b> pass</h1>
215
216 <p>
217 Once the analysis completed and all output file collected, you can easily concat and compress all files into a single output file to ease downstream analysis with:
218 </p>
219
220 <code>
221 zcat permutations.chunk*.txt.gz | gzip -c > permutations.all.chunks.txt.gz
222 </code>
223
224 <p>
225 After having performed a permutation pass on the data and concatenating the output files, you end up with a file with 10 columns and M lines with M being the total number of tested molecular phenotypes.
226 For instance, if you tested 1,000 molecular phenotypes, it means that you'll get 1,000 lines in the output files.
227 Hereafter a short example:
228 </p>
229
230 <code>
231 ENSG00000237438.1 7966 0.93436 2871.25 376.072 snp_22_17542810 25350 5.48728e-13 0.000999001 4.38131e-09<br>
232 ENSG00000177663.8 8165 0.936488 3667.52 392.593 snp_22_17497295 -68549 0.000167489 0.376623 0.361675<br>
233 ENSG00000183307.3 8279 1.03157 2719.43 369.406 snp_22_17587975 -14282 3.11324e-08 0.000999001 6.78199e-05<br>
234 ENSG00000069998.8 8466 1.04834 2472.36 358.392 snp_22_17639837 -6340 5.01155e-11 0.000999001 1.26653e-07<br>
235 ENSG00000093072.10 8531 0.934923 5838.18 406.904 snp_22_17699299 -39826 9.028e-05 0.246753 0.245706<br>
236 </code>
237
238 <p>
239 In this file, the 10 columns correspond to:
240 <ol>
241 <li>ID of the tested molecular phenotype (in this particular case, the gene ID)</li>
242 <li>Number of variants tested in cis for this phenotype</li>
243 <li>MLE of the shape1 parameter of the Beta distribution</li>
244 <li>MLE of the shape2 parameter of the Beta distribution</li>
245 <li>Dummy [To be described later]</li>
246 <li>ID of the best variant found for this molecular phenotypes (i.e. with the smallest p-value)</li>
247 <li>Distance between the molecular phenotype - variant pair</li>
248 <li>The <b>nominal</b> p-value of association that quantifies how significant from 0, the regression coefficient is</li>
249 <li>A first <b>permutation</b> p-value directly obtained from the permutations with the direct method. This is basically a corrected version of the nominal p-value that accounts for the fact that multiple variants are tested per molecular phenotype.
250 <li>A second <b>permutation</b> p-value obtained via beta approximation. We advice to use this one in any downstream analysis.
251 </ol>
252 </p>
253 </div>
254
255 <div class="item" id="cis_map8">
256 <h1>Checking that the experiment went well</h1>
257
258 <p>
259 To check that the beta approximated permutation p-values are well estimated, load the data in R and make the following plot:
260 </p>
261
262 <code>
263 R> d = read.table("permutations.all.chunks.txt.gz", hea=F, stringsAsFactors=F)<br>
264 R> colnames(d) = c("pid", "nvar", "shape1", "shape2", "dummy", "sid", "dist", "npval", "ppval", "bpval")<br>
265 R> plot(d$ppval, d$bpval, xlab="Direct method", ylab="Beta approximation", main="Check plot")<br>
266 R> abline(0, 1, col="red")<br>
267 </code>
268
269 <img WIDTH=40% src="../img/fastqtl_checks.jpg"></img>
270
271 <br>
272 <p>
273 The expectation is to get all the points along the diagonal as on the plot above.
274 This shows that unsignificant beta pproximated p-values are well estimated and therefore that the estimations went well.
275 </p>
276 </div>
277
278 <div class="item" id="cis_map9">
279 <h1>Controlling for multiple phenotypes being tested</h1>
280 <p>
281 Once obtained all permutation p-values for all the tested molecular phenotypes, we need now to account for the fact that many molecular phenotypes are tested whole genome in order to determine the significant QTLs.
282 To do so, we propose here 3 approaches; from the most to the least stringent.
283 First, load the data in R using:
284 </p>
285
286 <code>
287 R> d = read.table("permutations.all.chunks.txt.gz", hea=F, stringsAsFactors=F)<br>
288 R> colnames(d) = c("pid", "nvar", "shape1", "shape2", "dummy", "sid", "dist", "npval", "ppval", "bpval")<br>
289 </code>
290
291 <br>
292 <h3>Bonferroni correction</h3>
293 <p>
294 Look <a href="http://en.wikipedia.org/wiki/Bonferroni_correction">here</a> for some details on the correction and <a href="https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html">here</a> for some details on the R function used in this example.
295 To apply the Bonferroni correction on the permutation p-values derived from beta approximation, use:
296 </p>
297
298 <code>
299 <b>R></b> d$bonferroni = p.adjust(d$bpval, method="bonferroni")
300 </code>
301
302 <p>
303 Then, you can extract all significant MP-QTL pairs at &alpha;=0.05 and write them to a file <b>permutations.all.chunks.bonferroni.txt</b> using:
304 </p>
305
306 <code>
307 <b>R></b> write.table(d[which(d$bonferroni <= 0.05), c(1,6)], "permutations.all.chunks.bonferroni.txt", quote=F, row.names=F, col.names=T)
308 </code>
309
310 <br>
311 <h3>Benjamini & Hochberg correction</h3>
312 <p>
313 Look <a href="http://en.wikipedia.org/wiki/False_discovery_rate">here</a> for some details on the correction and <a href="https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html">here</a> for some details on the R function used in this example.
314 To apply the Benjamini & Hochberg correction on the permutation p-values derived from beta approximation, use:
315 </p>
316
317 <code>
318 <b>R></b> d$bh = p.adjust(d$bpval, method="fdr")
319 </code>
320
321 <p>
322 Then, you can extract all significant MP-QTL pairs with a 10% false discovery rate (FDR) for instance and write them to a file <b>permutations.all.chunks.benjamini.txt</b> using:
323 </p>
324
325 <code>
326 <b>R></b> write.table(d[which(d$bh <= 0.10), c(1,6)], "permutations.all.chunks.benjamini.txt", quote=F, row.names=F, col.names=T)
327 </code>
328
329 <br>
330 <h3>Storey & Tibshirani correction</h3>
331 <p>
332 Look <a href="http://en.wikipedia.org/wiki/False_discovery_rate">here</a> for some details on the correction and <a href="http://svitsrv25.epfl.ch/R-doc/library/qvalue/html/qvalue.html">here</a> for some details on the R function used in this example.
333 To install the R/qvalue package on your system, look <a href="http://www.bioconductor.org/packages/release/bioc/html/qvalue.html">here</a>.
334 Then to apply the Storey & Tibshirani correction on the permutation p-values derived from beta approximation, use:
335 </p>
336
337 <code>
338 <b>R></b> library(qvalue)
339 <b>R></b> d$st = qvalue(d$bpval)$qvalues
340 </code>
341
342 <p>
343 Then, you can extract all significant MP-QTL pairs with a 10% false discovery rate (FDR) for instance and write them to a file <b>whole_analysis.storey.permutations</b> using:
344 </p>
345
346 <code>
347 <b>R></b> write.table(d[which(d$st <= 0.10), c(1,6)], "permutations.all.chunks.storey.txt", quote=F, row.names=F, col.names=T)
348 </code>
349 </div>
350
351 <div class="log"><script type="text/javascript"> print_last_modif_date("$Date: 2015-04-01 16:11:24 +0200 (Wed, 01 Apr 2015) $"); </script></div>
352 </div>
353 </body>
354 </html>