Debian Codegraph @

docs/tutorial-cgp/de_novo.html

raw
1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
2 <html><head><title>Predicting Genes with AUGUSTUS</title>
3 <link rel="stylesheet" type="text/css" href="augustus.css">
4 <script src="tutorial.js" type="text/javascript"></script>
5 </head><body>
6 <font size=-1>
7 Navigate to <a href="index.html">main AugCGP tutorial</a>.
8 <a href="cactus.html">Cactus alignments and assembly Hubs</a>.
9 <a href="rnaseq.html">AugCGP with RNA-Seq</a>.
10 <a href="liftover.html">Annotation transfer with AugCGP</a>.
11 <a href="rnaseq+liftover.html">Combining RNA-Seq and annotation evidence</a>.
12 <a href="hgm.html">Cross-species consistency of gene sets</a>.
13 </font>
14 <div align="right">Show <a href="javascript:allOn()">all</a> / <a href="javascript:allOff()">no</a> details.</div>
15
16 <h1><i>De novo</i> comparative gene prediction with Augustus-cgp</h1>
17
18 In this tutorial, we run Augustus-cgp in its most basic form: <i>de novo</i>. That means
19 only the raw genomes and the alignment of the genomes are input, but no extrinsic evidence, e.g. from transcriptome data.
20 In most applications, however, RNA-Seq data will be available and it is recommended to incorporate them to obtain a better accuracy.
21
22 <h2 id="loadFA">1. Load genomes into an SQLite database</h2>
23
24 <span class="assignment">Prepare</span> a text file with a list of species names and location of
25 the corresponding genome fasta files.<br><br>
26
27 <a href="javascript:onoff('cft')" class="dlink"><span id="cft" title="cftd" class="dcross">[+]</span>
28 <span class="dtitle">file format...</span></a> <br>
29 <div id="cftd" class="details" style="display:none;">
30 <pre class="code">
31 name_of_genome_1 path/to/genome_1
32 name_of_genome_2 path/to/genome_2
33 ...
34 name_of_genome_N path/to/genome_N
35 </pre>
36 </div>
37 </br>
38
39 <pre class="code">
40 for f in $PWD/genomes/*.fa; do echo -ne "$(basename $f .fa)\t$f\n"; done >genomes.tbl
41 </pre>
42
43 The file <span class="result"><tt>genomes.tbl</tt></span> will now look like this (except for the parent directory)<br>
44 <pre class="code">
45 bosTau8 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/bosTau8.fa
46 canFam3 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/canFam3.fa
47 galGal4 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/galGal4.fa
48 hg38 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/hg38.fa
49 mm10 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/mm10.fa
50 monDom5 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/monDom5.fa
51 rheMac3 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/rheMac3.fa
52 rn6 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/rn6.fa
53 </pre>
54
55 <span class="assignment">Load</span> the genomes into an SQLite database
56 <pre class="code">
57 while read line
58 do
59 species=$(echo "$line" | cut -f 1)
60 genome=$(echo "$line" | cut -f 2)
61 load2sqlitedb --noIdx --species=$species --dbaccess=vertebrates.db $genome
62 done &ltgenomes.tbl
63 </pre>
64
65 <span class="assignment">Finalize</span> database by creating indices on the tables
66 <pre class="code">
67 load2sqlitedb --makeIdx --dbaccess=vertebrates.db
68 </pre>
69 You can <span class="assignment">check</span> if loading was successful with following
70 database query
71 <pre class="code">
72 sqlite3 -header -column vertebrates.db "\
73 SELECT speciesname, \
74 sum(end-start+1) AS 'genome length',\
75 count(*) AS '# chunks',\
76 count(distinct seqnr) AS '# seqs'\
77 FROM genomes natural join speciesnames\
78 GROUP BY speciesname;"
79 </pre>
80 that returns a summary of the genomes in the database, e.g. their sizes (total number of bases per genome),
81 number of sequences per genome, etc.
82 <pre class="code">
83 speciesname genome length # chunks # seqs
84 ----------- ------------- ---------- ----------
85 bosTau8 156091 4 1
86 canFam3 184728 4 1
87 galGal4 149999 3 1
88 hg38 210155 5 1
89 mm10 178393 4 1
90 monDom5 540519 11 1
91 rheMac3 220640 5 1
92 rn6 99944 2 1
93 </pre>
94
95 <h2 id="runAug">2. Run AUGUSTUS in CGP mode</h2>
96
97 <span class="assignment">Create</span> a new folder for the de novo experiments and
98 <span class="assignment">switch</span> to the new directory
99 <pre class="code">
100 mkdir augCGP_denovo
101 cd augCGP_denovo
102 </pre>
103 For convenience <span class="assignment">assign</span> each alignment chunk to a job ID by
104 creating softlinks
105 <pre class="code">
106 num=1
107 for f in ../mafs/*.maf; do ln -s $f $num.maf; ((num++)); done
108 </pre>
109 <span class="assignment">Run</span> Augustus in cgp mode on all alignment chunks in parallel (~3min).<br>
110 <span class="assignment">Use</span> the option <tt>--softmasking=1</tt> in cases where the genomes are soft-masked.<br>
111 The parameter <tt>--species</tt> is the same as in the standard version of Augustus. You can take
112 the species identifier that best represents your clade,<br>
113 (e.g. <tt>--species=human</tt> for mammalian clades, <tt>--species=fly</tt> for fly clades, <tt>--species=chicken</tt> for bird clades, ...)<br>
114 A complete list of species identifiers can be found in
115 the directory <tt>augustus/config/species</tt>.
116 <pre class="code">
117 for ali in *.maf; do
118 id=${ali%.maf} # remove .maf suffix
119 augustus \
120 --species=human \
121 --softmasking=1 \
122 --treefile=../tree.nwk \
123 --alnfile=$ali \
124 --dbaccess=../vertebrates.db \
125 --speciesfilenames=../genomes.tbl \
126 --alternatives-from-evidence=0 \
127 --/CompPred/outdir=pred$id &gt aug$id.out 2&gt err$id.out &
128 done
129 </pre>
130 This will generate the folders <span class="result"><tt>pred*/</tt></span> (one for each alignment chunk)
131 that contain gff files with gene predictions for each input genome.
132 <pre class="code">
133 bosTau8.cgp.gff
134 canFam3.cgp.gff
135 galGal4.cgp.gff
136 hg38.cgp.gff
137 mm10.cgp.gff
138 monDom5.cgp.gff
139 rheMac3.cgp.gff
140 rn6.cgp.gff
141 </pre>
142
143 Note that the parallelization with the bash '&' command above is quite simple and rather for demonstration purposes.<br>
144 For real applications with several hundreds or thousands of alignment chunks, we recommend to
145 run job arrays on a compute cluster.
146
147 <h2 id="merge">3. Merge gene predictions from parallel runs</h2>
148 <pre class="code">
149 mkdir joined_pred
150 while read line
151 do
152 species=$(echo "$line" | cut -f 1)
153 find pred* -name "${species}.cgp.gff" >${species}_gtfs.lst;
154 joingenes -f ${species}_gtfs.lst -o joined_pred/$species.gff
155 done &lt ../genomes.tbl
156 </pre>
157 This will create the folder <span class="result"><tt>joined_pred/</tt></span>
158 with the final gene predictions for each input genome.
159 <pre class="code">
160 bosTau8.gff
161 canFam3.gff
162 galGal4.gff
163 hg38.gff
164 mm10.gff
165 monDom5.gff
166 rheMac3.gff
167 rn6.gff
168 </pre>
169
170 <h2 id="makeBeds">4. Upload gene predictions into the assembly hub</h2>
171 <span class="assignment">Convert</span> the final gene predictions from gff to BED format and place
172 each BED file in a separate folder with the name of the corresponding genome. It is important that directory names are consistent with the names in the HAL alignment.
173 <pre class="code">
174 for f in joined_pred/*.gff
175 do
176 mkdir "$(basename $f .gff)"
177 gtf2bed.pl <$f >$(basename $f .gff)/augCGP_denovo.bed --itemRgb=225,0,0
178 done
179 </pre>
180 Specify any RGB color you like for the track with option <tt>--itemRgb</tt>, e.g. <span style="color:rgb(255,0,0);">225,0,0.</span><br>
181 The name of the current directory (i.e. <tt>augCGP_denovo</tt>) will be used as track name on the browser.<br>
182
183 <span class="assignment">Switch</span> back to the main working directory <tt>data/</tt>
184 <pre class="code">
185 cd ..
186 </pre>
187 and rerun the <tt>hal2assemblyHub.py</tt> script. Include gene tracks with option <tt>--bedDirs</tt>
188
189 <pre class="code">
190 hal2assemblyHub.py vertebrates.hal vertHub --lod \
191 --alignability --gcContent \
192 --hub vertCompHub --shortLabel VertebratesCompHub \
193 --bedDirs augCGP_denovo \
194 --tabBed \
195 --maxThreads=10 --longLabel "Vertebrates Comparative Assembly Hub"
196 </pre>
197 You can also include gene tracks from other exercises by passing a comma-separated list of directories e.g.
198 <tt>--bedDirs refseq,augCGP_denovo,augCGP_rnaseq,augCGP_liftover,...</tt><br><br>
199
200 <span class="assignment">Repeat</span> <a href="cactus.html#loadHub">4. Load the hub and browser the alignment</a>.<br>
201 </body></html>