| 1 |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"> |
| 2 |
<html><head><title>Predicting Genes with AUGUSTUS</title> |
| 3 |
<link rel="stylesheet" type="text/css" href="augustus.css"> |
| 4 |
<script src="tutorial.js" type="text/javascript"></script> |
| 5 |
</head><body> |
| 6 |
<font size=-1> |
| 7 |
Navigate to <a href="index.html">main AugCGP tutorial</a>. |
| 8 |
<a href="cactus.html">Cactus alignments and assembly Hubs</a>. |
| 9 |
<a href="rnaseq.html">AugCGP with RNA-Seq</a>. |
| 10 |
<a href="liftover.html">Annotation transfer with AugCGP</a>. |
| 11 |
<a href="rnaseq+liftover.html">Combining RNA-Seq and annotation evidence</a>. |
| 12 |
<a href="hgm.html">Cross-species consistency of gene sets</a>. |
| 13 |
</font> |
| 14 |
<div align="right">Show <a href="javascript:allOn()">all</a> / <a href="javascript:allOff()">no</a> details.</div> |
| 15 |
|
| 16 |
<h1><i>De novo</i> comparative gene prediction with Augustus-cgp</h1> |
| 17 |
|
| 18 |
In this tutorial, we run Augustus-cgp in its most basic form: <i>de novo</i>. That means |
| 19 |
only the raw genomes and the alignment of the genomes are input, but no extrinsic evidence, e.g. from transcriptome data. |
| 20 |
In most applications, however, RNA-Seq data will be available and it is recommended to incorporate them to obtain a better accuracy. |
| 21 |
|
| 22 |
<h2 id="loadFA">1. Load genomes into an SQLite database</h2> |
| 23 |
|
| 24 |
<span class="assignment">Prepare</span> a text file with a list of species names and location of |
| 25 |
the corresponding genome fasta files.<br><br> |
| 26 |
|
| 27 |
<a href="javascript:onoff('cft')" class="dlink"><span id="cft" title="cftd" class="dcross">[+]</span> |
| 28 |
<span class="dtitle">file format...</span></a> <br> |
| 29 |
<div id="cftd" class="details" style="display:none;"> |
| 30 |
<pre class="code"> |
| 31 |
name_of_genome_1 path/to/genome_1 |
| 32 |
name_of_genome_2 path/to/genome_2 |
| 33 |
... |
| 34 |
name_of_genome_N path/to/genome_N |
| 35 |
</pre> |
| 36 |
</div> |
| 37 |
</br> |
| 38 |
|
| 39 |
<pre class="code"> |
| 40 |
for f in $PWD/genomes/*.fa; do echo -ne "$(basename $f .fa)\t$f\n"; done >genomes.tbl |
| 41 |
</pre> |
| 42 |
|
| 43 |
The file <span class="result"><tt>genomes.tbl</tt></span> will now look like this (except for the parent directory)<br> |
| 44 |
<pre class="code"> |
| 45 |
bosTau8 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/bosTau8.fa |
| 46 |
canFam3 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/canFam3.fa |
| 47 |
galGal4 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/galGal4.fa |
| 48 |
hg38 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/hg38.fa |
| 49 |
mm10 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/mm10.fa |
| 50 |
monDom5 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/monDom5.fa |
| 51 |
rheMac3 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/rheMac3.fa |
| 52 |
rn6 /var/www/augustus/htdocs/binaries/tutorial-cgp/data/genomes/rn6.fa |
| 53 |
</pre> |
| 54 |
|
| 55 |
<span class="assignment">Load</span> the genomes into an SQLite database |
| 56 |
<pre class="code"> |
| 57 |
while read line |
| 58 |
do |
| 59 |
species=$(echo "$line" | cut -f 1) |
| 60 |
genome=$(echo "$line" | cut -f 2) |
| 61 |
load2sqlitedb --noIdx --species=$species --dbaccess=vertebrates.db $genome |
| 62 |
done <genomes.tbl |
| 63 |
</pre> |
| 64 |
|
| 65 |
<span class="assignment">Finalize</span> database by creating indices on the tables |
| 66 |
<pre class="code"> |
| 67 |
load2sqlitedb --makeIdx --dbaccess=vertebrates.db |
| 68 |
</pre> |
| 69 |
You can <span class="assignment">check</span> if loading was successful with following |
| 70 |
database query |
| 71 |
<pre class="code"> |
| 72 |
sqlite3 -header -column vertebrates.db "\ |
| 73 |
SELECT speciesname, \ |
| 74 |
sum(end-start+1) AS 'genome length',\ |
| 75 |
count(*) AS '# chunks',\ |
| 76 |
count(distinct seqnr) AS '# seqs'\ |
| 77 |
FROM genomes natural join speciesnames\ |
| 78 |
GROUP BY speciesname;" |
| 79 |
</pre> |
| 80 |
that returns a summary of the genomes in the database, e.g. their sizes (total number of bases per genome), |
| 81 |
number of sequences per genome, etc. |
| 82 |
<pre class="code"> |
| 83 |
speciesname genome length # chunks # seqs |
| 84 |
----------- ------------- ---------- ---------- |
| 85 |
bosTau8 156091 4 1 |
| 86 |
canFam3 184728 4 1 |
| 87 |
galGal4 149999 3 1 |
| 88 |
hg38 210155 5 1 |
| 89 |
mm10 178393 4 1 |
| 90 |
monDom5 540519 11 1 |
| 91 |
rheMac3 220640 5 1 |
| 92 |
rn6 99944 2 1 |
| 93 |
</pre> |
| 94 |
|
| 95 |
<h2 id="runAug">2. Run AUGUSTUS in CGP mode</h2> |
| 96 |
|
| 97 |
<span class="assignment">Create</span> a new folder for the de novo experiments and |
| 98 |
<span class="assignment">switch</span> to the new directory |
| 99 |
<pre class="code"> |
| 100 |
mkdir augCGP_denovo |
| 101 |
cd augCGP_denovo |
| 102 |
</pre> |
| 103 |
For convenience <span class="assignment">assign</span> each alignment chunk to a job ID by |
| 104 |
creating softlinks |
| 105 |
<pre class="code"> |
| 106 |
num=1 |
| 107 |
for f in ../mafs/*.maf; do ln -s $f $num.maf; ((num++)); done |
| 108 |
</pre> |
| 109 |
<span class="assignment">Run</span> Augustus in cgp mode on all alignment chunks in parallel (~3min).<br> |
| 110 |
<span class="assignment">Use</span> the option <tt>--softmasking=1</tt> in cases where the genomes are soft-masked.<br> |
| 111 |
The parameter <tt>--species</tt> is the same as in the standard version of Augustus. You can take |
| 112 |
the species identifier that best represents your clade,<br> |
| 113 |
(e.g. <tt>--species=human</tt> for mammalian clades, <tt>--species=fly</tt> for fly clades, <tt>--species=chicken</tt> for bird clades, ...)<br> |
| 114 |
A complete list of species identifiers can be found in |
| 115 |
the directory <tt>augustus/config/species</tt>. |
| 116 |
<pre class="code"> |
| 117 |
for ali in *.maf; do |
| 118 |
id=${ali%.maf} # remove .maf suffix |
| 119 |
augustus \ |
| 120 |
--species=human \ |
| 121 |
--softmasking=1 \ |
| 122 |
--treefile=../tree.nwk \ |
| 123 |
--alnfile=$ali \ |
| 124 |
--dbaccess=../vertebrates.db \ |
| 125 |
--speciesfilenames=../genomes.tbl \ |
| 126 |
--alternatives-from-evidence=0 \ |
| 127 |
--/CompPred/outdir=pred$id > aug$id.out 2> err$id.out & |
| 128 |
done |
| 129 |
</pre> |
| 130 |
This will generate the folders <span class="result"><tt>pred*/</tt></span> (one for each alignment chunk) |
| 131 |
that contain gff files with gene predictions for each input genome. |
| 132 |
<pre class="code"> |
| 133 |
bosTau8.cgp.gff |
| 134 |
canFam3.cgp.gff |
| 135 |
galGal4.cgp.gff |
| 136 |
hg38.cgp.gff |
| 137 |
mm10.cgp.gff |
| 138 |
monDom5.cgp.gff |
| 139 |
rheMac3.cgp.gff |
| 140 |
rn6.cgp.gff |
| 141 |
</pre> |
| 142 |
|
| 143 |
Note that the parallelization with the bash '&' command above is quite simple and rather for demonstration purposes.<br> |
| 144 |
For real applications with several hundreds or thousands of alignment chunks, we recommend to |
| 145 |
run job arrays on a compute cluster. |
| 146 |
|
| 147 |
<h2 id="merge">3. Merge gene predictions from parallel runs</h2> |
| 148 |
<pre class="code"> |
| 149 |
mkdir joined_pred |
| 150 |
while read line |
| 151 |
do |
| 152 |
species=$(echo "$line" | cut -f 1) |
| 153 |
find pred* -name "${species}.cgp.gff" >${species}_gtfs.lst; |
| 154 |
joingenes -f ${species}_gtfs.lst -o joined_pred/$species.gff |
| 155 |
done < ../genomes.tbl |
| 156 |
</pre> |
| 157 |
This will create the folder <span class="result"><tt>joined_pred/</tt></span> |
| 158 |
with the final gene predictions for each input genome. |
| 159 |
<pre class="code"> |
| 160 |
bosTau8.gff |
| 161 |
canFam3.gff |
| 162 |
galGal4.gff |
| 163 |
hg38.gff |
| 164 |
mm10.gff |
| 165 |
monDom5.gff |
| 166 |
rheMac3.gff |
| 167 |
rn6.gff |
| 168 |
</pre> |
| 169 |
|
| 170 |
<h2 id="makeBeds">4. Upload gene predictions into the assembly hub</h2> |
| 171 |
<span class="assignment">Convert</span> the final gene predictions from gff to BED format and place |
| 172 |
each BED file in a separate folder with the name of the corresponding genome. It is important that directory names are consistent with the names in the HAL alignment. |
| 173 |
<pre class="code"> |
| 174 |
for f in joined_pred/*.gff |
| 175 |
do |
| 176 |
mkdir "$(basename $f .gff)" |
| 177 |
gtf2bed.pl <$f >$(basename $f .gff)/augCGP_denovo.bed --itemRgb=225,0,0 |
| 178 |
done |
| 179 |
</pre> |
| 180 |
Specify any RGB color you like for the track with option <tt>--itemRgb</tt>, e.g. <span style="color:rgb(255,0,0);">225,0,0.</span><br> |
| 181 |
The name of the current directory (i.e. <tt>augCGP_denovo</tt>) will be used as track name on the browser.<br> |
| 182 |
|
| 183 |
<span class="assignment">Switch</span> back to the main working directory <tt>data/</tt> |
| 184 |
<pre class="code"> |
| 185 |
cd .. |
| 186 |
</pre> |
| 187 |
and rerun the <tt>hal2assemblyHub.py</tt> script. Include gene tracks with option <tt>--bedDirs</tt> |
| 188 |
|
| 189 |
<pre class="code"> |
| 190 |
hal2assemblyHub.py vertebrates.hal vertHub --lod \ |
| 191 |
--alignability --gcContent \ |
| 192 |
--hub vertCompHub --shortLabel VertebratesCompHub \ |
| 193 |
--bedDirs augCGP_denovo \ |
| 194 |
--tabBed \ |
| 195 |
--maxThreads=10 --longLabel "Vertebrates Comparative Assembly Hub" |
| 196 |
</pre> |
| 197 |
You can also include gene tracks from other exercises by passing a comma-separated list of directories e.g. |
| 198 |
<tt>--bedDirs refseq,augCGP_denovo,augCGP_rnaseq,augCGP_liftover,...</tt><br><br> |
| 199 |
|
| 200 |
<span class="assignment">Repeat</span> <a href="cactus.html#loadHub">4. Load the hub and browser the alignment</a>.<br> |
| 201 |
</body></html> |