Debian Codegraph @

format_docs/pdb/ztxt.txt

raw
1 The zTXT Format
2 ---------------
3
4 The zTXT format is relatively straightforward. The simplest zTXT contains a
5 Palm database header, followed by zTXT record #0, followed by the compressed
6 data. The compressed data can be in one of two formats: one long data stream,
7 or split into chunks for random access. If there are any bookmarks, they occupy
8 the record immediately after the compressed data. If there are any annotations,
9 the annotation index occupies the record immediately after the bookmarks with
10 each annotation in the index having a record immediately after the annotation
11 index. Here are diagrams of a simple zTXT and a full featured zTXT:
12
13 DB Header
14 0 Record 0
15 1
16 2
17 3
18 ... Compressed Data
19 36
20 37
21 38
22
23 DB Header
24 0 Record 0
25 1
26 2
27 3
28 ... Compressed Data
29 36
30 37
31 38
32 39 Bookmarks
33 40 Annotation Index
34 41 Annotation 1
35 42 Annotation 2
36 43 Annotation 3
37
38
39 Compression Modes
40 -----------------
41
42 zTXT version 1.40 and later supports two modes of compression. Mode 1 is a
43 random access mode, and mode 2 consists of one long data stream. Both modes
44 work on 8K (the default record size) blocks of text.
45
46 Please note, however, that as of Weasel Reader version 1.60 the old style
47 (mode 2) zTXT format is no longer supported. makeztxt and libztxt still support
48 creating these documents for backwards compatibility, but you should not use
49 mode 2 if possible.
50
51
52 Mode 1
53 ------
54
55 In mode one, 8K blocks of text are compressed into an equal number of blocks of
56 compressed data. Using the Z_FULL_FLUSH flush mode with zLib allows for random
57 access among the blocks of data. In order for this to function, the first block
58 must be decompressed first, and after that any block in the file may be
59 decompressed in any order. In mode 1, the blocks of compressed data will likely
60 not all have the same size.
61
62
63 Mode 2
64 ------
65
66 In zTXT versions before 1.40, this was the only method of compression. This
67 mode involves compressing the entire input buffer into a single output buffer
68 and then splitting the resulting buffer into 8K segments. This mode requires
69 that all of the compressed data be decompressed in one pass. Since there are no
70 real 'blocks' of data, the resulting output can be of any blocksize, though
71 typically the default of 8K should be fine. The advantage to mode 2 is that it
72 will give about 10% - 15% more compression.
73
74
75 zTXT Record #0 Definition (version 1.44)
76 ----------------------------------------
77
78 Record 0 provides all of the information about the zTXT contents. Be sure it is
79 correct, lest firey death rain down upon your program.
80
81 typedef struct zTXT_record0Type {
82 UInt16 version;
83 UInt16 numRecords;
84 UInt32 size;
85 UInt16 recordSize;
86 UInt16 numBookmarks;
87 UInt16 bookmarkRecord;
88 UInt16 numAnnotations;
89 UInt16 annotationRecord;
90 UInt8 flags;
91 UInt8 reserved;
92 UInt32 crc32;
93 UInt8 padding[0x20 - 24];
94 } zTXT_record0;
95
96
97 Structure Elements
98 ------------------
99
100 UInt16 version;
101
102 This is mostly just informational. Your program can figure out what features
103 might be available from the version. However, the remaining parts of the
104 structure are designed such that their value will be 0 if that particular
105 feature is not present, so that is the correct way to test. The version is
106 stored as two 8 bit integers. For example, version 1.42 is 0x012A.
107
108 UInt16 numRecords;
109
110 This is the number of DATA records only and does not include record 0,
111 bookmarks, or annotations. With compression mode 1, this is also the number of
112 uncompressed text records. With mode 2, you must decompress the file to figure
113 out how many text records there will be.
114
115 UInt32 size;
116
117 The size in bytes of the uncompressed data in the zTXT. Check this value with
118 the amount of free storage memory on the Palm to make sure there's enough room
119 to decompress the data in full or in part.
120
121 UInt16 recordSize;
122
123 recordSize is the size in bytes of a text record. This field is important, as
124 the size of text and decompression buffers is based on this value. It is used
125 by Weasel to navigate though the text so it can map absolute offsets to record
126 numbers. 8192 is the default. With compression mode 1, this is the amount of
127 data inside each compressed record (except maybe the last one), but the actual
128 compressed records will likely have varying sizes. In mode 2, both compressed
129 records and the resulting text records are all of this size (except, again, the
130 last record).
131
132 UInt16 numBookmarks;
133
134 The definitive count of how many bookmarks are stored in the bookmark index
135 record. See the section on bookmarks below.
136
137 UInt16 bookmarkRecord;
138
139 If there are any bookmarks, this is set to the record index number that
140 contains the bookmark listing, otherwise it is 0.
141
142 UInt16 numAnnotations;
143
144 Like the bookmark count, this is the definitive count of how many annotations
145 are in the annotation index and how many annotation records follow it. See the
146 section on annotation below.
147
148 UInt16 annotationRecord;
149
150 If there are any annotations, this is set to the record index number that
151 contains the annotation index, otherwise it is 0.
152
153 UInt8 flags;
154
155 These flags indicate various features of the zTXT database. flags is a bitmask
156 and at present the only two defined bits are:
157
158 ZTXT_RANDOMACCESS (0x01)
159 If the zTXT was compressed according to the method in mode 1, then it
160 supports random access and this should be set.
161 ZTXT_NONUNIFORM (0x02)
162 Setting this bit indicates that the text records within the zTXT database
163 are not of uniform length. That is, when the blocks of text are
164 decompressed they will not have identical block sizes. If this is not set,
165 the compressed blocks are assumed to all have the same size when
166 decompressed (typically 8K) except for the last block which can be smaller.
167
168 UInt32 crc32;
169
170 A CRC32 value for checking data integrity. This value is computer over all text
171 data record only and does not include record 0 nor any bookmark/annotation
172 records. The current implementation in makeztxt/Weasel computes this value
173 using the crc32 function in zLib which should be the standard CRC32 definition.
174
175 UInt8 padding[0x20 - 24];
176
177 zTXT record zero is 32 bytes in length, so the unused portion is padded.
178
179
180 zTXT Bookmarks
181 --------------
182
183 zTXT bookmarks are stored in a simple array in a record at the end of a zTXT.
184 The format is as follows:
185
186 #define MAX_BMRK_LENGTH 20
187
188 typedef struct GPlmMarkType {
189 UInt32 offset;
190 Char title[MAX_BMRK_LENGTH];
191 } GPlmMark;
192
193 In the structure, offset is counted as an absolute offset into the text. The
194 bookmarks must be sorted in ascending order.
195
196 If there are no bookmarks, then the bookmark index does not exist. When the
197 user creates the first bookmark, the record containing the index will then be
198 created. If there are annotations, when the bookmark record is created it must
199 go before the annotation index. This will require incrementing annotationRecord
200 in record 0 to point to the new record index.
201
202 Similarly, when all bookmarks are deleted the bookmark index record is also
203 deleted. If there are annotations, annotationRecord in record 0 must be
204 decremented to point to the new index.
205
206
207 zTXT Annotations
208 ----------------
209
210 zTXT annotations have a format almost identical to that of the bookmark index:
211
212 typedef struct GPlmAnnotationType {
213 UInt32 offset;
214 Char title[MAX_BMRK_LENGTH];
215 } GPlmAnnotation;
216
217 Like the bookmarks, offset is an absolute offset into the text. The annotation
218 index is organized just as the bookmarks are, as a single array in a record.
219 Note that this structure does NOT store the actual annotation text.
220
221 The text of each annotation is stored in its own record immediately following
222 the index. So, the first annotation in the index will occupy the first record
223 following the index, and the second annotation will be in the second record
224 following the index, and so on. The text of each annotation is limited to
225 4096 bytes.
226