Debian Codegraph @

doc/hacking-page.html

raw ◀ ▶
Source reconstructed at patch state 0 of 1 — not the installed (fully patched) source.
1 Debian patch touch this file
1 <html>
2 <head>
3 <title>netrik hacker's manual: page handling</title>
4 </head>
5 <body>
6
7 <h1 align="center">netrik hacker's manual<br />>========================&lt;</h1>
8
9 <p>
10 [This file contains a description of the page handling system. See hacking.txt
11 or <a href="hacking.html">hacking.html</a> for an overview of the
12 manual.]
13 </p>
14
15 <p>
16 The page handling system is responsible for the central browser functionality:
17 Loading pages to be displayed, loading new pages when links are followed etc.
18 This also includes all page history handling, as all page loads either affect
19 the page list (history), or depend on it, or both; and all history commands
20 involve a page load.
21 </p>
22
23 <p>
24 The page handling is also a central component, because it invokes all of the
25 other modules: The file loader
26 (<a href="hacking-load.html">hacking-load.*</a>) is used to fetch a
27 new document (file) if necessary, and the layout engine
28 (<a href="hacking-layout.html">hacking-layout.*</a>) to prepare it for
29 rendering; the pager (<a href="hacking-pager.html">hacking-pager.*</a>
30 is (indirectly) invoked to display the new page; and the link handling
31 mechanism tells when and what page to load. Moreover, most of the modules are
32 also coupled to the page handling, because they use the page structure handled
33 by the page loading mechanism.
34 </p>
35
36 <p>
37 So the page handling is really the central component, controlling the program
38 flow. Thus it can only partially be located in an own file (page.c); part of
39 the page handling has to be done in main() directly.
40 </p>
41
42 <a name="loadPage" id="loadPage">
43
44 <h2>load_page()</h2>
45
46 <p>
47 load_page() ist the main function of load.c, and does most of the work. It is
48 responsible for loading a page so that it will be shown in the pager next time
49 <a href="hacking-pager.html#display">display()</a> (see hacking-pager.*) is
50 called.
51 </p>
52
53 <p>
54 The exact actions necessary to achieve that vary on the nature of the load
55 operation. In the most basic operation mode it requires adding a new entry to
56 the page list and loading a new document to use. Sometimes however there are
57 only changes to the page list while the document is reused, or the page
58 history stays unchanged while some existing entry is reactivated and the
59 settings reloaded. The case of reloading some document while the page list
60 isn't altered, is possible as well.
61 </p>
62
63 <p>
64 load_page() takes almost the same arguments as layout(): A base URL, which is
65 usually the URL of the current page, which is used as the base when following a
66 realtive link or so; a main URL (as string), which can be absolute or relative,
67 in the latter case to be combined with the base URL to form an absolute target
68 URL; an optional form item, which tells where to find the form data of a form
69 to submit while loading a new document; the page width telling how to layout
70 loaded pages; and an error handle informing whether some problem occured while
71 loading the document.
72 </p>
73
74 <p>
75 In case a new document needs to be loaded, all of these parameters are simply
76 passed on to layout(), which is responsible for actually loading a document
77 from a file or an HTTP server, as well as invoking all necessary layouting
78 passes so that the document can be actually rendered and displayed in the pager.
79 This process is described in
80 <a href="hacking-layout.html">hacking-layout.*</a>.
81 </p>
82
83 <p>
84 load_page() takes an additional "reference" parameter. This one may refer to
85 some entry in the page list (normally the currently used entry), which contains an
86 already loaded document to reuse. In this case no document load needs to be
87 performed (i.e. layout() isn't called), as the layouting data of the reference
88 page is reused. (Most of the other parameters aren't used in this case.) This
89 feature is used when jumping to an anchor inside a document -- it doesn't
90 require loading a new document.
91 </p>
92
93 <p>
94 First action (even before loading the document) is creating a page descriptor
95 -- regardless whether a new document is loaded or existing layouting data is
96 reused.
97 </p>
98
99 <p>
100 This however is left out if the page is reloaded from history (indicated by
101 "url" being NULL), and thus already has a descriptor somewhere in the page
102 list.
103 </p>
104
105 <h3>Page List Handling</h3>
106
107 <a name="pageList" id="pageList">
108
109 <p>
110 The page list (history) is a global variable (of type "struct Page_list"),
111 which basically consists of an array of pointers to individual page
112 descriptors. It also stores the current number of entries in the list, as well
113 es the current active entry. (The one that describes the page visible in the
114 pager.)
115 </p>
116
117 <p>
118 The list has one entry for each page in the page history. Normally the last
119 entry is the visible page, but other pages are also possible after going back
120 in history.
121 </p>
122
123 <a name="pageStruct" id="pageStruct">
124
125 <p>
126 Each page descriptor is of type "struct Page". This struct contains:
127 </p>
128
129 <ul> <li>
130 The "layout" pointer, which points to a descriptor with the document's
131 layouting data created by <a href="hacking-layout.html#layout">layout()</a>.
132 </li> <li>
133 The page URL in the split URL structure "url"
134 </li> <li>
135 The current position of the pager, stored as the index of the first line
136 visible on the screen ("pager_pos")
137 </li> <li>
138 The cursor position in "cursor_x" and "cursor_y"
139 </li> <li>
140 The hashes "url_fingerprint" and "text_fingerprint", used to recognize
141 whether the link is identical to a link in the page (necessary when reloading
142 a page from history)
143 </li> <li>
144 The link number of the link currently active in the pager ("active_link")
145 </li> <li>
146 The optionl anchor number of an anchor to jump to when starting the pager
147 ("active_link")
148 </li> <li>
149 The "mark" flag indicating that a return mark has been set on this page
150 </li> </ul>
151
152 </a> <!-- pageStruct -->
153
154 <p>
155 The page list is manipulated exclusively by load_page() and its helper
156 functions. load_page() is also responsible for keeping the list up to date, so
157 it always contains exactly those entries that shall be used by the history
158 commands.
159 </p>
160
161 <p>
162 Thus, before a new page descriptor is created, the page list needs to be
163 adjusted.
164 </p>
165
166 <p>
167 Normally a new entry is simply appended at the end of the list; however, there
168 are several other cases.
169 </p>
170
171 <p>
172 When loading some new page while not being at the last entry in
173 the page list (after going back to some older page from the page history), all
174 entries after the current one have to be discarded.
175 </p>
176
177 <p>
178 Moreover, if the current page is an internal page (either a page loaded from
179 stdin, or an error page), it isn't to be kept in history; in that case, we go
180 further back until we find the last normal page entry.
181 </p>
182
183 <p>
184 All entries after the current or the last normal one are then cleared by
185 calling free_page() in a loop.
186 </p>
187
188 <p>
189 Afterwards, the new page descriptor is created.
190 </p>
191
192 <p>
193 When revisiting some page from history, it also may be necessary to delete internal
194 pages, if leaving such. The last non-internal page is determined (starting with
195 the end of the list), and all following are deleted.
196 </p>
197
198 </a> <!-- pageList -->
199
200 <h4>add_page()</h4>
201
202 <p>
203 The add_page() function (in url-history.c) is responsible for actually creating
204 the new page list entry. The list is a "struct Page_list", and contains the
205 following information:
206 </p>
207
208 <ul> <li>
209 "num" is the number of entries currently stored in the list
210 </li> <li>
211 "pos" is the entry number of the entry corresponding to the currently visible
212 page
213 </li> <li>
214 The history entries themselfs are stored in "page", which is an array of
215 pointers to the page descriptors of all pages.
216 </li> </ul>
217
218 <p>
219 The new entry is added at the position indicated by "page_list.pos".
220 </p>
221
222 <p>
223 First the array is resized to the new history size -- the history now will end
224 with the new entry generated. Then a new page descriptor is created and the
225 pointer to it is stored at the proper position (the last list entry).
226 </p>
227
228 <p>
229 After creating the descriptor, some default values are set. (Pager position etc.)
230 </p>
231
232 <h3>Loading</h3>
233
234 <p>
235 Now having the page descriptor, we need to get the layouting data someway so that
236 the page can be displayed in the pager.
237 </p>
238
239 <p>
240 Again, the standard case is loading a new document using
241 <a href="hacking-layout.html#layout">layout()</a>. The pointer to the
242 layouting data returned by that function is simply stored in the page
243 descriptor. The URL is extracted from the layouting data and stored in the
244 explicit "url" pointer of the page descriptor, which is necessary in case the
245 layout data will be discarded (when loading another document), but the page is kept
246 in history.
247 </p>
248
249 <h4>Local Links</h4>
250
251 <p>
252 As mentioned before, it's also possible instead of loading a new document, to
253 reuse existing layouting data of another one by passing a "reference" page. The
254 primary application of that is when following a link that points to some anchor
255 in the same document -- that doesn't require reloading the whole document, but
256 just jumping to the anchor. Of course, it is also used when returning to the
257 previous page after following such a local link, or going forward again.
258 </p>
259
260 <p>
261 The (pointer to) the layout data descriptor is simply copied from the page list
262 entry indicated by the given "reference" parameter.
263 </p>
264
265 <p>
266 As layout() and thus also
267 <a href="hacking-load.html#initLoad">init_load()</a> (see
268 hacking-load.*) isn't used in this case,
269 <a href="hacking-load.html#mergeUrls">merge_urls()</a> has to be
270 called directly. If URL merging fails here, load_page() returns immediately; no
271 page descriptor is created and nothing else is changed.
272 </p>
273
274 <p>
275 Also, if some anchor is active in the reference page,
276 <a href="hacking-links.html#highlight">highlight_link()</a> needs to
277 be used to remove the highlighting, to get a "clean" item tree.
278 </p>
279
280 <a name="reactivating" id="reactivating">
281
282 <h3>Reactivating Link</h3>
283
284 <p>
285 When revisiting a page from history, and some link was active when we last left
286 that page, that link is reactivated for convenience. This is non-trivial in
287 case the page contents changed in the meantime -- we need a heuristic to
288 find out which is the right link.
289 </p>
290
291 <p>
292 The best matching link is determined by examining several properties of the
293 links: The link number, the position on the page (x and y), and two hashes
294 identifying the link url and the text describing the link.
295 </p>
296
297 <p>
298 The properties of each link are stored in its link structure while they are
299 extracted in
300 <a href="hacking-layout.html#parseStruct">parse_struct()</a> (see
301 hacking-layout.*): The position is stored in each link in "x_start" and
302 "y_start" anyways. The hashes are extracted with fingerprint() (which just
303 generates a very simple (fast!) checksum of a string part (memory range)
304 specified by its starting and ending address), and stored in "url_fingerprint"
305 and "text_fingerprint". The link number is implicit.
306 </p>
307
308 <p>
309 Each time a link is activated with
310 <a href="hacking-links.html">activate_link</a> (described in
311 hacking-links.*), the properties of the link are copied to the page descriptor,
312 so that they can be compared against the actual links when revisiting the page
313 later. The hashes are stored in dedicated "url_fingerprint" and
314 "text_fingerprint" again. The position is stored in "cursor_x" and "cursor_y".
315 (Presently, those values are set *only* when a link is activated, and solely
316 for the sake of the link retrieval -- this will change as soon as real cursor
317 handling is implemented...)
318 </p>
319
320 <p>
321 When revisiting a page, as a first step we do a quick test whether the link
322 with the old number is still the same. If all five conditions are fulfilled
323 (i.e. the link number is still valid, and the other four values are identical),
324 we know nothing has changed, and need no further processing.
325 </p>
326
327 <p>
328 If some of the conditions fails, on the other hand, we need to test all links
329 on the page using the heuristics. First deviations are calculated seperately
330 for each property: "dev_url" and "dev_text" are flags, which eqal 0 if the hash
331 in the tested link is identical to the one of the old link, and 1 otherwise.
332 "dev_x" is also a flag, being 0 if the horizontal position is identical.
333 "dev_y" and "dev_num" are different: These are float numbers (also in the range
334 0 to 1) describing a relative deviation, i.e. the ratio between the number of
335 lines/links off and the total number of lines/links in the (new) page.
336 </p>
337
338 <p>
339 All of these are finally combined to form the total deviation, which is just a
340 weighted sum of all; the lower the value, the better the link matches.
341 </p>
342
343 <p>
344 All links are tested in a loop, and the best match is then stored as the new
345 "active_link". (Instead of testing all, we could limit the search to a smaller
346 range by pre-testing some of the criteria. However, a situation were this would
347 make a difference is unrealistic; it would be just a waste of code, meaning
348 wasted developement time, increased binary size, and most important additional
349 error sources and worse maintainability.)
350 </p>
351
352 <p>
353 At start "best" is set to -1 (i.e. no link), and "best_deviation" to 1.0. Thus,
354 only links with a deviation below or eqal 1 are considered at all; if none was
355 found, "active_link" is assigned -1, and no link will be activated. This is
356 important to ensure that only links will be activated which are actually
357 *similar* to the original one, not activating some unrelated random link which
358 happens to be nearest the old one.
359 </p>
360
361 <p>
362 Of course, this heuristic isn't perfect by any means; however, it is certainly
363 more than sufficient. Anything better would be waste of performance and
364 developement efforts. Recognition whether a candidate is worth considering
365 should be robust. Choosing the best if there are multiple similar candidates is
366 suboptimal, but such a situation is really unlikely; and if it occurs, the
367 question which is really the best link is disputable alyways...
368 </p>
369
370 <p>
371 The files test/mod[12].html are provided for testing the heuristics. These need
372 to be loaded alternatively; this is easy with Hurd, but not possible with
373 classical Unix systems; thus, it's necessary to use a web server with a CGI
374 script, or tcplisten:
375 </p>
376
377 <p><pre>
378 while true; do for i in test/mod*.html; do cat test/stat_only.http |tcplisten 8000; done; done
379 </pre></p>
380
381 <p>
382 (Point netrik to http://localhost:8000 and use ^R command.)
383 </p>
384
385 </a> <!-- reactivating -->
386
387 <h3>Anchors</h3>
388
389 <p>
390 If the URL contains a fragment identifier, the corresponding anchor is
391 retrieved from the anchor list, and stored in "page->active_anchor"; this is
392 described under <a href="hacking-links.html#anchors">Anchors</a>
393 in hacking-links.*. The pager then jumps to the anchor position and highligts
394 it upon startup.
395 </p>
396
397 </a> <!--loadPage-->
398
399 <h2>Handling in main()</h2>
400
401 <p>
402 Although load_page() does a great part of the work, some things have to be
403 taken care of in main(); particularily determining in which manner the new page
404 is to be loaded (reuse current document or load new one), and clearing the
405 layout data of an old document before loading a new one. Maybe that could be
406 done in load_page() too; however, it's not worth considering that now, as it
407 will probably need to be handled completely different with the planned new
408 basic program structure. (Using an event queue and a main dispatcher.)
409 </p>
410
411 <p>
412 Another thing that needs to be handled in main() is initiating a page load in
413 reaction to commands given by the user inside the pager (or on the command
414 promt), which often can't be clearly seperated from the load operation itself.
415 </p>
416
417 <p>
418 If the user activates the command prompt (by typing ':' inside the pager) and
419 issues the ":e" or ":E" command, the URL is extracted from the command, and
420 load_page() is called to load the desired new page. The URL of the current page
421 is used as base for a relative URL with ":e". With ":E" (and also for ":e", if
422 the current page is internal), no base is used; the URL is always interpreted
423 absolutely.
424 </p>
425
426 <p>
427 These commands never pass a reference page, i.e. always involve loading a new
428 document. (Even though it is possible to jump to a local anchor using ":e
429 #anchor".)
430 </p>
431
432 <p>
433 If a link/form control was activated by pressing &lt;return> on a selected link
434 inside the pager, the action depends on the link or from element type. For
435 normal links load_page() is used with the current URL as base, just as with
436 ":e". The link URL is extracted from the text item containing the link with
437 <a href="hacking-links.html#getLink">get_link()</a>, by help of the
438 "<a href="hacking-links.html#linkList">link_list</a>" structure. This
439 process is described under
440 <a href="hacking-links.html#followLink">Following Links</a> in
441 hacking-links.*.
442 </p>
443
444 <p>
445 The only difference to the ":e" command (except for the way of getting the
446 target URL) is that the current page is passed as "reference" to load_page() if
447 the URL starts with '#' (i.e. points to an anchor in the same document), so
448 that the document isn't reloaded in that case, but only the anchor is
449 activated.
450 </p>
451
452 <p>
453 Form submit buttons are quite similar to normal links. First,
454 <a href="hacking-links.html#getForm">get_form_item()</a> (also in
455 hacking-links.*) is used to retrieve the item (from the structure tree) that
456 describes to the form in which the button resides. This is used to get the
457 form's submit address ("action") first. Having this, load_page() is used to do
458 the submit; the form item is passed as the "form" argument. (And passed on to
459 <a href="hacking-load.html#initLoad">init_load()</a> there; see
460 hacking-load.*.) This is both to tell init_load() that a form is to be
461 submitted, and where to find the form data. init_load() (and its sub-functions)
462 then take care of extracting the form data (using
463 <a href="hacking-links.html#urlEncode">url_encode()</a> or
464 <a href="hacking-links.html#mimeEncode">mime_encode()</a> from
465 forms.c, also described in hacking-links.*) and submitting it to the server.
466 The resulting response page is loaded just like any other document.
467 </p>
468
469 <p>
470 Other form controls do not issue a load operation, but only adjust the form
471 value appropriately. (This is described under
472 <a href="hacking-links.html#manipulating">Manipulating</a> in
473 hacking-links.*.)
474 </p>
475
476 <p>
477 The 'u', 'U' and 'c' pager commands also aren't really load operations, but
478 they are described here because they involve similar actions as the
479 preparations for a page load.
480 </p>
481
482 <p>
483 If 'u' was typed, the link URL is retrieved the same way like when following a
484 link, and then just printed to the screen.
485 </p>
486
487 <p>
488 'U' is similar. Instead of printing the (relative) link URL directly, it merges
489 it with the current page URL, thus getting the same absolute target URL that
490 would be used if the link was actually followed.
491 </p>
492
493 <p>
494 'c' simply prints the "full_url" component of the current page URL.
495 </p>
496
497 <p>
498 If a history command was given, load_page() is called with the (split) URL
499 taken from the requested "page_list" entry. We know which entry to take by
500 "page_list.pos", which is set the desired new value before returning from the
501 pager.
502 </p>
503
504 <p>
505 If the history entry refers to the same HTML document as the one displayed up
506 to now, the current page descriptor is passed as "reference". To determine
507 whether it is the same document, we need to check if all entries between the
508 old and the new one (regardles whether the new one is before or after the old
509 one in history) have "local" URLs, i.e. if the newer of the two entries was
510 created only by following links to local anchors from the older one.
511 </p>
512
513 </body>
514 </html>