Debian Codegraph @

manual/externals.xml

raw
1 <?xml version="1.0" encoding="utf-8"?>
2 <!DOCTYPE chapter SYSTEM "docbook-dtd-45/docbookx.dtd">
3
4 <chapter id="externals">
5 <title>External Callbacks and Processors</title>
6
7 <para>
8 The <link linkend="external">EXTERNAL rule types</link> spawn and pass a window to an external process,
9 and expect a modified window in return. The external process is spawned only once the first time the rule hits,
10 is then sent an initial message of the current protocol version, and subseqently passed only windows.
11 It is expected that the reply from a window is another window, or a null reply if there are no changes.
12 </para>
13
14 <para>
15 What follows is a description of the protocol using C++ structs. For those who want a more hands-on example,
16 the source tree scripts/CG3_External.pm and scripts/external.pl and test/T_External/* is a working example of
17 calling a Perl program that simply adds a tag to every reading.
18 </para>
19
20 <para>
21 All datatypes correspond to the C and C++ &lt;stdint.h&gt; types, and no endianness conversion is performed;
22 it is assumed that the external program is native to the same arch as CG-3 is running on.
23 Fields marked 'const' denote they may not be changed by the external.
24 Notice that you may not change the number of cohorts or readings, but you may change the number of tags per reading.
25 </para>
26
27 <section id="external-datatypes">
28 <title>Protocol Datatypes</title>
29
30 <screen>
31 uint32_t protocol_version = 7226;
32 uint32_t null_response = 0;
33
34 struct Text {
35 uint16_t length;
36 char *utf8_bytes;
37 };
38
39 enum READING_FLAGS {
40 R_WAS_MODIFIED = (1 &lt;&lt; 0),
41 R_INVISIBLE = (1 &lt;&lt; 1),
42 R_DELETED = (1 &lt;&lt; 2),
43 R_HAS_BASEFORM = (1 &lt;&lt; 3),
44 };
45
46 struct Reading {
47 uint32_t reading_length; // sum of all data here plus data from all tags
48 uint32_t flags;
49 Text *baseform; // optional, depends on (flags &amp; R_HAS_BASEFORM)
50
51 uint32_t num_tags;
52 Text *tags;
53 };
54
55 enum COHORT_FLAGS {
56 C_HAS_TEXT = (1 &lt;&lt; 0),
57 C_HAS_PARENT = (1 &lt;&lt; 1),
58 };
59
60 struct Cohort {
61 uint32_t cohort_length; // sum of all data here plus data from all readings
62 const uint32_t number; // which cohort is this
63 uint32_t flags;
64 uint32_t parent; // optional, depends on (flags &amp; C_HAS_PARENT)
65 Text wordform;
66
67 const uint32_t num_readings;
68 Reading *readings;
69
70 Text *text; // optional, depends on (flags &amp; C_HAS_TEXT)
71 };
72
73 struct Window {
74 uint32_t window_length; // sum of all data here plus data from all cohorts
75 const uint32_t number; // which window is this
76
77 const uint32_t num_cohorts;
78 Cohort *cohorts;
79 };
80 </screen>
81 </section>
82
83 <section id="external-dataflow">
84 <title>Protocol Flow</title>
85
86 <orderedlist numeration="arabic">
87 <listitem>
88 <para>
89 Initial packet is simply the protocol_version. This is used to detect when an external may be out of date.
90 If an external cannot reliably handle a protocol, I recommend that it terminates to avoid subtle bugs.
91 Protocol version is only sent once and no response is allowed.
92 </para>
93 </listitem>
94 <listitem>
95 <para>
96 Every time an EXTERNAL rule hits, a Window is sent. If you make no changes to the window, send a null_response.
97 If you do change the window, you must compose a whole Window as response.
98 If you change anything in a Reading, you must set the R_WAS_MODIFIED flag on the Reading.
99 If you change a Cohort's wordform, that automatically sets the R_WAS_MODIFIED flags on all Readings.
100 You must send some response for every window.
101 </para>
102 </listitem>
103 <listitem>
104 <para>
105 When CG-3 is shutting down, the final thing it sends to the external is a null_reponse.
106 Use this to clean up if necessary. Any data output after the null_response is ignored.
107 </para>
108 </listitem>
109 </orderedlist>
110 </section>
111
112 </chapter>