Debian Codegraph @

doc/dapl_registry_design.txt

raw
1 DAT Registry Subsystem Design v. 0.90
2 -------------------------------------
3
4 =================
5 Table of Contents
6 =================
7
8 * Table of Contents
9 * Referenced Documents
10 * Introduction
11 * Goals
12 * Provider API
13 * Consumer API
14 * Registry Design
15 + Registry Database
16 + Provider API pseudo-code
17 + Consumer API pseudo-code
18 + Platform Specific API pseudo-code
19
20 ====================
21 Referenced Documents
22 ====================
23
24 uDAPL: User Direct Access Programming Library, Version 1.0. Published
25 6/21/2002. http://www.datcollaborative.org/uDAPL_062102.pdf. Referred
26 to in this document as the "DAT Specification".
27
28 ============
29 Introduction
30 ============
31
32 The DAT architecture supports the use of multiple DAT providers within
33 a single consumer application. Consumers implicitly select a provider
34 using the Interface Adapter name parameter passed to dat_ia_open().
35
36 The subsystem that maps Interface Adapter names to provider
37 implementations is known as the DAT registry. When a consumer calls
38 dat_ia_open(), the appropriate provider is found and notified of the
39 consumer's request to access the IA. After this point, all DAT API
40 calls acting on DAT objects are automatically directed to the
41 appropriate provider entry points.
42
43 A persistent, administratively configurable database is used to store
44 mappings from IA names to provider information. This provider
45 information includes: the file system path to the provider library
46 object, version information, and thread safety information. The
47 location and format of the registry is platform dependent. This
48 database is know as the Static Registry (SR). The process of adding a
49 provider entry is termed Static Registration.
50
51 Within each DAT consumer, there is a per-process database that
52 maps from ia_name -> provider information. When dat_ia_open() is
53 called, the provider library is loaded, the ia_open_func is found, and
54 the ia_open_func is called.
55
56 =====
57 Goals
58 =====
59
60 -- Implement the registration mechanism described in the uDAPL
61 Specification.
62
63 -- The DAT registry should be thread safe.
64
65 -- On a consumer's performance critical data transfer path, the DAT
66 registry should not require any significant overhead.
67
68 -- The DAT registry should not limit the number of IAs or providers
69 supported.
70
71 -- The user level registry should be tolerant of arbitrary library
72 initialization orders and support calls from library initialization
73 functions.
74
75 ============
76 Provider API
77 ============
78
79 Provider libraries must register themselves with the DAT registry.
80 Along with the Interface Adapter name they wish to map, they must
81 provide a routines vector containing provider-specific implementations
82 of all DAT APIs. If a provider wishes to service multiple Interface
83 Adapter names with the same DAT APIs, it must register each name
84 separately with the DAT registry. The Provider API is not exposed to
85 consumers.
86
87 The user level registry must ensure that the Provider API may be
88 called from a library's initialization function. Therefore the
89 registry must not rely on a specific library initialization order.
90
91 DAT_RETURN
92 dat_registry_add_provider(
93 IN DAT_PROVIDER *provider )
94
95 Description: Allows the provider to add a mapping. It will return an
96 error if the Interface Adapter name already exists.
97
98 DAT_RETURN
99 dat_registry_remove_provider(
100 IN DAT_PROVIDER *provider )
101
102 Description: Allows the Provider to remove a mapping. It will return
103 an error if the mapping does not already exist.
104
105 ============
106 Consumer API
107 ============
108
109 Consumers that wish to use a provider library call the DAT registry to
110 map Interface Adapter names to provider libraries. The consumer API is
111 exposed to both consumers and providers.
112
113 DAT_RETURN
114 dat_ia_open (
115 IN const DAT_NAME device_name,
116 IN DAT_COUNT async_event_qlen,
117 INOUT DAT_EVD_HANDLE *async_event_handle,
118 OUT DAT_IA_HANDLE *ia_handle )
119
120 Description: Upon success, this function returns an DAT_IA_HANDLE to
121 the consumer. This handle, while opaque to the consumer, provides
122 direct access to the provider supplied library. To support this
123 feature, all DAT_HANDLEs must be pointers to a pointer to a
124 DAT_PROVIDER structure.
125
126 DAT_RETURN
127 dat_ia_close (
128 IN DAT_IA_HANDLE ia_handle )
129
130 Description: Closes the Interface Adapter.
131
132 DAT_RETURN
133 dat_registry_list_providers(
134 IN DAT_COUNT max_to_return,
135 OUT DAT_COUNT *entries_returned,
136 OUT DAT_PROVIDER_INFO *(dat_provider_list[]) )
137
138 Description: Lists the current mappings.
139
140 ===============
141 Registry Design
142 ===============
143
144 There are three separate portions of the DAT registry system:
145
146 * Registry Database
147
148 * Provider API
149
150 * Consumer API
151
152 We address each of these areas in order. The final section will
153 describe any necessary platform specific functions.
154
155 Registry Database
156 -----------------
157
158 Static Registry
159 ................
160
161 The Static Registry is a persistent database containing provider
162 information keyed by Interface Adapter name. The Static Registry will
163 be examined once when the DAT library is loaded.
164
165 There is no synchronization mechanism protecting access to the Static
166 Registry. Multiple readers and writers may concurrently access the
167 Static Registry and as a result there is no guarantee that the
168 database will be in a consistent format at any given time. DAT
169 consumers should be aware of this and not run DAT programs when the
170 registry is being modified (for example, when a new provider is being
171 installed). However, the DAT library must be robust enough to recognize
172 an inconsistent Static Registry and ignore invalid entries.
173
174 Information in the Static Registry will be used to initialize the
175 registry database. The registry will refuse to load libraries for DAT
176 API versions different than its DAT API version. Switching API
177 versions will require switching versions of the registry library (the
178 library explicitly placed on the link line of DAPL programs) as well
179 as the header files included by the program.
180
181 Set DAT_NO_STATIC_REGISTRY at compile time if you wish to compile
182 DAT without a static registry.
183
184 UNIX Registry Format
185 .....................
186
187 The UNIX registry will be a plain text file with the following
188 properties:
189 * All characters after # on a line are ignored (comments).
190 * Lines on which there are no characters other than whitespace
191 and comments are considered blank lines and are ignored.
192 * Non-blank lines must have seven whitespace separated fields.
193 These fields may contain whitespace if the field is quoted
194 with double quotes. Within fields quoated with double quotes,
195 the following are valid escape sequences:
196
197 \\ backslash
198 \" quote
199
200 * Each non-blank line will contain the following fields:
201
202 - The IA Name.
203 - The API version of the library:
204 [k|u]major.minor where "major" and "minor" are both integers
205 in decimal format. Examples: "k1.0", "u1.0", and "u1.1".
206 - Whether the library is thread-safe:
207 [threadsafe|nonthreadsafe]
208 - Whether this is the default section: [default|nondefault]
209 - The path name for the library image to be loaded.
210 - The version of the driver: major.minor, for example, "5.13".
211
212 The format of any remaining fields on the line is dependent on the API
213 version of the library specified on that line. For API versions 1.0
214 and 1.1 (both kDAPL and uDAPL), there is only a single additional
215 field, which is:
216
217 - An optional string with instance data, which will be passed to
218 the loaded library as its run-time arguments.
219
220 This file format is described by the following grammar:
221
222 <entry-list> -> <entry> <entry-list> | <eof>
223 <entry> -> <ia-name> <api-ver> <thread-safety> <default-section>
224 <lib-path> <driver-ver> <ia-params> [<eor>|<eof>] |
225 [<eor>|<eof]
226 <ia-name> -> string
227 <api-ver> -> [k|u]decimal.decimal
228 <thread-safety> -> [threadsafe|nonthreadsafe]
229 <default-section> -> [default|nondefault]
230 <lib-path> -> string
231 <driver-ver> -> decimal.decimal
232 <ia-params> -> string
233 <eof> -> end of file
234 <eor> -> newline
235
236 The location of this file may be specified by setting the environment
237 variable DAT_CONF. If DAT_CONF is not set, the default location will
238 be /etc/dat.conf.
239
240 Windows Registry Format
241 .......................
242
243 Standardization of the Windows registry format is not complete at this
244 time.
245
246 Registry Database Data Structures
247 .................................
248
249 The Registry Database is implemented as a dictionary data structure that
250 stores (key, value) pairs.
251
252 Initially the dictionary will be implemented as a linked list. This
253 will allow for an arbitrary number of mappings within the resource
254 limits of a given system. Although the search algorithm will have O(n)
255 worst case time when n elements are stored in the data structure, we
256 do not anticipate this to be an issue. We believe that the number of
257 IA names and providers will remain relatively small (on the order of
258 10). If performance is found to be an issue, the dictionary can be
259 re-implemented using another data structure without changing the
260 Registry Database API.
261
262 The dictionary uses IA name strings as keys and stores pointers to a
263 DAT_REGISTRY_ENTRY structure, which contains the following
264 information:
265
266 - provider library path string, library_path
267 - DAT_OS_LIBRARY_HANDLE, library_handle
268 - IA parameter string, ia_params
269 - DAT_IA_OPEN_FUNC function pointer, ia_open_func
270 - thread safety indicator, is_thread_safe
271 - reference counter, ref_count
272
273 The entire registry database data structure is protected by a single
274 lock. All threads that wish to query/modify the database must posses
275 this lock. Serializing access in this manner is not expected to have a
276 detrimental effect on performance as contention is expected to be
277 minimal.
278
279 An important property of the registry is that entries may be inserted
280 into the registry, but no entries are ever removed. The contents of
281 the static registry are used to populate the initially empty registry
282 database. Since these mapping are by definition persistent, no
283 mechanism is provided to remove them from the registry database.
284
285 NOTE: There is currently no DAT interface to set a provider's IA
286 specific parameters. A solution for this problem has been proposed for
287 uDAPL 1.1.
288
289 Registry Database API
290 .....................
291
292 The static variable Dat_Registry_Db is used to store information about
293 the Registry Database and has the following members:
294
295 - lock
296 - dictionary
297
298 The Registry Database is accessed via the following internal API:
299
300 Algorithm: dat_registry_init
301 Input: void
302 Output: DAT_RETURN
303 {
304 initialize Dat_Registry_Db
305
306 dat_os_sr_load()
307 }
308
309 Algorithm: dat_registry_insert
310 Input: IN const DAT_STATIC_REGISTRY_ENTRY sr_entry
311 Output: DAT_RETURN
312 {
313 dat_os_lock(&Dat_Registry_Db.lock)
314
315 create and initialize DAT_REGISTRY_ENTRY structure
316
317 dat_dictionary_add(&Dat_Registry_Db.dictionary, &entry)
318
319 dat_os_unlock(&Dat_Registry_Db.lock)
320 }
321
322 Algorithm: dat_registry_search
323 Input: IN const DAT_NAME_PTR ia_name
324 IN DAT_REGISTRY_ENTRY **entry
325 Output: DAT_RETURN
326 {
327 dat_os_lock(&Dat_Registry_Db.lock)
328
329 entry gets dat_dictionary_search(&Dat_Registry_Db.dictionary, &ia_name)
330
331 dat_os_unlock(&Dat_Registry_Db.lock)
332 }
333
334 Algorithm: dat_registry_list
335 Input: IN DAT_COUNT max_to_return
336 OUT DAT_COUNT *entries_returned
337 OUT DAT_PROVIDER_INFO *(dat_provider_list[])
338 Output: DAT_RETURN
339 {
340 dat_os_lock(&Dat_Registry_Db.lock)
341
342 size = dat_dictionary_size(Dat_Registry_Db.dictionary)
343
344 for ( i = 0, j = 0;
345 (i < max_to_return) && (j < size);
346 i++, j++ )
347 {
348 initialize dat_provider_list[i] w/ j-th element in dictionary
349 }
350
351 dat_os_unlock(&Dat_Registry_Db.lock)
352
353 *entries_returned = i;
354 }
355
356 Provider API pseudo-code
357 ------------------------
358
359 + dat_registry_add_provider()
360
361 Algorithm: dat_registry_add_provider
362 Input: IN DAT_PROVIDER *provider
363 Output: DAT_RETURN
364 {
365 dat_init()
366
367 dat_registry_search(provider->device_name, &entry)
368
369 if IA name is not found then dat_registry_insert(new entry)
370
371 if entry.ia_open_func is not NULL return an error
372
373 entry.ia_open_func = provider->ia_open_func
374 }
375
376 + dat_registry_remove_provider()
377
378 Algorithm: dat_registry_remove_provider
379 Input: IN DAT_PROVIDER *provider
380 Output: DAT_RETURN
381 {
382 dat_init()
383
384 dat_registry_search(provider->device_name, &entry)
385
386 if IA name is not found return an error
387
388 entry.ia_open_func = NULL
389 }
390
391 Consumer API pseudo-code
392 ------------------------
393
394 * dat_ia_open()
395
396 This function looks up the specified IA name in the ia_dictionary,
397 loads the provider library, retrieves a function pointer to the
398 provider's IA open function from the provider_dictionary, and calls
399 the providers IA open function.
400
401 Algorithm: dat_ia_open
402 Input: IN const DAT_NAME_PTR name
403 IN DAT_COUNT async_event_qlen
404 INOUT DAT_EVD_HANDLE *async_event_handle
405 OUT DAT_IA_HANDLE *ia_handle
406 Output: DAT_RETURN
407
408 {
409 dat_registry_search(name, &entry)
410
411 if the name is not found return an error
412
413 dat_os_library_load(entry.library_path, &entry.library_handle)
414
415 if the library fails to load return an error
416
417 if the entry's ia_open_func is invalid
418 {
419 dl_os_library_unload(entry.library_handle)
420 return an error
421 }
422
423 (*ia_open_func) (name,
424 async_event_qlen,
425 async_event_handle,
426 ia_handle);
427 }
428
429 * dat_ia_close()
430
431 Algorithm: dat_ia_close
432 Input: IN DAT_IA_HANDLE ia_handle
433 IN DAT_CLOSE_FLAGS ia_flags
434 Output: DAT_RETURN
435 {
436 provider = DAT_HANDLE_TO_PROVIDER(ia_handle)
437
438 (*provider->ia_close_func) (ia_handle, ia_flags)
439
440 dat_registry_search(provider->device_name, &entry)
441
442 dat_os_library_unload(entry.library_handle)
443 }
444
445 + dat_registry_list_providers()
446
447 Algorithm: dat_registry_list_providers
448 Input: IN DAT_COUNT max_to_return
449 OUT DAT_COUNT *entries_returned
450 OUT DAT_PROVIDER_INFO *(dat_provider_list[])
451 Output: DAT_RETURN
452 {
453 validate parameters
454
455 dat_registry_list(max_to_return, entries_returned, dat_provider_list)
456 }
457
458 Platform Specific API pseudo-code
459 --------------------------------
460
461 Below are descriptions of platform specific functions required by the
462 DAT Registry. These descriptions are for Linux.
463
464 Each entry in the static registry is represented by an OS specific
465 structure, DAT_OS_STATIC_REGISTRY_ENTRY. On Linux, this structure will
466 have the following members:
467
468 - IA name string
469 - API version
470 - thread safety
471 - default section
472 - library path string
473 - driver version
474 - IA parameter string
475
476 The tokenizer will return a DAT_OS_SR_TOKEN structure
477 containing:
478
479 - DAT_OS_SR_TOKEN_TYPE value
480 - string with the fields value
481
482 The tokenizer will ignore all white space and comments. The tokenizer
483 will also translate any escape sequences found in a string.
484
485 Algorithm: dat_os_sr_load
486 Input: n/a
487 Output: DAT_RETURN
488 {
489 if DAT_CONF environment variable is set
490 static_registry_file = contents of DAT_CONF
491 else
492 static_registry_file = /etc/dat.conf
493
494 sr_fd = dat_os_open(static_registry_file)
495
496 forever
497 {
498 initialize DAT_OS_SR_ENTRY entry
499
500 do
501 {
502 // discard blank lines
503 dat_os_token_next(sr_fd, &token)
504 } while token is newline
505
506 if token type is EOF then break // all done
507 // else the token must be a string
508
509 entry.ia_name = token.value
510
511 dat_os_token_next(sr_fd, &token)
512
513 if token type is EOF then break // all done
514 else if token type is not string then
515 {
516 // an error has occurred
517 dat_os_token_sync(sr_fd)
518 continue
519 }
520 else if ( dat_os_convert_api(token.value, &entry.api) fails )
521 {
522 // an error has occurred
523 dat_os_token_sync(sr_fd)
524 continue
525 }
526
527 dat_os_token_next(sr_fd, &token)
528
529 if token type is EOF then break // all done
530 else if token type is not string then
531 {
532 // an error has occurred
533 dat_os_token_sync(sr_fd)
534 continue
535 }
536 else if ( dat_os_convert_thread_safety(token.value, &entry.thread_safety) fails )
537 {
538 // an error has occurred
539 dat_os_token_sync(sr_fd)
540 continue
541 }
542
543 dat_os_token_next(sr_fd, &token)
544
545 if token type is EOF then break // all done
546 else if token type is not string then
547 {
548 // an error has occurred
549 dat_os_token_sync(sr_fd)
550 continue
551 }
552 else if ( dat_os_convert_default(token.value, &entry.default) fails )
553 {
554 // an error has occurred
555 dat_os_token_sync(sr_fd)
556 continue
557 }
558
559 dat_os_token_next(sr_fd, &token)
560
561 if token type is EOF then break // all done
562 else if token type is not string then
563 {
564 // an error has occurred
565 dat_os_token_sync(sr_fd)
566 continue
567 }
568
569 entry.lib_path = token.value
570
571 dat_os_token_next(sr_fd, &token)
572
573 if token type is EOF then break // all done
574 else if token type is not string then
575 {
576 // an error has occurred
577 dat_os_token_sync(sr_fd)
578 continue
579 }
580 else if ( dat_os_convert_driver_version(token.value, &entry.driver_version) fails )
581 {
582 // an error has occurred
583 dat_os_token_sync(sr_fd)
584 continue
585 }
586
587 dat_os_token_next(sr_fd, &token)
588
589 if token type is EOF then break // all done
590 else if token type is not string then
591 {
592 // an error has occurred
593 dat_os_token_sync(sr_fd)
594 continue
595 }
596
597 entry.ia_params = token.value
598
599 dat_os_token_next(sr_fd, &token)
600
601 if token type is EOF then break // all done
602 else if token type is not newline then
603 {
604 // an error has occurred
605 dat_os_token_sync(sr_fd)
606 continue
607 }
608
609 if ( dat_os_sr_is_valid(entry) )
610 {
611 dat_registry_insert(entry)
612 }
613 }
614
615 dat_os_close(sr_fd)
616 }
617
618 Algorithm: dat_os_library_load
619 Input: IN const DAT_NAME_PTR *library_path
620 OUT DAT_LIBRARY_HANDLE *library_handle
621 Output: DAT_RETURN
622 {
623 *library_handle = dlopen(library_path);
624 }
625
626 Algorithm: dat_os_library_unload
627 Input: IN const DAT_LIBRARY_HANDLE library_handle
628 Output: DAT_RETURN
629 {
630 dlclose(library_handle)
631 }