####################################################################### # # # DAPL Shared Memory Design # # # # James Lentini # # jlentini at users.sourceforge.net # # # # Created 09/17/2002 # # Updated 01/21/2005 # # Version 0.04 # # # ####################################################################### Contents -------- 0. Introduction 1. Referenced Documents 2. Requirements 3. Interface 4. Implementation Options Introduction ------------ This document describes the design of shared memory registration for the DAPL reference implementation (RI). Implementing shared memory support completely within the DAPL RI would not be an ideal solution. A more robust and efficient implementation can be acheived by HCA vendors that integrate a DAT provider into their software stack. Therefore the RI will not contain an implementation of this feature. Referenced Documents -------------------- kDAPL: Kernel Direct Access Programming Library, Version 1.2. uDAPL: User Direct Access Programming Library, Version 1.2. Available in the DAPL SourceForge repository at [doc/api/kDAPL_spec.pdf] and [doc/api/uDAPL_spec.pdf]. Collectively referred to in this document as the "DAT Specification". InfiniBand Access Application Programming Interface Specification, Version 1.2, 4/15/2002. Available in the DAPL SourceForge repository at [doc/api/IBM_access_api.pdf]. Referred to in this document as the "IBM Access API Specification". Mellanox IB-Verbs API (VAPI) Mellanox Software Programmers Interface for InfiniBand Verbs. Available in the DAPL SourceForge repository at [doc/api/MellanoxVerbsAPI.pdf]. Referred to in this document as the "VAPI API Specification". InfiniBand Architecture Specification, Volumes 1 and 2, Release 1.2, Available from http://www.infinibandta.org/ Referred to in this document as the "Infiniband Specification". Requirements ------------ The DAT shared memory model can be characterized as a peer-to-peer model since the order in which consumers register a region is not dictated by the programming interface. The DAT API function used to register shared memory is: DAT_RETURN dat_lmr_create ( IN DAT_IA_HANDLE ia_handle, IN DAT_MEM_TYPE mem_type, IN DAT_REGION_DESCRIPTION region_description, IN DAT_VLEN length, IN DAT_PZ_HANDLE pz_handle, IN DAT_MEM_PRIV_FLAGS mem_privileges, OUT DAT_LMR_HANDLE *lmr_handle, OUT DAT_LMR_CONTEXT *lmr_context, OUT DAT_RMR_CONTEXT *rmr_context, OUT DAT_VLEN *registered_length, OUT DAT_VADDR *registered_address ); where a DAT_REGION_DESCRIPTION is defined as: typedef union dat_region_description { DAT_PVOID for_va; DAT_LMR_HANDLE for_lmr_handle; DAT_SHARED_MEMORY for_shared_memory; } DAT_REGION_DESCRIPTION; In the case of a shared memory registration the DAT consumer will set the DAT_MEM_TYPE flag to DAT_MEM_TYPE_SHARED_VIRTUAL and place a cookie in the DAT_REGION_DESCRIPTION union's DAT_SHARED_MEMORY member. The DAT_SHARED_MEMORY type is defined as follows: typedef struct dat_shared_memory { DAT_PVOID virtual_address; DAT_LMR_COOKIE shared_memory_id; } DAT_SHARED_MEMORY; Unlike the DAT peer-to-peer model, the Infiniband shared memory model requires a master-slave relationship. A memory region must first be registered using the Register Memory Region verb with subsequent registrations made using the Register Shared Memory Region verb. The later is implemented in the IBM OS Access API as: ib_int32_t ib_mr_shared_register_us( ib_hca_handle_t hca_handle, ib_mr_handle_t *mr_handle, /* IN-OUT: could be changed */ ib_pd_handle_t pd_handle, /* IN */ ib_uint32_t access_control, /* IN */ ib_uint32_t *l_key, /* OUT */ ib_uint32_t *r_key, /* OUT: if remote access needed */ ib_uint8_t **va ); /* IN-OUT: virt. addr. to register */ The important parameter is the memory region handle which must be the same as an already registered region. Two requirements are implied by this difference between the DAT and Infiniband models. First, DAPL implementations need a way to determine the first registration of a shared region. Second implementations must map DAT_LMR_COOKIE values to memory region handles both within and across processes. To satisfy the above requirements DAPL must maintain this information in a system wide database. The difficulty of implementing such a database at the DAT provider level is the reason the RI's shared memory code is meant to be temporary. Such a database is much better suited as part of the HCA vendor's software stack, specifically as part of their HCA driver. If DAPL was based on a master-slave model like InfiniBand, the implementation of shared memory would be straight forward. Specifically the complexity is a result of the consumer being responsible for specifying the DAT_LMR_COOKIE values. If the DAPL spec. were changed to allow the provider and not the consumer to specify the DAT_LMR_COOKIE value, the implementation of this feature would be greatly simplified. Since the DAPL API already requires consumers to communicate the DAT_LMR_COOKIE values between processes, such a change places minimal additional requirements on the consumer. The dapl_lmr_query call could easily be adapted to allow the consumer to query the provider for a given LMR's DAT_LMR_COOKIE value. The only spec changes needed would be to add a DAT_LMR_COOKIE member to the DAT_LMR_PARAM structure and a DAT_LMR_FIELD_LMR_COOKIE constant to the DAT_LMR_PARAM_MASK enumeration. A provider could then store the given LMR's memory region handle in this value, greatly simplifying the implementation of shared memory in DAPL. Interface --------- To allow the database implementation to easily change, the RI would use a well defined interface between the memory subsystem and the database. Conceptually the database would contain a single table with the following columns: [ LMR Cookie ][ MR Handle ][ Reference Count ][ Initialized ] where the LMR Cookie column is the primary key. The following functions would be used to access the database: DAT_RETURN dapls_mrdb_init ( void ); Called by dapl_init(.) to perform any necessary database initialization. DAT_RETURN dapls_mrdb_exit ( void ); Called by dapl_fini(.) to perform any necessary database cleanup. DAT_RETURN dapls_mrdb_record_insert ( IN DAPL_LMR_COOKIE cookie ); If there is no record for the specified cookie, an empty record is added with a reference count of 1 and the initialized field is set to false. If a record already exists, the function returns an error. DAT_RETURN dapls_mrdb_record_update ( IN DAPL_LMR_COOKIE cookie, IN ib_mr_handle_t mr_handle ); If there is a record for the specified cookie, the MR handle field is set to the specified mr_handle value and the initialized field is set to true. Otherwise an error is returned. DAT_RETURN dapls_mrdb_record_query ( IN DAPL_LMR_COOKIE cookie, OUT ib_mr_handle_t *mr_handle ); If there is a record for the specified cookie and the initialized field is true, the MR handle field is returned and the reference count field is incremented. Otherwise an error is returned. DAT_RETURN dapls_mrdb_record_dec ( IN DAPL_LMR_COOKIE cookie ); If there is a record for the specified cookie, the reference count field is decremented. If the reference count is zero after the decrement, the record is removed from the database. Otherwise an error is returned. The generic algorithms for creating and destroying a shared memory region are: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: CreateShared Inputs: ia_handle pz_handle address length lmr_cookie privileges Outputs: lmr_handle lmr_context registered_address registered_length forever { if dapls_mrdb_record_insert(cookie) is successful { if dapl_lmr_create_virtual is not successful dapls_mrdb_record_dec(cookie) return error else if dapls_mrdb_record_update(cookie, lmr->mr_handle) is not successful dapls_mrdb_record_dec(cookie) return error else break } else if dapls_mrdb_record_query(cookie, mr_handle) is successful { if ib_mrdb_shared_register_us is not successful dapls_mrdb_record_dec(cookie) return error else break } } +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: FreeShared Inputs: lmr Outputs: if dapls_ib_mr_deregister(lmr) is successful dapls_mrdb_record_dec(lmr->cookie) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Implementation Options ---------------------- As described above the crucial functionality needed to implement shared memory support is a system wide database for mapping LMR cookies to memory region handles. The following designs represent some of the options for implementing such a database. Adding a database increases the complexity of DAPL from both an implementor and user's perspective. These designs should be evaluated on the degree to which they minimize the additional complexity while still providing a robust solution. File System Database -------------------- Employing a database that is already part of the system would be ideal. One option on Linux is to use the file system. An area of the file system could be set aside for the creation of files to represent each LMR cookie. The area of the file system could be specified through a hard coded value, an environment variable, or a configuration file. A good candidate would be a DAPL subdirectory of /tmp. Exclusive file creation is available through the creat(2) system call in Linux. The standard I/O interface (fopen(3), etc.) does not support this feature making porting difficult. However porting to other environments is not a goal of this design since the entire scheme is only a temporary solution. Determining when to delete the files is a difficult problem. A reference count is required to properly remove a file once all the memory regions it represents are deregistered. The synchronization mechanism necessary for maintaining the reference count is not easily implemented. As an alternative, a script could be provided to clean up the database by removing all the files. The script would need to be run before any DAPL consumers were started to ensure a clean database. The disadvantage of using a script is that no DAPL instances can be running when it is used. Another option would be to store the process ID (PID) of the process that created the file as part of the file's contents. Upon finding a record for a given LMR cookie value, a DAPL instance could determine if there was a process with the same PID in the system. To accomplish this the kill(2) system call could be used (ex. kill(pid, 0) ). This method of validating the record assumes that all DAPL instances can signal one another and that the PID values do not wrap before the check is made. Another difficulty with this solution is choosing an accessible portion of the file system. The area must have permissions that allow all processes using DAPL to access and modify its contents. System administrators are typically reluctant to allow areas without any access controls. Typically such areas are on a dedicated file system of a minimal size to ensure that malicious or malfunctioning software does not monopolize the system's storage capacity. Since very little information will be stored in each file it is unlikely that DAPL would need a large amount of storage space even if a large number of shared memory regions were in use. However since a file is needed for each shared region, a large number of shared registrations may lead to the consumption of all a file system's inodes. Again since this solution is meant to be only temporary this constraint may be acceptable. There is also the possibility for database corruption should a process crash or deadlock at an inopportune time. If a process creates file x and then crashes all other processes waiting for the memory handle to be written to x will fail. The database interface could be implemented as follows: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_init Inputs: Outputs: return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_exit Inputs: Outputs: return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_insert Inputs: cookie Outputs: file_name = convert cookie to valid file name fd = exclusively create file_name if fd is invalid return failure if close fd fails return failure return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_update Inputs: cookie mr_handle Outputs: file_name = convert cookie to valid file name fd = open file_name if fd is invalid return failure if write mr_handle to file_name fails return failure if close fd fails return failure return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_query Inputs: cookie Outputs: mr_handle file_name = convert cookie to valid file name fd = open file_name if fd is invalid return failure if read mr_handle from file_name fails return failure if close fd fails return failure return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_dec Inputs: cookie Outputs: return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Daemon Database --------------- The database could be maintained by a separate daemon process. The DAPL instances would act as clients of the daemon server and communicate with the daemon through the various IPC mechanisms available on Linux: Unix sockets, TCP/IP sockets, System V message queues, FIFOs, or RPCs. As with the file system based database, process crashes can potentially cause database corruption. While the portability of this implementation will depend on the chosen IPC mechanism, this approach will be at best Unix centric and possibly Linux specific. The database interface could be implemented as follows: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_init Inputs: Outputs: initialize IPC mechanism +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_exit Inputs: Outputs: shutdown IPC mechanism +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_insert Inputs: cookie Outputs: if send insert message for cookie fails return error if receive insert response message fails return error if insert success return success else return error +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_update Inputs: cookie mr_handle Outputs: if send update message for cookie and mr_handle fails return error else return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_query Inputs: cookie Outputs: mr_handle if send query message for cookie fails return error else if receive query response message with mr_handle fails return error else return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_dec Inputs: cookie Outputs: if send decrement message for cookie fails return error else return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Shared Memory Database ---------------------- The database could be maintained in an area of memory shared by all DAPL instances running on a system. Linux supports the System V shared memory functions shmget(2), shmctl(2), shmat(2), and shmdt(2). A hard coded key_t value could be used so that each DAPL instance attached to the same piece of shared memory. The size of the database would be constrained by the size of the shared memory region. Synchronization could be achieved by using atomic operations targeting memory in the shared region. Such a design would suffer from the corruption problems described above. If a process crashed there would be no easy way to clean up its locks and roll back the database to a consistent state. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_init Inputs: Outputs: attach shared region +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_exit Inputs: Outputs: detach shared region +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_insert Inputs: cookie Outputs: lock database if db does not contain cookie add record for cookie unlock database +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_update Inputs: cookie mr_handle Outputs: lock database if db contains cookie update record's mr_handle unlock database +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_query Inputs: cookie Outputs: mr_handle lock database if db contains cookie set mr_handle to record's value increment record's reference count unlock database +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_dec Inputs: cookie Outputs: lock database if db contains cookie decrement record's reference count if reference count is 0 remove record unlock database +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Kernel Module Database ---------------------- If the DAT library were integrated with an HCA vendor's software stack, the database could be managed by the HCA driver. Placing the database in the kernel would alleviate the synchronization problems posed by multiple processes. Since memory registration operations already involve a transition into the kernel, no extra overhead would be incurred by this design. The RI could include a kernel module with this functionality as a temporary solution. The module could identify itself as a character device driver and communicate with user level processes through an ioctl(2). The driver could also create an entry in the proc file system to display the database's contents for diagnostic purposes. A major benefit of a kernel based implementation is that the database can remain consistent even in the presence of application errors. Since DAPL instances communicate with the driver by means of ioctl(2) calls on a file, the driver can be arrange to be informed when the file is closed and perform any necessary actions. The driver is guaranteed to be notified of a close regardless of the manner in which the process exits. The database could be implemented as a dictionary using the LMR cookie values as keys. The following pseudo-code describes the functions needed by the kernel module and the database interface. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: KernelModuleInit Inputs: Outputs: dictionary = create_dictionary() create_proc_entry() create_character_device_entry() +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: KernelModuleExit Inputs: Outputs: remove_character_device_entry() remove_proc_entry() fee_dictionary(dictionary) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: DeviceOpen Inputs: file Outputs: dev_data = allocate device data file->private_data = dev_data +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: DeviceClose Inputs: file Outputs: dev_data = file->private_data for each record in dev_data { RecordDecIoctl } deallocate dev_data +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: RecordInsertIoctl Inputs: file cookie Outputs: lock dictionary if cookie is not in dictionary insert cookie into dictionary unlock dictionary +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: RecordUpdateIoctl Inputs: file cookie mr_handle Outputs: dev_data = file->private_data lock dictionary if cookie is in dictionary add record reference to dev_data update mr_handle unlock dictionary +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: RecordQueryIoctl Inputs: file cookie Outputs: mr_handle dev_data = file->private_data lock dictionary if cookie is in dictionary add record reference to dev_data retrieve mr_handle unlock dictionary +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: RecordDecIoctl Inputs: file cookie Outputs: dev_data = file->private_data remove record reference from dev_data lock dictionary if cookie is in dictionary decrement reference count if reference count is 0 remove record unlock dictionary +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_init Inputs: Outputs: fd = open device file if fd is invalid return error else return success +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_exit Inputs: Outputs: close fd +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_insert Inputs: cookie Outputs: ioctl on fd +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_update Inputs: cookie mr_handle Outputs: ioctl on fd +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_query Inputs: cookie Outputs: mr_handle ioctl on fd +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Function: dapls_mrdb_record_dec Inputs: cookie Outputs: ioctl on fd +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+