| 1 |
STRING TOKENIZER Requirements |
| 2 |
================ |
| 3 |
|
| 4 |
## Overview |
| 5 |
The STRING TOKENIZER provides the functionality of splitting a STRING into multiples tokens |
| 6 |
|
| 7 |
## Exposed API |
| 8 |
```C |
| 9 |
extern STRING_TOKEN_HANDLE StringToken_GetFirst(const char* source, size_t length, const char** delimiters, size_t n_delims); |
| 10 |
extern bool StringToken_GetNext(STRING_TOKEN_HANDLE token, const char** delimiters, size_t n_delims); |
| 11 |
extern const char* StringToken_GetValue(STRING_TOKEN_HANDLE token); |
| 12 |
extern size_t StringToken_GetLength(STRING_TOKEN_HANDLE token); |
| 13 |
extern const char* StringToken_GetDelimiter(STRING_TOKEN_HANDLE token); |
| 14 |
extern int StringToken_Split(const char* source, size_t length, const char** delimiters, size_t n_delims, bool include_empty, char*** tokens, size_t* token_count); |
| 15 |
extern void StringToken_Destroy(STRING_TOKEN_HANDLE token); |
| 16 |
``` |
| 17 |
|
| 18 |
### StringToken_GetFirst |
| 19 |
```c |
| 20 |
extern STRING_TOKEN_HANDLE StringToken_GetFirst(const char* source, size_t length, const char** delimiters, size_t n_delims); |
| 21 |
``` |
| 22 |
|
| 23 |
**SRS_STRING_TOKENIZER_09_001: [** If `source` or `delimiters` are NULL, or `n_delims` is zero, the function shall return NULL **]** |
| 24 |
|
| 25 |
**SRS_STRING_TOKENIZER_09_002: [** If any of the strings in `delimiters` are NULL, the function shall return NULL **]** |
| 26 |
|
| 27 |
**SRS_STRING_TOKENIZER_09_003: [** A STRING_TOKEN structure shall be allocated to hold the token parameters **]** |
| 28 |
|
| 29 |
**SRS_STRING_TOKENIZER_09_004: [** If the STRING_TOKEN structure fails to be allocated, the function shall return NULL **]** |
| 30 |
|
| 31 |
**SRS_STRING_TOKENIZER_09_005: [** The source string shall be split in a token starting from the beginning of `source` up to occurrence of any one of the `demiliters`, whichever occurs first in the order provided **]** |
| 32 |
|
| 33 |
**SRS_STRING_TOKENIZER_09_006: [** If the source string does not have any of the `demiliters`, the resulting token shall be the entire `source` string **]** |
| 34 |
|
| 35 |
**SRS_STRING_TOKENIZER_09_007: [** If any failure occurs, all memory allocated by this function shall be released **]** |
| 36 |
|
| 37 |
|
| 38 |
### StringToken_GetNext |
| 39 |
```c |
| 40 |
extern bool StringToken_GetNext(STRING_TOKEN_HANDLE token, const char** delimiters, size_t n_delims); |
| 41 |
``` |
| 42 |
|
| 43 |
**SRS_STRING_TOKENIZER_09_008: [** If `token` or `delimiters` are NULL, or `n_delims` is zero, the function shall return false **]** |
| 44 |
|
| 45 |
**SRS_STRING_TOKENIZER_09_009: [** If the previous token already extended to the end of `source`, the function shall return false **]** |
| 46 |
|
| 47 |
**SRS_STRING_TOKENIZER_09_010: [** The next token shall be selected starting from the position in `source` right after the previous delimiter up to occurrence of any one of `demiliters`, whichever occurs first in the order provided **]** |
| 48 |
|
| 49 |
**SRS_STRING_TOKENIZER_09_011: [** If the source string, starting right after the position of the last delimiter found, does not have any of the `demiliters`, the resulting token shall be the entire remaining of the `source` string **]** |
| 50 |
|
| 51 |
**SRS_STRING_TOKENIZER_09_012: [** If a token was identified, the function shall return true **]** |
| 52 |
|
| 53 |
|
| 54 |
### StringToken_GetValue |
| 55 |
```c |
| 56 |
extern const char* StringToken_GetValue(STRING_TOKEN_HANDLE token); |
| 57 |
``` |
| 58 |
|
| 59 |
**SRS_STRING_TOKENIZER_09_013: [** If `token` is NULL the function shall return NULL **]** |
| 60 |
|
| 61 |
**SRS_STRING_TOKENIZER_09_014: [** The function shall return the pointer to the position in `source` where the current token starts. **]** |
| 62 |
|
| 63 |
|
| 64 |
### StringToken_GetLength |
| 65 |
```c |
| 66 |
extern size_t StringToken_GetLength(STRING_TOKEN_HANDLE token); |
| 67 |
``` |
| 68 |
|
| 69 |
**SRS_STRING_TOKENIZER_09_015: [** If `token` is NULL the function shall return zero **]** |
| 70 |
|
| 71 |
**SRS_STRING_TOKENIZER_09_016: [** The function shall return the length of the current token **]** |
| 72 |
|
| 73 |
|
| 74 |
### StringToken_GetDelimiter |
| 75 |
```c |
| 76 |
extern const char* StringToken_GetDelimiter(STRING_TOKEN_HANDLE token); |
| 77 |
``` |
| 78 |
|
| 79 |
**SRS_STRING_TOKENIZER_09_017: [** If `token` is NULL the function shall return NULL **]** |
| 80 |
|
| 81 |
**SRS_STRING_TOKENIZER_09_018: [** The function shall return a pointer to the delimiter that defined the current token, as passed to the previous call to `StringToken_GetNext()` or `StringToken_GetFirst()` **]** |
| 82 |
|
| 83 |
**SRS_STRING_TOKENIZER_09_019: [** If the current token extends to the end of `source`, the function shall return NULL **]** |
| 84 |
|
| 85 |
|
| 86 |
### StringToken_Destroy |
| 87 |
```c |
| 88 |
extern void StringToken_Destroy(STRING_TOKEN_HANDLE token); |
| 89 |
``` |
| 90 |
|
| 91 |
**SRS_STRING_TOKENIZER_09_020: [** If `token` is NULL the function shall return **]** |
| 92 |
|
| 93 |
**SRS_STRING_TOKENIZER_09_021: [** Otherwise the memory allocated for STRING_TOKEN shall be released **]** |
| 94 |
|
| 95 |
|
| 96 |
### StringToken_Split |
| 97 |
```c |
| 98 |
extern int StringToken_Split(const char* source, size_t length, const char** delimiters, size_t n_delims, bool include_empty, char*** tokens, size_t* token_count); |
| 99 |
``` |
| 100 |
|
| 101 |
**SRS_STRING_TOKENIZER_09_022: [** If `source`, `delimiters`, `token` or `token_count` are NULL, or `n_delims` is zero the function shall return a non-zero value **]** |
| 102 |
|
| 103 |
**SRS_STRING_TOKENIZER_09_023: [** `source` (up to `length`) shall be split into individual tokens separated by any of `delimiters` **]** |
| 104 |
|
| 105 |
**SRS_STRING_TOKENIZER_09_024: [** All NULL tokens shall be ommited if `include_empty` is not TRUE **]** |
| 106 |
|
| 107 |
**SRS_STRING_TOKENIZER_09_025: [** The tokens shall be stored in `tokens`, and their count stored in `token_count` **]** |
| 108 |
|
| 109 |
**SRS_STRING_TOKENIZER_09_026: [** If any failures splitting or storing the tokens occur the function shall return a non-zero value **]** |
| 110 |
|
| 111 |
**SRS_STRING_TOKENIZER_09_027: [** If no failures occur the function shall return zero **]** |
| 112 |
|