STRING TOKENIZER Requirements ================ ## Overview The STRING TOKENIZER provides the functionality of splitting a STRING into multiples tokens ## Exposed API ```C extern STRING_TOKEN_HANDLE StringToken_GetFirst(const char* source, size_t length, const char** delimiters, size_t n_delims); extern bool StringToken_GetNext(STRING_TOKEN_HANDLE token, const char** delimiters, size_t n_delims); extern const char* StringToken_GetValue(STRING_TOKEN_HANDLE token); extern size_t StringToken_GetLength(STRING_TOKEN_HANDLE token); extern const char* StringToken_GetDelimiter(STRING_TOKEN_HANDLE token); extern int StringToken_Split(const char* source, size_t length, const char** delimiters, size_t n_delims, bool include_empty, char*** tokens, size_t* token_count); extern void StringToken_Destroy(STRING_TOKEN_HANDLE token); ``` ### StringToken_GetFirst ```c extern STRING_TOKEN_HANDLE StringToken_GetFirst(const char* source, size_t length, const char** delimiters, size_t n_delims); ``` **SRS_STRING_TOKENIZER_09_001: [** If `source` or `delimiters` are NULL, or `n_delims` is zero, the function shall return NULL **]** **SRS_STRING_TOKENIZER_09_002: [** If any of the strings in `delimiters` are NULL, the function shall return NULL **]** **SRS_STRING_TOKENIZER_09_003: [** A STRING_TOKEN structure shall be allocated to hold the token parameters **]** **SRS_STRING_TOKENIZER_09_004: [** If the STRING_TOKEN structure fails to be allocated, the function shall return NULL **]** **SRS_STRING_TOKENIZER_09_005: [** The source string shall be split in a token starting from the beginning of `source` up to occurrence of any one of the `demiliters`, whichever occurs first in the order provided **]** **SRS_STRING_TOKENIZER_09_006: [** If the source string does not have any of the `demiliters`, the resulting token shall be the entire `source` string **]** **SRS_STRING_TOKENIZER_09_007: [** If any failure occurs, all memory allocated by this function shall be released **]** ### StringToken_GetNext ```c extern bool StringToken_GetNext(STRING_TOKEN_HANDLE token, const char** delimiters, size_t n_delims); ``` **SRS_STRING_TOKENIZER_09_008: [** If `token` or `delimiters` are NULL, or `n_delims` is zero, the function shall return false **]** **SRS_STRING_TOKENIZER_09_009: [** If the previous token already extended to the end of `source`, the function shall return false **]** **SRS_STRING_TOKENIZER_09_010: [** The next token shall be selected starting from the position in `source` right after the previous delimiter up to occurrence of any one of `demiliters`, whichever occurs first in the order provided **]** **SRS_STRING_TOKENIZER_09_011: [** If the source string, starting right after the position of the last delimiter found, does not have any of the `demiliters`, the resulting token shall be the entire remaining of the `source` string **]** **SRS_STRING_TOKENIZER_09_012: [** If a token was identified, the function shall return true **]** ### StringToken_GetValue ```c extern const char* StringToken_GetValue(STRING_TOKEN_HANDLE token); ``` **SRS_STRING_TOKENIZER_09_013: [** If `token` is NULL the function shall return NULL **]** **SRS_STRING_TOKENIZER_09_014: [** The function shall return the pointer to the position in `source` where the current token starts. **]** ### StringToken_GetLength ```c extern size_t StringToken_GetLength(STRING_TOKEN_HANDLE token); ``` **SRS_STRING_TOKENIZER_09_015: [** If `token` is NULL the function shall return zero **]** **SRS_STRING_TOKENIZER_09_016: [** The function shall return the length of the current token **]** ### StringToken_GetDelimiter ```c extern const char* StringToken_GetDelimiter(STRING_TOKEN_HANDLE token); ``` **SRS_STRING_TOKENIZER_09_017: [** If `token` is NULL the function shall return NULL **]** **SRS_STRING_TOKENIZER_09_018: [** The function shall return a pointer to the delimiter that defined the current token, as passed to the previous call to `StringToken_GetNext()` or `StringToken_GetFirst()` **]** **SRS_STRING_TOKENIZER_09_019: [** If the current token extends to the end of `source`, the function shall return NULL **]** ### StringToken_Destroy ```c extern void StringToken_Destroy(STRING_TOKEN_HANDLE token); ``` **SRS_STRING_TOKENIZER_09_020: [** If `token` is NULL the function shall return **]** **SRS_STRING_TOKENIZER_09_021: [** Otherwise the memory allocated for STRING_TOKEN shall be released **]** ### StringToken_Split ```c extern int StringToken_Split(const char* source, size_t length, const char** delimiters, size_t n_delims, bool include_empty, char*** tokens, size_t* token_count); ``` **SRS_STRING_TOKENIZER_09_022: [** If `source`, `delimiters`, `token` or `token_count` are NULL, or `n_delims` is zero the function shall return a non-zero value **]** **SRS_STRING_TOKENIZER_09_023: [** `source` (up to `length`) shall be split into individual tokens separated by any of `delimiters` **]** **SRS_STRING_TOKENIZER_09_024: [** All NULL tokens shall be ommited if `include_empty` is not TRUE **]** **SRS_STRING_TOKENIZER_09_025: [** The tokens shall be stored in `tokens`, and their count stored in `token_count` **]** **SRS_STRING_TOKENIZER_09_026: [** If any failures splitting or storing the tokens occur the function shall return a non-zero value **]** **SRS_STRING_TOKENIZER_09_027: [** If no failures occur the function shall return zero **]**