dktok(3) - Linux man page

Name

dktok - Tokenizer

Synopsis

#include <dktok.h>

int fct(void *d, void *tpv, char *s, int *ec);

dk_tokenizer_t *
dktok_new(
  size_t b, char **q, char *s, char *w, char *n, char c,
  dk_fct_tokenizer *f, void *d
);

void dktok_start(dk_tokenizer_t *tp);

int dktok_add(dk_tokenizer_t *tp, char c);

int dktok_stop(dk_tokenizer_t *tp);

int dktok_get_error_code(dk_tokenizier_t *tp);

unsigned long dktok_get_lineno(dk_tokenizer_t *tp);

void dktok_reset_error_code(dk_tokenizer_t *tp);

Description

The dktok module splits incoming data from a stream into tokens and allows to process the tokens by using a processing function.

The fct() function is an example for a processing function. The d parameter is a pointer to additional data used for processing, tpv is a pointer to a tokenizer, s is a pointer to the buffer containing the token, ec is a pointer to a variable which can be set if an error occures. The function must return 1 on success, 0 on errors. If the function returns 0 input processing is aborted.

The dktok_new() function creates a new tokenizer. The parameter b specifies the length of the largest token. The q parameter points to an array containing strings. Each string contains a bracket pair (opening and closing bracket). The array must be finished by a NULL pointer. The s parameter is a string containing all one-character-tokens (i.e. mathematical operators). The w parameter is a string containing all white-space characters. The processing function is specified as parameter f, additional data as specified in d is passed as parameter d to the processing callback function. The function returns a valid pointer on success, NULL on error. On success the tokenizer structure must be released after usage by the dktok_delete() function.

The dktok_start() function must be invoked before feeding data into the tokenizer.

The dktok_add() function feeds data into the tokenizer. If the return value is positive (>0) we can continue feeding the tokenizer.

The dktok_stop() function must be run after providing the last input to process the final token and to flush all buffers.

The dktok_get_error_code() retrieves the error code established by the processing callback function.

The dktok_get_lineno() function returns the current line number. This can be used to generate error messages.

The dktok_reset_error_code() function resets the internal error code.

Author

Dirk Krause

Copyright And License

Copyright © 2001-2008, Dirk Krause All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above
  copyright notice, this list of conditions and the
  following disclaimer.

* Redistributions in binary form must reproduce the above
  copyright notice, this list of conditions and the following
  disclaimer in the documentation and/or other materials
  provided with the distribution.

* Neither the name of the Dirk Krause nor the names of
  contributors may be used to endorse or promote products
  derived from this software without specific prior written
  permission.