#include <set>#include <map>#include <list>#include "freeling/windll.h"#include "freeling/morfo/language.h"#include "freeling/regexp.h"
Include dependency graph for tokenizer.h:
This graph shows which files directly or indirectly include this file:
Go to the source code of this file.
Classes | |
| class | freeling::tokenizer |
| Class tokenizer implements a token splitter, which converts a string into a sequence of word objects, according to a set of tokenization rules read from aconfiguration file. More... | |
Namespaces | |
| namespace | freeling |
