Error with analyze ** glibc detected *** /usr/local/bin/analyzer: double free or corruption (out):

Submitted by Josu on Mon, 09/19/2016 - 11:20
Forums

Helo.

I'm trying to process a Conll File with the analyze script.

The command I use:

task01.posttask.v1.0/corpora/test/es/auto/CESS-CAST-A_11476_20000614.auto_freeling | analyze -f /usr/local/share/freeling/config/es_semeval.cfg

Response of the system:

NEC activated since coreference or semantic graph was requested.
UKB sense disambiguation activated since coreference or semantic graph was requested.
Input mode switched to 'doc' since coreference or semantic graph was requested.
*** glibc detected *** /usr/local/bin/analyzer: double free or corruption (out): 0x000000000d628f80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7da26)[0x7fe83b6f7a26]
/usr/local/lib/libfreeling-4.0.so(_ZN8freeling8dep_tree18rebuild_node_indexEv+0x24d)[0x7fe83c75533d]
/usr/local/lib/libfreeling-4.0.so(_ZN8freeling8sentence12set_dep_treeERKNS_8dep_treeEi+0x35d)[0x7fe83c7587cd]
/usr/local/lib/libfreeling-4.0.so(_ZNK8freeling11dep_treeler10Treeler2FLERNS_8sentenceERKN7treeler9DepVectorISsEERKNS3_3srl10PredArgSetE+0x475)[0x7fe83c88daa5]
/usr/local/lib/libfreeling-4.0.so(_ZNK8freeling11dep_treeler7analyzeERNS_8sentenceE+0x12d)[0x7fe83c88e20d]
/usr/local/lib/libfreeling-4.0.so(_ZNK8freeling9processor7analyzeERSt4listINS_8sentenceESaIS2_EE+0x26)[0x7fe83c7706c6]
/usr/local/lib/libfreeling-4.0.so(_ZNK8freeling9processor7analyzeERNS_8documentE+0x26)[0x7fe83c770706]
/usr/local/lib/libfreeling-4.0.so(_ZNK8freeling8analyzer7analyzeERNS_8documentE+0x19)[0x7fe83c764d39]
/usr/local/bin/analyzer[0x40b4ec]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7fe83b69b7ed]
/usr/local/bin/analyzer[0x40bc75]

I tried to attach the config files but the spam filter blocked my post

Your problem is that the options "ConllInputConfig" and "ConllOutputConfig" that you used in the config file do not exist.

The "analyze" program is just an example of how to use freeling, and it does not cover all its capabilities.

In particular, it supports a limited number of I/O formats. CoNLL input and output formats accepted by analyzer are not configurable. If you want to check which format it is, use "--output conll" option. If you use the same format for input with "--input conll", it will work. Defining unexisting options in the config file will not work.

If you want to customize your conll I/O format (e.g. to use some specific column order) you need to use your own main program (or adapt analyzer.cc) to instantiate a conll_input and conll_output classes with the required configuration file.

I had already added the options to config.h and changed the main.cc to get the expected behaviour. It seems that it takes the expected Conll configuration files but then the reported crash happens.

These are the changes in main.cc

//---- Create input handler for requested format
io::input_handler* create_input_handler(config *cfg) {

io::input_handler *inp;
if (cfg->InputFormat==INP_CONLL) inp = new io::input_conll(cfg->ConllInConfig);
else if (cfg->InputFormat==INP_FREELING) inp = new io::input_freeling();
else inp = NULL;

return inp;
}

and
/
/---- Create output handler for requested format
io::output_handler* create_output_handler(config *cfg) {

io::output_handler *out;
if (cfg->OutputFormat==OUT_TRAIN) {
out = new io::output_train();
}
else if (cfg->OutputFormat==OUT_CONLL) {
out = new io::output_conll(cfg->ConllOutConfig);
out->load_tagset(cfg->TAGSET_TagsetFile);
}

did you try with default conll formats used by freeling?

just to rule out the problem is in the parser...

Or better, can you send me the modified main.cc and config.h?

I found the problem:

CoNLL format requires that word IDs start at 1 for each sentence, and your file starts word numbering at 0.
When creating dependency trees, each word points to the ID of its head, and the sentence root has head=0. Thus, no word can have ID=0.

Documentation for all CoNLL shared tasks involving dependency parsing define the format that way.