Problems installing Freeling 4.1 to use it with Python

Submitted by ivansiiito on Thu, 10/11/2018 - 21:53
Forums

Hi!

I'm trying to use Freeling 4.1 in Python (in Windows 10), but I'm having trouble trying to install it from sources. I'm following the installation manual and, with lots of effort (I'm not used to do these things at all), I managed to do up to the cmake command.

However, when I try the nmake install I'm having problems:

[ 30%] Building CXX object src/libfreeling/CMakeFiles/freeling.dir/language.cc.obj
language.cc
Unknown compiler version - please run the configure tests and report the results
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(365): warning C4267: 'return': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(394): warning C4267: 'return': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(402): warning C4267: 'return': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(460): warning C4018: '>': no coinciden signed/unsigned
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(464): warning C4018: '>': no coinciden signed/unsigned
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(523): error C2039: 'list': no es un miembro de 'freeling::word::iterator'
C:\PROGRA~1\FREELI~1.1-N\src\include\freeling/morfo/language.h(382): note: vea la declaración de 'freeling::word::iterator'
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(523): error C2955: 'std::iterator': el uso de clase plantilla requiere una lista de argumentos de plantilla
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\include\xutility(4293): note: vea la declaración de 'std::iterator'
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(523): error C2039: '++': no es un miembro de 'std::iterator<_Category,_Ty,_Diff,_Pointer,_Reference>'
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(564): error C2039: 'list': no es un miembro de 'freeling::word::const_iterator'
C:\PROGRA~1\FREELI~1.1-N\src\include\freeling/morfo/language.h(411): note: vea la declaración de 'freeling::word::const_iterator'
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(564): error C2039: 'const_iterator': no es un miembro de '`global namespace''
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(564): error C3083: 'const_iterator': el símbolo situado a la izquierda de '::' debe ser un tipo
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(564): error C2039: '++': no es un miembro de '`global namespace''
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(681): warning C4267: 'return': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1337): warning C4267: '=': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1340): warning C4267: '=': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1381): warning C4267: '=': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1384): warning C4267: '=': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1598): warning C4267: '+=': conversión de 'size_t' a 'int'; posible pérdida de datos
C:\Program Files\FreeLing-4.1-new\src\libfreeling\language.cc(1604): warning C4267: 'return': conversión de 'size_t' a 'int'; posible pérdida de datos
NMAKE : fatal error U1077: 'C:\PROGRA~2\MIB055~1\2017\PROFES~1\VC\Tools\MSVC\1415~1.267\bin\Hostx64\x64\cl.exe' : código devuelto '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\bin\HostX64\x64\nmake.exe"' : código devuelto '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\bin\HostX64\x64\nmake.exe"' : código devuelto '0x2'
Stop.

Apparently there is a problem when compiling the file language.cc, when reaching the line 523 (the line starting with this->...):
/// Generic preincrement, for all cases
word::iterator& word::iterator::operator++() {
do {
this->list::iterator::operator++();
} while (type!=ALL and (*this)!=iend and (*this)->is_selected(kbest)!=(type==SELECTED) );
return (*this);
}

I assume there is some problem regarding the version of the .h file and something else. But I'm completely unable to continue. I have been two days installing and uninstalling things, reading many web pages about similar errors, but I'm completely clueless.

I would appreciate very much if anybody could help me somehow.

Thanks!

Please follow the installation instructions from the manual. If you want to use python you need to install from source:

https://talp-upc.gitbooks.io/freeling-4-1-user-manual/content/installat…

Before attempting to compile freeling, make sure you follow all the steps about "installing requirements on windows"

Then follow carefully the steps about "building freeling on windows"

You also need to read "building and using APIs on windows"

This procedure will work on FreeLing source packages you can download from gitHub releases. It may not work if you are using latest development version from gitHub master

ivansiiito

Mon, 10/15/2018 - 19:39

Hi again and thanks a lot for your answer.

I have been trying the whole process from the very beginning, and I realized that I did some things wrong. However, after uninstalling several versions of Visual Studio, reinstalling again the Microsoft Visual Studio Community 2015 and checking and trying some other stuff I have manage to follow exactly what the installation guide says without any problem (appart from some compiler warnings).

Everything goes alright until I do the final nmake install. Then, this is what happens:

C:\Program Files\FreeLing-4.1\build>nmake install
Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation. All rights reserved.
[ 13%] Built target foma
[ 27%] Built target treeler
[ 91%] Built target freeling
[ 92%] Built target threaded_analyzer
[ 93%] Built target analyzer_client
[ 94%] Built target analyzer
[ 95%] Built target build-dict
[ 96%] Built target fusion-mw
[ 97%] Built target dicc2phon
[ 98%] Built target gz-cat
[ 99%] Built target convert_model
[ 99%] Building CXX object APIs/python2/CMakeFiles/_pyfreeling.dir/freeling_pythonAPIPYTHON_wrap.cxx.obj
freeling_pythonAPIPYTHON_wrap.cxx
c:\python27\include\pyconfig.h(57): warning C4068: unknown pragma
c:\python27\include\pyport.h(187): fatal error C1189: #error: "Python needs a typedef for Py_ssize_t in pyport.h."
NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe"' : return code '0x2'
Stop.

I can provide you any information about what I have done to reach up to this point if that's convenient, but I suppose this might have something to do with my Python environment. I have tried to google for this error but I haven't found anything relevant. Any ideas?

Moltes gràcies!

BR Iván Arias Rodríguez

Yes it looks like some mismatch between SWIG generated code and your python installation.

I can't help you about that...

But maybe it would be wise to start moving to python3 (python2 will finish support in 2020, and python3 is really much better)

Hello and thanks again,

Following your advice, I started using Python 3.7. And finally I managed to compile your code. Apart from fixing my Python environment, I had some problems because I hade copied the FreeLing package using a path that contained spaces (I put it in C:\Program Files\FreeLing-4.1\, and finally had to move it to C:\FreeLing-4.1\).

I will try the FreeLing Python package and I'll see whether I'm able to make it work or not.

Thanks a lot!

BR Iván Arias Rodríguez

ivansiiito

Fri, 10/19/2018 - 14:54

Hello again,

During the last days, I have been trying to make FreeLing work in Python after being able of compiling it, but I haven't been able at all. What is worse, thinking that the problem could come from the SWIG library being installed in a path that contains spaces, I moved it directly to C:\swigwin. I changed the corresponding parameters and tried to recompile everything. But I haven't been able to make it compile again, even if I apparently reverted all the changes. I don't understand what's going on here. The error I get is this:
[...]
[100%] Linking CXX executable convert_model.exe
[100%] Built target convert_model
Scanning dependencies of target pyfreeling_swig_compilation
[100%] Swig source
(1) : Error: Unable to find 'swig.swg'
(3) : Error: Unable to find 'python.swg'
C:\FreeLing-4.1\APIs\python3\freeling_pythonAPI.i(46) : Error: Unable to find 'std_string.i'
C:\FreeLing-4.1\APIs\python3\freeling_pythonAPI.i(47) : Error: Unable to find 'std_wstring.i'
C:\FreeLing-4.1\APIs\python3\..\common\templates.i(8) : Error: Unable to find 'std_list.i'
C:\FreeLing-4.1\APIs\python3\..\common\templates.i(9) : Error: Unable to find 'std_vector.i'
C:\FreeLing-4.1\APIs\python3\..\common\templates.i(10) : Error: Unable to find 'std_map.i'
C:\FreeLing-4.1\APIs\python3\..\common\templates.i(11) : Error: Unable to find 'std_pair.i'
C:\FreeLing-4.1\APIs\python3\..\common\templates.i(14) : Error: Unable to find 'std_set.i'
NMAKE : fatal error U1077: 'echo' : return code '0x1'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe"' : return code '0x2'
Stop.

I don't know the reason why it doesn't find swig.swg and python.swg, because they are in the same place they have always been. I have even set the environment variable SWIG_LIB, pointing to the swigwin\Lib folder (and some others, I have tried all kind of things). But I always get the same error. If I execute swig.exe -swiglib the result is the expected swigwin\Lib folder. Any ideas?

I have been full-time over a week trying to make FreeLing work in Python and so far, I'm quite desperate. Considering the idea that I won't ever be able to make it work this way, is there any easier way to pass a text file full of sentences to FreeLing and get the results?

Any help on this would be highly appreciated.

Thanks!

BR Iván Arias Rodríguez

I don't know what may be going on.. You may have messed something up somewhere...

Building your own program makes sense when you want to build an application that uses FreeLing as a module.  If you only want FreeLing output, you don't need to build a python program.  FreeLing provides a quite flexible front-end that can produce results for most cases.

Check the user manual https://talp-upc.gitbook.io/freeling-4-1-user-manual/analyzer
(on windows, the script is called "analyze.bat" and not just "analyze", but it is used in the same way and accepts the same options --extensively described in the manual--. Also, check the output of "analyze.bat --help")

 

 

ivansiiito

Fri, 10/19/2018 - 17:51

Yes, I think most probably at some point I broke something and I didn't realize.

Well, the point is that I wanted to compare your tagger/lemmatizer with one that I'm creating as part of my PhD (which doesn't work that well, but it is not supervised, and doesn't need any tagged corpus to train with). I have coded everything with Python, so it would be really handy if I could just manage both taggers within the same Python program.

Anyway, I have tried the analyzer.bat that comes in the precompiled Windows package, and most probably I can just get the result as a text file and parse it with Python. In fact, I had already tried the analyzer.bat but I mistakenly thought it didn't work, most probably because I didn't wait long enough for the answer. I hope I can work with FreeLing this way.

Thanks a lot for your help!