FreeLing 4.0 on Windows 10 to use in Java

Submitted by mmineo on Fri, 12/09/2016 - 21:38
Forums

I'm following the instructions here: https://github.com/TALP-UPC/FreeLing/blob/master/APIs/java/README

1. Install java
Done

2. Download and install swig (http://www.swig.org/)
Done

3. Run swig with freeling_javaAPI.i to generate the API SWIG will generate a directory edu/upc/freeling with the Java API for all modules, and a file freeling_javaAPI.cxx with the JNI side of the API.

mkdir -p edu/upc/freeling4

swig -java -c++ -package edu.upc.freeling4 -outdir edu/upc/freeling4 -o freeling_javaAPI.cxx -I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib\java" -I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib\std" -I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib" freeling_javaAPI.i

4. Compile java code in edu/upc/freeling and build a file freeling.jar with the results.

javac .\edu\upc\freeling4\*.java
jar -cf freeling.jar .\edu\upc\freeling4\

5. Compile freeling_javaAPI.cxx into a DLL (using MSVC or any other compiler). You'll need to provide paths to freeling and treeler include directories, and to Java headers (jni.h etc)

cl /LD freeling_javaAPI.cxx /I"C:\Program Files\Java\jdk1.8.0_91\include" /I"C:\Program Files\Java\jdk1.8.0_91\include\win32" /I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\include" /I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\boost\include" /I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\icu\include" /I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\zlib\include"

But I'm getting this error: https://codeshare.io/G6QbNa (sorry, I can't put all the error here)

What can I do?
Thanks!

It seems your compiler is complaining about things like "not", "and", which are valid C++.
So, my guess is that your compiler is not correctly configured.

You are either trying to compile as C (not C++) or maybe you need to add some flag to have it accept this C++ keywords. You probably need to look on your compiler's manual or a forum about it to find out the appropriate options.

If you are using MSVC, it may be helful to have a look at the options in the project files used to compile FreeLing.

I tried with both g++ and cl.exe comman line compiler and both of them give me the same error related with 'getpid'

Using g++
g++ freeling_javaAPI.cxx
-I"C:\Program Files\Java\jdk1.8.0_91\include"
-I"C:\Program Files\Java\jdk1.8.0_91\include\win32"
-I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\include"
-I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\boost\include"
-I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\icu\include"
-I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\zlib\include"
-std=c++11

error
C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\include/freeling/morfo/util.h:162:94: error: 'getpid' was not declared in this scope

Using cl.exe
cl /EHsc /LD freeling_javaAPI.cxx
/I"C:\Program Files\Java\jdk1.8.0_91\include"
/I"C:\Program Files\Java\jdk1.8.0_91\include\win32" /I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\include"
/I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\boost\include"
/I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\icu\include"
/I"C:\Users\Usuario\Documents\Tesis\Nueva carpeta\freeling\dependencies\zlib\include" /TP

error
c:\users\usuario\documents\tesis\nueva carpeta\freeling\include\freeling/morfo/util.h(162): error C3861: 'getpid': identifier not found

The error list is bigger, but I think that is something related with this.
Any idea?

Thanks in advance

To compile FreeLing headers in windows, you need to define compiler symbol WIN32 (.e.g, with g++ you would use command line option -DWIN32)

It may be necessary to add other options. Please check options used in MSVC project file provided with FreeLing sources.

I've been trying to build the DLL for several hours with no luck.

I'll explain here what I did, I hope you can help me.

I started to read again the README files (I think they are outdated, what they state is not correct, eg. paths to files are wrong).

I opened Visual Studio 2015 and opened the solution situated in C:\Users\Usuario\Documents\Tesis\compilacion\freeling\msvc\13.0\FreeLing.sln

I had to update projects to my VS version, added all the dependencies and libraries, also had to install a new version of boost and use those libraries, the one included in the ZIP file doesn't work with my VS version. I clicked on 'Build Solution' and I got 6 'unresolved external symbol' errors

Here you can see the first one
unresolved external symbol "__declspec(dllimport) public: int __thiscall boost::icu_regex_traits::translate(int,bool)const " (__imp_?translate@icu_regex_traits@boost@@QBEHH_N@Z) referenced in function "class boost::u16_to_u32_iterator > >,int> __fastcall boost::re_detail_106200::re_is_set_member > >,int>,int,class boost::icu_regex_traits,unsigned __int64>(class boost::u16_to_u32_iterator > >,int>,class boost::u16_to_u32_iterator > >,int>,struct boost::re_detail_106200::re_set_long const *,struct boost::re_detail_106200::regex_data const &,bool)" (??$re_is_set_member@V?$u16_to_u32_iterator@V?$_String_const_iterator@V?$_String_val@U?$_Simple_types@_W@std@@@std@@@std@@H@boost@@HVicu_regex_traits@2@_K@re_detail_106200@boost@@YI?AV?$u16_to_u32_iterator@V?$_String_const_iterator@V?$_String_val@U?$_Simple_types@_W@std@@@std@@@std@@H@1@V21@0PBU?$re_set_long@_K@01@ABU?$regex_data@HVicu_regex_traits@boost@@@01@_N@Z)

I didn't know what to do with that, so I decided to try to build the DLL for Java on Windows using the command line.
Reading the README file I found this:

3. Open msvc project named msvc/10.0/swig/java/freeling_javaAPI
Click right button over freeling_javaAPI.i

That project does not exists.

So I decided to build the DLL by my means, and my idea is to build it as it is created in Mac and Linux, so I followed the Makefile and did those steps in Windows.

1.
mkdir -p edu/upc/freeling4
2.
swig -java -c++ -package edu.upc.freeling4 -outdir edu/upc/freeling4 -o freeling_javaAPI.cxx
-I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib\java"
-I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib\std"
-I"C:\Users\Usuario\Documents\Tesis\swigwin-3.0.10\Lib" freeling_javaAPI.i
3.
javac .\edu\upc\freeling4\*.java
4.
jar -cf freeling.jar .\edu\upc\freeling4\

Steps 5 and 6 following http://www.mingw.org/wiki/sampledll
g++ -c -DBUILDING_EXAMPLE_DLL example_dll.cpp
g++ -shared -o example_dll.dll example_dll.o -Wl,--out-implib,libexample_dll.a

and http://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/
z:\Users\mpayne\Documents\MinGWDLL>gcc -c -o add.o add.c -D ADD_EXPORTS
z:\Users\mpayne\Documents\MinGWDLL>gcc -o add.dll add.o -s -shared -Wl,--subsystem,windows
5.
g++ -c -o .\freeling_javaAPI.o .\freeling_javaAPI.cxx
-I"C:\Program Files\Java\jdk1.8.0_91\include"
-I"C:\Program Files\Java\jdk1.8.0_91\include\win32"
-I"C:\Users\Usuario\Documents\Tesis\command_line\FreeLing-4.0\src\include"
-I"C:\Boost\include\boost-1_62"
-I"C:\Users\Usuario\Documents\Tesis\command_line\FreeLing-4.0\dependencies\icu\include"
-I"C:\Users\Usuario\Documents\Tesis\command_line\FreeLing-4.0\src\libfoma"
-I"C:\Users\Usuario\Documents\Tesis\command_line\FreeLing-4.0\dependencies\zlib\include"
-I"C:\Users\Usuario\Documents\Tesis\command_line\FreeLing-4.0\src\libtreeler"
-std=c++11
6.
g++ -o .\freeling_javaAPI.dll -s -shared .\freeling_javaAPI.o

and I got this long list of errors https://codeshare.io/21jQn5

Finally, I decided to build the DLL using a MacBook, maybe I was lucky doing a cross compiling.
I installed in my Mac a g++ cross compiler from Mac to Windows and edited the Makefile to work with these new configurations

# Compile the C++ API
libfreeling_javaAPI.dylib: freeling_javaAPI.cxx
$(GCC) /Users/martinmineo/Desarrollo/Tesis/Entorno/FreeLing/APIs/javaWIN/lib/libfreeling.dll -mdll -o libfreeling_javaAPI.dll freeling_javaAPI.cxx -I/Users/martinmineo/src/gcc-4.8.5/libjava/classpath/include -I/Users/martinmineo/src/gcc-4.8.5/libjava/include -I/Users/martinmineo/src/gcc-4.8.5/libjava -I$(FREELINGDIR)/include -I$(FREELINGDIR)/include/treeler -I$(FREELINGDIR)/src/include -I$(FREELINGDIR)/src/libfoma -I$(FREELINGDIR)/src/libtreeler -I$(JAVADIR)/include -I/opt/local/include/ -I$(JAVADIR)/include/linux -I/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers -std=c++11
$(JAVADIR)/bin/javac $(PACKAGE)/*.java
$(JAVADIR)/bin/jar -cf freeling.jar $(PACKAGE)

and got almost the same error as in Windows: https://codeshare.io/5Dd4Z5

All this is making me think that the error is in FreeLing and not in my side. But I'm not sure, could you please tell me what to do?

Many thanks!
And sorry for the long post

FreeLing is a C++ version developed in Linux.

You want to use it on windows, from java, and via a cross-compiler in Mac, and if it does not work, you blame the original code???
The easier explanation is that there are too many intermediate steps, and an error in any of them may break things.

Also, you are using the wrong README. In the latest version (4.0) there is no mention to
"3. Open msvc project named msvc/10.0/swig/java/freeling_javaAPI".
Please stick to the appropriate documentation for each version.

Finally, "unresolved external" means that the linker is missing a library, so, you need to add it to the libraries to be linked.

E.g. "unresolved external symbol "__declspec(dllimport) public: int __thiscall boost::icu_regex_traits::tr.."
refers to a symbol in libicu library (which is used by libboost)

So, you need to add this library to the path (which you may need to change, if you used a different boost version).

Sorry if I offended you.

I could compile Freeling in MSVC folder and then create the DLL file.

To do it, I had to install Visual Sturdio 2013 (v12)

Thanks!

Repository contains source code.
Binaries are packaged in the download section.

APIs for other languages are not included in binaries, because they are dependent on the installed version of java/python/perl/ruby/... So, each user needs to build the API to match his system.