Cross-compiling NDN projects for Raspberry Pi » History » Revision 31
Revision 30 (Wentao Shang, 04/10/2014 11:56 AM) → Revision 31/48 (Wentao Shang, 04/10/2014 11:56 AM)
Cross-compiling NDN projects for Raspberry Pi
=============================================
Note: before reading this document, you should already be familiar with the basic concepts of compiling and linking (especially the linking part). If not, you may be interested in reading this great book: [Linkers and Loaders](http://www.amazon.com/Linkers-Kaufmann-Software-Engineering-Programming/dp/1558604960/ref=sr_1_1?ie=UTF8&qid=1394130356&sr=8-1&keywords=linker+and+loader)
Basic idea
----------
Remember to compile a C/C++ project, we need the source code for the project, the header files for the included libraries, the binary objects of the libraries, and the compiler tools (gcc, as, ld, etc.). The combination of the last three things together is referred to as a _building environment_. Cross-compiling is no different. To cross compile a project, we first need to setup the building environment and then build the source code in that environment.
Difference between "native compiling" and "cross compiling"
--------
The biggest difference is that for native compiling, you build the binaries that will run on the same platform where you build them. For cross compiling, however, you build the binaries on one platform (called _build_ platform) and run them on another platform (called _host_ or _target_ platform). The platforms may differ in the operating systems (Windows vs. Linux) and/or the CPU architectures (x86_64 vs. arm32).
Raspberry Pi platform information
--------
Raspberry Pi runs on *ARMv6* CPU, which is a 32bit chip with hardware float-point support (abbreviated as *armhf*). There are many operating systems available. The one we are going to use is called *Raspbian*, which is a port of the Debian "wheezy" Linux distribution.
Creating compiling toolchain for Raspberry Pi
--------
To prepare a building environment, we need to get the gcc/g++ compiler toolchain that will generate binaries for the armhf platform. Raspbian already provided a set of compiling tools on their official [github](https://github.com/raspberrypi/tools). However, by the time of this writing, those tools only run on 32bit Linux systems. If we want to use 64bit Linux as the build platform, we may need to build our own gcc/g++ toolchain.
The tool we are going to use to build our gcc is [ct-ng](http://crosstool-ng.org/). It is designed to run on Linux but can be adjusted via some hacks to run on MacOSX. It is pretty easy to use and you may find this [article](http://www.kitware.com/blog/home/post/426) very helpful when building your own toolchain.
Tip: you may use the configuration file from Raspbian [github](https://github.com/raspberrypi/tools/blob/master/configs/bcm2708hardfp-ct-ng.config), which will load the "official" configurations for the platform.
Getting libraries ready
--------
After we have the toolchain, the next step is to gather the libraries, including the header files (_.h_ files) and the binaries (_.a, .so_ files). The libraries used by the NDN projects include *openssl, Boost, sqlite3, crypto++*. Those libraries may recursively depend on other libraries and it will be a big headache to compile all of them from source code and resolve the dependencies manually. Fortunately, Raspbian has a public package repo containing the binaries for most packages available on Debian. So the easy walk-around is to _install_ those libraries directly on Raspberry Pi using _apt-get_ and then copy the relevant files down to our build machine.
On Raspbian, all the header files are in the */usr/include* folder, while the binary objects for the libraries are scattered in many places, such as */usr/lib, /lib*, etc. A simple way to find the path of a library is to run the following command:
ldconfig -p | grep _library_name_
*ldconfig -p* will show all the dynamic linking libraries currently available on the system and their locations in the file system.
You may copy all the binaries into the same folder. For example, on the build machine you may have the folder _~/pi/_ and inside that folder there are two sub-folders _./include_ and _./lib_. You can copy all the header files into the include folder and the binaries into the lib folder.
Building the source code
--------
The final step is to compile the projects using the environment we created. The basic idea is to call the cross toolchain and link against the cross-compiled libraries. To do that, we need to export a set of shell variables that are used by the *make* command. For example, we can export the following variables in the shell:
export AS=/path/to/your/toolchain/as
export LD=/path/to/your/toolchain/ld
export CC=/path/to/your/toolchain/cc
export CXX=/path/to/your/toolchain/c++
This will tell *make* to use the toolchain we specify instead of the default toolchain.
In order to point the toolchain to the correct location to search libraries, we also need to export *CFLAGS/CPPFLAGS* and *LDFLAGS* variables, which will be provided to *gcc* as options:
export CFLAGS="-I/path/to/your/include/folder -L/path/to/your/lib/folder -Wl,-rpath=/path/to/your/lib/folder"
export LDFLAGS="-L/path/to/your/lib/folder -Wl,-rpath=/path/to/your/lib/folder"
Note that we repeat the *LDFLAGS* in the *CFLAGS*. This is because some Makefile script may combine the compiling and linking steps together and use only *CFLAGS*. (This actually happens when building the NDNx project.)
Another important option is the _rpath_ option. This specifies where the linker will search for recursive dependencies.
Notes on Waf build system
--------
*waf* is a great build system. But sometimes it can be too smart. When it tries to detect libraries like openssl and boost, it will create a small temporary source code and try to compile it. In most cases, it will also try to run the compiled program in order to check library linkage. This is problematic for cross-compiling since the compiled binaries cannot be run on the compiling machine (they are targeted to a different architecture!).
Fortunately, this behavior is configurable and easy to change. Just look into the *./.waf-tools/* folder under your project root directory, which is the location for scripts that perform the library checking tasks. Go through those files, look for statements like 'self.check_cc ()' function calls, and change the parameter 'execute=True' to 'execute=False'. This tells *waf* not to execute the compiled program during library checking process.