Project

General

Profile

Task #1988

Compile NFD on Android NDK

Added by Mohamed Hail about 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Android
Start date:
Due date:
% Done:

100%

Estimated time:
80.00 h

Description

Cross-compile NFD and necessary dependencies on Android platform with NDK.


Related issues

Blocks NFD-android - Task #1989: Basic NFD control app for AndroidClosedAlex Afanasyev01/27/2015

Actions
Blocks NFD-android - Task #2356: Run NFD on unrooted Android phoneClosedIvan Yeo

Actions
#1

Updated by Jeff Thompson about 6 years ago

  • Project changed from ndn-js to NFD
#2

Updated by Jeff Thompson about 6 years ago

  • Priority changed from High to Normal
#3

Updated by Jeff Thompson about 6 years ago

Hello Mohamed. I started working on compiling NFD in the Android NDK.

Has anyone else on the NFD team looked into compiling NFD on Android?

  • Jeff T
#4

Updated by Junxiao Shi about 6 years ago

  • Subject changed from NFD for Android? to Compile NFD on Android NDK
  • Description updated (diff)
  • Category set to Build
  • Status changed from New to In Progress
  • Assignee set to Yi Huang
  • Target version set to Unsupported
  • Start date deleted (09/19/2014)
  • Estimated time set to 80.00 h
#5

Updated by Junxiao Shi about 6 years ago

  • Blocks Task #1989: Basic NFD control app for Android added
#6

Updated by Junxiao Shi about 6 years ago

  • Tracker changed from Feature to Task

@Yi is working on Android cross-compiling. His assignment is limited to compiling and running NFD. Once this task completes, Android user needs to use a bash application to control NFD.

#7

Updated by Mohamed Hail about 6 years ago

Hi Jeff,

do you know, how long time do you need for that. I remember the CCNx was able to run on Android! Are there a lot to do to compile NFD on Android NDK?

Mohamed

#8

Updated by Jeff Thompson about 6 years ago

Yi, do you have an update for cross-compiling?

#9

Updated by Yi Huang about 6 years ago

The libraries in dependency are compiled (boost, crypto++, sqlite3). But I had some difficulty compiling ndn-cxx library. Waf is not generating correct command to check crypto++.

From the config.log:

----------------------------------------
Checking Crypto++ lib
562
----------------------------------------
Checking if CryptoPP library works
==>
#include <cryptopp/config.h>

int main(int argc, char **argv) {
    (void)argc; (void)argv;
    return 0;
}

<==
[1/2] ^[[32mcxx: build/.conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/test.cpp -> build/.              
conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/testbuild/test.cpp.1.o
^[[0m
['arm-linux-androideabi-g++', '-O2', '--sysroot=/Users/yihuang/nfd/android-compile/android-ndk-r10/  
platforms/android-19/arch-arm', '-I/opt/local/include', '-DHAVE_SQLITE3=1', '../test.cpp', '-c', '-  
o', '/Users/yihuang/nfd/ndn-cxx/build/.conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/testbuild/test.   
cpp.1.o']
[2/2] ^[[33mcxxprogram: build/.conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/testbuild/test.cpp.1.o -> 
build/.conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/testbuild/testprog
^[[0m
['arm-linux-androideabi-g++', '/Users/yihuang/nfd/android-compile/android-ndk-r10/platforms/android- 
19/arch-arm', 'test.cpp.1.o', '-o', '/Users/yihuang/nfd/ndn-cxx/build/.                              
conf_check_52ad30e0e3d6c12cb52f09bc6f61f03c/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '- 
L/opt/local/lib', '-lcryptopp']
err: /Users/yihuang/nfd/android-compile/android-ndk-r10/toolchains/arm-linux-androideabi-4.8/        
prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/   
bin/ld: error: cannot open crtbegin_dynamic.o: No such file or directory
/Users/yihuang/nfd/android-compile/android-ndk-r10/toolchains/arm-linux-androideabi-4.8/prebuilt/    
darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld:     
fatal error: /Users/yihuang/nfd/android-compile/android-ndk-r10/platforms/android-19/arch-arm: pread 
failed: Is a directory
collect2: error: ld returned 1 exit status

The second command shouldn't be looking like that. the second part should be "--sysroot=/Users/yihuang/nfd/android-compile/android-ndk-r10/platforms/android-19/arch-arm". But I haven't found an option when running waf that can make the change.

#10

Updated by Junxiao Shi about 6 years ago

Try modifying crypto++ detection tool .waf-tools/cryptopp.py to insert the arguments you need.

#11

Updated by Alex Afanasyev about 6 years ago

You should add --sysroot... settings to LDFLAGS environment variable too. CXXFLAGS is only active for compilation stage and is not used during linking (which is failing).

#12

Updated by Alex Afanasyev about 6 years ago

I have finally compiled cxx library, but it required a number of tweaks. i have pushed my updates to http://gerrit.named-data.net/#/c/1271/

Haven't tried nfd yet.

#13

Updated by Alex Afanasyev about 6 years ago

With http://gerrit.named-data.net/#/c/1272/ was able to compile NFD. Will stop at this point :)

#14

Updated by Jeff Burke almost 6 years ago

Now that this compiles, what is remaining to have this tested and available for use?

#15

Updated by Yi Huang almost 6 years ago

I am working on testing it on a real Android device.

#16

Updated by Lan Wang almost 6 years ago

what's the current status?

#17

Updated by Yi Huang almost 6 years ago

Alex and I are working on this issue since the existing modification does not work for me.

On Android end, I found a way to run cross-compiled C binary using an Android Application wrapper.

#18

Updated by Yi Huang almost 6 years ago

I have set up a VM to do the cross-compilation and Alex has build NFD on it. I am now trying to use my Android app (as a wrapper of NFD native binaries) to run it on a unrooted Android device.

There is one thing I am not sure with. Since NFD cannot run as root in unrooted Android devices, I cannot use default unix socket (not enough permissions). To use a unix socket in user space, do I have to create the socket before NFD starts? Or it is sufficient just to change the unix socket path in config file?

#19

Updated by Junxiao Shi almost 6 years ago

Since NFD cannot run as root in unrooted Android devices, I cannot use default unix socket (not enough permissions). To use a unix socket in user space, do I have to create the socket before NFD starts? Or it is sufficient just to change the unix socket path in config file?

Creating the socket before NFD starts still requires you to have root privilege.

I suggest defining an alternate location for the UNIX socket where only NFD.apk can write, but other apps can access.

It /data/data/net.named-data.nfd feasible for this purpose?

An alternative is to disable UNIX socket altogether and use TCP exclusively.

#20

Updated by Alex Afanasyev almost 6 years ago

As I remember, Java doesn't have (without using native code) UNIX socket support, so I would agree with Junxiao that this can be simply disabled and TCP could be default transport for local communications.

#21

Updated by Yi Huang almost 6 years ago

NFD still does not work when Unix socket is disabled.

I've tried both setting "listen" to "no" and comment out the entire unix section. Here are the error message:

When I set "listen" to "no":

; The unix section contains settings of UNIX stream faces and channels.
unix
{
  listen no ; set to 'no' to disable UNIX stream listener, default 'yes'
  ;path /var/run/nfd.sock ; UNIX stream listener path
}

The output was:

root@maguro:/nfd # ./nfd --config nfd.conf
1416000778.908610 INFO: [StrategyChoice] Set default strategy /localhost/nfd/strategy/best-route/%FD%02
1416000778.910013 INFO: [StrategyChoiceEntry] Set strategy /localhost/nfd/strategy/best-route/%FD%02 for / prefix
1416000778.913584 INFO: [InternalFace] registering callback for /localhost/nfd/fib
1416000778.914499 INFO: [InternalFace] registering callback for /localhost/nfd/faces
1416000778.915385 INFO: [InternalFace] registering callback for /localhost/nfd/strategy-choice
1416000778.916392 INFO: [InternalFace] registering callback for /localhost/nfd/status
1416000778.917277 INFO: [FaceTable] Added face id=1 remote=internal:// local=internal://
1416000778.996592 WARNING: [CommandValidator] Wildcard identity is intended for demo purpose only and SHOULD NOT be used in production environment
1416000778.996866 INFO: [CommandValidator] Giving privilege "faces" to identity wildcard
1416000778.997080 INFO: [CommandValidator] Giving privilege "fib" to identity wildcard
1416000778.997324 INFO: [CommandValidator] Giving privilege "strategy-choice" to identity wildcard
1416000778.998331 INFO: [TablesConfigSection] Setting CS max packets to 65536
1416000779.029886 FATAL: [NFD] boost::filesystem::canonical: No such file or directory: "/var/run"

When I comment out entire Unix section, the output was:

root@maguro:/nfd # ./nfd --config nfd.conf
1416003371.089335 INFO: [StrategyChoice] Set default strategy /localhost/nfd/strategy/best-route/%FD%02
1416003371.090769 INFO: [StrategyChoiceEntry] Set strategy /localhost/nfd/strategy/best-route/%FD%02 for / prefix
1416003371.094340 INFO: [InternalFace] registering callback for /localhost/nfd/fib
1416003371.095255 INFO: [InternalFace] registering callback for /localhost/nfd/faces
1416003371.096171 INFO: [InternalFace] registering callback for /localhost/nfd/strategy-choice
1416003371.097239 INFO: [InternalFace] registering callback for /localhost/nfd/status
1416003371.098154 INFO: [FaceTable] Added face id=1 remote=internal:// local=internal://
1416003371.177653 WARNING: [CommandValidator] Wildcard identity is intended for demo purpose only and SHOULD NOT be used in production environment
1416003371.177927 INFO: [CommandValidator] Giving privilege "faces" to identity wildcard
1416003371.178141 INFO: [CommandValidator] Giving privilege "fib" to identity wildcard
1416003371.178385 INFO: [CommandValidator] Giving privilege "strategy-choice" to identity wildcard
1416003371.179362 INFO: [TablesConfigSection] Setting CS max packets to 65536
1416003371.211405 FATAL: [NFD] resolve: Invalid argument
#22

Updated by Junxiao Shi almost 6 years ago

note-21 behavior with "listen=no" is due to Bug #2188.

note-21 behavior without unix section is caused by some other problem which might be specific to Android NDK.

Please comment out the try-catch statements in main.cpp, and obtain a stacktrace to find out where is the exception thrown.

#23

Updated by Yi Huang almost 6 years ago

I have commented out all try/catch in main.cpp

The error message is:

root@maguro:/ndn # ./nfd --config nfd.conf
1416372692.342025 INFO: [StrategyChoice] Set default strategy /localhost/nfd/strategy/best-route/%FD%02
1416372692.354141 INFO: [StrategyChoiceEntry] Set strategy /localhost/nfd/strategy/best-route/%FD%02 for / prefix
1416372692.361678 INFO: [InternalFace] registering callback for /localhost/nfd/fib
1416372692.364974 INFO: [InternalFace] registering callback for /localhost/nfd/faces
1416372692.367172 INFO: [InternalFace] registering callback for /localhost/nfd/strategy-choice
1416372692.369399 INFO: [InternalFace] registering callback for /localhost/nfd/status
1416372692.372024 INFO: [FaceTable] Added face id=1 remote=internal:// local=internal://
1416372692.452895 WARNING: [CommandValidator] Wildcard identity is intended for demo purpose only and SHOULD NOT be used in production environment
1416372692.453140 INFO: [CommandValidator] Giving privilege "faces" to identity wildcard
1416372692.453628 INFO: [CommandValidator] Giving privilege "fib" to identity wildcard
1416372692.454360 INFO: [CommandValidator] Giving privilege "strategy-choice" to identity wildcard
1416372692.455581 INFO: [TablesConfigSection] Setting CS max packets to 65536
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  resolve: Invalid argument
Segmentation fault
#24

Updated by Junxiao Shi almost 6 years ago

Please obtain a stacktrace, not just the error type. You may need to use GDB or its NDK equivalent.

#25

Updated by Yi Huang almost 6 years ago

There is no such thing like gdb built-in on Android since no native code should be run from command line. I'll try to cross compile gdb on Android then.

#26

Updated by Junxiao Shi almost 6 years ago

There is no such thing like gdb built-in on Android

This statement is false. See http://www.kandroid.org/ndk/docs/NDK-GDB.html

#27

Updated by Yi Huang almost 6 years ago

ndk-gdb works on the computer side. It is not like the gdb we used on our development boxes. There are three requirements that we should meet to use ndk-gdb and we simply don't have the first one:

Your application is built with the 'ndk-build' script:
    Building with the legacy "make APP=<name>" method is not supported by ndk-gdb.
#28

Updated by Yi Huang almost 6 years ago

I took Alex's advice and commented out "#if not defined(__FreeBSD__)" blocks in "core/resolver.hpp".

Looks like NFD is now running on Android (I don't see any error message and when Ctrl-C, log "[NFD] Caught signal 'Interrupt', exiting..." is displayed and NFD exits). However, I currently have no idea on what to do to make sure NFD is running correctly because we turned unix socket off and NRD can't even work without unix socket.

Here is the result when I run NFD with cross-compiled valgrind:

==3028== Memcheck, a memory error detector
==3028== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==3028== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==3028== Command: ./nfd --config ./nfd.conf
==3028==
946688329.031616 INFO: [StrategyChoice] Set default strategy /localhost/nfd/strategy/best-route/%FD%02
946688329.148376 INFO: [StrategyChoiceEntry] Set strategy /localhost/nfd/strategy/best-route/%FD%02 for / prefix
946688330.169555 INFO: [InternalFace] registering callback for /localhost/nfd/fib
946688330.274322 INFO: [InternalFace] registering callback for /localhost/nfd/faces
946688330.327728 INFO: [InternalFace] registering callback for /localhost/nfd/strategy-choice
946688330.352569 INFO: [InternalFace] registering callback for /localhost/nfd/status
946688330.414367 INFO: [FaceTable] Added face id=1 remote=internal:// local=internal://
946688336.265686 WARNING: [CommandValidator] Wildcard identity is intended for demo purpose only and SHOULD NOT be u
946688336.282440 INFO: [CommandValidator] Giving privilege "faces" to identity wildcard
946688336.294342 INFO: [CommandValidator] Giving privilege "fib" to identity wildcard
946688336.295196 INFO: [CommandValidator] Giving privilege "strategy-choice" to identity wildcard
946688336.324249 INFO: [TablesConfigSection] Setting CS max packets to 1024
946688337.138366 WARNING: [CommandValidator] Wildcard identity is intended for demo purpose only and SHOULD NOT be u
946688337.139404 INFO: [CommandValidator] Giving privilege "faces" to identity wildcard
946688337.359161 INFO: [CommandValidator] Giving privilege "fib" to identity wildcard
946688337.363708 INFO: [CommandValidator] Giving privilege "strategy-choice" to identity wildcard
946688337.507202 INFO: [FaceTable] Added face id=255 remote=null:// local=null://
946688339.089782 INFO: [FaceTable] Added face id=254 remote=contentstore:// local=contentstore://
946688558.058013 INFO: [NFD] Caught signal 'Interrupt', exiting...
==3028==
==3028== HEAP SUMMARY:
==3028==     in use at exit: 114,740 bytes in 481 blocks
==3028==   total heap usage: 7,886 allocs, 7,405 frees, 836,935 bytes allocated
==3028==
==3028== LEAK SUMMARY:
==3028==    definitely lost: 1,124 bytes in 6 blocks
==3028==    indirectly lost: 113,506 bytes in 471 blocks
==3028==      possibly lost: 110 bytes in 4 blocks
==3028==    still reachable: 0 bytes in 0 blocks
==3028==         suppressed: 0 bytes in 0 blocks
==3028== Rerun with --leak-check=full to see details of leaked memory
==3028==
==3028== For counts of detected and suppressed errors, rerun with: -v
==3028== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
#29

Updated by Alex Afanasyev almost 6 years ago

You can just take ndn-cxx commit that is not yet merged http://gerrit.named-data.net/#/c/1524/

Or simply update ndn-cxx source code manually to use different transport in the default constructor.

#30

Updated by Yi Huang almost 6 years ago

Alex and I are able to run nfd, nrd, and nfdc on Android.

The only problem we saw is that DNS resolution in nfdc does not work.

After starting nfd and nrd, I have registered prefix "ndn/edu/arizona" with "tcp4://128.196.203.36:6363" using nfdc (The IP address is the address for hobo.cs.arizona.edu, but DNS resolution is not working). Then "ndn-tlv-peek -p /ndn/edu/arizona" is able to print response "NDN TLV Ping Response".

Alex has prepared an archive that contains cross-compiled binaries and configuration files modified to disable unix socket and set default transport to tcp. Where should I upload this archive to?

The wiki page (http://redmine.named-data.net/projects/ndn-embedded/wiki/Android) will be updated soon.

#31

Updated by Junxiao Shi over 5 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

In 20150107 conference call, Alex confirms that NFD can be successfully cross-compiled and can run on rooted Android phone. Thus, this Task is complete.

#2356 is a continuation of this Task which aims that running NFD on an unrooted Android phone.

#32

Updated by Junxiao Shi over 5 years ago

  • Blocks Task #2356: Run NFD on unrooted Android phone added
#33

Updated by Alex Afanasyev over 5 years ago

  • Project changed from NFD to ndn-embedded
  • Category changed from Build to Android
  • Target version deleted (Unsupported)

Also available in: Atom PDF