jndn

Hi Andrew. I don't have any experience with Maven. Can it be used without locking everyone into using it in order to compile the project?

Jeff, not if done right. We could maintain separate Ant and Maven build scripts (Maven's would be in pom.xml; I already have one created locally) or we could try to keep the Ant script as is and try to use it from within Maven (I'll have to do a bit of research on this but I think it can be done). Any preference?

I chose Ant because it can be install simply with
sudo apt-get install ant

Is Maven that easy to install?

Yes, I think 'sudo apt-get install maven'. The advantage to using Maven comes when you integrate jndn with other projects; just like with NPM in Node, you add the dependency to your project (in Maven's case pom.xml instead of NPM's package.json) and Maven will pull it in. But to do that it has to be registered in that central repository.

One caveat: ideally the integration tests (that rely on the NFD or protobuf) should be in a separate directory; I'm having trouble recommending a best practice but this article (http://www.agile-engineering.net/2011/06/seperating-maven-unit-integration-tests.html) recommends the following dir structure: src, tests, integration-tests

I'll take a look at the unit vs. integration tests. Admittedly, right now they are mixed together.

I tried your pom.xml, but when I run mvn test it doesn't find org.sqlite.JDBC. Can the pom.xml automatically download the sqlite dependency and link to it for the tests?

I had to look it up (http://stackoverflow.com/a/1172371) because I was about to say that you had to hardcode the versions, but that is incorrect; in actuality it's much like NPM where you can set version wildcards to pull the latest release. For now we can probably hardcode it to 3.8.7 and later figure out what versions will always work.

As for not including the dependency, you could always deploy the NDN JAR without the SQLite JAR and it would just fail if it couldn't find the right classes; but by default it would be there, which is what we want, right?

Since the code dynamically loads the SQLite JAR, you are corrected that we could drop the dependency. I only tried this so that I could run the tests which require it? How can pom.xml get the SQLite JAR onto the classpath so that the tests can load it?

Is it a test dependency or a dependency actually needed by jndn code? I'll get on Skype if you want to discuss.

Jeff, I looked into how Maven would do the equivalent of the current build.xml (i.e. check if protobuf is available to optionally compile certain files). After thinking it through, I think our approach was wrong because we were trying to replicate build script functionality (Ant) in a dependency tree (Maven). I found that Maven already has a dependency registered for protobuf (groupId: com.google.protobuf) so we could test incorporating that and maybe just as a test dependency so it isn't required for the final JAR (from what I remember it was only required for some samples?). What do you think?

Protobuf is not just for samples but also the ProtobufTlv support in the src utils folder. The "build script" thinking was so that a developer is not forced to use Maven in their own project. They can use Maven to build ndn.jar, but then use it as they wish. If Maven can't conditionally include ProtobufTlv, do we have to move it out of ndn.jar?

Maybe the best solution is to include the dependency to protobuf by default. But if that's a problem for the developer to support a command-line option for mvn to exclude the protobuf-dependent files. I read some docs on Maven profiles, but it is still not clear. Can we make a profile that includes the protobuf files, then have a command-line option to turn it off?

To first post: Maven can always conditionally include stuff either using profiles or classifiers (from what I read it seems like classifiers are the preferred method for this type of thing because the actual output file name will reflect the inclusion (e.g. jndn-1.0-protobuf.jar with classifiers vs. jndn-1.0.jar with profiles; I don't have much of an opinion on this but thought you should know).

To second post: yes, we could create two profiles, a 'with-protobuf' that is activated by default and a 'without-protobuf' that would run when you do something like 'mvn package -P without-protobuf'.

It may be helpful to put the tentative POM up on a branch so we can comment/test on the real thing.

To second post: yes, we could create two profiles, a 'with-protobuf' that is activated by default and a 'without-protobuf' that would run when you do something like 'mvn package -P without-protobuf'.

Yes, that sounds exactly right. Are you proposing to put up a tentative POM? I could test it on our deployment platforms.

If you put the POM you've been working on a branch, I will add the profiles and we can go from there.

See the following in branch issue/2734-Add-Maven-support which I've been working with. It adds the protobuf dependency and removes the related excludes.

https://github.com/named-data/jndn/blob/issue/2374-Add-Maven-support/pom.xml

Hi Andrew,

I made some bug fixes in a branch. Can you check if "mvn test" works for you now? Pull the latest from GitHub and:

git checkout issue/2375-use-Maven

(The unit tests currently require NFD to be running.)

Hi again Andrew,

I had trouble running mvn -Pwithout-protobuf compile. Then in the without-protobuf section I removed the target and source versions and it worked:
https://github.com/named-data/jndn/commit/25164fab5c769d0da41662eacc25c7d3f4790581

Does that sound right? Do we need the target and source versions? (They are not in the other profiles.)

Jeff, works great! All tests pass in about 59 seconds on an NFD-running machine; are we at a point where we could pull out the tests that require external dependencies (e.g. file system, NFD) to an integration-tests directory? This may be the time before all the tests get intermixed with each other and require a gargantuan effort to separate out.

Also, I have one addition to the POM; I specified the compiler version so that Linux boxes like mine won't try to compile with JDK 1.3 and fail (see properties https://github.com/named-data/jndn/pull/4).

In other news, I finally figured out the bugs in the deploy process to OSSRH and now jndn is showing up in the Central Repository. The good thing now is that that was a one time setup; if we use the POM I created we should just be able to 'mvn deploy' and new versions will be released publicly.

Here's the Maven Central Repository link: http://search.maven.org/#search%7Cga%7C1%7Cnet.named-data

Thanks for the update. Please see my question above for trouble with mvn -Pwithout-protobuf compile. I still have the problem with your updated pom.xml. Do we need 'target' and 'source' in the without-protobuf profile?

... I should say that I have javac 1.8. Even the main profile fails. Can you update pom.xml to support Java 1.8 too? (Can we say version >= 1.7 to future-proof it?)

are we at a point where we could pull out the tests that require external dependencies (e.g. file system, NFD)

Yes, let's move them to integration-tests. These are TestFaceInterestMethods, TestFilePrivateKeyStorage, TestIdentityMethods, TestPolicyManager and TestVerificationRules.

Only TestFaceInterestMethods needs NFD.

I've been trying to replicate what you're seeing but can't; when I 'mvn deploy' I have to have maven.compiler.source and maven.compiler.target set or Maven tries to compile at 1.3 for some reason and fails (which doesn't make sense because maven-compiler-plugin says it defaults to 1.5). What I am thinking is that we have to specify at least a minimum source/target combo or systems like mine will fail.

Some more info on source/target (http://docs.oracle.com/javase/6/docs/technotes/tools/windows/javac.html): bottom line is you are specifying the version the code will compile on (source) and the minimum version the code will run on (target). According to the link, the JVMs are backwards compatible so if you specify the minimum version you need to compile/run (e.g. 1.5) then it will run on 1.8. I think a good idea would be to set it to a version that allows the most freedom for JVMs (something low like 1.5) and only raise it if we start adding Java 8 specific features. Thoughts?

I'm getting this problem on OS X 10.9 with package maven3 from MacPorts. If you're on Linux or Windows then maybe you don't see it.

The link you sent is for javac. But doesn't Maven have its own logic for checking the compiler version?

I'd like to stay with minimum version 1.7 since I think we need that version of the Java libraries.

From what I understand, the values under properties/maven.compiler.source and properties/maven.compiler.target get passed to maven-compiler-plugin which in turn passes them as the -source and -target parameters when it calls javac.

Why don't we set the top-level properties (like in my PR) to 1.7 and remove them from the profile section--does that work? If it doesn't then we can debug from there?

(BTW, I already moved the integration-tests in my branch, so you don't have to do it. https://github.com/named-data/jndn/tree/issue/2375-use-Maven . I'll look at your last message.)

I'm getting the problem even in the default profile which doesn't have its own source and target. Nonetheless I removed them in without-protobuf, and the same problem. When I run

javac -target 1.7 -source 1.7

the javac is happy. But when I run mvn compile, I get the error:

Fatal error compiling: invalid target release: 1.7

Do I know for a fact that Maven runs the same executable that I get when I run javac?

Weird... Can you check your JAVA_HOME or run with 'mvn -X' to see what JDK Maven is using? Maybe it decided to use an earlier version, different than the one your terminal looks at?

Good tip. JAVA_HOME was not set. I searched my hard drive and I found an installation of Java 6 at /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK. I temporarily blasted it and ran mvn compile which said:

JAVA_HOME is not defined correctly.
We cannot execute /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/bin/java

Maven must set JAVA_HOME to look in /System/Library by default.

I had run the OS X updater for Java to install Java 8, and assumed (ha ha ha) that it would update its default Java installation. Anyway, I set JAVA_HOME and Maven works now. Sorry for the rabbit hole.

I merged your pom.xml into the Maven branch.

Next question: How do we get Maven to compile the files in 'examples' so that it can be run be something like the following?

java -cp $CLASSPATH:examples-classes:target/jndn-0.2.jar net.named_data.jndn.tests.TestEncodeDecodeData

Yeah, Maven does so much it's almost impossible to know what's going on under the hood. Once you are ready to deploy make sure you set up your Sonatype account like I described above--you shouldn't have to go through everything I went through but I do know you will have to have an account.

For examples I would recommend creating a series of profiles, one for each example, that would compile only the necessary files (using includes or excludes) and then run them with exec-maven-plugin in the test phase (see http://www.vineetmanohar.com/2009/11/3-ways-to-run-java-main-from-maven/). This way, you could run an example with 'mvn test -P my-example'. For clarity, you could even make a sub-POM with all of these profiles in the examples directory and users would have to cd there to run the maven commands.

Thanks for the feedback. Do you have time to give me an example profile to run TestEncodeDecodeFibEntry? I ask because this example needs FibEntryProto and also the protobuf-java dependency. (I can figure out the simpler profiles from there.) If you don't have time, let me know and I'll give it a try.

Hi Andrew. Thanks for the pom.xml but I couldn't get it to work on OS X. 'mvn install' failed because gpg was not installed. I installed the MacPorts gnupg package but then 'mvn install' failed again with 'secret key not available'.

I don't think we want users to have to install and configure gpg just to run an example. The Ant version (and the other CCL libraries) let the user try an example without running 'install', which helps with experimentation. Is there a way for the examples pom.xml to use the jndn jar which was created by 'mvn package'? Or is there another way to run an example without configuring gpg?

Makes sense; I wouldn't want to install it either. The problem is that the OSSRH deployment requirements use the GPG plugin (and others) to build and these plugins are overriding the normal compiler plugin. I vote that we move all the deployment specific plugins into a separate profile so that deployers must enter 'mvn deploy -P deploy' to run all the extra stuff that OSSRH requires; this way, casual developers can run 'mvn install' with no issues.

Thanks for the quick feedback. Sounds like a good solution. Can you attach a revised pom.xml?

Great! The pom.xml in examples works now. Since we only need to run examples and not deploy, can we remove the 'version' section? I'm worried that it will get out of date.

I think you can remove it because it should inherit from the parent POM (according to http://maven.apache.org/pom.html).

When I removed version, Maven complained. So I simply set the version to 'test'. Also, to avoid confusion I changed the profile name to be spelled the same as the Java class such as TestEncodeDecodeFibEntry. I added all the profiles. Every profile is the same except for the name and Java class. This is OK, but I must ask if it is simple in Maven to use a variable so that we only have one generic profile.

https://github.com/named-data/jndn/blob/b1a8288dd814762881b2114c3f10b7463ed41b41/examples/pom.xml#L51

It makes some assumptions that they are all in the same package, etc., but that simplifies the POM a bit.

That pom.xml for building examples works great. I merged the changes into the main branch so that I could tag release v0.3. When I run mvn deploy -P ossrh, I get a bunch of errors like:

src/net/named_data/jndn/sync/ChronoSync2013.java:[23,27] package com.google.protobuf does not exist

I suspect that the ossrh profile doesn't have the dependencies like protobuf. If you need to test it by making the deployment for 0.3, go ahead.

... I should ask you to first pull the latest from the main jndn branch so that it is the same pom.xml with the changes to the version, etc.

Hi Andrew. I think I fixed it. I added the protobuf dependency section to the ossrh profile. Is that right? https://github.com/named-data/jndn/commit/3d695ce4e54d769340857f7c814d223daaee30be

Now I have to tackle gpg.