Project

General

Profile

Actions

Task #4573

closed

Detect architecture-specific location for Boost libs on Ubuntu

Added by Eric Newberry about 6 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Build
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:

Description

.waf-tools/boost.py needs to be modified in each package to search for Boost libraries in the /usr/lib/[arch] directory on Ubuntu. Currently, the script is hardcoded for amd64 and i386. However, we should detect the architecture automatically to provide easier cross-platform support. At the moment, this is necessary for armhf and aarch64 support.


Related issues 1 (0 open1 closed)

Blocks NFD - Task #4488: Add ARM slaves to JenkinsClosedEric Newberry

Actions
Actions #1

Updated by Eric Newberry about 6 years ago

  • Blocks Task #4488: Add ARM slaves to Jenkins added
Actions #2

Updated by Eric Newberry about 6 years ago

  • Subject changed from Add Boost library locations for to Add Boost library locations for ARM
Actions #3

Updated by Junxiao Shi about 6 years ago

search for Boost libraries in the /usr/lib/arm-linux-gnueabihf directory (for armhf)

None of these folders should appear in the build script. Moreover, folders for x86 and amd64 should be removed. Instead, use dpkg-architecture -qDEB_HOST_MULTIARCH to determine the multiarch tuple.
https://github.com/named-data/ppa-packaging/issues/15#issuecomment-269842891

Actions #4

Updated by Alex Afanasyev about 6 years ago

I don't think we should put distribution-specific commands inside the build script. For PPA scripts, it is ok, as they are for the specific Linux distribution type.

Actions #5

Updated by Junxiao Shi about 6 years ago

I don't think we should put distribution-specific commands inside the build script.

Is there any non-Debian-derived distribution that use a library directory with multiarch tupe?

Actions #6

Updated by Eric Newberry about 6 years ago

I agree with Alex here. What happens if we run these commands on macOS? We might get weird values for the directory or there might (theoretically) be another program with the same name that produces different output.

Actions #7

Updated by Junxiao Shi about 6 years ago

What happens if we run these commands on macOS?

You should never run dpkg-architecture on a non-Debian-derived system, nor should you add a path like /usr/lib/tuple because this path is only on Debian.

there might (theoretically) be another program with the same name that produces different output

This shouldn't be a problem. Otherwise, same can be said for pkg-config and g++.

Actions #8

Updated by Eric Newberry about 6 years ago

Junxiao Shi wrote:

What happens if we run these commands on macOS?

You should never run dpkg-architecture on a non-Debian-derived system, nor should you add a path like /usr/lib/tuple because this path is only on Debian.

I don't think we can say with 100% certainty that this path format for libraries is only used on Debian.

Then we also have the task of detecting the specific distribution of Linux we're running on from Python, which could prove to be difficult.

Actions #9

Updated by Junxiao Shi about 6 years ago

I don't think we can say with 100% certainty that this path format for libraries is only used on Debian.

As I asked on note-5, please give an example on a non-Debian-derived system using multiarch tuples.
Without a specific example, I'll be against adding any new tuples, but I won't object keeping the existing two tuples.

Then we also have the task of detecting the specific distribution of Linux we're running on from Python, which could prove to be difficult.

The existence of dpkg-architecture indicates Debian (see also note-7 second part). It is part of dpkg-dev package that is a dependency of build-essential package, so NFD won't gain a new dependency.

Actions #10

Updated by Eric Newberry about 6 years ago

Junxiao Shi wrote:

Then we also have the task of detecting the specific distribution of Linux we're running on from Python, which could prove to be difficult.

The existence of dpkg-architecture indicates Debian (see also note-7 second part). It is part of dpkg-dev package that is a dependency of build-essential package, so NFD won't gain a new dependency.

Yes, but how do we easily check for the existence of dpkg-architecture from within Python?

Actions #11

Updated by Alex Afanasyev about 6 years ago

Yes, but how do we easily check for the existence of dpkg-architecture from within Python?

That's actually not a big problem. This can be done as detecting any other program (like pkg-config) in waf.

Actions #12

Updated by Eric Newberry about 6 years ago

  • Status changed from New to Code review
  • % Done changed from 0 to 10

I submitted a patchset for NFD (https://gerrit.named-data.net/#/c/4654/) implementing automatic architecture detection using dpkg-architecture.

Actions #13

Updated by Eric Newberry about 6 years ago

I've submitted a pull request for this upstream (https://github.com/waf-project/waf/pull/2146).

I'll work on commits for the rest of the projects actively built on Jenkins soon.

Actions #14

Updated by Eric Newberry about 6 years ago

  • % Done changed from 10 to 100

I've pushed changes for all relevant projects built on Jenkins in the last year (NFD, ndn-cxx, ndn-tools, NLSR, ChronoSync, ChronoShare, ndn-atmos, ndn-group-encrypt, ndncert, ndns, and repo-ng). I am uncertain if ndnSIM will also need this patch.

Actions #15

Updated by Eric Newberry about 6 years ago

  • % Done changed from 100 to 60

One of the waf maintainers pointed out that Utils.subprocess.check_output was only added in Python 2.7. We require >=2.6, so I'll modify all the changes to work on 2.6.

Actions #16

Updated by Davide Pesavento about 6 years ago

Eric Newberry wrote:

We require >=2.6, so I'll modify all the changes to work on 2.6.

*We* actually require >= 2.7, and we already use check_output elsewhere, but it's better to use the same patch that upstream will eventually merge.

Actions #17

Updated by Alex Afanasyev about 6 years ago

I don't care about python 2.6, it is dead. Python 2.7 is on life support also, but as long as our code works with it (with a few non-critical exception), I'm fine.

Actions #18

Updated by Eric Newberry about 6 years ago

Davide Pesavento wrote:

Eric Newberry wrote:

We require >=2.6, so I'll modify all the changes to work on 2.6.

*We* actually require >= 2.7, and we already use check_output elsewhere, but it's better to use the same patch that upstream will eventually merge.

We should update the ndn-cxx INSTALL.rst then, because it says Python >= 2.6.

Actions #19

Updated by Alex Afanasyev about 6 years ago

We should update the ndn-cxx INSTALL.rst then, because it says Python >= 2.6.

Sure.

Actions #20

Updated by Eric Newberry about 6 years ago

  • % Done changed from 60 to 70

My pull request has been merged into upstream. I'll begin integrating the upstream code into the remaining projects that have not already had their change merged.

Actions #21

Updated by Eric Newberry about 6 years ago

  • Subject changed from Add Boost library locations for ARM to Detect architecture-specific location for Boost libs on Ubuntu
  • Description updated (diff)
Actions #22

Updated by Eric Newberry about 6 years ago

  • % Done changed from 70 to 100

All changes have been updated and are ready for review.

Actions #23

Updated by Eric Newberry about 6 years ago

Changes have been merged except for ndn-cxx (failed due to Jenkins issues) and projects that appear broken for other reasons (ChronoShare, ndn-atmos, and ndn-group-encrypt).

Actions #24

Updated by Eric Newberry about 6 years ago

ndn-group-encrypt has been replaced with name-based-access-control, so I've pushed a change for that project (Alex abandoned the one for ndn-group-encrypt).

Actions #25

Updated by Eric Newberry about 6 years ago

  • Status changed from Code review to Closed

All changes have been merged.

Actions

Also available in: Atom PDF