Task #5040: Deploy CentOS 8 Jenkins agents - NFD - NDN project issue tracking system

Actions

Copy link

Task #5040

closed

Deploy CentOS 8 Jenkins agents

Added by Davide Pesavento over 4 years ago. Updated about 4 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Davide Pesavento

Category:

Build

Target version:

22.02

Start date:

Due date:

% Done:

100%

Estimated time:

Description

Installing the NDN software stack on CentOS 7.x is a pain due to the very old toolchain and the version of various libraries that are shipped by default with the distribution. The SANDIE project is quickly migrating to CentOS 8, so we can stop supporting CentOS 7 asap and support CentOS 8 instead.

This requires setting up two CentOS 8 Jenkins agents and some changes to our CI scripts. After that, all CentOS 7 agents can be retired.

Related issues 3 (0 open — 3 closed)

Actions

Copy link

Updated by Davide Pesavento over 4 years ago

There are CentOS 8 Vagrant boxes from generic and bento.

Actions

Copy link

Updated by Md Ashiqur Rahman over 4 years ago

Yes, I'm already setting up one from generic. This should be done by today.

Actions

Copy link

Updated by Md Ashiqur Rahman over 4 years ago

Ok, so I setup the vm with the following packages installed:

    sudo yum -qy install epel-release
    sudo yum -qy repolist
    sudo yum -qy install epel-release
    sudo yum config-manager --set-enabled PowerTools
    sudo yum -qq update
    sudo yum -qy groupinstall 'Development Tools'
    sudo yum install -qy pkgconfig python3 openssl-devel libpcap-devel sqlite-devel boost-devel libasan
    sudo alternatives --set python /usr/bin/python3

After that, I manually wanted to test ndn-cxx and got an error:

[vagrant@centos8 ndn-cxx]$ ./waf configure
Setting top to                           : /home/vagrant/ndn-cxx 
Setting out to                           : /home/vagrant/ndn-cxx/build 
Building static library                  : no 
Building shared library                  : yes 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
Checking C++ compiler version            : 8.2.1 
Checking supported CXXFLAGS              : -std=c++14 
Checking supported LINKFLAGS             : -fuse-ld=gold 
Checking for std::to_string              : yes 
Checking for program 'doxygen'           : not found 
Checking for program 'tar'               : /usr/bin/tar 
Checking for program 'sphinx-build'      : not found 
Checking for program 'sh'                : /usr/bin/sh 
Checking for library pthread             : yes 
Checking for library rt                  : yes 
Checking for function getpass            : yes 
Checking for netlink                     : yes 
Checking for NETLINK_EXT_ACK             : yes 
Checking for IFA_FLAGS                   : yes 
Checking for program 'pkg-config'        : /usr/bin/pkg-config 
Checking for 'sqlite3'                   : yes 
Checking for OpenSSL version             : "OpenSSL 1.1.1 FIPS  11 Sep 2018" 
Checking if OpenSSL library works        : yes 
Checking for program 'dpkg-architecture' : not found 
Checking boost includes                  : 1.66.0 
Checking boost libs                      : lib stacktrace_backtrace not found in /usr/lib64 
Checking boost libs                      : boost_stacktrace_basic 
Checking for boost linkage               : ok 
Checking boost libs                      : boost_system boost_program_options boost_chrono boost_date_time boost_filesystem boost_thread boost_log boost_stacktrace_basic 
Checking the flags needed to use pthreads : -pthread 
Checking for boost linkage                : ok 
Checking supported CXXFLAGS               : -O2 -g -pedantic -Wall -Wextra -Wnon-virtual-dtor -Wno-unused-parameter -fdiagnostics-color 
Checking supported LINKFLAGS              : -Wl,-O1 
'configure' finished successfully (7.461s)
[vagrant@centos8 ndn-cxx]$ ./waf
Waf: Entering directory `/home/vagrant/ndn-cxx/build'
[  1/203] Compiling ndn-cxx/impl/common-pch.hpp
[  2/203] Compiling libndn-cxx.pc.in
[  3/203] Compiling tools/wrapper/ndnsec-delete.sh
[  4/203] Compiling tools/wrapper/ndnsec-keygen.sh
[  5/203] Compiling tools/wrapper/ndnsec-install-cert.sh
[  6/203] Compiling tools/wrapper/ndnsec-certgen.sh
[  7/203] Compiling tools/wrapper/ndnsec-cert-gen.sh
[  8/203] Compiling tools/wrapper/ndnsec-get-default.sh
[  9/203] Compiling tools/wrapper/ndnsec-export.sh
[ 10/203] Compiling tools/wrapper/ndnsec-key-gen.sh
[ 11/203] Compiling tools/wrapper/ndnsec-set-default.sh
[ 12/203] Compiling tools/wrapper/ndnsec-cert-install.sh
[ 13/203] Compiling tools/wrapper/ndnsec-import.sh
[ 14/203] Compiling tools/wrapper/ndnsec-unlock-tpm.sh
[ 15/203] Compiling tools/wrapper/ndnsec-dump-certificate.sh
[ 16/203] Compiling tools/wrapper/ndnsec-ls-identity.sh
[ 17/203] Compiling tools/wrapper/ndnsec-sign-req.sh
[ 18/203] Compiling tools/wrapper/ndnsec-list.sh
[ 19/203] Compiling tools/wrapper/ndnsec-cert-dump.sh
[ 20/203] Compiling ndn-cxx/name-component.cpp
[ 21/203] Compiling ndn-cxx/name.cpp
cc1plus: error: one or more PCH files were found, but they were invalid
cc1plus: error: use -Winvalid-pch for more information
cc1plus: fatal error: /home/vagrant/ndn-cxx/build/ndn-cxx.2: No such file or directory
compilation terminated.

cc1plus: error: one or more PCH files were found, but they were invalid
cc1plus: error: use -Winvalid-pch for more information
cc1plus: fatal error: /home/vagrant/ndn-cxx/build/ndn-cxx.2: No such file or directory
compilation terminated.

Waf: Leaving directory `/home/vagrant/ndn-cxx/build'
Build failed
 -> task in 'ndn-cxx' failed with exit status 1 (run with -v to display more information)
 -> task in 'ndn-cxx' failed with exit status 1 (run with -v to display more information)

Actions

Copy link

Updated by Davide Pesavento over 4 years ago

Md Ashiqur Rahman wrote:

Ok, so I setup the vm with the following packages installed:
    sudo yum -qy install epel-release
    sudo yum -qy repolist

What are these?

sudo yum -qy install epel-release

This is installed twice?

sudo alternatives --set python /usr/bin/python3

Ah yes... python... https://gitlab.com/ita1024/waf/issues/2259

cc1plus: error: one or more PCH files were found, but they were invalid
cc1plus: error: use -Winvalid-pch for more information

Please do what it says and report back, because it could be many things and the error message doesn't say.

Actions

Copy link

Updated by Md Ashiqur Rahman over 4 years ago

Davide Pesavento wrote:

Md Ashiqur Rahman wrote:
Ok, so I setup the vm with the following packages installed:
    sudo yum -qy install epel-release
    sudo yum -qy repolist
What are these?

These are required to properly enable PowerTools and then install devel packages.

sudo yum -qy install epel-release
This is installed twice?

Yes, I'll remove the duplicate.

sudo alternatives --set python /usr/bin/python3
Ah yes... python... https://gitlab.com/ita1024/waf/issues/2259

cc1plus: error: one or more PCH files were found, but they were invalid
cc1plus: error: use -Winvalid-pch for more information

Please do what it says and report back, because it could be many things and the error message doesn't say.

Noted.

Actions

Copy link

Updated by Davide Pesavento over 4 years ago

Ehm... you read my comment wrong. I was quoting you and then commenting/asking about the part quoted directly above my comment... I think you've reversed the order.

Actions

Copy link

Updated by Md Ashiqur Rahman over 4 years ago

So I ran with the mentioned flag in verbose and got this: Please point out if I'm missing anything.

[vagrant@centos8 ndn-cxx]$ CXXFLAGS="-Winvalid-pch" ./waf configure
Setting top to                           : /home/vagrant/ndn-cxx 
Setting out to                           : /home/vagrant/ndn-cxx/build 
Building static library                  : no 
Building shared library                  : yes 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
Checking C++ compiler version            : 8.2.1 
Checking supported CXXFLAGS              : -std=c++14 
Checking supported LINKFLAGS             : -fuse-ld=gold 
Checking for std::to_string              : yes 
Checking for program 'doxygen'           : not found 
Checking for program 'tar'               : /usr/bin/tar 
Checking for program 'sphinx-build'      : not found 
Checking for program 'sh'                : /usr/bin/sh 
Checking for library pthread             : yes 
Checking for library rt                  : yes 
Checking for function getpass            : yes 
Checking for netlink                     : yes 
Checking for NETLINK_EXT_ACK             : yes 
Checking for IFA_FLAGS                   : yes 
Checking for program 'pkg-config'        : /usr/bin/pkg-config 
Checking for 'sqlite3'                   : yes 
Checking for OpenSSL version             : "OpenSSL 1.1.1 FIPS  11 Sep 2018" 
Checking if OpenSSL library works        : yes 
Checking for program 'dpkg-architecture' : not found 
Checking boost includes                  : 1.66.0 
Checking boost libs                      : lib stacktrace_backtrace not found in /usr/lib64 
Checking boost libs                      : boost_stacktrace_basic 
Checking for boost linkage               : ok 
Checking boost libs                      : boost_system boost_program_options boost_chrono boost_date_time boost_filesystem boost_thread boost_log boost_stacktrace_basic 
Checking the flags needed to use pthreads : -pthread 
Checking for boost linkage                : ok 
'configure' finished successfully (6.758s)
[vagrant@centos8 ndn-cxx]$ ./waf -v
Waf: Entering directory `/home/vagrant/ndn-cxx/build'
[  1/203] Compiling ndn-cxx/impl/common-pch.hpp
23:02:50 runner ['/usr/bin/g++', '-std=c++14', '-Winvalid-pch', '-fPIC', '-pthread', '-x', 'c++-header', '-I.', '-I..', '-I/usr/include', '-DNDEBUG', '-DBOOST_STACKTRACE_DYN_LINK', '-DBOOST_LOG_DYN_LINK', '-include', '../ndn-cxx/impl/common-pch.hpp', '/home/vagrant/ndn-cxx/ndn-cxx/impl/common-pch.hpp', '-c', '-o/home/vagrant/ndn-cxx/build/ndn-cxx.2.gch']
[  2/203] Compiling libndn-cxx.pc.in
[  3/203] Compiling tools/wrapper/ndnsec-list.sh
[  4/203] Compiling tools/wrapper/ndnsec-cert-dump.sh
[  5/203] Compiling tools/wrapper/ndnsec-ls-identity.sh
[  6/203] Compiling tools/wrapper/ndnsec-cert-gen.sh
[  7/203] Compiling tools/wrapper/ndnsec-install-cert.sh
[  8/203] Compiling tools/wrapper/ndnsec-get-default.sh
[  9/203] Compiling tools/wrapper/ndnsec-export.sh
[ 10/203] Compiling tools/wrapper/ndnsec-key-gen.sh
[ 11/203] Compiling tools/wrapper/ndnsec-set-default.sh
[ 12/203] Compiling tools/wrapper/ndnsec-cert-install.sh
[ 13/203] Compiling tools/wrapper/ndnsec-import.sh
[ 14/203] Compiling tools/wrapper/ndnsec-unlock-tpm.sh
[ 15/203] Compiling tools/wrapper/ndnsec-keygen.sh
[ 16/203] Compiling tools/wrapper/ndnsec-certgen.sh
[ 17/203] Compiling tools/wrapper/ndnsec-dump-certificate.sh
[ 18/203] Compiling tools/wrapper/ndnsec-delete.sh
[ 19/203] Compiling tools/wrapper/ndnsec-sign-req.sh
[ 20/203] Compiling ndn-cxx/security/transform/step-source.cpp
23:02:58 runner ['/usr/bin/g++', '-std=c++14', '-Winvalid-pch', '-fPIC', '-pthread', '-include', '/home/vagrant/ndn-cxx/build/ndn-cxx.2', '-I.', '-I..', '-I/usr/include', '-DNDEBUG', '-DBOOST_STACKTRACE_DYN_LINK', '-DBOOST_LOG_DYN_LINK', '../ndn-cxx/security/transform/step-source.cpp', '-c', '-o/home/vagrant/ndn-cxx/build/ndn-cxx/security/transform/step-source.cpp.2.o']
cc1plus: warning: /home/vagrant/ndn-cxx/build/ndn-cxx.2.gch: had text segment at different address
cc1plus: error: one or more PCH files were found, but they were invalid
cc1plus: fatal error: /home/vagrant/ndn-cxx/build/ndn-cxx.2: No such file or directory
compilation terminated.

Waf: Leaving directory `/home/vagrant/ndn-cxx/build'
Build failed
 -> task in 'ndn-cxx' failed with exit status 1: 
    {task 140284355010640: cxx step-source.cpp -> step-source.cpp.2.o}
['/usr/bin/g++', '-std=c++14', '-Winvalid-pch', '-fPIC', '-pthread', '-include', '/home/vagrant/ndn-cxx/build/ndn-cxx.2', '-I.', '-I..', '-I/usr/include', '-DNDEBUG', '-DBOOST_STACKTRACE_DYN_LINK', '-DBOOST_LOG_DYN_LINK', '../ndn-cxx/security/transform/step-source.cpp', '-c', '-o/home/vagrant/ndn-cxx/build/ndn-cxx/security/transform/step-source.cpp.2.o']

Actions

Copy link

Updated by Davide Pesavento over 4 years ago

Uhm, weird, I have no idea at the moment.

Actions

Copy link

Updated by Davide Pesavento over 4 years ago

Btw,

sudo yum -qy groupinstall 'Development Tools'

does not seem to be strictly required. You can just install gcc-c++ and git-core the regular way.

Actions

Copy link

#10

Updated by Davide Pesavento over 4 years ago

Blocked by Bug #5042: Injecting PKG_CONFIG_PATH in os.environ doesn't work added

Actions

Copy link

#11

Updated by Davide Pesavento over 4 years ago

Target version set to 22.02

Actions

Copy link

#12

Updated by Davide Pesavento over 4 years ago

Davide Pesavento wrote:

Uhm, weird, I have no idea at the moment.

Looks like https://bugzilla.redhat.com/show_bug.cgi?id=1721553

Actions

Copy link

#13

Updated by Davide Pesavento over 4 years ago

Davide Pesavento wrote:

Looks like https://bugzilla.redhat.com/show_bug.cgi?id=1721553

Given that there's been no progress on this bug for months, I'd suggest we explicitly disable PCH on CentOS 8 in our CI scripts, and warn users of this bug (and its workaround) in the installation instructions.

Actions

Copy link

#14