Skip to content

Commit

Permalink
Speed up cibuildwheel builds
Browse files Browse the repository at this point in the history
This is done by removing the need to compile or install anything from
boost. For quite a while, boost_system has been header-only (and only
indirectly used, via asio), so references to it are removed. That left
boost_program_options as the only library that spead2 depends on - and
only for command-line tools, not the wheel. So add
`--without-program-options` configure option to allow that dependency to
be skipped, and use it from the Python build.

That allowed the unpacked boost to be "installed" just with a symlink to
the include files rather than a proper installation.

Closes #145.
  • Loading branch information
bmerry committed Aug 18, 2023
1 parent 80d6d2f commit d4233ff
Show file tree
Hide file tree
Showing 5 changed files with 32 additions and 18 deletions.
22 changes: 15 additions & 7 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,25 @@ CXXFLAGS="$save_CXXFLAGS"

### Required libraries

SPEAD2_CHECK_LIB([boost/program_options.hpp], [boost_program_options],
[boost::program_options::option_description foo],
[], [AC_MSG_ERROR([boost_program_options is required])])
SPEAD2_CHECK_LIB([boost/system/system_error.hpp], [boost_system], [boost::system::error_code ec],
[], [AC_MSG_ERROR([boost_system is required])])
SPEAD2_CHECK_HEADER([boost/asio.hpp], [boost_system,pthread], [boost::asio::io_service io_service],
SPEAD2_CHECK_HEADER([boost/asio.hpp], [pthread], [boost::asio::io_service io_service],
[], [AC_MSG_ERROR([boost_asio is required])])
SPEAD2_CHECK_HEADER([libdivide.h], [], [libdivide::divider<unsigned long long> div],
[], [AC_MSG_ERROR([libdivide is required])])

### Optional libraries/features

SPEAD2_ARG_WITH(
[program_options],
[AS_HELP_STRING([--without-program-options], [Do not build utility programs])],
[SPEAD2_USE_PROGRAM_OPTIONS],
[SPEAD2_CHECK_LIB(
[boost/program_options.hpp], [boost_program_options],
[boost::program_options::option_description foo],
[SPEAD2_USE_PROGRAM_OPTIONS=1],
[AC_MSG_ERROR([boost_program_options is required unless --without-program-options is given])]
)]
)

# pkg-config files for libibverbs are quite new, so fall back to searching
# system paths.
IBV_CFLAGS=""
Expand Down Expand Up @@ -219,7 +226,7 @@ SPEAD2_ARG_WITH(

### Determine libraries to link against and include paths

LIBS="-lboost_system -lpthread -ldl"
LIBS="-lpthread -ldl"
SPEAD2_CFLAGS=""
AS_IF([test "x$SPEAD2_USE_PCAP" = "x1"], [
LIBS="$PCAP_LIBS $LIBS"
Expand Down Expand Up @@ -266,6 +273,7 @@ AM_CONDITIONAL([SPEAD2_USE_IBV], [test "x$SPEAD2_USE_IBV" = "x1"])
AM_CONDITIONAL([SPEAD2_USE_CUDA], [test "x$SPEAD2_USE_CUDA" = "x1"])
AM_CONDITIONAL([SPEAD2_USE_GDRAPI], [test "x$SPEAD2_USE_GDRAPI" = "x1"])
AM_CONDITIONAL([SPEAD2_USE_CAP], [test "x$SPEAD2_USE_CAP" = "x1"])
AM_CONDITIONAL([SPEAD2_USE_PROGRAM_OPTIONS], [test "x$SPEAD2_USE_PROGRAM_OPTIONS" = "x1"])

AC_SUBST(SPEAD2_MAJOR, m4_esyscmd([python3 gen/get_version.py major]))
AC_SUBST(SPEAD2_MINOR, m4_esyscmd([python3 gen/get_version.py minor]))
Expand Down
15 changes: 8 additions & 7 deletions doc/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,19 +72,20 @@ Python install from source
^^^^^^^^^^^^^^^^^^^^^^^^^^
Installing from source requires a modern C++ compiler supporting C++11 (GCC
4.8+ or Clang 3.5+, although only GCC 11.2 and Clang 14.0 are tested and support
for older compilers may be dropped) as well as Boost (including compiled
libraries), libdivide, and the Python development headers. At the moment only
GNU/Linux and OS X get tested but other POSIX-like systems should work too.
There are no plans to support Windows.
for older compilers may be dropped) as well as Boost 1.69+
(only headers are required), libdivide, and the Python development headers.
At the moment only GNU/Linux and OS X get tested but other POSIX-like systems
should work too. There are no plans to support Windows.

Installation works with standard Python installation methods.

Installing spead2 for C++
-------------------------
spead2 requires a modern C++ compiler supporting C++11 (see above for supported
compilers) as well as Boost (including compiled libraries) and libdivide. At
the moment only GNU/Linux and OS X get tested but other POSIX-like systems
should work too. There are no plans to support Windows.
compilers) as well as Boost 1.69+ (including the compiled boost_program_options
library) and libdivide. At the moment only GNU/Linux and OS X get tested but
other POSIX-like systems should work too. There are no plans to support
Windows.

The C++ API uses the standard autoconf installation flow i.e.:

Expand Down
5 changes: 2 additions & 3 deletions manylinux/before_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,8 @@ unset LDFLAGS
# Install boost
wget --progress=dot:mega https://boostorg.jfrog.io/artifactory/main/release/1.81.0/source/boost_1_81_0.tar.bz2 -O /tmp/boost_1_81_0.tar.bz2
tar -C /tmp -jxf /tmp/boost_1_81_0.tar.bz2
cd /tmp/boost_1_81_0
./bootstrap.sh --prefix=/usr --with-libraries=program_options,system
./b2 cxxflags=-fPIC link=static install
# Quick-n-dirty approach (much faster than doing the install, which copies thousands of files)
ln -s /tmp/boost_1_81_0/boost /usr/include/boost

# Install rdma-core
wget --progress=dot:mega https://github.com/linux-rdma/rdma-core/releases/download/v44.0/rdma-core-44.0.tar.gz -O /tmp/rdma-core-44.0.tar.gz
Expand Down
4 changes: 3 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ def initialize_options(self):

def run(self):
self.mkpath(self.build_temp)
subprocess.check_call(os.path.abspath("configure"), cwd=self.build_temp)
subprocess.check_call(
[os.path.abspath("configure"), "--without-program-options"], cwd=self.build_temp
)
config = configparser.ConfigParser()
config.read(os.path.join(self.build_temp, "python-build.cfg"))
for extension in self.extensions:
Expand Down
4 changes: 4 additions & 0 deletions src/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ bin_PROGRAMS = spead2_recv spead2_send spead2_bench
check_PROGRAMS = spead2_unittest
TESTS = spead2_unittest

if SPEAD2_USE_PROGRAM_OPTIONS

if SPEAD2_USE_IBV
bin_PROGRAMS += mcdump
mcdump_SOURCES = mcdump.cpp
Expand All @@ -35,6 +37,8 @@ spead2_send_LDADD = -lboost_program_options $(LDADD)
spead2_bench_SOURCES = spead2_bench.cpp spead2_cmdline.cpp
spead2_bench_LDADD = -lboost_program_options $(LDADD)

endif

if SPEAD2_USE_CAP
bin_PROGRAMS += spead2_net_raw
spead2_net_raw_SOURCES = spead2_net_raw.cpp
Expand Down

0 comments on commit d4233ff

Please sign in to comment.