Build from Source
This document provides instructions for building Infinity from source, as well as building and running unit and functional tests.
Infinity can be natively compiled on Linux only. For non-Linux operating systems, build it using Docker.
Prerequisites
1. Replace the native Out-Of-Memory killer
Building the entire project, particularly during the link stage, requires considerable RAM, which can cause the host machine to become unresponsive due to the sluggishness of the kernel's Out-Of-Memory (OOM) killer. To mitigate this, we recommend installing earlyoom to improve the host's responsiveness.
2. Limit the number of concurrent link processes
Configure cmake
to limit the number of concurrent link processes. For example:
-DCMAKE_JOB_POOLS:STRING='link=4'
Recommended link pool size settings are as follows:
1
: 6 GB of RAM2
: 16 GB of RAM3
: 32 GB of RAM
3. Set the cmake
build type
The cmake
build type (CMAKE_BUILD_TYPE
) can be one of the following:
Debug
: Suitable for daily development.
No inline; with symbol info; with address sanitizer.
Typically ~10x slower thanRelWithDebInfo
orRelease
.RelWithDebInfo
: Suitable for performance analysis.
Optimizes with-O2
; with symbol information.Release
: Suitable for project releases. Optimizes with-O3
; without symbol information.
The built executables are significantly smaller than those ofRelWithDebInfo
.
The following procedures set CMAKE_BUILD_TYPE
to Debug
. Change it as you see necessary.
Build the source code
- On Linux using Docker
- On Ubuntu
This section provides instructions for building Infinity from source on Linux using Docker.
- Download the source code:
git clone https://github.com/infiniflow/infinity.git
- Build the source code using Docker:
cd infinity && mkdir cmake-build-debug
TZ=$(readlink -f /etc/localtime | awk -F '/zoneinfo/' '{print $2}')
docker run -d --name infinity_build -e TZ=$TZ -v $PWD:/infinity -v /boot:/boot infiniflow/infinity_builder:centos7_clang18
docker exec infinity_build bash -c "cd /infinity/cmake-build-debug && cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON .. && cmake --build . -t infinity"
- Start up the Infinity server:
sudo mkdir -p /var/infinity && sudo chown -R $USER /var/infinity
ulimit -n 500000
./cmake-build-debug/src/infinity
This section provides instructions for building Infinity from source on Ubuntu.
- Install necessary dependencies:
- Ubuntu 22.04
- Ubuntu 24.04
sudo apt update && sudo apt install git wget unzip software-properties-common
wget https://github.com/Kitware/CMake/releases/download/v3.29.0/cmake-3.29.0-linux-x86_64.tar.gz
tar zxvf cmake-3.29.0-linux-x86_64.tar.gz
sudo cp -rf cmake-3.29.0-linux-x86_64/bin/* /usr/local/bin && sudo cp -rf cmake-3.29.0-linux-x86_64/share/* /usr/local/share && rm -rf cmake-3.29.0-linux-x86_64
wget https://github.com/ninja-build/ninja/releases/download/v1.12.1/ninja-linux.zip
unzip ninja-linux.zip && sudo cp ninja /usr/local/bin && rm ninja
wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && sudo ./llvm.sh 18 && rm llvm.sh
sudo add-apt-repository -P ppa:ubuntu-toolchain-r/test
sudo add-apt-repository -P ppa:mhier/libboost-latest
sudo apt update && sudo apt install libc++-18-dev clang-tools-18 flex libboost1.81-dev liblz4-dev zlib1g-dev libevent-dev python3-dev autoconf
wget https://github.com/jemalloc/jemalloc/archive/refs/tags/5.3.0.tar.gz
tar zxvf 5.3.0.tar.gz && cd jemalloc-5.3.0 && ./autogen.sh && CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --enable-static --disable-libdl --enable-prof --enable-prof-libunwind --disable-initial-exec-tls && sudo make -j install && cd ..
sudo ldconfig && sudo rm -rf jemalloc-5.3.0 5.3.0.tar.gz
sudo ln -s /usr/lib/llvm-18/bin/clang-scan-deps /usr/bin/clang-scan-deps
sudo ln -s /usr/bin/clang-format-18 /usr/bin/clang-format
sudo ln -s /usr/bin/clang-tidy-18 /usr/bin/clang-tidy
sudo ln -s /usr/bin/llvm-symbolizer-18 /usr/bin/llvm-symbolizer
sudo ln -s /usr/lib/llvm-18/include/x86_64-pc-linux-gnu/c++/v1/__config_site /usr/lib/llvm-18/include/c++/v1/__config_site
sudo apt update && sudo apt install -y git wget lsb-release software-properties-common
wget https://github.com/Kitware/CMake/releases/download/v3.29.0/cmake-3.29.0-linux-x86_64.tar.gz
tar zxvf cmake-3.29.0-linux-x86_64.tar.gz
sudo cp -rf cmake-3.29.0-linux-x86_64/bin/* /usr/local/bin && sudo cp -rf cmake-3.29.0-linux-x86_64/share/* /usr/local/share && rm -rf cmake-3.29.0-linux-x86_64
wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && sudo ./llvm.sh 18 && rm llvm.sh
sudo apt install -y ninja-build clang-tools-18 flex libc++-18-dev libboost1.83-dev liblz4-dev zlib1g-dev libevent-dev python3-dev autoconf
wget https://github.com/jemalloc/jemalloc/archive/refs/tags/5.3.0.tar.gz
tar zxvf 5.3.0.tar.gz && cd jemalloc-5.3.0 && ./autogen.sh && CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --enable-static --disable-libdl --enable-prof --enable-prof-libunwind --disable-initial-exec-tls && sudo make -j install && cd ..
sudo ldconfig && sudo rm -rf jemalloc-5.3.0 5.3.0.tar.gz
sudo ln -s /usr/lib/llvm-18/bin/clang-scan-deps /usr/bin/clang-scan-deps
sudo ln -s /usr/bin/clang-format-18 /usr/bin/clang-format
sudo ln -s /usr/bin/clang-tidy-18 /usr/bin/clang-tidy
sudo ln -s /usr/bin/llvm-symbolizer-18 /usr/bin/llvm-symbolizer
sudo ln -s /usr/lib/llvm-18/include/x86_64-pc-linux-gnu/c++/v1/__config_site /usr/lib/llvm-18/include/c++/v1/__config_site
- Download the source code:
git clone https://github.com/infiniflow/infinity.git
- Build the source code:
You must also install simde
v0.7.4+ if using Ubuntu 22.04 on an ARM architecture.
sudo apt install libsimde-dev
If your installed version is below v0.7.4, download the include files directly from github and replace them.
git config --global --add safe.directory infinity
cd infinity && mkdir cmake-build-debug && cd cmake-build-debug
export CC=/usr/bin/clang-18
export CXX=/usr/bin/clang++-18
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON ..
cmake --build . -t infinity
- Start up the Infinity server:
sudo mkdir -p /var/infinity && sudo chown -R $USER /var/infinity
ulimit -n 500000
./cmake-build-debug/src/infinity
Build and run unit tests
- Using Docker on Linux
- on Ubuntu
- With code coverage enabled
docker exec infinity_build bash -c "cd /infinity/cmake-build-debug && cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON .. && cmake --build . -t test_main"
./cmake-build-debug/src/test_main
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON ..
cmake --build . -t test_main
./cmake-build-debug/src/test_main
-
Install the
Gcovr
dependency:pip install gcovr
-
Build and run the unit tests with code coverage enabled:
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON -DCODE_COVERAGE=ON ..
cmake --build . -t test_main
find . -name "*.gcda" | xargs rm -f
./cmake-build-debug/src/test_main -
Use Gcovr to generate summarized code coverage results:
cd ./cmake-build-debug/src/CMakeFiles/unit_test.dir
gcovr --gcov-executable "llvm-cov gcov" -r "YOUR_ABSOLUTE_PATH_OF_THE_PROJECT/infinity/src" --gcov-exclude-directories ${PWD}'/unit_test' . --html unit_test_html.html
Build and run functional tests
- Using Docker on Linux
- on Ubuntu
- With code coverage enabled
-
Build and start up the Infinity server:
docker exec infinity_build bash -c "cd /infinity/cmake-build-debug && cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON .. && cmake --build . -t infinity"
./cmake-build-debug/src/infinity -
Install Python sdk of infinity:
pip install infinity-sdk==0.5.0.dev1
-
Run the functional tests:
python3 tools/run_pytest_parallel.py
-
Build and start up the Infinity server with code coverage enabled:
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON ..
cmake --build . -t infinity
./cmake-build-debug/src/infinity -
Install Python sdk of infinity:
pip install infinity-sdk==0.5.0.dev1
-
Run the functional tests:
python3 tools/run_pytest_parallel.py
-
Build and start up the Infinity server with code coverage enabled:
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_VERBOSE_MAKEFILE=ON -DCODE_COVERAGE=ON ..
cmake --build . -t infinity
find . -name "*.gcda" | xargs rm -f
./cmake-build-debug/src/infinity -
Install Python sdk of infinity:
pip install infinity-sdk==0.5.0.dev1
-
Run the functional tests:
python3 tools/run_pytest_parallel.py
-
Shut down the Infinity server:
kill -15 `pidof infinity`
-
Use Gcovr to generate summarized code coverage results:
cd ./cmake-build-debug/src/CMakeFiles
gcovr --gcov-executable "llvm-cov gcov" -r "YOUR_ABSOLUTE_PATH_OF_THE_PROJECT/infinity/src" --gcov-exclude-directories ${PWD}'/unit_test' . --html function_test_html.html