feat(build): make TBB dependency configurable

[gripleaf]

Purpose

Linked issue: N/A

Make the oneTBB dependency optional through the PAIMON_USE_TBB CMake option.

• Keep PAIMON_USE_TBB=ON by default to preserve existing behavior.
• Skip resolving, building, including, and linking TBB when PAIMON_USE_TBB=OFF.
• Encapsulate TBB concurrent containers under Paimon utility headers.
• Provide C++17 standard-library implementations for builds without TBB.
• Select the implementation at compile time:
  • concurrent_hash_map.h exposes the TBB implementation when TBB is enabled and includes std_concurrent_hash_map.h otherwise.
  • concurrent_bounded_queue.h exposes the TBB implementation when TBB is enabled and includes std_concurrent_bounded_queue.h otherwise.
• Keep the class names, template parameters, and exposed methods consistent between the TBB and standard-library implementations.
• Keep the ConcurrentBoundedQueue method names and relevant type aliases compatible with the corresponding tbb::concurrent_bounded_queue API.
• Remove direct exposure of TBB queue types from the asynchronous key-value producer and consumer.

Tests

• Built paimon_objlib with PAIMON_USE_TBB=ON.
• Built paimon_objlib with PAIMON_USE_TBB=OFF.
• Ran the concurrent bounded queue tests with TBB enabled: 2 tests passed.
• Ran the concurrent bounded queue tests with TBB disabled: 2 tests passed.
• Compiled the concurrent hash map test translation unit with TBB enabled and disabled.
• Ran pre-commit checks for all changed files.
• Ran git diff --check.

The complete unit and integration test suites were not run.

API and Format

This change does not affect the public API under include/paimon/.

It does not change the storage format or protocol.

The new concurrent container implementations are internal utilities under src/paimon/common/utils/.

Documentation

Updated docs/source/building.rst to document:

• The PAIMON_USE_TBB CMake option.
• TBB being enabled by default.
• Building Paimon without resolving, building, or linking TBB.
• TBB_SOURCE only being considered when TBB is enabled.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-5)

[lxy-9602]

Before starting the review, I’d like to better understand the specific motivation behind this PR — for example, which platform or downstream project is unable to use TBB, and what the performance trade-offs are when disabling it. BTW, the current CI also doesn’t seem to cover the PAIMON_USE_TBB=OFF branch.

[gripleaf]

Before starting the review, I’d like to better understand the specific motivation behind this PR — for example, which platform or downstream project is unable to use TBB, and what the performance trade-offs are when disabling it. BTW, the current CI also doesn’t seem to cover the PAIMON_USE_TBB=OFF branch.

The motivation is not platform compatibility. We integrate Paimon C++ as a DSO to isolate its large dependency graph from the existing service. However, both the service and the DSO bring their own TBB runtime and worker pool, which can cause thread oversubscription. PAIMON_USE_TBB=OFF allows the DSO to avoid introducing the second TBB runtime.

You are also right that CI currently does not cover PAIMON_USE_TBB=OFF; I will add a dedicated non-TBB build/test configuration.

[zjw1111]

@gripleaf Thanks for the clarification. If the main concern is that the Paimon DSO embeds a second TBB runtime, could this be addressed by building Paimon against the same shared TBB used by the host service, for example with:

-DTBB_SOURCE=SYSTEM -DTBB_ROOT=/path/to/the/host/tbb/prefix

The existing dependency resolver already supports TBB_SOURCE=SYSTEM. If both the host and the Paimon DSO resolve to the same compatible libtbb.so, this should avoid embedding the bundled static TBB into the DSO and should let them share one runtime. Of course, this would need to be verified from the final DT_NEEDED entries / ldd output, since TBB_SOURCE=SYSTEM by itself does not guarantee that the selected library is shared or identical to the host copy.

Also, Paimon currently appears to use TBB only for concurrent_hash_map and concurrent_bounded_queue; the producer/consumer threads themselves are created with std::async. Could you provide profiling or thread/runtime evidence showing that the second TBB copy actually initializes another worker pool and causes measurable oversubscription? That would help establish whether a non-TBB implementation is necessary, rather than simply reusing the host shared TBB.