第22章 打包与分发

pybind11 模块的分发涉及 Python 包的标准流程，但需要处理 C++ 编译和平台兼容性问题。本章介绍 setuptools 集成、wheel 构建和跨平台分发。

22.1 setuptools 集成

setuptools 是 Python 包管理的事实标准，pybind11 模块通过它进行构建和分发。

setup.py 基本结构

import os
import sys
from pathlib import Path

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext

import pybind11
pybind11_include = pybind11.get_include()


class BuildExt(build_ext):
    """自定义构建命令：添加 pybind11 包含路径"""
    def build_extensions(self):
        for ext in self.extensions:
            ext.include_dirs.append(pybind11_include)
        super().build_extensions()


setup(
    name="my_pybind11_module",
    version="1.0.0",
    author="Your Name",
    author_email="you@example.com",
    description="A pybind11 example module",
    long_description="",
    ext_modules=[
        Extension(
            "my_module",
            ["src/my_module.cpp"],
            include_dirs=[pybind11_include],
            language="c++",
            extra_compile_args=["-std=c++17"],
        )
    ],
    cmdclass={"build_ext": BuildExt},
    zip_safe=False,
)

pyproject.toml（现代方式）

[build-system]
requires = ["setuptools>=45", "pybind11>=2.10", "cmake>=3.15"]
build-backend = "setuptools.build_meta"

[project]
name = "my_pybind11_module"
version = "1.0.0"
description = "A pybind11 example module"
readme = "README.md"
requires-python = ">=3.8"
classifiers = [
    "Programming Language :: C++",
    "Programming Language :: Python :: 3",
]

[tool.setuptools]
packages = ["my_module"]

[tool.setuptools.build_meta]
build-backend = "scikit_build_core.build"

CMake-based 构建（推荐）

cmake_minimum_required(VERSION 3.15)
project(my_pybind11_module LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(pybind11 REQUIRED)

pybind11_add_module(my_module src/my_module.cpp)

set(PYBIND11_MODULE_VERSION "1.0.0")
target_compile_definitions(my_module PRIVATE PYBIND11_MODULE_VERSION=${PYBIND11_MODULE_VERSION})

[build-system]
requires = ["scikit-build-core>=0.5", "pybind11~=2.10"]
build-backend = "scikit_build_core.build"

[project]
name = "my_pybind11_module"

[tool.scikit-build]
cmake.minimum-version = "3.15"
cmake.build-type = "Release"

关键洞察：CMake 是构建 pybind11 模块的推荐方式，特别是对于复杂项目。scikit-build-core 提供了更好的 Python 集成。

22.2 wheel 构建（sdist vs wheel）

wheel 是 Python 的二进制分发格式，比源码分发（sdist）更快。

构建 wheel

python -m build --sdist

python -m build --wheel

ls dist/

分离式构建

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.package-dir]
"" = "src"

纯 Python fallback

try:
    # 尝试导入 C++ 模块
    from . import _my_module_cpp as impl
except ImportError:
    # Fallback 到纯 Python 实现
    impl = None
    import warnings
    warnings.warn("C++ module not available, using pure Python fallback")

def compute(x):
    if impl is None:
        return _python_fallback_compute(x)
    return impl.compute(x)


def _python_fallback_compute(x):
    """纯 Python 回退实现"""
    # 简化实现
    return x * 2

class ComputeFallback:
    """当 C++ 模块不可用时的纯 Python 回退"""

    def compute(self, x):
        """简化版计算"""
        return x * 2

    def process(self, data):
        """简化版处理"""
        return [self.compute(x) for x in data]

关键洞察：wheel 是编译好的二进制包，安装速度快但平台相关。sdist 需要在目标机器上编译，但跨平台。提供纯 Python fallback 可以在没有 C++ 编译器的平台上运行。

22.3 多平台 wheel（manylinux, auditwheel）

跨平台二进制分发需要处理 Linux 的 manylinux 标准。

manylinux 标准

标准	glibc 版本	Python 版本
manylinux2014	glibc 2.17	2.7, 3.5-3.11
manylinux_2_17	glibc 2.17	3.6+
manylinux_2_24	glibc 2.24	3.6+

cibuildwheel 自动构建

[tool.cibuildwheel]
build-verbosity = "1"

platforms = ["linux", "macos", "windows"]

[tool.cibuildwheel.linux]
before-build = "pip install cmake"
repair-wheel-command = "auditwheel repair --wheel-dir {wheel_dir} {wheel}"

[tool.cibuildwheel.macos]
architectures = ["x86_64", "arm64"]

[tool.cibuildwheel.windows]

name: Build wheels

on:
  release:
    types: [published]

jobs:
  build_linux:
    runs-on: ubuntu-latest
    container: manylinux2014_x86_64

    steps:
    - uses: actions/checkout@v3

    - name: Build wheels
      uses: pypa/cibuildwheel@v2.16
      env:
        CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014

    - name: Upload wheels
      uses: actions/upload-artifact@v3
      with:
        name: wheels-linux
        path: ./dist/*.whl

  build_macos:
    runs-on: macos-latest

    steps:
    - uses: actions/checkout@v3

    - name: Build wheels
      uses: pypa/cibuildwheel@v2.16
      env:
        CIBW_SKIP: "*-win_*"

    - name: Upload wheels
      uses: actions/upload-artifact@v3
      with:
        name: wheels-macos
        path: ./dist/*.whl

auditwheel 检查与修复

auditwheel show my_module-1.0.0-cp311-cp311-manylinux_2_17_x86_64.whl

auditwheel repair --wheel-dir ./dist my_module-1.0.0-cp311-cp311-manylinux_2_17_x86_64.whl

关键洞察：cibuildwheel 自动化了跨平台 wheel 构建的复杂性。它使用 Docker 处理 Linux 构建，确保 manylinux 兼容性。

22.4 纯 Python Fallback 实现

对于没有 C++ 编译器的平台，提供纯 Python 实现作为后备。

设计模式

from abc import ABC, abstractmethod


class ComputeEngine(ABC):
    """计算引擎抽象接口"""

    @abstractmethod
    def compute(self, x: float) -> float:
        pass

    @abstractmethod
    def process_array(self, data: list) -> list:
        pass


class CppComputeEngine(ComputeEngine):
    """C++ 实现"""

    def __init__(self):
        from . import _cpp_engine
        self._impl = _cpp_engine

    def compute(self, x: float) -> float:
        return self._impl.compute(x)

    def process_array(self, data: list) -> list:
        return self._impl.process_array(data)


class PythonComputeEngine(ComputeEngine):
    """纯 Python 回退实现"""

    def compute(self, x: float) -> float:
        return x * 2.0  # 简化实现

    def process_array(self, data: list) -> list:
        return [x * 2.0 for x in data]

import warnings

def get_engine():
    """获取可用的计算引擎"""
    try:
        from ._cpp_engine import CppComputeEngine
        return CppComputeEngine()
    except ImportError:
        warnings.warn(
            "C++ module not available, using pure Python fallback. "
            "Performance may be reduced.",
            RuntimeWarning
        )
        return PythonComputeEngine()


engine = get_engine()

def compute(x):
    return engine.compute(x)

def process_array(data):
    return engine.process_array(data)

关键洞察：纯 Python fallback 增加了维护成本，但保证了最大兼容性。只在必要时使用，并明确文档化性能差异。

22.5 版本管理

语义版本（SemVer）规范了版本号的意义。

版本规范

版本格式	说明	示例
Major	不兼容的 API 变更	1.0.0 → 2.0.0
Minor	向后兼容的功能添加	1.0.0 → 1.1.0
Patch	向后兼容的 bug 修复	1.0.0 → 1.0.1

绑定版本管理

version.cpp

#include <pybind11/pybind11.h>

namespace py = pybind11;

const char* module_version = "1.2.3";

PYBIND11_MODULE(versioned_module, m) {
    m.doc() = "Example module";

    // 导出版本信息
    m.def("get_version", []() { return module_version; });

    // 使用编译时宏
#ifdef MODULE_VERSION
    m.def("get_compile_version", []() { return MODULE_VERSION; });
#endif
}

__version__ = "1.2.3"

def get_cpp_version():
    try:
        from . import _my_module
        return getattr(_my_module, '__version__', 'unknown')
    except ImportError:
        return None

关键洞察：版本号应该准确反映兼容性变化。Major 版本变化意味着使用者可能需要修改代码，Minor 版本变化是安全的新增，Patch 版本是安全的问题修复。

22.6 依赖管理

明确声明和管理模块依赖。

setup.py 依赖

setup(
    name="my_pybind11_module",
    version="1.0.0",

    # 运行时依赖
    install_requires=[
        "numpy>=1.20.0",
    ],

    # 构建时依赖
    setup_requires=[
        "pybind11>=2.10.0",
        "cmake>=3.15",
    ],

    # 可选依赖
    extras_require={
        "dev": [
            "pytest>=7.0",
            "coverage>=6.0",
        ],
        "完整": [
            "scipy>=1.9.0",
        ],
    },
)

pyproject.toml 依赖

[project]
name = "my_pybind11_module"
version = "1.0.0"
requires-python = ">=3.8"

dependencies = [
    "numpy>=1.20.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "coverage>=6.0",
]

[project.urls]
Homepage = "https://github.com/yourname/my_pybind11_module"

requirements.txt

numpy>=1.20.0

-r requirements.txt
pytest>=7.0
coverage>=6.0

关键洞察：明确声明依赖避免版本冲突。运行时依赖应该设置最低版本要求，但避免过高的上限。

22.7 conda 包创建

conda 是科学计算常用的包管理器，可以分发编译好的二进制包。

conda-build 配置

package:
  name: my-pybind11-module
  version: "1.0.0"

source:
  path: .

build:
  number: 0
  script: python -m pip install . -vv

requirements:
  host:
    - python
    - pip
    - pybind11
    - cmake
    - numpy
  run:
    - python
    - numpy

test:
  imports:
    - my_module

about:
  home: https://github.com/yourname/my_pybind11_module
  license: MIT
  summary: A pybind11 example module

构建 conda 包

conda install conda-build

conda build recipe/

conda install --use-local my-pybind11-module

anaconda upload build/dist/*.tar.bz2

conda-forge 集成

{% set name = "my-pybind11-module" %}
{% set version = "1.0.0" %}

package:
  name: {{ name|lower }}
  version: {{ version }}

source:
  git_url: https://github.com/yourname/my-pybind11-module.git
  git_rev: {{ version }}

build:
  number: 0
  script: {{ PYTHON }} -m pip install . -vv

requirements:
  host:
    - python
    - pip
    - pybind11
  run:
    - python
    - numpy

test:
  imports:
    - my_module

about:
  home: https://github.com/yourname/my-pybind11-module
  license: MIT
  license_family: MIT
  summary: A pybind11 example module

关键洞察：conda 包提供了预编译的二进制分发，适合科学计算环境。conda-forge 是社区维护的 conda 包仓库，可以让你的包被广泛分发。

打包与分发总结：

方式	适用场景	优点	缺点
wheel	PyPI 分发	安装快，跨 Python 版本	仅限特定平台
sdist	源码分发	真正的跨平台	需要编译环境
conda	conda 用户	预编译，科学计算友好	生态系统较小
pure Python fallback	无编译器环境	最大兼容性	性能降低

实战建议：优先使用 cibuildwheel 构建跨平台 wheel。使用 pyproject.toml 管理构建配置。提供纯 Python fallback 以支持无编译器环境。