PanoWeave

PanoWeave is a C++ library with optional Python bindings for stitching multiple images into an equirectangular panorama, using Basalt camera calibration data. The library utilizes the OpenCV T-API for end-to-end GPU-accelerated processing, with optional CUDA support.

The stitching process requires geometric and extrinsic parameters of the camera system, as well as current depth information. While depth maps improve the quality of blended regions, they can be set to a uniform value for simpler use cases. When calibration data is available, PanoWeave can also correct photometric distortions such as lens vignetting and sensor response, and normalizes exposure across all input images for consistent brightness in the final panorama.

Dependencies

PanoWeave requires the following tools and libraries:

• C++17 compatible compiler
• OpenCV 4 (with CUDA support, when using CUDA)
• spdlog
• pybind11 (optional, for Python bindings)
• pybind11-stubgen (optional, for Python type stubs)
• Doxygen (optional, for documentation generation)

Usage

Integrating into Your Project

To use PanoWeave in your own project, simply add it as a git submodule and include it as a CMake subdirectory.

1. Add as a git submodule:

   mkdir thirdparty
   cd thirdparty
   git submodule add https://github.com/RoblabWh/PanoWeave
   git submodule update --init --recursive

2. Update your CMakeLists.txt:

   ...
   add_subdirectory(thirdparty/PanoWeave)
   ...
   add_executable(${PROJECT_NAME} ...)
   target_link_libraries(${PROJECT_NAME} PUBLIC PanoWeave)
   ...

Example Applications

PanoWeave includes two minimal examples: one for interactive video stitching and another for benchmarking performance. Both are available in C++ and Python.

First, build the project:

cmake -Bbuild -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel $(nproc)

To run the C++ example:

./build/example/stitch_video --help
./build/example/bench_stitcher --help

To run the Python example, first set the PYTHONPATH:

export PYTHONPATH="$(realpath build):$PYTHONPATH"
 
./example/stitch_video.py --help
./example/bench_stitcher.py --help

Configuration

PanoWeave offers CMake options to adjust the build to your needs.

To set options during configuration, use:

cmake -Bbuild -DCMAKE_BUILD_TYPE=Release -DUSE_CUDA=ON

Alternatively, set options in your CMakeLists.txt before adding the subdirectory. This allows you to change defaults or FORCE specific settings:

...
set(USE_CUDA ON CACHE BOOL "Use CUDA by default")
set(EXAMPLES OFF CACHE BOOL "Never build examples" FORCE)
add_subdirectory(thirdparty/PanoWeave)
...

Available options (default values in parentheses):

• USE_CUDA (ON if CUDA is detected): Build with CUDA support instead of T-API
• PYBINDS (OFF if subdirectory): Build Python bindings with pybind11
• EXAMPLES (OFF if subdirectory): Build example applications
• DOCS (OFF if subdirectory): Generate documentation with Doxygen

Documentation

PanoWeave offers documentation for both C++ and Python APIs. C++ documentation is generated with Doxygen, while Python stubs are created for IDE support.

Doxygen

You can generate HTML documentation by building the PanoWeaveDocs CMake target. This target is created automatically if Doxygen is available during configuration and can be built on demand.

1. Build documentation:

   cmake --build build --target PanoWeaveDocs

2. View documentation under build/html in your browser, for example:

   firefox build/html/index.html

If Doxygen was not found during configuration:

1. Install Doxygen:

   sudo apt-get install doxygen

2. Reconfigure and rebuild documentation:

   cmake -Bbuild
   cmake --build build --target PanoWeaveDocs

3. Open build/html/index.html in your browser.

Python stubs

During a CMake build with enabled Python bindings the stubs are generated by default, as long as pybind11-stubgen is available. The generated .pyi gets parsed by IDEs and similar to provide highlighting and descriptions.

To generate Python stubs (if not already present):

1. Install pybind11-stubge:

   pip install pybind11-stubgen

2. Reconfigure and rebuild the project:

   cmake -Bbuild
   cmake --build build