Developing SIPI¶

Using an IDE¶

CLion¶

If you are using the CLion IDE, note that code introspection in the CLion editor may not work until it has run CMake. Open the project root directory (which contains CMakeLists.txt) and let CLion configure the project automatically.

For Nix-based development, launch CLion from inside the Nix shell so it inherits all required environment variables and dependencies:

nix develop
clion .

Running Locally¶

A dedicated local development config is provided at config/sipi.localdev-config.lua. It points imgroot at the bundled test images and uses small cache limits (1 MB, 10 files) so IIIF requests work out of the box and cache eviction is easy to observe.

Start the server¶

# Reproducible (what CI runs) — build through Nix, then run the built binary
just nix-build
just nix-run

Or, for the inner-loop dev workflow (non-reproducible — does not match CI outputs byte-for-byte):

nix develop
cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug -DCODE_COVERAGE=ON
cmake --build build --parallel
./build/sipi --config config/sipi.localdev-config.lua

The server starts on http://localhost:1024.

Try some requests¶

# Fetch an IIIF image with a transformation (creates a cache entry)
# Note: requests that need no processing (same format, full size, no rotation)
# are served directly from the original file and bypass the cache.
curl http://localhost:1024/unit/gradient-stars.tif/full/max/0/default.jpg -o /tmp/test.jpg

# Prometheus metrics (cache counters, gauges, no auth required)
curl http://localhost:1024/metrics

# Cache file list via Lua API (requires admin credentials from config)
curl -u admin:Sipi-Admin http://localhost:1024/api/cache

Make several different image requests to fill the cache past its 1 MB / 10 file limits and watch the eviction metrics change:

# Format conversions (TIF → JPG) trigger caching — all well under 2 MB
curl http://localhost:1024/unit/gradient-stars.tif/full/max/0/default.jpg -o /dev/null
curl http://localhost:1024/unit/lena512.tif/full/max/0/default.jpg -o /dev/null
curl http://localhost:1024/unit/cielab.tif/full/max/0/default.jpg -o /dev/null

# Resized requests also trigger caching
curl http://localhost:1024/unit/MaoriFigure.jpg/full/200,/0/default.jpg -o /dev/null
curl http://localhost:1024/unit/MaoriFigureWatermark.jpg/full/200,/0/default.jpg -o /dev/null

curl http://localhost:1024/metrics | grep sipi_cache

Available configs¶

Config file	Purpose
`config/sipi.config.lua`	Production-like defaults (`./images` imgroot, 20 MB cache)
`config/sipi.localdev-config.lua`	Local development (test images, tiny cache, DEBUG logging)
`config/sipi.test-config.lua`	Automated test suite

Pre-commit hook¶

nix develop (and direnv-driven shell loads) automatically point Git at the repo-tracked hook directory .githooks/ via git config core.hooksPath .githooks. The pre-commit hook runs scripts/shttps-context-check.sh on commits that touch shttps/ and refuses commits that introduce a SIPI→shttps leak. Working outside the dev shell? Run the same git config line by hand. The mandatory gate is CI; the local hook is fast-feedback parity.

Writing Tests¶

We use two test frameworks: GoogleTest for unit tests and pytest for end-to-end tests.

Unit Tests¶

Unit tests live in test/unit/ and use GoogleTest with ApprovalTests. Tests are organized by component:

test/unit/configuration/ - Configuration parsing tests
test/unit/filenamehash/ - Filename hashing tests
test/unit/iiifparser/ - IIIF URL parser tests
test/unit/sipiimage/ - Image processing tests
test/unit/shttps/ - HTTP server utility tests
test/unit/logger/ - Logger tests
test/unit/handlers/ - HTTP handler tests

Run all unit tests (inside the Nix sandbox via doCheck = enableTests in package.nix — just nix-build fails if any unit test fails):

just nix-build

Run a specific test binary directly, from the dev-shell inner loop:

nix develop
cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug -DCODE_COVERAGE=ON
cmake --build build --parallel
cd build && test/unit/iiifparser/iiifparser

Rust End-to-End Tests¶

Rust-based e2e tests live in test/e2e-rust/ and use reqwest for HTTP requests, serde_json for JSON validation, and insta for golden baseline snapshots. They cover IIIF compliance, server behaviour, and upload functionality.

Run Rust e2e tests:

just rust-test-e2e

The recipe resolves the sipi binary via $SIPI_BIN with a canonical default of ./result/bin/sipi (the Nix artifact path). Override SIPI_BIN to point at a dev-shell-built binary if you are iterating on cmake locally.

Sequential execution required

Tests must run with --test-threads=1 because each test starts its own SIPI server instance on a unique port. The recipe handles this automatically.

Hurl HTTP Contract Tests¶

Declarative HTTP contract tests live in test/hurl/ and use Hurl. Each .hurl file describes a sequence of HTTP requests and expected responses.

Run Hurl tests:

just hurl-test

Current test files:

file_access.hurl — File access and permission checks
lua_endpoints.hurl — Lua script endpoint responses
missing_sidecar.hurl — Behaviour when sidecar files are absent
sqlite_api.hurl — SQLite API endpoint tests
video_knora_json.hurl — Video metadata JSON responses

Requires Hurl binary

Hurl is available inside nix develop. Outside Nix, install it from hurl.dev.

Smoke Tests¶

Smoke tests live in test/smoke/ and run against a Docker image. They verify basic server functionality after a Docker build:

make test-smoke

Approval Tests¶

Approval tests live in test/approval/ and use snapshot-based testing for regression detection.

Managing Dependencies¶

External library sources are vendored in vendor/ and tracked with Git LFS. The manifest cmake/dependencies.cmake is the single source of truth for versions, download URLs, and SHA-256 hashes.

See Building: Vendored Dependencies for setup instructions and update/add workflows.

Quick reference:

make vendor-download    # fetch all archives
make vendor-verify      # check SHA-256 integrity
make vendor-checksums   # print hashes for manifest updates

Commit Message Schema¶

We use Conventional Commits. These prefixes drive release-please to automatically determine SemVer bumps and generate changelogs — using the correct prefix is required, not optional.

type(scope): subject
body

Types:

feat - new feature (SemVer minor)
fix - bug fix (SemVer patch)
docs - documentation changes
style - formatting, no code change
refactor - refactoring production code
test - adding or refactoring tests
build - changes to build system or dependencies
chore - miscellaneous maintenance
ci - continuous integration changes
perf - performance improvements

Breaking changes are indicated with !:

feat!: remove deprecated API endpoint

Example:

feat(HTTP server): support more authentication methods