Contributing & development

distillPDF is a young project, and feedback on real PDFs is the fastest way to make it better. This page covers what helps most and how to build and test from source.

What feedback helps most

PDF extraction breaks in document-specific ways, so the most valuable contribution is showing where the output is wrong on PDFs you actually care about.

1. Try it on your PDFs. Run distillpdf or the Python API over your own documents and compare the HTML or Markdown against the source.
2. Open an issue with a failing file. A small PDF that reproduces the problem, plus a note on what the output should look like, is far more actionable than a description alone. File issues at github.com/kkollsga/distillpdf/issues.
3. Pull requests are welcome. See the development setup below.

Development setup

distillPDF is a Rust crate (lopdf) exposed to Python through PyO3 + maturin, so building from source needs a Rust toolchain and a Python environment.

git clone https://github.com/kkollsga/distillpdf
cd distillpdf
maturin develop --release    # build + install into the current venv

--release matters: the debug build is much slower for extraction work.

maturin develop vs the wheel build

maturin develop is convenient for iterating, but its editable layout can leave a stale compiled module loaded so that source edits silently don't take effect. The test runner (tests/run.sh) avoids this by building a wheel and force-reinstalling it, which is exactly what CI does. If a code change doesn't seem to apply, rebuild through tests/run.sh.

Running the tests

The test suite lives in tests/ and uses pytest. The simplest path is the dev-loop script, which rebuilds and reinstalls distillpdf before running the suite:

bash tests/run.sh

Pass --no-build to skip the rebuild and run the suite against whatever is already installed:

bash tests/run.sh --no-build

If you have already built and installed distillpdf yourself, you can invoke pytest directly:

pytest tests/ -q

Test dependencies beyond distillpdf itself are pytest, reportlab, and pillow (see tests/requirements-dev.txt); reportlab and pillow are used to generate fixtures.

Test-suite layout

CI runs entirely on data the project owns, so the committed suite is self-contained and passes on a fresh clone with nothing downloaded.

• tests/demo/ — a self-contained demo PDF (demo.pdf, plus demo_platypus.pdf) with a committed ground-truth file (demo_groundtruth.json) for an end-to-end structure check.
• tests/corpus_tables/ — a synthetic table corpus (grid, boxed, ruled, borderless, varying column counts) with a ground_truth.json, gating table extraction.
• tests/fixtures_pdf/ — smaller generated fixtures for targeted checks (figures, footnotes, front matter, heading traps, and so on).

Fixtures are generated by committed scripts (gen_demo.py, gen_tables.py, gen_fixtures.py), so there is no opaque binary to trust — the inputs are reproducible from source.

Third-party corpora self-skip

Deeper coverage uses third-party PDF corpora, the local-only deep gates, and the competitor head-to-head benchmarks, which all live under the gitignored benchmarking/ tree and are not part of a fresh clone. Their tests self-skip when the corpora are absent and run only when the corpora are present locally.

Some tests also skip based on what is compiled in or installed — for example, the OCR tests skip the fast-tier checks when the Tesseract feature is not built into the wheel, and skip the accurate-tier checks when the optional VLM dependencies are missing. See OCR for scanned PDFs and OCR runtime setup for how those tiers are wired.

Building the docs locally

This documentation is built with MkDocs (Material theme). To preview it with live reload:

pip install -r docs/requirements.txt
mkdocs serve

Then open the local URL that mkdocs serve prints.

Design notes

For the internals behind the typed element tree and the document model, see the design notes and the .dpdf format reference.

License

distillPDF is MIT licensed — see LICENSE. You can use it anywhere, including commercial and closed-source projects.