Heuristic:Duckdb Duckdb Test Development Guidelines
| Knowledge Sources | |
|---|---|
| Domains | Testing, Code_Quality |
| Last Updated | 2026-02-07 12:00 GMT |
Overview
DuckDB strongly prefers sqllogictest (.test files) over C++ unit tests; fast tests (~1 min) must pass before PRs, while slow tests (.test_slow) run in the ~1 hour all-unit suite.
Description
DuckDB uses a two-tier test system: fast unit tests (`make unit`, ~1 minute) and all unit tests (`make allunit`, ~1 hour). Tests should primarily be written using the sqllogictest framework (`.test` files) rather than C++ unit tests. The sqllogictest format is declarative, easier to write and maintain, and provides better test isolation. C++ tests should only be used when testing exotic behavior like concurrent connections.
Usage
Apply these guidelines when writing tests for any DuckDB change. All PRs must include tests, and the fast unit tests must pass before submission. Use this heuristic to decide between sqllogictest and C++ tests, and between fast and slow test classification.
The Insight (Rule of Thumb)
- Rule 1: Strongly prefer sqllogictest (`.test` files) over C++ unit tests. Only write C++ tests for concurrent connections or other exotic behavior.
- Rule 2: Test with different types: numerics, strings, and complex nested types.
- Rule 3: Test unexpected/incorrect usage, not just the happy path.
- Rule 4: Fast tests (`make unit`) must complete in ~1 minute. Ensure all new fast tests pass before sending a PR.
- Rule 5: Slow tests should be named `.test_slow` in sqllogictest or marked with `[.]` in C++ test groups.
- Rule 6: Aim for code coverage — check the coverage report and attempt to cover all code paths in fast unit tests.
- Rule 7: Attempt to trigger exceptions in tests. It is acceptable to leave some untriggered (e.g., out-of-memory), but large code branches should be covered.
- Trade-off: Thorough testing increases development time but prevents regressions.
Reasoning
The sqllogictest framework provides several advantages over C++ tests: it's declarative (easier to read/write), language-agnostic (tests can be reused across client implementations), and provides automatic result verification. The fast/slow split ensures CI remains responsive — developers get quick feedback from `make unit` while the full suite runs in nightly CI.
Code evidence from `CONTRIBUTING.md:58-66`:
* Unit tests can be written either using the sqllogictest framework (.test files)
or in C++ directly. We **strongly** prefer tests to be written using the
sqllogictest framework.
* `make unit` runs the **fast** unit tests (~one minute), `make allunit` runs
**all** unit tests (~one hour).
* Make sure **all** unit tests pass before sending a PR.
* Slower tests should be added to the **all** unit tests. You can do this by
naming the test file `.test_slow` in the sqllogictests, or by adding `[.]`
after the test group in the C++ tests.
* Look at the code coverage report of your branch and attempt to cover all code
paths in the fast unit tests.