Implementation:Alibaba MNN RapidJSON Reader

Knowledge Sources	Alibaba_MNN RapidJSON
Domains	JSON_Parsing, SAX
Last Updated	2026-02-10 12:00 GMT

Overview

3rd_party/rapidjson/reader.h (2230 lines) implements a SAX-style streaming JSON reader/parser for the vendored RapidJSON library within Alibaba MNN. Unlike the DOM-based GenericDocument, the reader fires events (callbacks) as it encounters JSON tokens, enabling low-memory streaming parse scenarios. It is also used internally by GenericDocument::Parse() to build the DOM tree.

Usage note: Vendored dependency used internally by MNN for JSON configuration parsing (model configs, LLM configs). Not directly imported by end users.

Key Class

GenericReader

template <typename SourceEncoding, typename TargetEncoding, typename StackAllocator = CrtAllocator>
class GenericReader {
public:
    typedef typename SourceEncoding::Ch Ch;

    GenericReader(StackAllocator* stackAllocator = 0, size_t stackCapacity = kDefaultStackCapacity);

    // Parse with handler receiving SAX events
    template <unsigned parseFlags, typename InputStream, typename Handler>
    ParseResult Parse(InputStream& is, Handler& handler);

    // Iterative (non-recursive) parsing
    template <unsigned parseFlags, typename InputStream, typename Handler>
    bool IterativeParseInit();

    template <unsigned parseFlags, typename InputStream, typename Handler>
    bool IterativeParseNext(InputStream& is, Handler& handler);

    bool IterativeParseComplete() const;

    // Error state
    bool HasParseError() const;
    ParseErrorCode GetParseError() const;
    size_t GetErrorOffset() const;
};

The default typedef is:

typedef GenericReader<UTF8<>, UTF8<>> Reader;

Handler Concept

The parser invokes methods on a handler object as it encounters JSON tokens:

concept Handler {
    bool Null();
    bool Bool(bool b);
    bool Int(int i);
    bool Uint(unsigned u);
    bool Int64(int64_t i);
    bool Uint64(uint64_t u);
    bool Double(double d);
    bool RawNumber(const Ch* str, SizeType length, bool copy);
    bool String(const Ch* str, SizeType length, bool copy);
    bool StartObject();
    bool Key(const Ch* str, SizeType length, bool copy);
    bool EndObject(SizeType memberCount);
    bool StartArray();
    bool EndArray(SizeType elementCount);
};

Each handler method returns bool; returning false aborts parsing.

Parse Flags

enum ParseFlag {
    kParseNoFlags = 0,
    kParseInsituFlag = 1,             // In-situ (destructive) parsing
    kParseValidateEncodingFlag = 2,   // Validate encoding of JSON strings
    kParseIterativeFlag = 4,          // Iterative (non-recursive) parsing
    kParseStopWhenDoneFlag = 8,       // Stop after parsing a complete JSON value
    kParseFullPrecisionFlag = 16,     // Use full precision double parsing
    kParseCommentsFlag = 32,          // Allow one-line and multi-line comments
    kParseNumbersAsStringsFlag = 64,  // Parse numbers as strings
    kParseTrailingCommasFlag = 128,   // Allow trailing commas
    kParseNanAndInfFlag = 256,        // Allow NaN and Inf
    kParseDefaultFlags = kParseNoFlags
};

SIMD Acceleration

The reader supports SIMD-optimized whitespace skipping and string scanning:

SSE4.2 -- Uses _mm_cmpistri for accelerated character class matching (enabled via RAPIDJSON_SSE42)
SSE2 -- Uses _mm_cmpeq_epi8 for byte-parallel comparisons (enabled via RAPIDJSON_SSE2)
NEON -- ARM NEON intrinsics for mobile/embedded platforms (enabled via RAPIDJSON_NEON)

Number Parsing

The reader supports parsing JSON numbers into int, unsigned, int64_t, uint64_t, and double types. With kParseFullPrecisionFlag, it uses the Strtod algorithm for precise double conversion. The internal strtod.h header provides this functionality.

Error Handling

Parsing errors are reported via ParseErrorCode with the byte offset of the error. The macro RAPIDJSON_PARSE_ERROR_NORETURN can be overridden to throw exceptions instead of using error codes.

Dependencies

allocators.h -- Memory allocator interfaces
stream.h -- Input stream abstractions
encodedstream.h -- Encoding-aware stream wrappers
internal/meta.h -- Template metaprogramming utilities
internal/stack.h -- Internal stack for parsing state
internal/strtod.h -- String-to-double conversion

License

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment