Heuristic:MarketSquare Robotframework browser GRPC Response Chunking
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Networking |
| Last Updated | 2026-02-12 04:00 GMT |
Overview
Workaround that chunks large HTTP response bodies into 3.5MB segments to avoid exceeding gRPC message size limits.
Description
The Browser library communicates between Python and Node.js via gRPC. gRPC has a default maximum message size (~4MB). When intercepting network responses with large bodies (e.g., downloading files, receiving large JSON payloads), the response body can exceed this limit and cause a gRPC error. The Node.js side works around this by splitting the response body into 3.5MB chunks and sending them as separate gRPC messages.
Usage
This heuristic is relevant when debugging network interception failures with large responses or when tuning the gRPC message size. If you see gRPC message size errors during `Wait For Response` or network observation keywords, this chunking mechanism is the mitigation.
The Insight (Rule of Thumb)
- Action: Split HTTP response bodies into chunks of 3,500,000 bytes (3.5MB) before sending over gRPC.
- Value: `chunkSize = 3500000` — chosen to be safely under the ~4MB gRPC default maximum message size.
- Trade-off: Large responses require multiple gRPC round trips. The Python side must reassemble the chunks.
Reasoning
The gRPC default maximum message size is approximately 4MB. Setting the chunk size to 3.5MB leaves sufficient headroom for gRPC message framing, headers, and the JSON metadata sent alongside the body. This value was empirically chosen to be safe across all platforms and gRPC versions while minimizing the number of chunks needed.
Code Evidence
From `node/playwright-wrapper/network.ts:95-100`:
const chunkSize = 3500000;
const responseChunks = [];
if (body && body.length > chunkSize) {
logger.info(`body.length: ${body.length}`);
for (let i = 0; i < body.length; i += chunkSize) {
const chunk = body.substring(i, i + chunkSize);