Implementation:Microsoft Playwright CssTokenizer
| Knowledge Sources | |
|---|---|
| Domains | CSS Parsing, Selector Engine |
| Last Updated | 2026-02-12 00:00 GMT |
Overview
Concrete tool for tokenizing CSS selector strings provided by the Playwright library.
Description
The `cssTokenizer.ts` module is a TypeScript port of the CSS tokenizer from the `parse-css` project (originally CC0 licensed). It converts CSS selector strings into a stream of typed tokens following the CSS specification. The tokenizer implements the full CSS tokenization algorithm including handling of whitespace, strings (single and double quoted), numbers (integer and float), dimensions, percentages, identifiers, hash tokens, functions, at-keywords, URLs, delimiters, and escape sequences. It correctly processes Unicode code points, surrogate pairs, and the BOM character. Key token types include `IdentToken`, `FunctionToken`, `StringToken`, `NumberToken`, `HashToken`, `WhitespaceToken`, `ColumnToken`, and `EOFToken`. The `InvalidCharacterError` class handles malformed input.
Usage
Use this module as the low-level CSS tokenizer for Playwright's selector engine. It is consumed by the CSS parser (`cssParser.ts`) which in turn is used by the selector parser to process CSS-based Playwright selectors.
Code Reference
Source Location
- Repository: Microsoft_Playwright
- File: packages/playwright-core/src/utils/isomorphic/cssTokenizer.ts
Signature
export interface CSSTokenInterface {
toSource(): string;
value: string | number | undefined;
}
export class InvalidCharacterError extends Error {
constructor(message: string);
}
export function tokenize(str: string): CSSTokenInterface[];
Import
import { tokenize, CSSTokenInterface, InvalidCharacterError } from '../utils/isomorphic/cssTokenizer';
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| str | string | Yes | CSS selector string to tokenize |
Outputs
| Name | Type | Description |
|---|---|---|
| tokens | CSSTokenInterface[] | Array of CSS tokens with type, value, and source reconstruction capability |
Usage Examples
import { tokenize } from 'playwright-core/lib/utils/isomorphic/cssTokenizer';
const tokens = tokenize('div.class > #id:nth-child(2n+1)');
for (const token of tokens) {
console.log(token.toSource(), token.value);
}
// Reconstructs the original CSS:
const reconstructed = tokens.map(t => t.toSource()).join('');