Implementation:ClickHouse ClickHouse StringViewHash

Knowledge Sources	ClickHouse
Domains	Hashing, SIMD
Last Updated	2026-02-08 00:00 GMT

Overview

A high-performance string hashing implementation using hardware CRC32 instructions and SIMD optimizations for fast hash table operations.

Description

This code provides optimized hash functions and comparison operators for `std::string_view` types, leveraging CPU-specific instructions for maximum performance. It implements both `CRC32Hash` (using hardware CRC32 instructions on x86, ARM, and s390x) and `StringViewHash64` (using CityHash). The CRC32-based hash is significantly faster on real datasets when used in hash tables, despite being theoretically lower quality than CityHash. The implementation also includes SIMD-optimized string equality and comparison operators using SSE2/AVX or ARM NEON instructions to compare 16 or 64 bytes at a time.

Usage

Use this when you need high-performance string hashing for hash tables, especially when working with large volumes of string data. The CRC32 variant is optimized for real-world performance in database operations. Use `StringViewHash` (defaults to CRC32 if available) for hash tables, and use the SIMD equality operators for fast string comparisons in performance-critical code.

Code Reference

Source Location

• Repository: ClickHouse
• File: base/base/StringViewHash.h
• Lines: 1-336

Signature

// Hash functors
struct CRC32Hash {
    size_t operator()(std::string_view x) const;
};

struct StringViewHash64 {
    size_t operator()(std::string_view x) const;
};

// Default hash (CRC32 if available, otherwise CityHash64)
struct StringViewHash : CRC32Hash {};

// SIMD-optimized comparison functions
inline bool compare16(const char * p1, const char * p2);
inline bool compare64(const char * p1, const char * p2);
inline bool memequalWide(const char * p1, const char * p2, size_t size);

// Comparison operators for string_view
inline bool operator==(std::string_view lhs, std::string_view rhs);
inline bool operator!=(std::string_view lhs, std::string_view rhs);
inline bool operator<(std::string_view lhs, std::string_view rhs);
inline bool operator>(std::string_view lhs, std::string_view rhs);

// Helper functions
inline UInt64 hashLen16(UInt64 u, UInt64 v);
inline UInt64 shiftMix(UInt64 val);
inline size_t hashLessThan8(const char * data, size_t size);
inline size_t hashLessThan16(const char * data, size_t size);

Import

#include <base/StringViewHash.h>

I/O Contract

Inputs

Name	Type	Required	Description
x	std::string_view	Yes	String to hash
p1, p2	const char *	Yes	Pointers to memory regions to compare
size	size_t	Yes	Length of memory regions

Outputs

Name	Type	Description
hash	size_t	64-bit hash value
equal	bool	True if strings/memory regions are equal
comparison	int	Comparison result (-1, 0, or 1)

Usage Examples

#include <base/StringViewHash.h>
#include <unordered_map>

// Use in hash table
std::unordered_map<std::string_view, int, StringViewHash> string_map;
string_map["hello"] = 1;
string_map["world"] = 2;

// Explicit CRC32 hashing
CRC32Hash crc_hasher;
size_t hash1 = crc_hasher(std::string_view("test"));

// CityHash64 for higher quality (slower)
StringViewHash64 city_hasher;
size_t hash2 = city_hasher(std::string_view("test"));

// Fast string comparison
std::string_view s1 = "hello world";
std::string_view s2 = "hello world";
if (s1 == s2) {  // Uses SIMD-optimized equality
    // Strings are equal
}

// Direct memory comparison
const char * data1 = "0123456789ABCDEF";  // 16 bytes
const char * data2 = "0123456789ABCDEF";
if (compare16(data1, data2)) {  // SIMD compare 16 bytes
    // Memory regions are equal
}

// Wide memory equality check
const char * long_str1 = "this is a longer string for testing";
const char * long_str2 = "this is a longer string for testing";
if (memequalWide(long_str1, long_str2, strlen(long_str1))) {
    // Strings are equal (uses SIMD for chunks)
}

// Sort strings (uses optimized comparison)
std::vector<std::string_view> strings = {"zebra", "apple", "banana"};
std::sort(strings.begin(), strings.end());  // Uses optimized operator<

Related Pages

• Principle:ClickHouse_ClickHouse_SIMD_String_Hashing