Principle:Apache Dolphinscheduler Connection Pool Management
| Knowledge Sources | |
|---|---|
| Domains | Connection_Management, Performance |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
A cached connection pool management system that maintains HikariCP-backed datasource pools per unique connection configuration, with Guava cache for automatic eviction of idle pools.
Description
The Connection Pool Management principle addresses the need for efficient, reusable database connections during task execution. DataSourceClientProvider maintains a Guava Cache<String, PooledDataSourceClient> with configurable expireAfterAccess eviction. When a pooled connection is requested, the provider checks the cache; on miss, it creates a new PooledDataSourceClient via the DataSourceChannel. The underlying BasePooledDataSourceClient configures a HikariCP pool with configurable maximum pool size, minimum idle connections, and validation query.
This two-level caching (Guava cache for client instances, HikariCP pool for JDBC connections) provides both efficient connection reuse within a task and efficient pool reuse across tasks hitting the same datasource.
Usage
Use DataSourceClientProvider.getPooledConnection() when executing SQL tasks that need efficient database access. The provider handles pool creation, caching, and lifecycle management automatically.
Theoretical Basis
The pool management combines two caching layers:
- Level 1 (Guava Cache): Caches PooledDataSourceClient instances by connection key
- Level 2 (HikariCP): Each client maintains a JDBC connection pool
getPooledConnection(dbType, connectionParam):
key = buildCacheKey(dbType, connectionParam)
client = guavaCache.get(key, () ->
channel.createPooledDataSourceClient(connectionParam, dbType)
)
return client.getConnection() // from HikariCP pool
// Eviction: expireAfterAccess triggers RemovalListener
onRemoval(key, client):
client.close() // closes HikariDataSource, releasing all connections