Principle:HKUDS AI Trader Crypto Data Fetching
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, Financial_Data, Cryptocurrency |
| Last Updated | 2026-02-09 14:00 GMT |
Overview
A data acquisition technique for retrieving daily cryptocurrency OHLCV data from financial APIs with support for data merging and backup.
Description
Crypto Data Fetching adapts the price data acquisition pattern for cryptocurrency markets. Unlike equities, crypto markets trade 24/7 and use different symbol conventions (e.g., BTC, ETH vs. USD pairs). The fetcher handles crypto-specific concerns: data backup before overwrite, merging new data with existing historical data, and conversion to a standardized format compatible with the trading agent pipeline.
The target universe includes major cryptocurrencies: BTC, ETH, XRP, SOL, ADA, SUI, LINK, AVAX, LTC, and DOT.
Usage
Use this principle when building a backtesting pipeline for cryptocurrency markets. It is the data acquisition step for the crypto variant of the trading agent, fetching daily OHLCV data for a fixed set of major cryptocurrencies.
Theoretical Basis
# Pseudocode for crypto data fetching
backup_existing_data()
for each symbol in [BTC, ETH, XRP, SOL, ADA, ...]:
new_data = API_GET(DIGITAL_CURRENCY_DAILY, symbol, market="USD")
existing_data = load_existing(daily_prices_{symbol}.json)
merged = merge(new_data, existing_data)
save(merged, daily_prices_{symbol}.json)
Key differences from equity fetching:
- Data backup: Existing data is backed up before overwriting
- Incremental merge: New data is merged with existing historical data
- 24/7 markets: No trading day filtering needed
- USD pairing: All prices quoted against USD