Implementation:Recommenders team Recommenders DKN Iterator

Knowledge Sources	Recommenders
Domains	Recommendation Systems, Knowledge Graphs, Data Loading
Last Updated	2026-02-10 00:00 GMT

Overview

DKNTextIterator is a specialized data loader for the Deep Knowledge-Aware Network (DKN) model, handling the multi-modal data format that combines news article word indices, entity indices, and user click history.

Description

The DKNTextIterator class extends BaseIterator to provide data loading capabilities tailored to the DKN model's unique requirements. DKN requires each training instance to contain a label, a candidate news article (represented by aligned word and entity indices), and the user's clicked news history (also represented by word and entity indices).

During initialization, the iterator loads two external files: a news feature file that maps news IDs to their word index and entity index arrays, and a user history file that maps user IDs to their lists of clicked news articles. User histories are truncated to a configurable history_size and padded with zeros if shorter. TensorFlow placeholders are created for candidate news indices, click news indices, candidate entity indices, click entity indices, and labels.

The load_data_from_file method reads instances line by line, parses labels and user/candidate news IDs, looks up pre-loaded feature dictionaries, and yields mini-batches. Incomplete final batches are padded to the full batch size by repeating earlier samples, ensuring consistent tensor shapes. The class also provides load_infer_data_from_file for inference-time document embedding generation, which loads news articles without user history context.

Usage

Use DKNTextIterator when working with the DKN model for knowledge-aware news recommendation. It handles the specialized data format where each news article must have both word-level and entity-level representations, and user click histories must be aligned with both feature types.

Code Reference

Source Location

• Repository: Recommenders
• File: recommenders/models/deeprec/io/dkn_iterator.py
• Lines: 1-373

Signature

class DKNTextIterator(BaseIterator):
    def __init__(self, hparams, graph, col_spliter=" ", ID_spliter="%"):

    def parser_one_line(self, line):
        # Returns: (label, candidate_news_index, click_news_index,
        #           candidate_news_entity_index, click_news_entity_index,
        #           impression_id)

    def load_data_from_file(self, infile):
        # Yields: (feed_dict, impression_id_list, data_size)

    def load_infer_data_from_file(self, infile):
        # Yields: (feed_dict, newsid_list, data_size)

    def gen_feed_dict(self, data_dict):
        # Returns: dict

    def gen_infer_feed_dict(self, data_dict):
        # Returns: dict

Import

from recommenders.models.deeprec.io.dkn_iterator import DKNTextIterator

I/O Contract

Inputs

Name	Type	Required	Description
hparams	object	Yes	Global hyper-parameters object containing batch_size, doc_size, history_size, news_feature_file, and user_history_file
graph	tf.Graph	Yes	The TensorFlow graph to which all created placeholders will be added
col_spliter	str	No	Column separator in one line (default: " ")
ID_spliter	str	No	ID separator in one line (default: "%")

Outputs

Name	Type	Description
load_data_from_file()	generator	Yields tuples of (feed_dict, impression_id_list, data_size) for each mini-batch
load_infer_data_from_file()	generator	Yields tuples of (feed_dict, newsid_list, data_size) for inference document embedding
parser_one_line()	tuple	Returns (label, candidate_news_index, click_news_index, candidate_news_entity_index, click_news_entity_index, impression_id)

TensorFlow Placeholders

Placeholder	Shape	Type	Description
labels	[None, 1]	tf.float32	Ground-truth labels
candidate_news_index_batch	[batch_size, doc_size]	tf.int64	Word indices for the candidate news article
click_news_index_batch	[batch_size, history_size, doc_size]	tf.int64	Word indices for user's clicked news articles
candidate_news_entity_index_batch	[batch_size, doc_size]	tf.int64	Entity indices for the candidate news article
click_news_entity_index_batch	[batch_size, history_size, doc_size]	tf.int64	Entity indices for user's clicked news articles

Usage Examples

Basic Usage

import tensorflow as tf
from recommenders.models.deeprec.io.dkn_iterator import DKNTextIterator

# Assume hparams is configured with:
#   hparams.batch_size = 64
#   hparams.doc_size = 10
#   hparams.history_size = 50
#   hparams.news_feature_file = "news_features.txt"
#   hparams.user_history_file = "user_history.txt"

graph = tf.Graph()
iterator = DKNTextIterator(hparams, graph)

# Load training data in mini-batches
train_file = "train_data.txt"
for batch_feed_dict, impression_ids, batch_size in iterator.load_data_from_file(train_file):
    # batch_feed_dict can be directly fed into sess.run()
    # impression_ids tracks which impressions are in this batch
    # batch_size indicates actual data size (may differ in last batch)
    pass

Inference Document Embedding

import tensorflow as tf
from recommenders.models.deeprec.io.dkn_iterator import DKNTextIterator

graph = tf.Graph()
iterator = DKNTextIterator(hparams, graph)

# Load news articles for document embedding inference
news_file = "news_features.txt"
for batch_feed_dict, newsid_list, batch_size in iterator.load_infer_data_from_file(news_file):
    # batch_feed_dict contains only candidate news indices (no user history)
    # newsid_list contains the news IDs in this batch
    pass