Implementation:Recommenders team Recommenders DeepRec Iterator

Knowledge Sources	Recommenders
Domains	Recommendation Systems, Data Loading, Deep Learning Infrastructure
Last Updated	2026-02-10 00:00 GMT

Overview

This module defines the abstract BaseIterator interface and the FFMTextIterator for loading FFM-format data into deep recommendation models such as xDeepFM.

Description

The module contains two classes:

BaseIterator is an abstract base class that defines the standard interface for all data iterators in the DeepRec framework. It declares four abstract methods: parser_one_line (parse a single line into feature values), load_data_from_file (read and parse data from a file), _convert_data (convert parsed data to numpy arrays), and gen_feed_dict (construct a TensorFlow feed dictionary from the converted data). All concrete iterator implementations in the DeepRec ecosystem inherit from this class.

FFMTextIterator implements the BaseIterator interface for Field-aware Factorization Machine (FFM) format data, as used by models like xDeepFM. It creates TensorFlow placeholders for labels, FM feature indices/values/shapes, and DNN feature indices/values/weights/shapes. The parser_one_line method parses field:feature:value triples from text lines. The _convert_data method builds both FM-style and DNN-style sparse feature representations as numpy arrays. Data is loaded in mini-batches to keep memory usage low, allowing large files to be used as input.

Usage

Use BaseIterator as the parent class when implementing a new data iterator for a DeepRec model. Use FFMTextIterator directly when working with xDeepFM or any model that requires FFM-format input data with field:feature:value triples.

Code Reference

Source Location

• Repository: Recommenders
• File: recommenders/models/deeprec/io/iterator.py
• Lines: 1-241

Signature

class BaseIterator(object):
    @abc.abstractmethod
    def parser_one_line(self, line):
    @abc.abstractmethod
    def load_data_from_file(self, infile):
    @abc.abstractmethod
    def _convert_data(self, labels, features):
    @abc.abstractmethod
    def gen_feed_dict(self, data_dict):


class FFMTextIterator(BaseIterator):
    def __init__(self, hparams, graph, col_spliter=" ", ID_spliter="%"):

    def parser_one_line(self, line):
        # Returns: (label, features, impression_id)

    def load_data_from_file(self, infile):
        # Yields: (feed_dict, impression_id_list, batch_size)

    def _convert_data(self, labels, features):
        # Returns: dict

    def gen_feed_dict(self, data_dict):
        # Returns: dict

Import

from recommenders.models.deeprec.io.iterator import BaseIterator
from recommenders.models.deeprec.io.iterator import FFMTextIterator

I/O Contract

Inputs (FFMTextIterator.__init__)

Name	Type	Required	Description
hparams	object	Yes	Global hyper-parameters with FEATURE_COUNT, FIELD_COUNT, and batch_size settings
graph	tf.Graph	Yes	The TensorFlow graph to which all created placeholders will be added
col_spliter	str	No	Column separator in one line (default: " ")
ID_spliter	str	No	ID separator in one line (default: "%")

Outputs

Name	Type	Description
load_data_from_file()	generator	Yields tuples of (feed_dict, impression_id_list, batch_size) for each mini-batch
parser_one_line()	tuple	Returns (label, features, impression_id) where features is a list of [field_idx, feature_idx, feature_value] triples

TensorFlow Placeholders (FFMTextIterator)

Placeholder	Shape	Type	Description
labels	[None, 1]	tf.float32	Ground-truth labels
fm_feat_indices	[None, 2]	tf.int64	FM feature sparse indices
fm_feat_values	[None]	tf.float32	FM feature values
fm_feat_shape	[None]	tf.int64	FM feature tensor shape
dnn_feat_indices	[None, 2]	tf.int64	DNN feature sparse indices
dnn_feat_values	[None]	tf.int64	DNN feature values (integer feature IDs)
dnn_feat_weights	[None]	tf.float32	DNN feature weights
dnn_feat_shape	[None]	tf.int64	DNN feature tensor shape

Class Hierarchy

BaseIterator (abstract)
 +-- FFMTextIterator         (FFM-format data for xDeepFM)
 +-- DKNTextIterator         (news data for DKN)
 +-- SequentialIterator      (sequential data for A2SVD, Caser, GRU, SLI_REC, SUM)
      +-- NextItNetIterator  (sequence-level predictions for NextItNet)

Usage Examples

Basic Usage with FFMTextIterator

import tensorflow as tf
from recommenders.models.deeprec.io.iterator import FFMTextIterator

# hparams must have FEATURE_COUNT, FIELD_COUNT, and batch_size
# Example: hparams.FEATURE_COUNT = 10000
#          hparams.FIELD_COUNT = 15
#          hparams.batch_size = 128

graph = tf.Graph()
iterator = FFMTextIterator(hparams, graph)

# Load training data in mini-batches from FFM-format file
# Each line format: label field1:feat1:val1 field2:feat2:val2 ...
train_file = "train_ffm.txt"
for feed_dict, impression_ids, batch_size in iterator.load_data_from_file(train_file):
    # feed_dict is ready for sess.run()
    pass

Implementing a Custom Iterator

from recommenders.models.deeprec.io.iterator import BaseIterator

class CustomIterator(BaseIterator):
    def parser_one_line(self, line):
        # Parse a single line of custom-format data
        cols = line.strip().split("\t")
        label = float(cols[0])
        features = [float(x) for x in cols[1:]]
        return label, features, 0

    def load_data_from_file(self, infile):
        # Read data in mini-batches
        pass

    def _convert_data(self, labels, features):
        # Convert to numpy arrays
        pass

    def gen_feed_dict(self, data_dict):
        # Build TensorFlow feed dictionary
        pass