Implementation:Ggml org Llama cpp Console Readline

Aspect	Detail
Implementation Name	Console Readline
Doc Type	Pattern Doc
Category	Input/Output
Workflow	Interactive_Chat
Applies To	llama.cpp
Status	Active

Overview

Description

This pattern documents two approaches to reading user input in llama.cpp chat applications: the simple approach using std::getline(std::cin, user) (as seen in simple-chat), and the advanced approach using console::readline(line, multiline_input) from the common library. The simple approach provides basic single-line input, while the advanced approach provides a full line-editing experience with cursor navigation, input history, multiline support, and cross-platform UTF-8 handling.

Usage

The simple approach is used in minimal examples where input complexity is not a concern. The advanced approach is used in the full llama-cli tool and other interactive applications that require richer input capabilities. Both approaches block until the user submits their input.

Code Reference

Attribute	Value
Source Location (simple)	`examples/simple-chat/simple-chat.cpp:159-165`
Source Location (advanced declaration)	`common/console.h:22`
Source Location (advanced implementation)	`common/console.cpp:1021-1040` (simple path), `common/console.cpp:743-1019` (advanced path)
Import (simple)	`#include <iostream>`
Import (advanced)	`#include "console.h"`

Signatures:

// Simple approach (C++ standard library)
std::getline(std::cin, user);

// Advanced approach (llama.cpp common library)
namespace console {
    void init(bool use_simple_io, bool use_advanced_display);
    void cleanup();
    void set_display(display_type display);
    bool readline(std::string & line, bool multiline_input);
}

I/O Contract

Simple approach (std::getline):

Direction	Name	Type	Description
Input	std::cin	`std::istream &`	Standard input stream
Output	user	`std::string &`	The line read from input (without newline)
Output	return (implicit)	`bool`	Stream state: false if EOF or error

Advanced approach (console::readline):

Direction	Name	Type	Description
Input/Output	line	`std::string &`	On return, contains the text entered by the user (with trailing newline)
Input	multiline_input	`bool`	If true, Enter continues input; backslash toggles behavior
Output	return	`bool`	True if more input lines are expected (multiline continuation)

Preconditions (advanced):

console::init(use_simple_io, use_advanced_display) must be called before first use
Terminal must support the required I/O modes

Postconditions:

The returned string contains the user's input
For the simple approach, an empty string indicates the user wants to end the conversation
For the advanced approach, a return value of false indicates input is complete for the current turn

Usage Examples

Simple approach (from simple-chat):

while (true) {
    // Display prompt
    printf("\033[32m> \033[0m");
    std::string user;
    std::getline(std::cin, user);

    if (user.empty()) {
        break;  // end conversation on empty input
    }

    // ... process user input ...
}

Advanced approach (from llama-cli pattern):

// Initialize console at startup
console::init(/* use_simple_io */ false, /* use_advanced_display */ true);

// In chat loop
std::string line;
console::set_display(DISPLAY_TYPE_USER_INPUT);
bool need_more = console::readline(line, /* multiline_input */ false);

while (need_more) {
    // Continue reading multiline input
    std::string next;
    need_more = console::readline(next, /* multiline_input */ false);
    line += next;
}

// Cleanup at shutdown
console::cleanup();

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment