Implementation:Testtimescaling Testtimescaling github io Json Paper Registration

Type	Pattern Doc (manual JSON edit)
Source	`papers.json:L1-6`, `.github/scripts/papers.json:L1-6`, `.github/scripts/update_arxiv_citations.py:L22-25`
Domains	Data_Management, Automation
Last Updated	2026-02-14

Overview

A manual JSON editing process for registering a new paper in the citation tracking system by updating two JSON files and one Python script with hardcoded arXiv IDs.

Description

This pattern documents the triple-update process required to register a new paper for automated citation tracking. Due to the current architecture, three separate files must be modified in a specific way.

The process is straightforward but error-prone because of the redundancy. Missing any of the three update locations will result in incomplete citation tracking. Contributors must update all three files in a single commit to maintain consistency.

File 1: Root papers.json

The primary JSON registry at the repository root. Contains an array of paper objects. A new entry is appended to this array.

File 2: Workflow .github/scripts/papers.json

An identical copy of the root registry, located alongside the GitHub Actions scripts. Must be kept in exact sync with the root copy.

File 3: Python script .github/scripts/update_arxiv_citations.py

The automation script that fetches citation counts from the Semantic Scholar API. At approximately lines 22-25, it contains a hardcoded Python list of arXiv IDs. The new paper's arXiv ID must be added to this list.

Known issue: The Python script does not read from either papers.json file. Instead, it uses a hardcoded list. This means that even if both JSON files are updated correctly, the citation counts will not be fetched unless the Python script is also updated. This is a known technical debt item.

Usage

Perform this registration after adding the paper to the comparison table (Step 3). All three files should be modified and committed together. After committing, proceed to integration verification (Step 5) to confirm the citation tracking pipeline works correctly with the new paper.

Code Reference

Source Location

Three files must be modified:

papers.json (repository root)
.github/scripts/papers.json
.github/scripts/update_arxiv_citations.py (lines 22-25)

Interface Specification

JSON Entry Format (for both papers.json files):

{
  "title": "<Full Paper Title>",
  "arxiv_id": "<XXXX.XXXXX>"
}

Current state of papers.json (both copies):

[
  {
    "title": "What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models",
    "arxiv_id": "2503.24235"
  }
]

Python script arXiv ID list format (at update_arxiv_citations.py:L22-25):

arxiv_ids = [
    "2503.24235",
    # Add new arXiv IDs here
]

Complete update process:

REGISTRATION INTERFACE
=======================

Input:
  - paper_title: string (exact paper title)
  - arxiv_id: string (format "XXXX.XXXXX")

Steps:
  1. OPEN papers.json (root)
  2. APPEND new object to the JSON array:
     {"title": "<paper_title>", "arxiv_id": "<arxiv_id>"}
  3. OPEN .github/scripts/papers.json
  4. APPEND the same object to keep in sync
  5. OPEN .github/scripts/update_arxiv_citations.py
  6. FIND the arxiv_ids list (approx. lines 22-25)
  7. ADD the new arxiv_id string to the list
  8. COMMIT all three files together

Output:
  - Updated papers.json (root)
  - Updated .github/scripts/papers.json
  - Updated .github/scripts/update_arxiv_citations.py

Import

No imports required. This is a manual file editing process. Use any text editor or the GitHub web interface.

I/O Contract

Inputs

Parameter	Type	Required	Description
paper_title	String	Yes	The exact title of the paper, matching what was used in the comparison table
arxiv_id	String	Yes	arXiv identifier in format `XXXX.XXXXX`

Outputs

Output	Type	Description
root_papers_json	File	Updated `papers.json` at repository root with new entry appended
scripts_papers_json	File	Updated `.github/scripts/papers.json` with identical new entry
update_script	File	Updated `.github/scripts/update_arxiv_citations.py` with new arXiv ID in hardcoded list

Usage Examples

Example 1: Adding a new paper to all three files

Step 1 -- Update root papers.json:

[
  {
    "title": "What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models",
    "arxiv_id": "2503.24235"
  },
  {
    "title": "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters",
    "arxiv_id": "2408.03314"
  }
]

Step 2 -- Update .github/scripts/papers.json identically:

[
  {
    "title": "What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models",
    "arxiv_id": "2503.24235"
  },
  {
    "title": "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters",
    "arxiv_id": "2408.03314"
  }
]

Step 3 -- Update .github/scripts/update_arxiv_citations.py:

arxiv_ids = [
    "2503.24235",
    "2408.03314",
]

Step 4 -- Commit all three files:

git add papers.json .github/scripts/papers.json .github/scripts/update_arxiv_citations.py
git commit -m "Register new paper: Scaling LLM Test-Time Compute (2408.03314)"

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment