Implementation:Langchain ai Langchain Core Compatibility Tests

Overview

Concrete tool for verifying backward compatibility of new langchain-core releases against published partner packages, provided by the test-prior-published-packages-against-new-core job in _release.yml.

Description

This job only executes when the release is for libs/core (it exits early otherwise). It uses a matrix strategy to test multiple partner packages in parallel with fail-fast: false.

For each partner in the matrix (currently [anthropic]), the job:

1. Checks out the repository at the current ref.
2. Identifies the latest stable tag for the partner package by querying remote tags matching langchain-<partner>*, filtering out pre-releases, and sorting by version.
3. Shallow-fetches that specific tag.
4. Checks out the partner and standard-tests directories at the published tag, replacing the current HEAD versions.
5. Installs the partner's test and integration test dependencies via uv sync.
6. Overrides the langchain-core dependency by installing the newly-built core wheel from dist/.
7. Runs make integration_tests for the partner package.

The job requires API key secrets (e.g., ANTHROPIC_API_KEY, OPENAI_API_KEY) for the integration tests to call real services.

Usage

This job runs automatically in the release pipeline only for langchain-core releases. It depends on build, release-notes, test-pypi-publish, and pre-release-checks.

Code Reference

Source Location: .github/workflows/_release.yml (lines 384-472)

Conditional Guard:

- name: Check if libs/core
  run: |
    if [ "${{ startsWith(inputs.working-directory, 'libs/core') }}" != "true" ]; then
      echo "Not in libs/core. Exiting successfully."
      exit 0
    fi

Matrix Strategy:

strategy:
  matrix:
    partner: [anthropic]
  fail-fast: false

Latest Tag Discovery and Partner Checkout:

# Identify latest stable tag (excluding pre-releases)
LATEST_PACKAGE_TAG="$(
  git ls-remote --tags origin "langchain-${{ matrix.partner }}*" \
  | awk '{print $2}' \
  | sed 's|refs/tags/||' \
  | grep -E '[0-9]+\.[0-9]+\.[0-9]+$' \
  | sort -Vr \
  | head -n 1
)"
echo "Latest package tag: $LATEST_PACKAGE_TAG"

# Shallow-fetch just that single tag
git fetch --depth=1 origin tag "$LATEST_PACKAGE_TAG"

# Checkout published partner files
rm -rf $GITHUB_WORKSPACE/libs/partners/${{ matrix.partner }}/*
rm -rf $GITHUB_WORKSPACE/libs/standard-tests/*
cd $GITHUB_WORKSPACE/libs/
git checkout "$LATEST_PACKAGE_TAG" -- standard-tests/
git checkout "$LATEST_PACKAGE_TAG" -- partners/${{ matrix.partner }}/
cd partners/${{ matrix.partner }}

# Sanity check
cat pyproject.toml | grep "version = "

Install and Test:

# Install partner dependencies
uv sync --group test --group test_integration

# Override core with the new build
uv pip install ../../core/dist/*.whl

# Run integration tests
make integration_tests

Invocation: Automatic, only for libs/core releases.

I/O Contract

Direction	Name	Type	Description
Input	working-directory	string	Must start with libs/core for this job to execute
Input	core dist/*.whl	File	Newly-built langchain-core wheel
Input	matrix.partner	string	Partner package name (e.g., anthropic)
Input	API key secrets	Secrets	ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.
Output	Test results	Pass/Fail	Integration tests for each partner must pass
Output	LATEST_PACKAGE_TAG	string (log)	The tag of the latest published partner package (e.g., langchain-anthropic==0.3.12)

Usage Examples

Example 1: Simulating the tag discovery locally

# Find latest stable langchain-anthropic tag
git ls-remote --tags origin "langchain-anthropic*" \
  | awk '{print $2}' \
  | sed 's|refs/tags/||' \
  | grep -E '[0-9]+\.[0-9]+\.[0-9]+$' \
  | sort -Vr \
  | head -n 1

Example 2: Testing locally with a new core wheel

# Build core
cd libs/core && uv build

# Go to partner
cd ../partners/anthropic
uv sync --group test --group test_integration
uv pip install ../../core/dist/*.whl

# Run tests
make integration_tests

Example 3: Adding a new partner to the matrix

strategy:
  matrix:
    partner: [anthropic, openai]
  fail-fast: false

Related Pages

• Principle:Langchain_ai_Langchain_Compatibility_Testing
• Environment:Langchain_ai_Langchain_Unit_Test_Network_Isolation