Implementation:Apache Spark K8s Entrypoint

Metadata	Value
Source	Repo: Apache Spark
Domains	Kubernetes
Type	API Doc
Related	Principle:Apache_Spark_K8s_Container_Lifecycle

Overview

Container entrypoint and decommission scripts for Spark on Kubernetes pod lifecycle management.

Description

Two scripts manage the container lifecycle in Spark on Kubernetes:

entrypoint.sh

The main container entrypoint (resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh, lines 1-119) routes execution based on the first argument:

• driver -- Launches spark-submit in client mode. Sets spark.driver.bindAddress and spark.executorEnv.SPARK_DRIVER_POD_IP from $SPARK_DRIVER_BIND_ADDRESS. The remaining arguments are passed through to spark-submit.
• executor -- Launches the JVM directly with org.apache.spark.scheduler.cluster.k8s.KubernetesExecutorBackend. The JVM is configured with:
  • Memory: -Xms$SPARK_EXECUTOR_MEMORY -Xmx$SPARK_EXECUTOR_MEMORY
  • Classpath: $SPARK_CLASSPATH:$SPARK_DIST_CLASSPATH
  • Parameters: --driver-url, --executor-id, --cores, --app-id, --hostname, --resourceProfileId, --podName
• Anything else -- Pass-through mode; the command and arguments are executed directly.

Before routing, the entrypoint performs initialization:

1. Checks for anonymous UID and creates a /etc/passwd entry if needed (for OpenShift compatibility).
2. Resolves JAVA_HOME if not set.
3. Builds SPARK_CLASSPATH from $SPARK_HOME/jars/*, $SPARK_EXTRA_CLASSPATH, $HADOOP_CONF_DIR, $SPARK_CONF_DIR, and the current working directory.
4. Collects SPARK_JAVA_OPT_* environment variables, sorts them numerically, and stores them as JVM options.
5. Exports PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON if set.
6. Sets SPARK_DIST_CLASSPATH from Hadoop if HADOOP_HOME is defined.

All commands are executed under /usr/bin/tini -s as PID 1 for proper signal handling and zombie reaping.

decom.sh

The decommission script (resource-managers/kubernetes/docker/src/main/dockerfiles/spark/decom.sh, lines 1-41) performs graceful executor shutdown:

1. Finds the executor Java process PID by searching for processes matching Executor in the java command output.
2. Sends SIGPWR to the executor JVM, triggering the Spark decommission handler.
3. Waits for the process to exit using tail --pid.
4. Exits after the process terminates.

This script is invoked by Kubernetes preStop hooks to enable graceful decommissioning during pod eviction or scale-down events.

Usage

These are container-internal scripts:

• Modify entrypoint.sh to customize container initialization (e.g., fetch credentials, install packages, configure additional environment).
• decom.sh is invoked by Kubernetes preStop hooks and generally does not need modification unless the decommission signal or wait behavior needs to change.

Code Reference

Item	Reference
Entrypoint script	resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh (L1-119)
Decommission script	resource-managers/kubernetes/docker/src/main/dockerfiles/spark/decom.sh (L1-41)
PID 1 init	/usr/bin/tini

Entrypoint Modes

First Argument	Action	Process Launched
driver	Spark driver launch	spark-submit --deploy-mode client
executor	Spark executor launch	java org.apache.spark.scheduler.cluster.k8s.KubernetesExecutorBackend
(anything else)	Pass-through	Direct execution of provided arguments

Key Environment Variables

Variable	Used By	Description
SPARK_DRIVER_BIND_ADDRESS	Driver	Address for the driver to bind to
SPARK_EXECUTOR_MEMORY	Executor	JVM heap size (-Xms/-Xmx)
SPARK_DRIVER_URL	Executor	RPC URL to reach the driver
SPARK_EXECUTOR_ID	Executor	Unique executor identifier
SPARK_EXECUTOR_CORES	Executor	Number of cores allocated to this executor
SPARK_APPLICATION_ID	Executor	Spark application identifier
SPARK_EXECUTOR_POD_IP	Executor	Pod IP used as executor hostname
SPARK_RESOURCE_PROFILE_ID	Executor	Resource profile for dynamic allocation
SPARK_EXECUTOR_POD_NAME	Executor	Kubernetes pod name
SPARK_JAVA_OPT_*	Both	Numbered JVM options (sorted numerically)
SPARK_EXTRA_CLASSPATH	Both	Additional classpath entries
SPARK_USER_NAME	Both	Username for anonymous UID passwd entry
PYSPARK_PYTHON	Both	Python interpreter path
HADOOP_HOME	Both	Hadoop installation for classpath discovery

Inputs and Outputs

Direction	Description
Inputs	Container environment variables, first argument (driver, executor, or other)
Outputs	Running Spark process within the container

Examples

Driver mode

/opt/entrypoint.sh driver \
  --conf spark.app.name=test \
  --class org.apache.spark.examples.SparkPi \
  local:///opt/spark/examples/jars/spark-examples.jar

Executor mode

# Typically invoked by the Kubernetes scheduler with environment variables set:
# SPARK_DRIVER_URL, SPARK_EXECUTOR_ID, SPARK_EXECUTOR_CORES,
# SPARK_APPLICATION_ID, SPARK_EXECUTOR_MEMORY, etc.
/opt/entrypoint.sh executor

Pass-through mode

# Run an arbitrary command in the Spark container
/opt/entrypoint.sh /bin/bash -c "echo hello"

Decommission flow

# Invoked by Kubernetes preStop hook:
# 1. Finds the executor Java process
# 2. Sends SIGPWR to trigger graceful decommission
# 3. Waits for the process to exit
/opt/decom.sh

The decommission script executes the following sequence:

# Find executor PID
WORKER_PID=$(ps -o pid,cmd -C java | grep Executor \
  | tail -n 1 | awk '{ sub(/^[ \t]+/, ""); print }' \
  | cut -f 1 -d " ")

# Send SIGPWR for graceful decommission
kill -s SIGPWR ${WORKER_PID}

# Wait for process to exit
tail --pid=${WORKER_PID} -f /dev/null

Related

• Principle:Apache_Spark_K8s_Container_Lifecycle
• Environment:Apache_Spark_Kubernetes_Runtime
• Heuristic:Apache_Spark_K8s_Container_Patterns