Implementation:Apache Spark Start Master
| Field | Value |
|---|---|
| Source | Repo Apache Spark |
| Domains | Deployment |
| Type | API Doc |
| Related | Principle:Apache_Spark_Master_Daemon_Management |
Overview
Shell script that starts the Spark standalone master daemon on the local machine.
Description
sbin/start-master.sh starts the Spark master process (org.apache.spark.deploy.master.Master) by:
- Sourcing configuration from spark-config.sh and load-spark-env.sh
- Determining the hostname via
hostname -fif not set - Setting default ports for RPC (7077) and Web UI (8080)
- Delegating to spark-daemon.sh for actual process management
The master listens for worker connections on the RPC port and provides a Web UI for cluster monitoring at the configured Web UI port.
Usage
Run on the designated master node to start the Spark standalone cluster coordinator. This must be executed before starting any workers, as workers need the master URL to register.
Code Reference
Source: sbin/start-master.sh (L1-69), sbin/spark-daemon.sh (L1-269)
Signature:
sbin/start-master.sh [--host <host>] [--port <port>] [--webui-port <port>]
Default environment variables:
| Variable | Default | Purpose |
|---|---|---|
| SPARK_MASTER_HOST | hostname -f | Network interface the master binds to |
| SPARK_MASTER_PORT | 7077 | RPC port for worker and driver connections |
| SPARK_MASTER_WEBUI_PORT | 8080 | HTTP port for the monitoring Web UI |
Underlying class: org.apache.spark.deploy.master.Master
I/O
| Direction | Description |
|---|---|
| Inputs | Optional CLI args (--host, --port, --webui-port), env vars from spark-env.sh |
| Outputs | Running Master JVM process, PID file at $SPARK_PID_DIR, log file at $SPARK_LOG_DIR, Web UI at http://<host>:8080 |
Examples
Default start (binds to hostname, port 7077, WebUI 8080):
./sbin/start-master.sh
Custom host and ports:
./sbin/start-master.sh --host 0.0.0.0 --port 7077 --webui-port 9090
Verify the master is running:
curl http://localhost:8080