Implementation:Apache Spark AmIpFilter

Knowledge Sources	Apache_Spark
Domains	Security, YARN, Web_UI
Last Updated	2026-02-08 22:00 GMT

Overview

Servlet filter that validates incoming web requests to the Spark UI originate from authorized YARN proxy hosts and handles redirection.

Description

AmIpFilter is a Jakarta Servlet `Filter` implementation annotated with Hadoop's `@Public` classification. It is copied from Hadoop 3.4.0's `org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter` and modified to use Jakarta Servlet APIs (migrated from javax.servlet). The filter maintains a cached set of proxy IP addresses (refreshed every 5 minutes) and checks each incoming request's remote address against this set. Requests from authorized proxy hosts pass through with the proxy user identity extracted from cookies. Requests from unauthorized sources are redirected to the active YARN ResourceManager proxy URL. It supports RM HA (High Availability) configurations by probing multiple RM URLs.

Usage

This filter is automatically configured in the Spark Web UI servlet container when running on YARN. It ensures that the Application Master's web UI is only accessible through the YARN web proxy, providing security isolation for Spark applications in multi-tenant YARN clusters.

Code Reference

Source Location

• Repository: Apache_Spark
• File: resource-managers/yarn/.../AmIpFilter.java
• Lines: 1-239

Signature

@Public
public class AmIpFilter implements Filter {

    public static final String PROXY_HOSTS = "PROXY_HOSTS";
    public static final String PROXY_URI_BASES = "PROXY_URI_BASES";
    public static final String PROXY_USER_COOKIE_NAME = "proxy-user";

    @Override
    public void init(FilterConfig conf) throws ServletException { ... }

    @Override
    public void doFilter(ServletRequest req, ServletResponse resp,
        FilterChain chain) throws IOException, ServletException { ... }

    protected Set<String> getProxyAddresses() throws ServletException { ... }

    public String findRedirectUrl() throws ServletException { ... }

    public boolean isValidUrl(String url) { ... }
}

Import

import org.apache.spark.deploy.yarn.AmIpFilter;

I/O Contract

Inputs

Name	Type	Required	Description
PROXY_HOSTS	FilterConfig param	Yes	Comma-separated list of proxy hostnames
PROXY_URI_BASES	FilterConfig param	Yes	Comma-separated list of proxy base URIs
RM_HA_URLS	FilterConfig param	No	Comma-separated RM HA URLs for failover
HttpServletRequest	Jakarta Servlet	Yes	Incoming HTTP request to filter

Outputs

Name	Type	Description
Pass-through	FilterChain	Authorized requests continue with proxy user principal set
Redirect	HTTP 302	Unauthorized requests redirect to YARN proxy URL

Usage Examples

Automatic YARN Integration

// AmIpFilter is typically configured automatically by Spark's YARN integration.
// Manual configuration in web.xml (for reference):
//
// <filter>
//   <filter-name>AmIpFilter</filter-name>
//   <filter-class>org.apache.spark.deploy.yarn.AmIpFilter</filter-class>
//   <init-param>
//     <param-name>PROXY_HOSTS</param-name>
//     <param-value>proxy1.example.com,proxy2.example.com</param-value>
//   </init-param>
//   <init-param>
//     <param-name>PROXY_URI_BASES</param-name>
//     <param-value>http://proxy1.example.com:8088/proxy/app_123,
//                  http://proxy2.example.com:8088/proxy/app_123</param-value>
//   </init-param>
// </filter>

Related Pages

• Principle:Apache_Spark_YARN_Web_Proxy_Security