A few months ago, the news broke that CVE-2024-0012 and CVE-2024-9474 were under active exploitation in Palo Alto’s firewall. Described as a combination of an auth bypass and command execution, successful exploitation led to root on the affected devices.
At Assetnote, we not only wrote checks for the vulnerability but also had a look into how the patch worked. This is our standard practice so we can proactively support our customers.
As we looked further into the architecture of the management interface, we suspected something was off, even post-patch. This is the story of how we discovered a zero-day auth bypass in the PAN-OS management interface.
As always, Assetnote clients were informed immediately via our Attack Surface Management platform of newly detected vulnerabilities after disclosure. We worked with our clients to swiftly mitigate their issues to prevent any in-the-wild exploits. Assetnote’s research team continues to perform original security research to inform our customers about zero-day vulnerabilities in their attack surface.
Why is the Architecture Suspicious?
Let’s look at how authentication is implemented to understand why the architecture is suspicious.
Your web request to the management interface is handled by three separate components: Nginx, Apache, and the PHP application itself. First, your web request hits the Nginx reverse proxy. If your request is on a port that indicates it’s destined for the management interface, PAN-OS sets a bunch of headers:
proxy_set_header X-Real-IP "";
proxy_set_header X-Real-Scheme "";
proxy_set_header X-Real-Port "";
proxy_set_header X-Real-Server-IP "";
proxy_set_header X-Forwarded-For "";
proxy_set_header X-pan-ndpp-mode "";
proxy_set_header Proxy "";
proxy_set_header X-pan-AuthCheck 'on';
The most important of these is (as you may guess) X-pan-AuthCheck: on, which indicates to downstream that we should require authentication. The Nginx configuration then goes through a bunch of location checks and selectively sets the auth check to off:
if ($uri ~ ^\/unauth\/.+$) {
set $panAuthCheck 'off';
}
if ($uri = /php/logout.php) {
set $panAuthCheck 'off';
}
This means that (for example), if we pass /unauth/foo/bar/baz, we will have X-pan-AuthCheck: off set, which indicates to downstream that we should not require authentication.
The request is then proxied to Apache. _Apache will re-normalize and re-process the request_, as well as applying a rewrite rule if matching:
<Location "/">
DirectorySlash off
RewriteEngine on
RewriteRule ^(.*)(\/PAN_help\/)(.*)\.(css|js|html|htm)$ $1$2$3.$4.gz [QSA,L]
AddEncoding gzip .gz
Options Indexes FollowSymLinks
Require all granted
</Location>
If the file requested is a PHP file, Apache will then pass through the request via mod_php FCGI, which enforces authentication based upon the header:
if (
$_SERVER['HTTP_X_PAN_AUTHCHECK'] != 'off'
&& $_SERVER['PHP_SELF'] !== '/CA/ocsp'
&& $_SERVER['PHP_SELF'] !== '/php/login.php'
&& stristr($_SERVER['REMOTE_HOST'], '127.0.0.1') === false
) {
}
So why is this suspicious? Our authentication is set at the Nginx level and based on HTTP headers, which we can set almost arbitrarily. Our request is then re-processed again in Apache, which may process the path or headers differently to Nginx! Then and only then is the request handed down to PHP.
If there is a difference between what Nginx thinks our request looks like and what Apache thinks our request looks like, we could achieve an authentication bypass. As it turns out, Apache has some weird path-processing behavior.
Weird Apache Behavior
If you have been following Orange’s latest research on Apache, you may be suspicious of the above RewriteRule:
<Location "/">
RewriteRule ^(.*)(\/PAN_help\/)(.*)\.(css|js|html|htm)$ $1$2$3.$4.gz [QSA,L]
</Location>
However, because this RewriteRule is in a location block, this rewrite happens in the so-called “per-dir” context, which foils most of the attacks mentioned in the blog post. However, Apache still has some interesting implementation details in how it processes this RewriteRule. To explain this better, let’s consider this simplified configuration:
<Location "/">
RewriteEngine On
RewriteRule (.*).abc$ $1 [L]
</Location>
This rule strips the .abc extension from the end of a request, so /foo/hello.html and /foo/hello.html.abc end up requesting the same file. If we send a request to /foo/hello.html.abc, what happens in order is as follows:
- mod_rewrite gets the requested file as foo/hello.html.abc and tests it against the rewrite rule.
- Since the rule matches, mod_rewrite does the substitution and ends up with foo/hello.html
- mod_rewrite then appends a slash as the front to get /foo/hello.html and does an “internal redirect” – this involves re-processing the request from the start as if the user had requested /foo/hello.html directly.
The fact that Apache issues an internal redirect to implement per-directory rewrite rules has a couple of interesting consequences, which are not well documented. For example, environment variables set during the original request processing are dropped.
Crucially for us, because the request is being processed twice, another effect is that the URL will be _url decoded twice_. Any information security professional will know about URL encoding – if we request either of:
/foo/hello.html
/foo/hello%25html
We will see the contents of /foo/hello.html since the request is URL decoded before processing. However, if we double encode and request /foo/hello%252ehtml, we will get a 404. This is because Apache only URL decodes once, and a file hello%25html doesn’t exist.
However, if we request /foo/hello%252ehtml.abc, we do, in fact, see the content of hello.html! The URL is being decoded twice, once on the initial request and once on the internal redirect. If the rule is applied multiple times, this can get even crazier; a request like /foo/hello%25252ehtml.abc.abc will work to triple-decode the URL, etc.
The Bug
Let’s return to PAN-OS and consider the following URL:
/unauth/%252e%252e/php/ztp_gate.php/PAN_help/x.css
Nginx will URL decode this once, getting /unauth/%2e%2e/php/ztp_gate.php/PAN_help/x.css, and decide no normalisation of the path needs to take place, since there is no .. sequence. Then Nginx will test this $uri against the paths, and see it hits this block:
if ($uri ~ ^\/unauth\/.+$) { set $panAuthCheck 'off';}
Since Nginx thinks we are in the unauth directory, it will set the X-pan-AuthCheck header to off, and proxy the request to Apache. Apache will receive the full URL again:
/unauth/%252e%252e/php/ztp_gate.php/PAN_help/x.css
Apache will also URL decode this once:
/unauth/2e%2e/php/ztp_gate.php/PAN_help/x.css
Apache also decides it does not need any normalization. However, it will see that it matches the RewriteRule specified:
RewriteRule ^(.*)(\/PAN_help\/)(.*)\.(css|js|html|htm)$ $1$2$3.$4.gz [QSA,L]
So it will rewrite the URL to:
/unauth/%2e%2e/php/ztp_gate.php/PAN_help/x.css.gz
Apache then issues an internal redirect, which means the request is reprocessed. Apache receives the request again, but this time, the path URL decoded looks like this:
/unauth/../php/ztp_gate.php/PAN_help/x.css.gz
This is a directory traversal, which we have to normalise! So Apache rewrites the URL internally to resolve the traversal and now the URL looks like this, which will be passed to PHP’s FCGI via mod_php.
/php/ztp_gate.php/PAN_help/x.css.gz
mod_php sees that this looks like a request for a path SCRIPT_FILENAME=/php/ztp_gate.php with some path info afterwards (PATH_INFO=/PAN_help/x.css.gz), so it will execute /php/ztp_gate.php. And since the X-pan-AuthCheck header was set to off by Nginx, no authentication is required!
This leads to a full authentication bypass in the PAN-OS management interface.
GET /unauth/%252e%252e/php/ztp_gate.php/PAN_help/x.css HTTP/1.1
Host: my.testing.environment
Connection: close
...
HTTP/1.1 200 OK
Date: Mon, 02 Dec 2024 02:34:21 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
<html>
<head>
<title>Zero Touch Provisioning</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
...
Conclusion
In this blog post, we have explored a suspicious (and quite common) architecture where authentication is enforced at a proxy later, but then the request is passed through a second layer with different behavior. Fundamentally, these architectures lead to header smuggling and path confusion, which can result in many impactful bugs!
This issue is fixed in PAN-OS 10.2.14, PAN-OS 11.0.7, PAN-OS 11.2.5, and all later PAN-OS versions. Internally, this was assigned as PAN-273971. Palo Alto recommends whitelisting IPs in the management interface to prevent this or similar vulnerabilities from being exploited over the internet.
about assetnote
Assetnote provides industry-leading attack surface management and adversarial exposure validation solutions, helping organizations identify and remediate security vulnerabilities before they can be exploited. Assetnote customers receive security alerts and mitigations at the same time to disclosure to third-party vendors.
In January 2025 Assetnote was acquired by Searchlight Cyber. Combined, the companies form a holistic platform for combating external threats through Continuous Threat Exposure Management.
Read the press release to find out more.
