A Saga of Code Executions on Zimbra

Zimbra is well known for its signature email product, Zimbra Collaboration Suite. Putting client-side vulnerabilities aside, Zimbra seems to have very little security history in the past. Its last critical bug was a Local File Disclosure back in 2013.

Recently with several new findings, it has been known that at least one potential Remote Code Execution exists in all versions of Zimbra. Specifically,

- Pre-Auth RCE on Zimbra <8.5.

- Pre-Auth RCE on Zimbra from 8.5 to 8.7.11.

- Auth'd RCE on Zimbra 8.8.11 and below with an additional condition that Zimbra uses Memcached. More on that in the next section.

Breaking Zimbra part 1

1. The XXE cavalry - CVE-2016-9924, CVE-2018-20160, CVE-2019-9670

Zimbra uses a large amount of XML handling for both its internal and external operations. With great XML usage comes great XXE vulnerabilities.

Back in 2016, another research has discovered CVE-2016-9924 with the bug locating in SoapEngine.chooseFaultProtocolFromBadXml(), which happens on the parsing of invalid XML requests. This code is used in all Zimbra instances version below 8.5. Note however, as there's no way to extract the output to the HTTP response, an out-of-band extraction method is required in exploiting it.

For more recent versions, CVE-2019-9670 works flawlessly where the XXE lies in the handling of Autodiscover requests. This can be applied on Zimbra from 8.5 to 8.7.11. And for the sake of completeness, CVE-2018-20160 is an XXE in the handling of XMPP protocol and an additional bug along CVE-2019-9670 is a prevention bypass in the sanitizing of XHTML documents which also leads to XXE, however they both require some additional conditions to trigger. These all allow direct file extraction through response.

It's worth to mention that exploiting out-of-band XXE on recent Java just got a lot harder due to a patch in the core FtpClient which makes it reject all FTP commands containing newline. This doesn't affect the exploits for the vulnerabilities mentioned above, but it did make some of my previous efforts to chain XXE with other bugs in vain.

On installation, Zimbra sets up a global admin for its internal SOAP communications, with the username 'zimbra' and a randomly generated password. These information are always stored in a local file named localconfig.xml. As such, a file-read vulnerability like XXE could potentially be catastrophic to Zimbra, since it allows an attacker to acquire the login information of a user with all the admin rights. This has been demonstrated as the case in a CVE-2013-7091 LFI exploit where under certain conditions, one could use such credentials to gain RCE.

However things have never been that easy. Zimbra manages user privileges via tokens, and it sets up an application model such that an admin token can only be granted to requests coming to the admin port, which by default is 7071. The aforementioned LFI exploit conveniently assumes we already have access to that port. But how often do you see the weirdo 7071 open to public?

2. SSRF to the rescue - CVE-2019-9621

If you can't access the port from public, let the application do it for you. The code at ProxyServlet.doProxy() does exactly what its name says, it proxies a request to another designated location. What's more, this servlet is available on the normal webapp and therefore accessible from public. Sweet! However the code has an additional protection, it checks whether the proxied target matches a set of predefined whitelisted domains. That is, unless the request is from an admin. Sounds right, an admin should be able to do what he wants.

(Un)Fortunately, the admin checks are flawed. First thing it checks is whether the request comes from port 7071. However it uses ServletRequest.getServerPort() to fetch the incoming port. This method returns a tainted input controllable by an attacker, which is the part after ':' in the Host header. What's more, after that the check for the admin token happens only if it is fetched from a parameter, meanwhile we can totally send a token via cookie! In short, if we send a request with 'foo:7071' Host header and a valid token in cookie, we can proxy a request to arbitrary targets that is otherwise only accessible to admins.

3. Pre-Auth RCE from public port

ProxyServlet still needs a valid token though, so how does this fit in a preauth RCE chain? Turns out Zimbra has a 'hidden' feature that can help us generate a normal user token under the special global 'zimbra' account. When we modify an ordinary SOAP AuthRequest which looks like this:

...<account by="name">tint0</account>...

into this:

...<account by="adminName">zimbra</account>...

Zimbra will then lookup all the admin accounts and proceed to check the password. This is actually quite surprising because Zimbra admins and users naturally reside in two different LDAP branches. A normal AuthRequest should only touch the normal user branch, never the other. If the application wants a token for an admin, it already has port 7071 for that.

Note that while this little trick could give us a token for the 'zimbra' user, this token doesn't have any of the admin flag in it as it's not coming from port 7071. This is when ProxyServlet jumps in, which will help us to proxy another admin AuthRequest to port 7071 and obtain a global admin token.

Now that we've got everything we need. The flow is to read the config file via XXE, generate a low-priv token through a normal AuthRequest, proxy an admin AuthRequest to the local admin port via ProxyServlet and finally, use the global admin token to upload a webshell via the ClientUploader extension.

Breaking Zimbra part 2

Zimbra has its own implementation of IMAP protocol, where it keeps a cache of the recently logged-in mailbox folders so that it doesn't have to load all the metadata from scratch next time. Zimbra serializes a user's mailbox folders to the cache on logging out and deserializes it when the same user logs in again.

It has three ways to maintain a cache: Memcached(network-based input), EhCache(memory-based) and file-based. If one fails, it tries the next in list. Of all of those, we can only hope to manipulate Memcached, and this is the condition of the exploit: Zimbra has to use Memcached as its caching mechanism. Even though Memcached is prioritized over the others, (un)fortunately on a single-server instance, the LDAP key zimbraMemcachedClientServerList isn't auto-populated, so Zimbra wouldn't know where the service is and will fail over to Ehcache. This is probably a bug in Zimbra itself, as Memcached service is up and running by default and that way it wouldn't have any data in it. On a multi-server install however, setting this key is expected as only Memcached can work accross many servers.

To check whether your Zimbra install is vulnerable, invoke this command on every node in the cluster and check if it returns a value:

$ zmprov gs `zmhostname` zimbraMemcachedClientServerList

This was assigned CVE-2019-6980. The deserialization process happens at ImapMemcachedSerializer.deserialize() and triggers on ImapHandler.doSELECT() i.e. when a user invoking an IMAP SELECT command. The IMAP port in most cases is publicly accessible, so we can safely assume the trigger of this exploit.

To bring this to RCE, one still needs to find a suitable gadget to form a chain. The twist is, none of the current public chains (ysoserial) works on Zimbra.

1. Making of a gadget

Of all the gadgets available, MozillaRhino1 particularly stands out as all classes in the chain are available on Zimbra's classpath. This chain is based on Rhino library version 1.7R2. Zimbra uses the lib yuicompressor version 2.4.2 for js compression, and yuicompressor is bundled with Rhino 1.6R7. The unfortunate thing is there's an internal bug in 1.6R7 that would break the MozillaRhino1 chain before it ever reaches code execution, so we're out of luck. The good thing is, thanks to the effort in attempting to get the original chain to work and to the blog post detailing the MozillaRhino1 chain [1], we learnt a lot about Rhino's internals and on our way to pop another gadget.

There are two main points. First, the class NativeJavaObject on deserialization will store all members of an object's class. Members refer to all elements that define a class such as variables and methods. In Rhino context, it also detects when there's a getter or setter member and if so, it declares and includes the corresponding bean as an additonal member of this class. Second, a call to NativeJavaObject.get() will search those members for a matching bean name and if one is found, invoke that bean's getter. These match the nature of one of the native 'gadget helpers' - TemplatesImpl.getOutputProperties(). Essentially if we can pass in the name 'outputProperties' in NativeJavaObject.get(), Rhino will invoke TemplatesImpl.getOutputProperties() which will eventually lead to the construction of a malicious class from our predefined bytecodes. Searching for a place that we can control the passed-in member name leads to the discovery of JavaAdapter.getObjectFunctionNames() (Thanks to the valuable help from @matthias_kaiser) and it's directly accessible from NativeJavaObject.readObject().

The chain is now available in ysoserial's payload storage under the name MozillaRhino2. It works all the way to the latest version (with some tweaks) and has some additional improvement over MozillaRhino1. One interesting thing I found while reading Matt's blog post is that OpenJDK 1.7.x always bundles with rhino as its scripting engine, which essentially means that these rhino gadgets may very well work natively on OpenJDK7 and below.

This discovery escalates the bug from a Memcached Injection into a Code Execution. To exploit it, query into the Memcached service, pop out any 'zmImap' key, replace its value with the serialized object from ysoserial and next time the corresponding user logins via IMAP, the deserialization will trigger.

2. Smuggling from HTTP to Memcached

RCE from port 11211 sounds fun, but less so practical. So again, we turn to SSRF for help. The idea is to use the HTTP request from SSRF to inject our defined data in Memcached. To accomplish this, first we need to control a field in the HTTP request that allows the injection of newlines (CRLF). This is because a CRLF in Memcached will denote the end of a command and allow us to start a new arbitrary command after that. Second, since we're pushing raw objects into Memcached, our controlled input also needs to be able to carry binary data.

Zimbra has quite a few SSRFs in itself, however there's only one place that suffices both conditions, and it happens to be the all-powerful ProxyServlet earlier.

For a successful smuggle from HTTP to Memcached protocol, you should see something like above under the hood. It has exactly 6 ERROR and 1 STORED, correlating to 6 lines of HTTP headers and our payload, which also means our payload was successfully injected.

3. RCE from public port

That said, things are different when we use SSRF to inject to Memcached. In this situation we could only inject data into the cache, not pop data out because HTTP protocol cannot parse Memcached response. So we have no idea what our targeted Memcached entry's key looks like, and we need to know the exact key to be able replace its value with our malicious payload.

Fortunately, the Memcached key for Zimbra Imap follows a structure that we can construct ourselves.

It follows the pattern

zmImap:<accountId>:<folderNo>:<modseq>:<uidvalidity>

with:
- accountId fetched from hex-decoding any login token
- folderNo the constant '2' if we target the user's Inbox folder
- modseq and uidvalidity obtained via IMAP as shown below

Now we have everything we need. Putting it together, the chain would be as follows:
- Get a user credentials
- Construct a Memcached key for that user following the above instructions
- Generate a ysoserial payload from the gadget MozillaRhino2, use it as the Memcached entry value.
- Inject the payload to Memcached via the SSRF. In the end, our payload should look like:

"set zmImap:61e0594d-dda9-4274-87d8-a2912470a35e:2:162:1 2048 3600 <size_of_object>" + "\r\n" + <object> + "\r\n"

- Login again via IMAP. Upon selecting the Inbox folder, the payload will get deserialized, followed by the RCE gadget.

The patches

Zimbra issued quite a number of patches, of which the most important are to fix XXEs and arbitrary deserialization. However the fix is only available for 8.7.11 and 8.8.x. If you happen to use an earlier version of Zimbra, consider upgrading to one of their supported version.

As a workaround, blocking public requests going to '/service/proxy*' would most likely break the RCE chains. Unfortunately there's none that I can think of that could block all the XXEs without also breaking some of Zimbra features.

Edit 30/04: Including a more specific workaround for the Autodiscover XXE and ProxyServlet SSRF which seem to be actively exploited. Locate {zimbra_home}/mailboxd/etc/service.web.xml.in, find the servlet tags named ProxyServlet and AutoDiscoverServlet, remove %%zimbraMailPort%% and %%zimbraMailSSLPort%% in both allowed.ports param and restart Zimbra. This would prevent public access to the affected components. Other than that, this thread provides some useful information suggested by savvy Zimbra user on how to clean an infected instance.

Author: Trinh Phuoc An