In-JVM Reverse Proxies on Domino

Sun Jul 04 17:04:01 EDT 2021

A while back, I added built-in reverse proxy capabilities to the Open Liberty Runtime, first as a standalone process on its own port and then within Domino's normal HTTP stack as an HttpService extension. I realized now that I didn't go much into the specifics, and I think those are interesting.

Commonalities

Though the two modes are very different in their final implementation, they share a common conceptual basis, with shared representations of what it is that's supposed to be reversed proxied to. The specifics are abstracted enough such that the core that any proxy implementation needs is an instance of ReverseProxyConfig, as provided by a ReverseProxyConfigProvider service implementation. In practice, this comes from a class that reads app configuration from a view in the admin NSF:

Reverse Proxy Targets view

That lets it determine which port to target, which connector headers to use, and which Domino servers host it. Importantly, the response-doc lines there are the context paths that the apps listen on.

Each individual app attached running on an app server here is given a context root, such as "exampleapp" or "jessehome". When a request comes in, the proxy checks to see if it starts with one of the context roots and then either routes the request to the backing app server or passes it along to Domino (with the mechanism for the latter differing between proxies). This allows all of the apps and Domino to have a unified URL space:

Reverse Proxy

On its own, that's basically what a reverse proxy always does, but this one has the advantage of being auto-configured based on the configuration in the admin NSF, and also deployed alongside or in Domino.

In-HTTP Reverse Proxy

The in-HTTP reverse proxy is implemented as an HttpService extension, which is a just-below-XPages-layer extension available on Domino generally. As I mentioned in my blog post about this the other month, the way it works is that the Domino HTTP server checks with these services to see if any of them want to lay claim to the incoming URL. That claim can be made by any aspect of the URL string - so, for example, the XPages service looks for ".xsp" and related URLs, plus any registered servlets and webapps. In my case, that fits perfectly with what I need here.

The proxy itself consists of one class, which is adapted from an open-source reverse proxy servlet. The proxy checks its target list for the incoming request. If one of those matches, it will then proxy along the request to the given backend. The specifics of implementing the reverse proxy in a servlet-type service are fiddly, but of note is the addition of the standard and WS-specific proxy headers, based on the configuration in the NSF.

This implementation lucks out in the default case: for it to pass unmatched requests to Domino, all it has to do is declare that it doesn't handle the URL, and the rest of the system will handle it from there.

Benefits

The big benefit to this approach is that it ends up being basically transparent to a normal Domino administration. Since it's running as an HttpService, it shares Domino's HTTP and HTTPS ports, and also uses the same TLS configuration. Though the runtinme spawns Libery servers in the background, those can be on high-numbered ports and not visible, so it ends up looking like the apps are running on Domino itself.

Drawbacks

The big drawback stems from the benefits above: since it shares Domino's HTTP ports, it also shares its limitations. Request matching and routing happens at the C level in Domino, so there's no possibility of supporting HTTP/2 or, I believe, WebSocket connections. In a lot of cases, that's not really a big deal, but it diminishes the benefits of using a superior app server.

Separate-Port Reverse Proxy

The standalone reverse proxy listens on its own HTTP and HTTPS ports and makes use of the Undertow project, which is an infrastructural piece of the Wildfly Java app server. In this sort of setup, you'd likely shift Domino's HTTP port to something downstream, like 8080, and configure this proxy to listen on port 80 and 443 itself, taking over main HTTP duties.

Though this listens on a separate port, it still runs from within the JVM of the HTTP service on Domino - it's just spawned in a separate thread as a listener. Since Undertow is designed to be this sort of in-server plumbing, it's configured with a pleasant-enough API, where you can add a number of "handler" types for incoming requests. Of particular note here is the PathHandler class, which lets you assign behavior based on prefix paths, and that's exactly what I do.

For each configured app, my code registers a prefix-based proxying handler, which lets Undertow do the job above of matching incoming URLs to their backend app.

I then set up a catchall proxy at /, which handles passing requests along to the port configured in the Domino server's server doc, along with built-in support for my reverse proxy secret DSAPI filter (which has been proving itself nicely in practice).

Benefits

The big benefit to this approach is that Undertow is much more capable than Domino's HTTP server when it comes to modern web technologies. Your apps can use HTTP/2 and WebSocket without issue, and I can be confident that it will also be updated for future changes down the line. It's also tried-and-true code: while my adapted proxy servlet seems fine in practice, I know for sure that Undertow is up for the task.

It also has a slew of other capabilities and a pluggable architecture that I don't use here, but it opens the door to other potential uses.

Drawbacks

The big drawback is the separate port, which increases the cognitive load for administration and is just "weird" for someone used to Domino's HTTP stack being the only entrypoint. And even if you're okay with doing that (which you should be - it works fine), it's non-obvious when you're trying to get a bead on how the server is configured. If you're tracking down what's actually listening on port 80, you can see that Domino is configured for some other port using normal Domino-admin knowledge, but there's nothing that would tell you to look in libertyadmin.nsf for a configuration for another reverse proxy on a separate thread. That's not wrong, and it's better than if I required you to mangle your names.nsf design, but it's opaque.

Conclusion

I suppose the main conclusion I'll hope you draw is that this stuff is in there and it's neat. It's also the sort of thing that would be adaptable to plenty of other uses: in addition to the Domino Open Liberty project itself being structured such that one could add non-Liberty/Java app runtimes, neither of these proxy techniques are inherently dependent on that project at all.

New Comment