PSA: Reverse-Proxy Regression in Domino 12.0.1

Wed Jan 19 17:08:42 EST 2022

  1. Putting Apache in Front of Domino
  2. Better Living Through Reverse Proxies
  3. Domino's Server-Side User Security
  4. A Partially-Successful Venture Into Improving Reverse Proxies With Domino
  5. PSA: Reverse-Proxy Regression in Domino 12.0.1

For a good while now, I've been making use of the HTTPEnableConnectorHeaders notes.ini property in Domino to allow my reverse proxies to have Domino "see" the real remote system. Though this feature is coarse-grained and is best paired with some tempering, it's served me well when used on appropriately-configured servers.

Unfortunately, HCL saw fit to remove this feature in 12.0.1, declaring it a security vulnerability. I don't think this made it into the release notes as such, but did eventually get patched into the "Components no longer included in this release" page for V12.

Obviously, the true problem here is that it makes my blog entries retroactively less useful. However, a secondary issue is that it will damage your applications in two main ways if you were making use of these headers:

Authentication

The $WSRU header allowed you to specify a username to act as for the duration of the web request, and this would be honored from the core: all legacy and Servlet components would see that as the true authenticated user. This header is essentially a quick-and-dirty SSO mechanism, and works similarly in that way to an LTPA token.

Fortunately, this header has some workarounds, though proper ones would likely require diving into some C:

  1. If you have a simple situation where you were using it to log in as a single known user, you could switch to sending HTTP Basic auth with that user's credentials
  2. If the server in front of Domino happens to have been written by IBM, you could potentially spin your own LTPA tokens on that side and have them trusted by Domino
  3. You could establish your own exchange between the proxy server and Domino to have Domino generate an SSO token for an arbitrary name for you and then pass that along (note: if you do that, use Domino JNA to do the heavy lifting)
  4. You could write a DSAPI filter to implement whatever type of authentication you like

With any of those, Domino will trust the user you provide it to the same extent it previously would trust the $WSRU header when configured to do so.

Note: if you do implement your own authentication, don't just re-use $WSRU. It appears that part (all?) of the change on the Domino side is to hard-strip the $WS* headers from the incoming request.

Remote IP Address, etc.

The other use, which I've used in a good many deployed apps, is to use $WSRA and friends to tell Domino what the true remote IP address is. When you do this, CGI item values like REMOTE_ADDR will reflect the external user's IP address instead of the proxy server's.

Since 9.0.1FP8, there's been a notes.ini property - HTTP_LOG_ACCESS_XFORWARDED_FOR - that will tell Domino to pay a little attention to the de-facto standard X-Forwarded-For header that proxies often use to tip an app server off to the proxy hops a request took to get to its destination. From what I can tell, this support remains limited to just writing to a new field in log entries in domlog.nsf. The in-app CGI variables will still reflect the reverse proxy's address.

Unfortunately, as I found when I was trying to make a DSAPI filter to properly trust that header, these request values are not writable in such filters. It might hypothetically be possible to get some of this with EM triggers juggling fields around for classic use and futzing with the low levels of the XPages stack, but even I wouldn't resort to that.

The upshot is that if, for example, you're storing REMOTE_ADDR in created documents or making use of ServletRequest#getRemoteHost, you'll have to alter your applications to also consider X-Forwarded-For, and the same goes for any other such fields you were using.

As above, if you do this, you can't use $WSRA et al, as I don't think they'll be visible in your app.

Commenter Photo

David Marko - Thu Jan 20 04:04:01 EST 2022

We are trying to run Domino 9.0.1FP10 application behind nginx (nginx on HTTPS, Domino on HTTP) but we encounter some serious issues. Main one is that any Domino serverside redirects we do from the code e.g. from WebQuerySave LotusScript we use print "[/some-url]" ... here we use relative URL but Domino creates absolute/full URL and uses incorrect scheme so redirect points to HTTP instead of HTTPS. We found that Domino adheres Host header comming from nginx but can't find a way how to force Domino to use https . We tried different headers like X-Forwarded-Proto, but no luck. Do you have any hint on that?

thanks David Marko

Commenter Photo

Karsten Lehmann - Thu Jan 20 04:41:30 EST 2022

@David: Not sure if this is the best solution, but we used nginx rules to rewrite the location HTTP headers and URLs in HTML pages returned by Domino, e.g. like this:

proxy_redirect   http://$host/    https://$host/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
sub_filter_types text/html text/css text/xml application/javascript text/javascript;
sub_filter 'http://www.mindoo.de' 'https://www.mindoo.de';
sub_filter 'http://www.mindoo.com' 'https://www.mindoo.com';
sub_filter 'http://blog.mindoo.de' 'https://blog.mindoo.de';
sub_filter 'http://blog.mindoo.com' 'https://blog.mindoo.com';
sub_filter 'http://mindoo.de' 'https://mindoo.de';
sub_filter 'http://$host' 'https://$host';
sub_filter_once off;
Commenter Photo

Mark Leusink - Thu Jan 20 04:48:13 EST 2022

David,

If I recall correctly we had a similar issue. To fix it we added this to the nginx config:

proxy_redirect http:// https://;

It will perform a replace on the Location header coming from the proxied server (Domino in this case).

Commenter Photo

Jesse Gallagher - Thu Jan 20 06:44:21 EST 2022

Karsten and Mark have the right of it, at least with 12.0.1 and above. That problem is one of the exact scenarios that these headers solved, but now you?ll have to fall back to in-flight HTML rewriting on the proxy server for it.

Commenter Photo

David Marko - Thu Jan 20 14:35:03 EST 2022

Just confirming, that proxy_redirect http:// https://; ... did the trick. Thanks a lot!!

New Comment