Richard Moy added an interesting comment to my last blog post, wishing that HCL would add Web Sockets support to the core. It's a popular requirement, and one that people have tried to work around. John Jardin added a web sockets server to our React application for our JavaScript session at IBM Think 2018, so I got experience of it then and saw the architecture involved.
Firstly, it's worth pointing out that, although I've not seen Richard's applications, I know they are built with JavaScript and REST services. As we'll see, that makes Web Sockets sensible. I'll also point out that although I've used web sockets, I'm not an expert in them.
Notes Client
There is actually the equivalent of web sockets in Notes Client applications, I'll call it "client sockets". Again though, most developers would probably not be aware of it. It's managed in the View properties, in the "Refresh Index" and "On Refresh" property on the Options tab of View properties.
- Any automatic "Refresh Index" option with any refresh display setting for "On Refresh" will enable "client sockets" - the server pushes the change to users immediately, whether the change is made on another client, on a browser, via an agent, whatever.
- Any automatic "Refresh Index" option with the "Display indicator" setting for "On Refresh" will enable client sockets only for the top left square of the Notes View: the server pushes the familiar blue arrow to say an update is available on the server. But the user gets to choose when to pull the update from the server.
- The "Manual" option for "Refresh Index" will turn off indexing on the server, and thus "client sockets".
Another point worth emphasising here is that all clients hold all of the view.
Domino "Just Does It For You" and, because it pre-dated pervasiveness of the web, let alone web sockets, it hides the similarity to web sockets from developers and also hides the complexity.
Web Sockets in Architecture
It's worth reviewing the architecture from our application at IBM Think and Engage.
Note that the Web Socket server is a layer between our React app and Node-RED. Node-RED was where all interfaces - the React app and the Watson Workspace integration - went through. The Watson Workspace integration pushed not only to the API gateway on Domino, but also to the Web Socket server. This meant any changes to the application were instantly available on all connected browsers, whether they were made from the current browser, from another browser or from Watson Workspace.
Note that we had no XPages interface and no Notes Client interface. Any changes there would not be pushed to the Web Socket server, so would never get pushed to the connected browser. They would only get pulled when a fresh request to the API gateway was made.
If memory serves me correctly, I don't think we were chunking the view either. A request to the API gateway passed the whole view. So filtering happened against the stored state in the Node.js app. This may have a relevance, I don't yet understand the inner workings of web sockets to offer a definitive answer. But it seems plausible that it might have an impact.
XPages
Things get more difficult in XPages. XPages does not have a live link to the view on the server. It keeps its state based on persistence settings, as I mentioned in the first blog post of this series. That may be in memory or on disk.
Updating the browser seems easy as a push, but it would need to be something separate from the XPages runtime. So how does it hook into the XPages runtime? What if you're using a repeat control that's a ViewEntryCollection or ViewNavigator? How easy would that be to push from the server? And what if you've created a repeat control based on a ViewEntryCollection that includes editable components or buttons to allow you to effectively have "InViewEdit"? John and I had that in our React app, but it worked because our web sockets server was updating the server-side local storage. How do you update the in memory storage of the XPages collection? And would you corrupt the partial refresh process by doing so?
And any updates also have to go through that web sockets part.
Java, Proton
Presumably there would also be similar challenges with Java frameworks and Proton.
Pub Sub
The Pub Sub functionality may be the key here.
Notes Client will get immediate updates, where coded, because Domino "Just Does It For You".
For your web applications, you code your own web socket server - in Node.js, Vert.x, whatever. You route all instances of your web applications through it. And you subscribe to the Pub Sub functionality for any CRUD changes to the underlying NSF, pushing the update to your web socket server. If your REST API that third parties want to connect to wants to offer subscribe functionality, again, that routes through your web socket server, and it handles it. If it doesn't, you just write to the NSF and the Domino server's pub sub pushes it to the subscriber which pushes it to the web socket server. If you have a basic web application that doesn't need web sockets, it continues as is.
Yes, this is more work. But it ensures your custom application has the custom requirements you want.
Maybe HCL - or the community on OpenNTF - can provide a starter module in the language of your choice. But I suspect it probably needs to be done by the community.
If you want HCL to build a web socket server into the core that addresses all use cases and doesn't break anything, and supports things like XPages because that's still being maintained, with a GUI and editors that makes it as easy as Notes developers expect, I doubt it'll happen any time soon.
f:websocket in JSF, why would it be so hard in Xpages?
Looks interesting, but there seem to be a lot of intersections to runtime and not something you just “switch on”. There are two reasons it’s likely to prove hard. The first being that developers will need to understand all the kind of things I’ve covered in this blog post to utilise it – you either force them to learn those concepts and make XPages harder, or you somehow have to come up with a way to hide that complexity. The second is HCL have no XPages developers and I don’t anticipate them employing anyone for XPages.
From my understanding, just adding it to the XPages runtime would not help, because the nHTTP task can not handle WebSocket protocol at the moment.
That’s why I think that the best approach would be that it would be possible to run an own JVM instance outside Domino’s HTTP task. This would make maintenance of running applications a lot easier, instead of restarting the complete task (which stops ALL applications), every application could be restarted by each own.
This also would allow to use “normal” Java dev toolchains like Maven etc.
I have written about this idea a year ago: http://hasselba.ch/blog/?p=2625
I can dream. We have thought about using the App Dev Pack to address the web socket need. However, I would think with Sametime moving forward and back as a Domino-based solution that the use case is there to support Web sockets.
My solution was to build a OSGi bundle with an embedded Jetty server, Atmoshphere as websocket transport Layer and Cettia on top of it to simplify the websocket handling and communication and this along Jersey for JAX-RS. And finally with Hazelcast I was able to made the Websocket messaging clustered. All this works great. The downsides are that I had to build an OAuth authentication to secure my REST-API’s and the Websockets and that a second webserver is running on its own port. This means that you have to handle CORS to be able to access the REST API and websockets served via nHTTP.
That’s a powerful stack. Jersey is less readily available in Domino than Apache Wink, but more widely used, so probably a preferable option. Building OAuth authentication and handling CORS are additional complications, but critical to understand for building a microservices architecture. Building the Java API for Watson Workspace was very instructive for helping me understand the OAuth dance and JWT tokens, and I’m sure would be useful if I had to build an OAuth authentication. Other learning meant I was able to take different approaches to authentication though.
Sounds like a very nice solution.
Alternativly, you can use LTPA tokens for an simple solution for authentication between Domino and “3rd Party” servers. Not the best choice, but a lot easier (while OAuth is the preferred way).
And the CORS problem can be solved with a reverse proxy.