Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

 
alt

Daniel Nashed

 

Correctly Stopping a Traveler Server

Daniel Nashed  24 October 2017 13:02:57
This is not new but I ran into this a couple of times on customer site. Specially on a Traveler HA server this becomes important.

Shutting down the Traveler servertask when the Domino server/service is stopped might lead to hang situations of the HTTP task.

The better way would be to shutdown the Traveler servertask first. But even that might lead to undesired results.


There is a special Traveler shutdown command "tell traveler shutdown" that can be used to let Traveler finish it's work and not accept any new requests before cleanly shutting down.


When you configured NTS_AUTOSTART_HTTP=true the Traveler task did start the HTTP task automatically but by default the HTTP task will not be shutdown automatically when you just shutdown the Traveler servertask.

If you configure NTS_AUTOSTOP_HTTP=true the Traveler servertask will automatically take care of shutting down the HTTP task in the right moment.


This is important for Traveler HA environments because the Traveler OSGi servlet will only connect to the local Traveler servertask which will than either process the request or forward it to another Traveler server (if another server holds the master monitor for the user).

So if you don't shutdown the HTTP task your load-balancer might still send requests to the Traveler server which reports back that the Traveler Server is not available ("IBM Traveler server is not available.").


That's an undesirable result for a load-balancer because usually the load-balance does not check for this status and might continue to send devices to that Traveler server.


It makes sense to configure the server to be either fully available or to not respond to HTTPS requests if the Traveler service is not available.

This is true if you shutdown the Traveler task or also during shutdown of the whole server.


My Domino start script for Linux allows to configure a pre-shutdown command which can be used to shutdown Traveler before you shutdown the whole server.


-- /etc/sysconfig/rc_domino_config --


# -- Command to execute before shutting down the Domino server --

DOMINO_PRE_SHUTDOWN_COMMAND="tell traveler shutdown"


# -- Delay before shutting down the Domino server after invoking the pre-shutdown command --

DOMINO_PRE_SHUTDOWN_DELAY=10


There is no simple solution for Windows because Domino is started as a Windows service.
But if you shutdown your server manually, you should shutdown the Traveler server and also use the notes.ini setting.


Sadly if you shutdown the whole server all processed will receive the quit command at the same time and will start to shutdown.
This means it could come to undesired timing issues when HTTP and Traveler do their shutdown at the same time.

A controlled Traveler server shutdown would be the cleaner solution.


Update to make it more clear:

Yes there is a big difference between using tell traveler quit and tell traveler shutdown!

If you quit a servertask the task has only limited time to continue to work with full resources available.
If a task or a server is in pending shutdown some resources are not available any more and cannot fully controlled terminate pending work.

Here is the decumentation for the shutdown command:

Shutdown [command]     - Stop accepting new work requests, allow current work to complete, and then Quit.  
If you specify a [command], then this command will be run after the server is idle but before it quits.

So it makes a lot of sense to use shutdown instead of quit.

------------


Another tip that might be still new for some customers. If your load-balancer or other monitoring software support authenticated (with user credentials), SSL enabled probes, there is a very nice feature in Traveler to allow an end to end check for your Traveler server availability.


HTTPS://traveler.acme.com/traveler?action=getStatus

Depending on your language settings you either get


a.) IBM Notes Traveler server is available.


b.) IBM Notes Traveler server is not available.

Or an error message for the end to end check for the user you specified. The user must be allowed to user Traveler and have a proper mailfile for this end to end test.

c.) The IBM Notes Traveler server cannot connect to your mail database mail/jdoe.nsf on server CN=domino.acme.com/OU=Srv/O=Acme.  Verify that your mail server mail database grants access to server CN=domino.acme.de/OU=Srv/O=Acme and is operational.  If this does not resolve the problem, your administrator may need to verify the network connection between the servers and that the IBM Notes Traveler server is allowed to access your mail server.


So I would recommend to
  • always use the "tell traveler shutdown" command first
  • enable HTTP Auto shutdown -> notes.ini NTS_AUTOSTOP_HTTP=true
  • use the options in my start script if you are running on Linux
  • have a look into the ?action=getStatus command if you didn't use that before
-- Daniel

Links

    Archives


    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]