Today I went to apply a patch to a customer’s Sametime Proxy server.  This is a server that’s been around for a few weeks.  I’ve logged into the SSC countless times in that time.  I launch Installation Manager (using “run as administrator”) and when it gets to the “sign on to SSC” part it fails saying it can’t connect.  I check the logs in /users/myname/appdata/local/temp/SSCLogs and find the error saying it can’t resolve <sschostname>:9443/console/deployment/login.  So I try that URL in a browser myself  and sure enough it does fail.

Well I can guess what that is and it’s an easy fix.  In Sametime we map virtual hosts for each application including the SSC containing the hostnames and ports used by that application.  So I went to check that the default_host virtual host used by the SSC had 9443 in it.

Go to SSC on the Deployment Manager server through a browser, try and login using my file repository account.  Login failed.  Try again. and again.  and again. and again. Type into notepad to make sure there’s no caps lock or language issues.  Failed again. This is worrying, no-one else has access right now so no-one has changed any password. I check the SystemOut.log for dmgr and there are errors in there and in the FFDC files saying Password is wrong.  OK.  No need to panic.  I’ve seen this before when Dmgr gets low on memory so first things first, let’s restart the box.  If in doubt, reboot WebSphere.  Server comes back up and still I can’t login.

OK so now I start to worry.  I go find the security.xml file in the config for the cell and decode the password stored in there (don’t ask how because I shouldn’t be able to but it’s possible).  The password says it’s what I think it is.  I really really don’t want to go down the path of changing that password even though I can disable security and do that because that’s going to have knock on effects all over the place….So - deep breath - let’s try this again from another machine.  I go to the SSC from my desktop this time instead of a browser on the DMGR server and it logs in perfectly first time using the name and password that was failing when I tried from the DMGR server.  Back to the browser on the server, login still fails.   This makes no sense.

So the issue isn’t the “wrong password” at all.  The issue is that the security on the SSC OS is preventing me logging in via a browser - I assume preventing the browser accessing the files on the file system in some way.  In addition the SSC was unable to sync any nodes or restart any servers (this was new) although it could tell status - until I restarted everything manually under my account.  This appears to be a problem with the services on the SSC accessing the file system on any of the OS even its own.  The customer is looking into all of that since the environment is tightly locked down and I can’t see anything.

When I finally got in (and yes I could use the LDAP alternative accounts I had in there) I added 9443 and 9080 to default_host under the hostname of the SSC and the Installation Manager ran fine.

Today’s lesson learned..DON’T PANIC!