What causes node secret mismatch errors to happen?

[ScottWilliamson]

I have a server running Windows Server 2016, WDS, and an Active Directory help desk tool for our users. I have no problem authenticating to the server. However, I keep seeing node secret mismatch verification errors from users trying to authenticate to the AD tool. I reset the node secret in RSA to correct this same issue 5 days ago.I refuse to believe that my only solution is to reset the node secret every couple days. Google and the RSA articles I've read have not helped me.  Is there any information regarding what can cause node secret errors to happen?

[EdwardDavis]

It depends on what the specific error condition is: using different node secrets (mismatch) ? cleared on server not on agent ? agent missing node secret ? 

One typical problem is permissions: the process creating the node secret on the agent has the permissions to write it, and later, a process that needs to read it, doesn't have read permissions. that is one common example....

Also there is no specific information about what 'AD tool' is. Not able to provide help on that. Some software uses embedded code to manage node secrets, but can only work with an older version of node secret format. So for example if the RSA Windows agent is what creates the node secret, perhaps the AD tool cannot read it correctly and might use an older format node secret.

One thing to try is: clear the node secret on both sides, then see if the AD tool can authenticate and create a new node secret on the fly. Now authenticate with the AD tool a second time (to prove the new node secret works). If that works, then put a copy of the new node secret in the location the RSA agent needs to see it, and then see if the RSA agent can authenticate. Or do the opposite of this type of test.

Summary, typical issues are:

-permissions

-node secret versions (old vs new)

[StevenSpicer]

Without knowing any details, it's hard to say what's going on in your environment, but often this is caused by someone or some process either removing the node secret file at the Agent end or copying another node secret file over the top of the valid one.  I've seen customers synch directory trees across hosts [to keep an application config consistent in a cluster, for example] thereby replacing the node secret file with the file from the source of the synch operation.

[ScottWilliamson]

Apologies. I am seeing node secrets mismatch. I have addressed this short term by clearing the node secret but the goal of my question is to drill down to the long term fix.

I don't think Manage Engine's AD360 SelfService Tool is the issue. It run as an intranet site to allow users to authenticate with RSA and manage reset and unlock functions of their own Active Directory account. It just references sdconf.rec for where to forward the provided passocde data. 

[EdwardDavis]

Manage Engine needs the sdconf.rec but it also needs to be able to handle reading and writing node secrets as well...and I see they are not a current partner***... if they are using older legacy code to read the node secret and send it along with the passcode, that might be the problem. Testing if Manage Engine can create the first node secret, and use it, would be my next step.

Note: the first authentication when there is no node secret on RSA AM server or agent is a 'freebie'. Auth will be successful, then a node secret is created, and now node secret must match. What matters is once the node secret file is created, can you do another authentication afterward, with the same process that did the first authentication ?

RSA agent will be able to read an old or new format. Manage Engine may only work with old format....or something along this thought process. 

***They were a partner in 2009 when we were still using old format node secrets, but I cannot find them on the current partner list+++ (I may be wrong). I suspect old RSA nugget code and old node secret formats. This is the case with some Microsoft products (TMG and UAG) as well. You need to use the older [thing] to create node secrets, and the newer RSA agent will happily use and read the old format, but can never create old format node secrets. In the case of TMG/UAG there is an sdtest tool to perform authentication tests, and it can create a node secret in old format. So you then copy that node secret to the location where the newer software 'RSA Auth Data directory' is, and both processes can use it. But it has to start as the old format so TMG/UAG don't get node secret mismatches.

+++Update: Manage Engine AD Self Service Plus is now Zoho Zoho Corporation - Technology Integrations 

If this still doesn't help fix the issue, open a support case and we can dig into it properly.

[RandyBelbin · Accepted solution]

Ed hit the nail on the head here.
UDP agents are identified by IP address in AM and you essentially have two agents running on the same server (ME and the Windows agent). ME is creating its own node secret (as an older agent, it doesn't know about the new format used by your Windows agent). The problem is your server has a node secret for this IP address already and reports back a mismatch.

For the two agents to co-exist, you'll need to use the node secret utility to load the node secret into the Windows agent.

Here is a link to a kb article that has some helpful information.

000030947 - Unable to Integrate Two Agents on the Same Host - Node Verification Mismatch 

You do not need to disable UAC, you should be able to run the node secret utility using the run as admin functionality instead. If you do disable UAC, it can be safely re-enabled after completing this process. The high level flow is this:
1. Find where Manage Engine is storing its node secret. Probably in C:\Windows\System32 or C:\Windows\SysWOW64 but could be in a ManageEngine directory. The file will be named securid with no extension.
2. Delete this file and then clear the node secret from the Windows agent using the Control Center.
3. Clear the node secret in Authentication Manager.
4. Perform an authentication using Manage Engine allowing it to negotiate a new node secret with AM.
5. Follow the steps in the KB to copy the node secret file over to the Windows agent using the node secret utility.

[RicardoArmand]

It's under Zoho Corp now.

[EdwardDavis]

Thanks, updated my earlier comment with a link to Zoho guides. 

[ScottWilliamson]

Thank everyone for your time and assistance. The processes Edward and Randy advised have corrected my issue.