Strange behavior AD FS Windows Server 2012 R2 after changing the service communications certificate

Yesterday I ran into a problem in my demo environment after I changed the AD FS service communications certificate. My old certificate wasn’t prepared for DRS (Device Registration Service) and since I wanted to test some things with DRS in combination with Office 365 I needed to replace the certificate with a new one which included the enterpriseregistration.domain.com UPN suffix.

However, after I requested a new SSL certificate and changed the service communications certificate in the AD FS management console and restarted the AD FS service I ran into the following problems:

1.       My Web Application Proxy (WAP) had problems creating a trust relationship with my AD FS server:

Log Name:                AD FS/Admin

Source:                      AD FS

Date:                          20-10-2014 16:56:16

Event ID:                   422

Task Category:         None

Level:                         Error

Keywords:                 AD FS

User:                          NETWORK SERVICE

Computer:                OP-WAP01

Description:              Unable to retrieve proxy configuration data from the Federation Service.

2.       I started receiving event id error 15021 on my AD FS server:

Log Name:                System

Source:                      HttpEvent

Date:                          20-10-2014 16:57:03

Event ID:                   15021

Task Category:         None

Level:                         Error

Keywords:                 Classic

User:                          N/A

Computer:                OP-DC01.demo.unifyit.nl

Description:             An error occurred while using SSL configuration for endpoint localhost:443. The error status code is contained within the returned data.

3.       I wasn’t able to login to Office 365 anymore using a federated identity from an internal client:

4.       The Device Registration Service (DRS) on my AD FS server didn’t want to start anymore:

Log Name:                DRS/Admin

Source:                      Device Registration Service

Date:                          20-10-2014 16:57:03

Event ID:                   124

Task Category:         None

Level:                         Error

Keywords:                 Service

User:                          DEMO\svc_adfs$

Computer:                 OP-DC01.demo.unifyit.nl

Description:              Could not determine the SSL port over which the AD FS service is listening on.

User Action:              Make sure that the AD FS service is running and that AD FS is configured correctly.

Since the AD FS service itself didn’t report any problems I first tried to remove the AD FS trust between the Web Application Proxy (WAP) server and the AD FS server. However, after I removed the trust and tried to recreate it using the “install-webapplicationproxy” cmdlet on the WAP server, I received the following error:

Because it took me some time to figure it out, I wanted to share my solution with you.

Apparently there is some strange behavior in Windows Server 2012 R2 if you want to change the AD FS service communications certificate. After you changed it, for some reason Windows Server doesn’t change the SSL certificates on the socket layer of the system. In my case the Certificate Hash or Thumbprint didn’t match between AD FS and the socket layer anymore:

As you can see, after typing in “cd cert:\LocalMachine\My” and then a “ls” or “dir”, my new certificate thumbprint/hash was 2A4BF86B8387BA006C7AC63183557F4D009FE7C4.

However, when I looked into the socket layer with the command “netsh http show sslcert”, it returned the following information:

Clearly the socket layer was still in the understanding that it needed to use my old certificate, which was already gone by now.

Since AD FS in this version of Windows Server is not running on IIS anymore, you can’t change the default certificate on the IIS website. You need to remove all SSL Certificate bindings and add new bindings with the new SSL certificate.

So before we do this, you need to save the output of the “netsh http show sslcert” command to a notepad. This is important because you need the appid, hostnameport, certstorename and sslctlstorename to recreate the SSL certificate bindings.

Note: It is not necessary but before I do anything related to an application with a service install, I always stop the related service and make sure I have a current backup of the server in place.  

After we have saved the needed information, we can remove the current SSL Certificate bindings by using the “netsh http delete sslcert” command. In this command you need to specify the hostnameport for the binding you want to remove. Since there are 3 bindings, we need to remove all of them. Also, you need to run your commandprompt or powershell commandprompt in elevated permissions. This is how the commands look like in my environment:

Now the old bindings are removed and you are ready to add the binding again with the SSL certificate that relates to the AD FS service communications certificate. You can verify if the removal of the SSL bindings where successful by running the “netsh http show sslcert” command again:

So, for the creation of the new SSL binding for AD FS you need the following information ready (you can take this out of the earlier created notepad file):

hostnameport	Unicode hostname and port for binding.
certhash	The SHA hash or thumbprint of the certificate. This hash is 20 bytes long and specified as a hex string.
appid	GUID to identify the owning application.
certstorename	Store name for the certificate. Required for Hostname based configurations. Defaults to MY for IP based configurations. Certificate must be stored in the local machine context.
sslctlstorename	Store name under LOCAL_MACHINE where SslCtlIdentifier is stored.

For some reason netsh doesn’t allow you to use the same syntax as showing and deleting the bindings. If you do this anyway, it prompt you the error “parameter is incorrect”. So, we need to create a session by entering the following commands:

netsh

http

This gets you in the http session of netsh. After this you can recreate the binding with the command  “add sslcert”. In my case this where the following command lines:

add sslcert hostnameport=sts.demo.unifyit.nl:443 certhash=2A4BF86B8387BA006C7AC63183557F4D009FE7C4 appid={5d89a20c-beab-4389-9447-324788eb944a} certstorename=MY sslctlstorename=AdfsTrustedDevices

add sslcert hostnameport=localhost:443 certhash=2A4BF86B8387BA006C7AC63183557F4D009FE7C4 appid={5d89a20c-beab-4389-9447-324788eb944a} certstorename=MY sslctlstorename=AdfsTrustedDevices

add sslcert hostnameport=sts.demo.unifyit.nl:49443 certhash=2A4BF86B8387BA006C7AC63183557F4D009FE7C4 appid={5d89a20c-beab-4389-9447-324788eb944a} certstorename=MY

After running these commands, you should get the following output:

Now, we want to verify if the bindings are restored in a proper manner. You can do this by entering “show sslcert” from within the netsh shell. This should give you the following output:

As you can see now, the bindings are successfully created and the Certificate Hash matches the SSL certificate installed on the AD FS server.

After this I started the AD FS and DRS Service on AD FS server again, recreated the proxy trust between the WAP and AD FS server again and all problems disappeared. 

I don’t know if this behavior is just default in Windows Server 2012 R2 or if it is a bug, but it’s important to always check if the new SSL certificate for the AD FS service communications matches with the SSL certificate binding on the socket layer of the AD FS server.

I hope this guide is useful for you when you encounter similar problems.

Good luck!