Fix Certificate Issues on Development Machines

Since we moved to the new AX (#Dyn365FO) we use virtual machines to perform development tasks which are provided as images by Microsoft and either deployed to Azure or downloaded and run via Hyper-V on a local server. At least the latter suffer an issue at some point (age) that relates to expired certificates and that might be hard to overcome if you start from scratch. No worries – here’s some guidance that should help you to fix the machine.

Symptoms

On your development machine you cannot open the application in the browser anymore and face an error message like

There is a problem with the server
Sorry, the server has encountered an error. It is either not available or it can't respond at this time. Please contact your system administrator.

If you check the event log using the Event Viewer you’ll find a warning message pointing to an ExpiredCertificateException there:

Process information: 
    Process ID: 14516 
    Process name: w3wp.exe 
    Account name: NT AUTHORITY\NETWORK SERVICE 
 
Exception information: 
    Exception type: ExpiredCertificateException 
    Exception message: Expired certificate for id 'C0E503DC8987D25B63897A7BE0B3E34BDCC89F41'.
   at Microsoft.Dynamics.AX.Configuration.CertificateHandler.LocalStoreCertificateHandler.GetCertificatesForId(String id)
etc.

Not only you cannot open the application via browsing the URL (typically https://usnconeboxax1aos.cloud.onebox.dynamics.com/) anymore if this happened because of expired certificates – starting a form using the debugger and unit test execution are unusable at that stage, too! When it first appeared a couple of months ago almost all the development machines in our organization were affected. That was a stressful day at work… With a lot of trial and error and the help of the wonderful people on yammer (follow to see some more of the history of this) I was able to create a very specific guide to fix it which only worked for the machines of this particular age.

Solution

This week, it happened again. Some newer images were affected and this time the old solution didn’t work on these machines. Basically, the old approach was to take an existing and not expired certificate that is there already and replace all references of the expired ones to this one. But what to do if there is no suitable certificate? Exactly – create one.

Find Certificates

You can see the certificates that are relevant here using Manage computer certificates from Windows Start menu. Navigate to Certificates – Local Computer > Personal > Certificates.

In the column Expiration Date you can easily identify the ones that recently expired, in this case

  • DeploymentsOnebox.DaxRunnerTokenUserCertificate.pfx
  • DeploymentsOnebox.LcsClientCertificate.pfx
  • DeploymentsOnebox.MRClientCertificate.pfx
  • DeploymentsOnebox.SessionAuthenticationCertificate.pfx

As far as I know there is no way to extend the validity of such certificates. So, we need to use a different one.
Also, it is mandatory that some of the properties equal the ones of the certificates that are in place but expired. So, the best approach is to create clones! You can use PowerShell to do so – special thanks go to Brad Bateman for the hint to the according article on docs.microsoft.com.

Identify Thumbprint of Expired Certificate

Certificates get accessed by their thumbprint which is a 40-digit hexadecimal value. You can see it by double-clicking the certificate in the certificates viewer and open the Details tab.

Unfortunately, we need it to be upper case and without blanks so this is the right time to open the files that need to be modified later already. You can use any text editor or event VS, my preferred one for such operations is Notepad++. Make sure to run it as Administrator so you can save the files later without any issues. All three files we need are located in C:\AOSService\webroot:

  • web.config
  • wif.config
  • wif.services.config

Use the first 4 digits or so to find the whole string in the web.config file. Make sure it’s the right one. Copy and note it in some other place (text file / OneNote / whatever you use for that). In my example here, this is 43082FE50B4D02562C89EA728B2363C598E84886 (and I searched for 4308).

Clone the Certificate

Use PowerShell (and Run as Administrator, of course) to execute the following command (and make sure to replace the thumbprint with the one you just identified):

Set-Location -Path "cert:\LocalMachine\My"
$OldCert = (Get-ChildItem -Path 43082FE50B4D02562C89EA728B2363C598E84886)
New-SelfSignedCertificate -CloneCert $OldCert -NotAfter (Get-Date).AddMonths(999)

999 is the number of months the certificate will be valid until. Should be fine for quite some time.
The execution of this creates some output – copy and note the thumbprint of the newly created certificate. In the certificate manager you can see the clone (you might have to Refresh after a right click on the folder on the left).

Update References

Use Notepad++ (slash the tool of your choice) to find/replace the old thumbprint by the new one in all three files mentioned and opened earlier. Backup the files before.

Repeat

Repeat this for all expired certificates. In the example there were four. I’d guess this differs from time to time. Don’t forget to save the files.

Reboot

It might be enough to start some services (IIS, Batch, SSIS, MR, SQL) on the machine but in such a case I prefer to simply reboot the whole thing which is faster than doing the restart one by one – if you do not have a suitable script for that around.

That’s it

AX should now be working again. Er, Dynamics 365 for Finance and Operations should be back in an operational state now 🙂

14 thoughts on “Fix Certificate Issues on Development Machines”

  1. Thank you for sharing this. Unfortunately, looks like my MicrosoftDynamicsAXDSCEncryptionCert certificate is also expired (and not listed in those config files)… and it isn’t listed in any of those config files. Since fixing my other certs isn’t doing the trick, I suspect I’ve got something else I need to do. If you (or anyone else) has managed this, please share!

    (I can abandon this development VM pretty soon, but we’re not QUITE ready for the next release yet, and would hate to have to set up a brand new one just for a couple of weeks use.)

    • Hi Brian,
      Even though MicrosoftDynamicsAXDSCEncryptionCert is expired on my development machine, too (since 10/27/2017), everything works fine there.
      A quick search for the certificate’s name on the web and docs.microsoft.com didn’t result in anything and on yammer there’s only one single entry talking about it – me asking for help with expired certificates initially.
      So my best advice is to double check if the fix for the other certficates was done correctly. Good luck!

  2. Hi, here is optimized script to get updated keys in one ps command.

    Get-ChildItem -Path “cert:\LocalMachine\My” |where {$_ -like “*DeploymentsOnebox*”} | ForEach-Object {echo $_ | Select Thumbprint; New-SelfSignedCertificate -CloneCert $_ -NotAfter (Get-Date).AddMonths(999) ;echo “—————————————-” }

  3. I am coming up with the new certificate not trusted and am never getting the install Icon. Also the wif.services.config did not have a reference to the old certificate number. I did not change it for that reason. The website is still returning a 500 error. I believe there is a problem in my execution of this process. Should the new certificate show as trusted?

    • Hi Don,
      I’m not sure which place it is you see that the certificate is trusted (or not). Also, the described fix works only for these expired certificates that are referenced by the config files (because what we do here is to create new ones and these need to be referenced instead of the expired ones). When you say that wif.services.config did not have a reference – did the other files? That would be fine – not all certificates are referenced in all three files. Also, you should be able to identify the issue causing certificate via the event log. The thumbprint can be seen in the error message – maybe that leads you to a better understanding what is wrong still.

  4. Write-Output “Rotating Certificates on OneBox VM”
    Set-Location -Path “cert:\LocalMachine\My”
    foreach($OldCert in Get-ChildItem -path Cert:\LocalMachine\My | Where {$_.NotAfter -lt $(get-date).AddMonths(2)})
    {
    $OldCert
    $NewCert = New-SelfSignedCertificate -CloneCert $OldCert -NotAfter (Get-Date).AddMonths(999)

    (Get-Content ‘C:\AOSService\webroot\web.config’).Replace($OldCert.Thumbprint, $NewCert.Thumbprint) | Set-Content ‘C:\AOSService\webroot\web.config’
    (Get-Content ‘C:\AOSService\webroot\wif.config’).Replace($OldCert.Thumbprint, $NewCert.Thumbprint) | Set-Content ‘C:\AOSService\webroot\wif.config’
    (Get-Content ‘C:\AOSService\webroot\wif.services.config’).Replace($OldCert.Thumbprint, $NewCert.Thumbprint) | Set-Content ‘C:\AOSService\webroot\wif.services.config’
    }
    Write-Output “IIS Reset…”
    iisreset

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.