Azure KeyVault – Authenticating with Certificates and Reading Secrets

You should never keep any confidential configuration information in an application configuration file. This include injecting sensitive information via web transformation files. Adding sensitive values via the AppService settings is not ideal either.

In all these cases you may leak sensitive information. For example via your source control. Or anyone with access to your subscription could get those secrets.

In a recent post we introduced Azure KeyVault as a solution for secure storage of confidential information. In this post we will take you through using KeyVault values in web applications.

There are two ways to authenticate a web application in KeyVault. You can store a simple client identifier and client secret in the app settings. This is a rather bad approach as it once again gives access to the KeyVault values.

A better is approach is to authenticate the web application using a certificate. This certificate is also deployed directly from KeyVault. This means neither the confidential information nor the keys to the vault are ever disclosed. This is why we choose this approach here.

There is one caveat though. In code you could still display the confidential information on a web page. Without code reviews this is still not a bullet proof solution.

Loading the access certificate for your application into KeyVault

You will be utilising the certificate store. So you must start a PowerShell in Administrator mode. Run all the following scripts in the same PowerShell session to retain the variables.

First we set some variables we are about to use.

$pwd = 'Password12!'
$vaultname = "demo-keyvault"
$dnsName = 'demo-application.example.org'
$dummyurl = "http://$dnsName/"

The dnsName (and dummyurl) need only be unique in you subscription. They do not need to be real existing urls. The password is used when exporting certificates but you do not need it after the script has run.

Next you need to create a certificate which you then export to a file and remove the certificate from the store.

$certStore = 'cert:LocalMachineMy'
$cert = New-SelfSignedCertificate -DnsName $dnsName -CertStoreLocation $certStore
$file = $dnsName + '.pfx'
$certPath = $certStore + '' + $cert.Thumbprint
$certPwd = ConvertTo-SecureString -String $pwd -Force -AsPlainText
Export-PfxCertificate -Cert $certPath -FilePath $file -Password $certPwd
Get-ChildItem $certPath | Remove-Item

At the time of writing there is an issue with certificates created using PowerShell (i.e. New-SelfSignedCertificate). It seems to be an issue with the provider used by PowerShell (CNG – Cryptographic New Generation).

To solve this issue download OpenSSL and install it.

Next we use OpenSSL to convert the certificate to a compatible version and store is as a .pfx file.

$openssl = "C:toolsopensslbinopenssl"
&$openssl pkcs12 -in "$dnsName.pfx" -nokeys -out "$dnsName.cer" -passin "pass:$pwd" | Out-Null
&$openssl pkcs12 -in "$dnsName.pfx" -nocerts -out "$dnsName.pem" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null
&$openssl rsa -inform PEM -in "$dnsName.pem" -out "$dnsName.rsa" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null
&$openssl pkcs12 -export -in "$dnsName.cer" -inkey "$dnsName.rsa" -out "$dnsName-converted.pfx" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null

Remove-Item "$dnsName.rsa"
Remove-Item "$dnsName.cer"
Remove-Item "$dnsName.pem"
Remove-Item "$dnsName.pfx"

$pfx = new-object System.Security.Cryptography.X509Certificates.X509Certificate2
$fullPath = Get-ChildItem "$dnsName-converted.pfx"
$pfx.import($fullPath.FullName,$pwd,"Exportable,PersistKeySet")
$export = $pfx.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12)
$certb64 = [System.Convert]::ToBase64String($export)
Remove-Item "$dnsName-converted.pfx"

If you are developing locally you need to install this certificate into your local store. Take note again not to use the Import-PfxCertificate cmdlet. It imports but using the certificate may lead to an “Invalid provider type” error.

$store = new-object System.Security.Cryptography.X509Certificates.X509Store("My","LocalMachine")
$store.open("MaxAllowed")
$store.add($pfx)
$store.close()

Now you have the certificate as a base64 encoded string. You will now add an application to Azure Active Directory using it. For this application you can create a service principal. This in turn is used to give access to KeyVault.

$app = New-AzureRmADApplication -DisplayName $dummyurl -HomePage $dummyurl -IdentifierUris $dummyurl -CertValue $certb64 -StartDate $cert.NotBefore -EndDate $cert.NotAfter
$sp = New-AzureRmADServicePrincipal -ApplicationId $app.ApplicationId
Set-AzureRmKeyVaultAccessPolicy -VaultName $vaultname -ServicePrincipalName $sp.ApplicationId -PermissionsToSecrets all -PermissionsToKeys all

Your application will use the certificate to authenticate against Azure AD. With this user (i.e. service principal) the application will access KeyVault. You need to make the certificate available for the web application. When running locally you have just imported the certificate to your local store.

In PowerShell run the following commands to retrieve the certificate thumbprint and application id. The latter is also known as client id. Both are used below.

$cert.thumbprint
$sp.ApplicationId

In Azure we will store the certificate in KeyVault and deploy it to your application using ARM.

$certSecureString = ConvertTo-SecureString -String $certb64 -AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName $vaultname -Name kvAccessCert -SecretValue $certSecureString -ContentType "application/x-pkcs12"

The resource definition to deploy the certificate is:

"variables": {
   "aspName": "kvdemo20170103",
   "kvName": "keyvault-bhany3pt7zg6k",
   "kvResourceGroup": "demosupport",
},
"resources": [{
   "apiVersion": "2015-08-01",
   "type": "Microsoft.Web/certificates",
   "location": "[resourceGroup().location]",
   "name": "webKvAccessCert",
   "properties": {
      "KeyVaultId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', variables('kvResourceGroup'), '/providers/Microsoft.KeyVault/vaults/', variables('kvName') )]",
      "KeyVaultSecretName": "kvAccessCert",
      "serverFarmId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Web/serverfarms/', variables('aspName') )]"
   }
}]

The name (webKvAccessCert) of the certificate needs to be unique per app service plan. If you are running this deployment more than once you may have to remove the certificate first. The deployment references the KeyVault by id. Use the name of the secret you used in the PowerShell above (kvAccessCert).

Important: If you have not already done so, give the WebApp-Resource Provider access to the vault. The ID is static across all of Azure:

Set-AzureRmKeyVaultAccessPolicy -VaultName demo-keyvault -ServicePrincipalName abfa0a7c-a6b6-4736-8310-5855508787cd -PermissionsToSecrets get

In order for the app service to load the certificate you must add an app setting.

Key: WEBSITE_LOAD_CERTIFICATES
Value: *

Use * to load all available certificates. Or you can use a comma seperated list of certificate thumbprints. Add this setting any way you’d like (ARM template, via portal or PowerShell). Make sure to restart the app service to load the application.

Using the certificate to access KeyVault

Install the following NuGet packages to use KeyVault from your application.

Microsoft.Azure.KeyVault
Microsoft.Azure.KeyVault.Extensions
Microsoft.Extensions.Logging.Console
Microsoft.IdentityModel.Clients.ActiveDirectory

You can use the following helper class for reading the certificate and creating a KeyVaultClient.

public static class KeyVaultUtility
{
   private static ClientAssertionCertificate AssertionCert
   {
      get;
      set;
   }

   private static object m_lockObj = new object ();
   public static void Initialize(string clientId, string certificateThumbprint, bool isSelfSignedCertificate = false)
   {
      lock (m_lockObj)
      {
         if (String.IsNullOrWhiteSpace(clientId))
         {
            throw new ArgumentException("Argument clientId is required.");
         }

         if (String.IsNullOrWhiteSpace(certificateThumbprint))
         {
            throw new ArgumentException("Argument certificateThumbprint is required.");
         }

         certificateThumbprint = Regex.Replace(certificateThumbprint, @"[^da-zA-z]", string.Empty).ToUpper();
         var certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);
         try
         {
            // If you are using a self-signed certificate you have to switch off validation
            bool onlyAllowValidCerts = !isSelfSignedCertificate;
            certStore.Open(OpenFlags.ReadOnly);
            var certCollection = certStore.Certificates.Find(X509FindType.FindByThumbprint, certificateThumbprint, onlyAllowValidCerts);
            if ((certCollection?.Count ?? 0) == 0)
            {
               StringBuilder sb = new StringBuilder();
               foreach (var cert in certStore.Certificates)
               {
                  sb.AppendLine($"{cert.Thumbprint} - '{cert.SubjectName.Name}',");
               }

               throw new ArgumentException($"Certificate not found using thumbprint: '{certificateThumbprint}'. Certificates found:rn{sb.ToString()}");
            }

            var certificate = certCollection[0];
            AssertionCert = new ClientAssertionCertificate(clientId, certificate);
         }
         finally
         {
            certStore.Close();
         }
      }
   }

   public static KeyVaultClient GetClient()
   {
      if (AssertionCert == null)
      {
         throw new Exception("Call Initialise before calling GetClient.");
      }

      return new KeyVaultClient(new KeyVaultClient.AuthenticationCallback((a, r, s) =&gt; GetAccessToken(a, r, s, AssertionCert)));
   }

   private static async Task GetAccessToken(string authority, string resource, string scope, ClientAssertionCertificate cert)
   {
      var context = new AuthenticationContext(authority, TokenCache.DefaultShared);
      var result = await context.AcquireTokenAsync(resource, cert).ConfigureAwait(false);
      return result.AccessToken;
   }
}

Note that this is configured to search in the CurrentUser certificate store. This is where the certificates are loaded in Azure. Locally you may be importing them to the LocalMachine store instead.

If you are using .NET Core and do not want to use the provider shown in the next section, you need use the IClientAssertionCertificate interface instead and implement your own implementation.

Using the values you retrieved from PowerShell above you can create an instance of the client:

KeyVaultUtility.Initialize(CLIENT_ID, CERT_THUMBPRINT, true);

var client = KeyVaultUtility.GetClient();
var secret = await client.GetSecretAsync("https://demo-keyvault.vault.azure.net", "key");

The last parameter is the key of the secret you want to retrieve. There are many moving parts to this approach. But using certificates deployed from KeyVault is the most secure approach.

You may run into errors such as:

AADSTS50027: Invalid JWT token. AADSTS50027: Invalid JWT token. Token format not valid.
Application with identifier 'xxx' not found.

Possible solutions are:

Make sure you don’t have hidden characters in your client id or thumbprint values from the copy paste. Either type them into a new notepad and copy that value. Or use a hex editor to check the string (e.g. Notepad++ with Hex Editor Plugin).
Make sure you have not create a new Azure Active Directory and added the application there. You must add the application into the default directory.
If you come across any other errors and solutions, feel free to contact us.

Using Configuration provider

ASP.NET Core brings a new extensible configuration provider model. This allows for automatic loading of secrest from KeyVault.

Install the following NuGet Package

Microsoft.Extensions.Configuration.AzureKeyVault

In your Startup.cs where the configuration is built add the following code:

var certificateThumbprint = Regex.Replace(CERT_THUMBPRINT, @"[^da-zA-z]", string.Empty).ToUpper();

var store = new X509Store(StoreLocation.CurrentUser);
using (var store = new X509Store(StoreName.My, StoreLocation.CurrentUser))
{
        store.Open(OpenFlags.ReadOnly);
        var cert = store.Certificates.Find(X509FindType.FindByThumbprint, certificateThumbprint, false);

        builder.AddAzureKeyVault(
                keyvault,
                clientID,
                cert.OfType&lt;X509Certificate&gt;().Single(),
                new EnvironmentSecretManager(env.ApplicationName));
}

It utilized the same two values (client id, certificate thumbprint) as the above code. Also note that you need to import your certificate to the right certificate store.

Microsoft provides a template class “EnvironmentSecretManager”. This class will only load secrest whose key begins with the application name.

public class EnvironmentSecretManager : IKeyVaultSecretManager
{
   private readonly string _appNamePrefix;
   public EnvironmentSecretManager(string appName)
   {
      _appNamePrefix = appName + "-";
   }

   public bool Load(SecretItem secret)
   {
      return secret.Identifier.Name.StartsWith(_appNamePrefix);
   }

   public string GetKey(SecretBundle secret)
   {
      return secret.SecretIdentifier.Name.Substring(_appNamePrefix.Length);
   }
}

This is by far the easiest way to inject configuration values from KeyVault to ASP.NET Core.

2 thoughts on “Azure KeyVault – Authenticating with Certificates and Reading Secrets”

Greg says:

January 16, 2019 at 23:30

Is it possible to use Azure KeyVault in this way with a web application that is not hosted in Azure?
azurectowpadmin says:

January 17, 2019 at 15:40

Hi Greg, yes absolutely. When developing solutions I run this locally. You just have to make sure the webapp has access to the certificate you are using (in Azure it’s the AppSetting WEBSITE_LOAD_CERTIFICATES). Depending on your server you may have to provide access to the account the web service is running with.