Azure KeyVault - Authenticating with Certificates and Reading Secrets

You should never keep any confidential configuration information in an application configuration file. This includes injecting sensitive information via web transformation files. Adding sensitive values via the AppService settings is not ideal either.

In all these cases, you may leak sensitive information, for example, via your source control. Or anyone with access to your subscription could get those secrets.

In a recent post, we introduced Azure KeyVault as a solution for secure storage of confidential information. In this post, we will take you through using KeyVault values in web applications.

There are two ways to authenticate a web application in KeyVault. You can store a simple client identifier and client secret in the app settings. This is a rather bad approach as it once again gives access to the KeyVault values.

A better approach is to authenticate the web application using a certificate. This certificate is also deployed directly from KeyVault. This means neither the confidential information nor the keys to the vault are ever disclosed. This is why we choose this approach here.

There is one caveat, though. In code, you could still display the confidential information on a web page. Without code reviews, this is still not a bulletproof solution.

Loading the access certificate for your application into KeyVault

You will be utilizing the certificate store. So you must start a PowerShell in Administrator mode. Run all the following scripts in the same PowerShell session to retain the variables.

First, we set some variables we are about to use:

$pwd = 'Password12!'
$vaultname = "demo-keyvault"
$dnsName = 'demo-application.example.org'
$dummyurl = "http://$dnsName/"

The dnsName (and dummyurl) need only be unique in your subscription. They do not need to be real existing URLs. The password is used when exporting certificates, but you do not need it after the script has run.

Next, you need to create a certificate, export it to a file, and remove the certificate from the store:

$certStore = 'cert:LocalMachineMy'
$cert = New-SelfSignedCertificate -DnsName $dnsName -CertStoreLocation $certStore
$file = $dnsName + '.pfx'
$certPath = $certStore + '\' + $cert.Thumbprint
$certPwd = ConvertTo-SecureString -String $pwd -Force -AsPlainText
Export-PfxCertificate -Cert $certPath -FilePath $file -Password $certPwd
Get-ChildItem $certPath | Remove-Item

At the time of writing, there is an issue with certificates created using PowerShell (New-SelfSignedCertificate). It seems to be an issue with the provider used by PowerShell (CNG - Cryptographic New Generation).

To solve this issue, download OpenSSL and install it.

Next, use OpenSSL to convert the certificate to a compatible version and store it as a .pfx file:

$openssl = "C:\tools\openssl\bin\openssl"
&$openssl pkcs12 -in "$dnsName.pfx" -nokeys -out "$dnsName.cer" -passin "pass:$pwd" | Out-Null
&$openssl pkcs12 -in "$dnsName.pfx" -nocerts -out "$dnsName.pem" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null
&$openssl rsa -inform PEM -in "$dnsName.pem" -out "$dnsName.rsa" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null
&$openssl pkcs12 -export -in "$dnsName.cer" -inkey "$dnsName.rsa" -out "$dnsName-converted.pfx" -passin "pass:$pwd" -passout "pass:$pwd" | Out-Null

Remove-Item "$dnsName.rsa"
Remove-Item "$dnsName.cer"
Remove-Item "$dnsName.pem"
Remove-Item "$dnsName.pfx"

$pfx = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$fullPath = Get-ChildItem "$dnsName-converted.pfx"
$pfx.Import($fullPath.FullName, $pwd, "Exportable,PersistKeySet")
$export = $pfx.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12)
$certb64 = [System.Convert]::ToBase64String($export)
Remove-Item "$dnsName-converted.pfx"

If you are developing locally, you need to install this certificate into your local store. Take note again not to use the Import-PfxCertificate cmdlet. It imports but using the certificate may lead to an “Invalid provider type” error:

$store = New-Object System.Security.Cryptography.X509Certificates.X509Store("My","LocalMachine")
$store.Open("MaxAllowed")
$store.Add($pfx)
$store.Close()

Now you have the certificate as a Base64-encoded string. You will now add an application to Azure Active Directory using it. For this application, you can create a service principal, which is used to give access to KeyVault.

$app = New-AzureRmADApplication -DisplayName $dummyurl -HomePage $dummyurl -IdentifierUris $dummyurl -CertValue $certb64 -StartDate $cert.NotBefore -EndDate $cert.NotAfter
$sp = New-AzureRmADServicePrincipal -ApplicationId $app.ApplicationId
Set-AzureRmKeyVaultAccessPolicy -VaultName $vaultname -ServicePrincipalName $sp.ApplicationId -PermissionsToSecrets all -PermissionsToKeys all

Your application will use the certificate to authenticate against Azure AD. With this user (i.e., service principal), the application will access KeyVault. You need to make the certificate available for the web application. When running locally, you have just imported the certificate to your local store.

In PowerShell, run the following commands to retrieve the certificate thumbprint and application ID (also known as client ID). Both are used below:

$cert.thumbprint
$sp.ApplicationId

In Azure, we will store the certificate in KeyVault and deploy it to your application using ARM:

$certSecureString = ConvertTo-SecureString -String $certb64 -AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName $vaultname -Name kvAccessCert -SecretValue $certSecureString -ContentType "application/x-pkcs12"

The resource definition to deploy the certificate is:

{
	"variables": {
		"aspName": "kvdemo20170103",
		"kvName": "keyvault-bhany3pt7zg6k",
		"kvResourceGroup": "demosupport"
	},
	"resources": [
		{
			"apiVersion": "2015-08-01",
			"type": "Microsoft.Web/certificates",
			"location": "[resourceGroup().location]",
			"name": "webKvAccessCert",
			"properties": {
				"KeyVaultId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', variables('kvResourceGroup'), '/providers/Microsoft.KeyVault/vaults/', variables('kvName'))]",
				"KeyVaultSecretName": "kvAccessCert",
				"serverFarmId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Web/serverfarms/', variables('aspName'))]"
			}
		}
	]
}

The name (webKvAccessCert) of the certificate needs to be unique per app service plan. If you are running this deployment more than once, you may have to remove the certificate first. The deployment references the KeyVault by ID. Use the name of the secret you used in the PowerShell script above (kvAccessCert).

Important: If you have not already done so, give the WebApp-Resource Provider access to the vault. The ID is static across all of Azure:

Set-AzureRmKeyVaultAccessPolicy -VaultName demo-keyvault -ServicePrincipalName abfa0a7c-a6b6-4736-8310-5855508787cd -PermissionsToSecrets get

In order for the app service to load the certificate, you must add an app setting:

Key: WEBSITE_LOAD_CERTIFICATES
Value: *

Use * to load all available certificates. Alternatively, you can use a comma-separated list of certificate thumbprints. Add this setting in any way you’d like (ARM template, via portal, or PowerShell). Make sure to restart the app service to load the application.

Using the Certificate to Access KeyVault

Install the following NuGet packages to use KeyVault from your application:

Microsoft.Azure.KeyVault
Microsoft.Azure.KeyVault.Extensions
Microsoft.Extensions.Logging.Console
Microsoft.IdentityModel.Clients.ActiveDirectory

You can use the following helper class for reading the certificate and creating a KeyVaultClient:

public static class KeyVaultUtility {
    private static ClientAssertionCertificate AssertionCert { get; set; }
    private static object m_lockObj = new object();

    public static void Initialize(string clientId, string certificateThumbprint, bool isSelfSignedCertificate = false) {
        lock (m_lockObj) {
            if (string.IsNullOrWhiteSpace(clientId)) {
                throw new ArgumentException("Argument clientId is required.");
            }

            if (string.IsNullOrWhiteSpace(certificateThumbprint)) {
                throw new ArgumentException("Argument certificateThumbprint is required.");
            }

            certificateThumbprint = Regex.Replace(certificateThumbprint, "[^da-zA-Z]", string.Empty).ToUpper();
            var certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);

            try {
                bool onlyAllowValidCerts = !isSelfSignedCertificate;
                certStore.Open(OpenFlags.ReadOnly);
                var certCollection = certStore.Certificates.Find(X509FindType.FindByThumbprint, certificateThumbprint, onlyAllowValidCerts);

                if ((certCollection?.Count ?? 0) == 0) {
                    StringBuilder sb = new StringBuilder();
                    foreach (var cert in certStore.Certificates) {
                        sb.AppendLine($"{cert.Thumbprint} - '{cert.SubjectName.Name}'");
                    }

                    throw new ArgumentException($"Certificate not found using thumbprint: '{certificateThumbprint}'. Certificates found:\n{sb}");
                }

                var certificate = certCollection[0];
                AssertionCert = new ClientAssertionCertificate(clientId, certificate);
            } finally {
                certStore.Close();
            }
        }
    }

    public static KeyVaultClient GetClient() {
        if (AssertionCert == null) {
            throw new Exception("Call Initialize before calling GetClient.");
        }

        return new KeyVaultClient(new KeyVaultClient.AuthenticationCallback((a, r, s) => GetAccessToken(a, r, s, AssertionCert)));
    }

    private static async Task<string> GetAccessToken(string authority, string resource, string scope, ClientAssertionCertificate cert) {
        var context = new AuthenticationContext(authority, TokenCache.DefaultShared);
        var result = await context.AcquireTokenAsync(resource, cert).ConfigureAwait(false);
        return result.AccessToken;
    }
}

Note that this is configured to search in the CurrentUser certificate store, where certificates are loaded in Azure. Locally, you may import them to the LocalMachine store instead.

If you are using .NET Core and do not want to use the provider shown in the next section, you need to use the IClientAssertionCertificate interface instead and implement your own version.

Using the values you retrieved from PowerShell above, you can create an instance of the client:

KeyVaultUtility.Initialize(CLIENT_ID, CERT_THUMBPRINT, true);
var client = KeyVaultUtility.GetClient();
var secret = await client.GetSecretAsync("https://demo-keyvault.vault.azure.net", "key");

The last parameter is the key of the secret you want to retrieve. Using certificates deployed from KeyVault is the most secure approach.

Troubleshooting Common Errors

• AADSTS50027: Invalid JWT token. Token format not valid.

  • Ensure no hidden characters exist in your client ID or thumbprint values from copy-paste.
  • Add these values to a plain-text editor and copy them again to ensure no formatting issues.

• Application with identifier ‘xxx’ not found.

  • Verify that you added the application to the default Azure Active Directory.

If you come across any other errors and solutions, feel free to contact us.

Using Configuration Provider

ASP.NET Core introduces a new extensible configuration provider model, allowing for the automatic loading of secrets from Azure Key Vault.

Installing the Required NuGet Package

Install the following NuGet package:

Microsoft.Extensions.Configuration.AzureKeyVault

Adding Configuration in Startup.cs

In your Startup.cs, where the configuration is built, add the following code:

var certificateThumbprint = Regex.Replace(CERT_THUMBPRINT, "[^0-9a-zA-Z]", string.Empty).ToUpper();

using (var store = new X509Store(StoreName.My, StoreLocation.CurrentUser))
{
    store.Open(OpenFlags.ReadOnly);
    var cert = store.Certificates.Find(X509FindType.FindByThumbprint, certificateThumbprint, false);

    builder.AddAzureKeyVault(
        keyvault,
        clientID,
        cert.OfType<X509Certificate2>().Single(),
        new EnvironmentSecretManager(env.ApplicationName)
    );
}

This utilizes the same two values (client ID and certificate thumbprint) as the code above. Ensure that you import your certificate into the appropriate certificate store.

Using EnvironmentSecretManager

Microsoft provides a template class EnvironmentSecretManager. This class loads only secrets whose keys begin with the application name.

public class EnvironmentSecretManager : IKeyVaultSecretManager
{
    private readonly string _appNamePrefix;

    public EnvironmentSecretManager(string appName)
    {
        _appNamePrefix = appName + "-";
    }

    public bool Load(SecretItem secret)
    {
        return secret.Identifier.Name.StartsWith(_appNamePrefix);
    }

    public string GetKey(SecretBundle secret)
    {
        return secret.SecretIdentifier.Name.Substring(_appNamePrefix.Length);
    }
}

Summary

Using this approach, you can easily inject configuration values from Azure Key Vault into your ASP.NET Core application. The EnvironmentSecretManager class ensures that only relevant secrets are loaded based on the application name. x