Hiding Secrets in appsettings.json – Using a Bridge in your ASP.Net Core Configuration (Part 4)

This is part 4 of a series where I have been looking at moving to a SOLID approach of implementing configuration binding in ASP.Net Core using a bridging class to remove the need for consumers of the configuration object to use IOptions<T>  or IOptionsSnapshot<T>. If you have arrived at this page from a search engine, I recommend looking at the previous posts Part 1Part 2 and Part 3 before moving onto this one.

In this post I move onto looking at injecting some functionality into the bridge class to decrypt settings and validate the settings read. Lastly I show registering the bridge class via multiple fine grained interfaces.

To follow along with this post, I suggest you download the full solution source code from the Github repo at https://github.com/configureappio/ConfiguarationBridgeCrypto as there is far too much code to display in this post.

Adding More DI Services

I will start with the changes to the Startup.cs ConfigureServices method which gives a structure to the changes we will be making.

The main highlights are:

  • A factory class is registered via its interface to decrypt values read from the settings
  • A class via its interface is registered to validate the settings
  • The bridge class is registered via an aggregate interface
  • A resolution lambda is registered for each of the component interfaces that make up the aggregate interface.

The classes and interfaces are now looked at in more detail below.

Encrypted Settings

For the purpose of this demonstration, I am assuming that for one reason or another,  the standard secure configuration providers such as Azure Key Vault cannot be used for one reason or another, so we are having to deal with encrypting the settings ourselves.

Heath Warning !!! 

In this demo, the encrypted settings are in the main appsettings.json. DO NOT DO THIS IN THE REAL WORLD!  Stick to the mantra that you should not put any secrets in your code source control. Always think, “Would I be OK with the source code repo going open source?”

In the source code, I have included code to read from an external file outside of the web code location so that the secrets are maintained outside of source code, but the settings could come from environmental variables or the command line. If copying the source code that accompanies this post, I suggest copying the appsettings.json to the location shown and removing outside of source code. It is up to you where to store it.

To keep things clean, all encrypted values are held in a dictionary within the AppSettings class in a property called Secrets. It will be the responsibility of the bridge class to decrypt the secrets and inject them into the properties exposed via the interfaces (more on this later).

The appsettings.json and matching MyAppSettings class will therefore look like this:

At this point, the DI container has been configured to return IOptionsSnapshot<MyAppSettings> by the services.Configure<MyAppSettings> line in the startup code.

The secrets dictionary key/value pairs have been encrypted using a hash for the key and AESManaged for the value. Both have then been Base64 encoded so that they can be cleanly represented in the JSON.

Decrypting the Settings

In order for the bridge class to decrypt the dictionary, we will need a class that will get injected into the bridge class via an ICryptoAlgorithm interface. To keep things flexible, we will use a factory pattern to create the decryptor instance.

In the example code, I register the results of calling the factory as a singleton that. However, you may want to register the factory and inject that into classes if you want to use multiple cryptographic algorithms or salt/password combinations.

Once we have the decryptor registered, we need to apply it to the settings. For this, we will have a SettingsDecryptor class that implements an ISettingsDecrypt interface.

This is registered with the DI container services ready to be used by the bridge class. The Decrypt method in the example takes a plain text version of the dictionary key then

  • hashes it with the method exposed by the injected decryptor,
  • looks up the hashed key (that is in the appsettings.json)
  • then decrypts the value for that key.

So with these pieces in place, we have the components for decrypting the settings in the bridge class.

Before looking at the bridge, we will look at injecting functionality to validate the settings.

Validating Settings

By injecting a settings validator into the bridge class, we have the ability to catch any problems before the rest of our code tries to use the values (encrypted or not).

The class implements an interface that validates the settings and returns a boolean to indicate success or failure. If validation has failed, an AggregateException instance is available that holds one or more validation exceptions.

There are more elegant ways in which this can be approached, but I used this approach for simplicity to illustrate the principle of injecting a validator into the bridge.

Once the validator is registered as a DI service, we are now ready to register the bridge class that takes both the decryptor and validator.

Take It To The Bridge

The example above is fairly self-explanatory at a high level. The constructor takes the IOptionsShapshot<MyAppSettings> to get the settings class that has been constructed by the DI service using the Options pattern. This gives us access to the bound object.

We then have a decryptor which is stored as an instance field for use by the property getters to decrypt values read from the settings object.

We then have a validator instance which we call immediately to validate the settings and throw an exception if there is a problem.

The three properties exposed are proxies to the underlying settings class, with decryption taking place where necessary.

Interface Segregation

The bridge class implements the IAppSettingsResolved interface which is an aggregate of three other interfaces.

This has been done to illustrate that the settings can be registered as multiple interfaces to allow for interface segregation as part of the SOLID approach. E.g. if a controller is only interested in the SQL Server connection string, it can just ask for that rather than the full  IAppSettingsResolved. This makes it easier to implement just the required functionality in any mocks when unit testing or any other implementation you may want to register.

The registration above uses the service for IAppSettingsResolved to resolve the three other interfaces.

Conclusion

Having described the working parts above, we can come back to the ConfigureSevices method to tie it together.

Here we have done the following

  • Registered an IOptionsSnapshot<MyAppSettings> using the Configure method to bind the “MyAppSettings” configuration section to an object instance
  • Registered a decryption algorithm
  • Registered a class instance to decrypt the key/value pairs in the Secrets dictionary using the algorithm
  • Registered a class instance to validate the settingsop
  • Registered a class instance to act as the bridge/proxy to the IOptionsSnapshot<MyAppSettings> and decrypt the values
  • Registered the resolved bridge class using its multiple exposed interfaces for finer grained use

With the last of these in place, our controllers and any other dependant classes can choose whether to get the settings as a whole via IAppSettingsResolved or one of the finer grained interfaces

  • IAppSettings – the non-encrypted values
  • ISqlConnectionString – the decrypted SQL connection string
  • IOracleConnectionString – the decrypted Oracle connection string

Taking Things Further

That wraps up this series of posts for now, but you may want to take things further now that the settings class can have functionality injected into it.  Possibilities include

  • Using connection string builders to create connection strings using multiple properties (some encrypted, some not) from the bound object
  • Using the ICryptoFactory instead of ICryptoAlgorithm to use multiple algorithms for different properties

Don’t forget, the full source code including a WPF app to encrypt the settings dictionary can the downloaded from the Github repo at https://github.com/configureappio/ConfiguarationBridgeCrypto

Thanks for reading.

12 thoughts on “Hiding Secrets in appsettings.json – Using a Bridge in your ASP.Net Core Configuration (Part 4)

  1. Hi Steve,

    Nice series of posts.
    I had too this odd feelings about using the IOptions or IOptionsSnapshot throughout because – as you said – I was only interested in the value of T.

    Have you seen the IConfigureOptions interface? I only discovered it las week and it made really easy to inject dependencies to resolve you options.

    Thanks for sharing your ideas
    All the best

    1. Hi Anderson,

      Thanks for the comment. I wasn’t aware of the IConfigureOptions and will do some reading up on it.

      Do you have an example of how this can be used to allow the controller constructor to just take T as a constructor parameter (where T is the populated class from services.Configure)? My concern would be that if T has to implement IConfigureOptions, it would create a dependency on Microsoft.Extensions.Options.dllfor anything that wants to use T. I am happy to look into this further though.

      Thanks

      Steve

      1. Hi Steve,

        I understand and share your concern about the unnecessary dependency, but I believe you can use the same technique you described in part 2.

        So, you could implement the option like that:

        public class MyOptions
        {
        public string ApplicationName { get; set; }
        public string EnvironmentName { get; set; }
        }

        public class MyOptionsSetup : IConfigureOptions
        {
        private readonly IHostingEnvironment _hostingEnvironment;

        public MyOptionsSetup(IHostingEnvironment hostingEnvironment)
        {
        _hostingEnvironment = hostingEnvironment;
        }

        public void Configure(MyOptions options)
        {
        options.EnvironmentName = _hostingEnvironment.EnvironmentName;
        }
        }


        And in your Startup class:

        public void ConfigureServices(IServiceCollection services)
        {
        services.AddMvc();

        services.Configure(Configuration.GetSection(“MyOptions”));
        services.AddSingleton<IConfigureOptions, MyOptionsSetup>();
        services.AddTransient((serviceProvider) => serviceProvider.GetService<IOptions>().Value);
        }

        The only problem I noticed is that I need to register it as singleton which is super weird. Even if I register as transient the dependency behaves as singleton and if I try to register as scoped I got an error.

        1. Hi Anderson, Thanks for the comment and sorry it’s taken a while to respond.

          While I can see how using IConfigureOptions is attractive as it has out-of-the-box support, I’m not a big fan of changing the property values directly on the configured object and my (admittedly biased) opinion is that using a bridge is cleaner as there is less ‘magic’ going on that is done by ASP.Net behind the scenes and it is a more SOLID approach.

          Based on your comment, I have put my thoughts about IConfigureOptions in my latest blog post https://stevetalkscode.co.uk/using-iconfigureoptions.

          Cheers

          Steve

  2. Hello!

    Thank you so much for your post, i’m searching for solution to encrypt keys in json settings and i’m trying to use your proposal.
    I’m working dotnet core in ubuntu. Unfurtunately i’m receive the folowing error when compile the project after download from git hub:

    /usr/share/dotnet/sdk/2.2.104/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644

    I not found a solution fo this.

    Do you have any idea?

  3. Hi,
    Great post! Thanks!

    I tried to integrate it into a console application as a hosted service. But it seems the IOptionsSnapshot isn’t but injecting IConfiguration works fine,
    e.g. var x=configuration.GetSection( “MyAppSettings” )
    .Get();

    Any idea why? Or is this feature only supported for ASP.NET applications?

    1. Hi Rudy,

      Not sure why it isn’t working.

      I’ve just run the following code in a .Net Core console application and getting the value back. Have a look a the following code to make sure all steps are in place.

      Steve


      public class Program
      {
      static void Main(string[] args)
      {
      var config = new ConfigurationBuilder()
      .AddJsonFile("appsettings.json")
      .Build();

      var serviceProvider = new ServiceCollection()
      .AddOptions()
      .Configure(config.GetSection("Settings"))
      .AddScoped(sp => sp.GetRequiredService>().Value)
      .BuildServiceProvider();

      var settings = serviceProvider.GetRequiredService();

      Console.WriteLine(settings.Value);
      Console.ReadKey();
      }
      }

      public interface ISettings
      {
      string Value { get; set; }
      }

      public class Settings : ISettings
      {
      public string Value { get; set; }
      }

      1. Hi Steve,
        Thanks for reply! 🙂
        I get the values but they don’t change when I modify the appsettings.json while the console app is running. In your code above .AddJsonFile(“appsettings.json”) is missing the reloadOnChange parameter.
        However, I got an answer from another post – his explanation is:

        “IOptionsSnapshot is registered as scoped in DI. With ASP.NET Core, a scope is created per request. This is handy since the config updates only change per request, not during. Since you likely have no scope created in your console app, it’s likely resolving the same instance each time you request it (with no updates reflected). I suggest you try using IOptionsMonitor which is a singleton registration and supports reflecting changes immediately.”

        I’ll check, if or how I can integrate the IOptionsMonitor instead. Any suggestions appreciated. 🙂

        1. Hi Rudy,

          Sorry – I misunderstood your original comment. I thought you meant that you were getting a null when requesting the IOptionsSnapshot which was why I send some basic code back.

          You are right that I missed the reloadOnChange parameter in that example.

          With regards to IOptionsSnapshot vs IOptionMonitor, it is going to come down to what you are doing in your application.

          If you want to retrieve the configuration value at the time you are calling then you can register a transient to retrieve the IOptionMonitor.CurrentValue

          In ASP.NET Core, the Scoped lifetime makes sense at it effectively has the same lifetime as a HttpRequest.

          In a console app, it is your responsibility to create a scope when you need it. Where this may be of use is where you have long-running functions running concurrently where you do not want config values changing mid-way through processing.

          In those scenarios, you may want to create a scope yourself and use the IOptionsSnapshot anyway.

          E.g.

          var serviceProvider = new ServiceCollection()
          .AddOptions()
          .Configure(config.GetSection(“Settings”))
          .AddScoped(sp => sp.GetRequiredService>().Value)
          .BuildServiceProvider();

          var scopeFactory = serviceProvider.GetRequiredService();

          using (var scopedServices = scopeFactory.CreateScope())
          {
          var settings = scopedServices.ServiceProvider.GetRequiredService();
          Console.WriteLine(settings.Value);
          Console.ReadKey();
          }

          Not a great example, but provided just to give you the syntax required.

          In reality, you would be creating the scope at the start of a new thread/TPL Task (using (var scopedServices = scopeFactory.CreateScope()))

          Andrew Lock has a good comparison (though in the context of ASP.NET Core) at https://andrewlock.net/creating-singleton-named-options-with-ioptionsmonitor/

          Hope this is of help

          Steve

  4. Hi Steve,
    Awesome article! I gleaned lots of valuable bits but I feel like i’m stuck in a recursive loop of problems involving appsettings.json for our on premise applications. I am attempting to move my “secrets” into the Azure Key Vault, the Vault is configured and I have access restricted just to my application using a Service Principle Identity. The initial problem is solved, my secrets are now in the Vault instead of appsettings.join. My new problem is that in order to access the Key Vault I have to provide the Service Principle Identity clientId\clientSecret to authenticate to the Vault, most guidance I’m finding is that the clientId\clientSecret get defined in the appsettings.json or more commonly providing the clientId\clientSecret aren’t addressed so I now have a “different” secret define on-disk in a config file, this seems insecure.

    Maybe having clientId\clientSecret in the config is OK since the Vault is heavily audited with alerting capabilities but it feels like just another username\password that should not be in a config file (definitely not in source control). I’ve read up on defining the clientId\clientSecret as environment variables but in our multi-client, multi-server, multi-system environment it’s not feasible to manually apply environment variables to every single service.

    Any tips on how to define\configure the Azure Key Vault clientId\clientSecret in an on-prem, non-interactive application would be greatly appreciated. I can definitely use the encrypted configuration mechanism you outline in this post but it then feels like leveraging the Azure Key Vault really isn’t any more secure than just having encrypted settings locally, other than it has 1 more layer of obfuscation. Cheers!
    Jason

    1. Hi Jason,

      Thanks for the comment.

      I agree that when using Azure Key Vault, you can end up with a ‘turtles all the way down’ scenario, where at some point you have to provide some credentials, so you are just adding multiple layers of obfuscation.

      The only suggestions I can provide are the following:

      (a) Have you looked at using certificate authentication instead of clientid/clientsecret? There is an overload AddAzureKeyVault(IConfigurationBuilder, String, String, X509Certificate2) where you can use the certificate instead of the client secret. Certificates are held securely in the operating system, so your credential secret is outside source control. However, given what you have said about the multiple clients/servers etc your may have the same deployment problems you state about environmental variables.

      (b) My next suggestion was to put in environmental variables, but in your question you say this is not possible. I assume that you are not using containers as they are not mentioned, as Docker secrets would be a similar way of staying outside of source control. These can be access via the AddKeyPerFile extension to the ConfigurationBuilder.

      (c) My next suggestion requires a bit of work. Given you have a multi-client, multi-server, multi-system environment, you could write a REST API service to access the secrets (over HTTPS of course). This service should be within your infrastructure and not accessible to the outside world (you could limit access to it by IP address, certificates, NT authentication if in Windows environment etc). The service would provide the secrets either directly or as a proxy to the Azure Key Vault. You could then write an implementation of IConfigurationSource/IConfigurationProvider to access the service. The downsides to this are (i) potentially a single point of failure unless load balanced and (ii) the latency especially if acting as a proxy to the Azure Key Vault (c) you will have to write your own polling mechanism if you want to support configuration reload while the application is running.

      (d) The last option is to use a JSON file, but put it outside of the project/deployment folder. You can use an IFileProvider to load the json file into the ConfigurationBuilder as shown in the GitHub demo. This file would not be source-controlled.

      As you indicate in your question, there are two factors at play here.

      The first is protecting the secrets in the deployed application. Ultimately, the secrets need to be somewhere to be usable, so it is a case of making it as difficult as possible for a bad agent to get their hands on the secrets. The analogy here is putting a lock on a car’s steering wheel. It will not stop a determined thief from taking the car, but it makes it much more difficult and therefore deters the opportunist.

      The second is keeping the secrets outside of source control. Even if encrypted, if the secrets are available in the repository, this could be a more dangerous attack vector than the deployment itself if the repo was compromised or made public on GitHub. By having the secrets in Azure Key Vault and the credentials to the vault in a certificate/environmental variables/a file outside of source control, this removes this attack vector.

      Whilst my answer does not completely address your issues, hopefully it gives you some food for thought.

      Regards

      Steve

Leave a Reply

Your email address will not be published. Required fields are marked *