An IStartup Gotcha – The Curious Incident of the Missing Log Provider

In this post I look at a problem I had with injecting values into the constructor of a class that implemented IStartup and why doing this does not work as expected.

Background

When learning a new technology, I like to get to know the inner workings of it to try and understand how my code will work and develop a set of my own best practices.

Two areas of ASP.Net Core that I have a lot of interest in are the startup pipeline and configuration. The latter of these has been the focus of my previous posts about how to set up configuration using a SOLID approach.

For the former, I have been reading several blog posts, the most illuminating of which has been Steve Gordon’s blog which I highly recommend a read of. (Full disclosure – Steve runs the local .Net South East user group that I regularly attend).

In particular, I found his post on the Startup process illuminating as it reveals a whole load of reflection work being done by the ConventionBasedStartup class when using a convention based Startup class.

Now whilst I appreciate that Microsoft wants to make the Startup class as simple as possible to implement out-of-the-box in a new project, I am not a massive fan of this approach once you know what you are doing. After reading Steve’s blog post, it seemed logical to refactor my Startup classes to implement the IStartup interface as this is the first thing that is checked when instantiating the class (as shown in Steve G’s gist here)

On reading through how the ConventionBasedStartup class constructs an implementation of IStartup, I came to the following conclusions:

  • There is a lot of reflection going on that, if IStartup was used, would not be required
  • It is not statically typed, so if signatures are incorrect in some way such as a typo, it will fail at runtime but not be caught at compile time
  • The ability to have multiple versions of the same method for different environments in the same class differentiated by name E.g. ConfigureServicesDevelopment, ConfigureServicesStaging and relying on the ASPNETCORE_ENVIRONMENT environmental variable feels clunky and doesn’t feel like a SOLID approach.

I can see why Microsoft has taken this approach as it is easy to provide boilerplate code that is easy to extend, but for me, the convention approach does not feel right for the Startup process (though I reserve the right to be hypocritical in that I have been using the convention based approach for years in letting ASP.Net MVC work out which controller is being used in routing, but hey, I’m not perfect! ).

Therefore, in my ASP.Net Core projects, I started to refactor the Startup classes to implement the IStartup interface.

This was the approach I took on a project I was working on for a client. I had recently added NLog and this was working nicely using the convention based Startup, but in an effort to clean up the code, I wanted to refactor to IStartup. However, for some strange reason, the NLog logger stopped working after refactoring.

Time to get the deerstalker and magnifying glass out.

How Using IStartup Differs from Convention Based Startup

At first glance, the Startup classes used in both approaches appear to be the same, but there is one significant difference.

In the convention based approach, you are free to add as many parameters as you want to the Configure method, safe in the knowledge that as long as you (or rather in most cases, the WebHost.CreateDefaultBuilder) have registered the interfaces or classes you require with the Dependency Injection services, then they will be injected into the method.

This is thanks to the ‘magic’ inside the ConventionBasedStartup. When implementing IStartup yourself, the signatures are set in the contract

Of particular note is the signature for the Configure method which only accepts a single parameter of type IApplicationBuilder.

Therefore, if you want to inject other types, you can’t just add them to the method parameters, so you need to find another way.

Adding ILoggerFactory to the Startup Constructor – Don’t Do This At Home

TL;DR You can skip this bit if you want as it describes how I got to the bottom of the problem. If you just want to know how to correctly access the ILoggerFactory, go to here

When writing code that implements an interface, if I need extra class instances that are not provided for in the method signature, my usual approach is to make an instance variable that is assigned from a constructor parameter.

So when I no longer had access to ILoggerFactory in the Configure method, I added it to the Startup constructor and created an instance variable that I could then accessed from inside the Configure method. So far, so normal. I did the NLog configuration in the Configure method and just expected things to work as they had before.

Alas, twas not to be …

At first I thought that I must have messed up something in the NLog configuration, but couldn’t see anything different. So I stepped through the code. I looked at the ILoggerFactory that had been injected into the constructor of the Startup.

There were two providers present, Console and Debug. I then ran through the code to where NLog had been added, and looked at the providers collection again. Yup, three providers with NLog being the last. All good.

I then ran though to the constructor of the controller where I was doing some logging where I had declared a parameter of ILogger. I looked at the Loggers collection on the logger and there were four loggers, but no sign of the NLog logger.

Somewhere along the pipeline, the ILoggerFactory had lost NLog and added two loggers, both of which were the ApplicationInsights logger.

For the client’s project I reverted back to the convention based approach so as not to hold up the development, but my curiosity got the better of me and decided to dig a bit more.

Compare and Contrast

At home, I created a simple project where I created two copies of the Startup class. One using the convention approach where ILoggerFactory is injected into the Configure method and one where I have the ILoggerFactory injected into the constructor of the Startup class and saved in an instance variable. I then used a compilation symbol to run with one or the other.

I created a dummy class that implemented ILoggerProvider rather than muddy the waters by using NLog and having to worry about its configuration. I then implemented the constructor based code as I had previously done, as shown below:

I ran through both scenarios using the Visual Studio debugger and got the same result.

  • When using convention based Startup class, there were five loggers, Debug, Console, 2 x Application Insights and my DummyProvider.
  • When using the IStartup implementation, there were just the four loggers – DummyProvider was missing.

So somehow, the logging factory instance in my Startup class is not the one that makes it to the controller.

The Correct Way to Access the Logger Factory

OK, so the constructor approach does not work. It then dawned on me – is there another way to get to the ILoggerFactory instance through the IApplicationBuilder that is provided via the IStartup.Configure(IApplicationBuilder app) method?

In short, yes! The app.ApplicationServices provides access the the IServiceProvider container where we can resolve services.

I ran the test again, and there was the DummyLogProvider.

The Logger Is There!`

Problem solved.!

Hiding Secrets in appsettings.json – Using a Bridge in your ASP.Net Core Configuration (Part 4)

This is part 4 of a series where I have been looking at moving to a SOLID approach of implementing configuration binding in ASP.Net Core using a bridging class to remove the need for consumers of the configuration object to use IOptions<T>  or IOptionsSnapshot<T>. If you have arrived at this page from a search engine, I recommend looking at the previous posts Part 1Part 2 and Part 3 before moving onto this one.

In this post I move onto looking at injecting some functionality into the bridge class to decrypt settings and validate the settings read. Lastly I show registering the bridge class via multiple fine grained interfaces.

To follow along with this post, I suggest you download the full solution source code from the Github repo at https://github.com/configureappio/ConfiguarationBridgeCrypto as there is far too much code to display in this post.

Adding More DI Services

I will start with the changes to the Startup.cs ConfigureServices method which gives a structure to the changes we will be making.

The main highlights are:

  • A factory class is registered via its interface to decrypt values read from the settings
  • A class via its interface is registered to validate the settings
  • The bridge class is registered via an aggregate interface
  • A resolution lambda is registered for each of the component interfaces that make up the aggregate interface.

The classes and interfaces are now looked at in more detail below.

Encrypted Settings

For the purpose of this demonstration, I am assuming that for one reason or another,  the standard secure configuration providers such as Azure Key Vault cannot be used for one reason or another, so we are having to deal with encrypting the settings ourselves.

Heath Warning !!! 

In this demo, the encrypted settings are in the main appsettings.json. DO NOT DO THIS IN THE REAL WORLD!  Stick to the mantra that you should not put any secrets in your code source control. Always think, “Would I be OK with the source code repo going open source?”

In the source code, I have included code to read from an external file outside of the web code location so that the secrets are maintained outside of source code, but the settings could come from environmental variables or the command line. If copying the source code that accompanies this post, I suggest copying the appsettings.json to the location shown and removing outside of source code. It is up to you where to store it.

To keep things clean, all encrypted values are held in a dictionary within the AppSettings class in a property called Secrets. It will be the responsibility of the bridge class to decrypt the secrets and inject them into the properties exposed via the interfaces (more on this later).

The appsettings.json and matching MyAppSettings class will therefore look like this:

At this point, the DI container has been configured to return IOptionsSnapshot<MyAppSettings> by the services.Configure<MyAppSettings> line in the startup code.

The secrets dictionary key/value pairs have been encrypted using a hash for the key and AESManaged for the value. Both have then been Base64 encoded so that they can be cleanly represented in the JSON.

Decrypting the Settings

In order for the bridge class to decrypt the dictionary, we will need a class that will get injected into the bridge class via an ICryptoAlgorithm interface. To keep things flexible, we will use a factory pattern to create the decryptor instance.

In the example code, I register the results of calling the factory as a singleton that. However, you may want to register the factory and inject that into classes if you want to use multiple cryptographic algorithms or salt/password combinations.

Once we have the decryptor registered, we need to apply it to the settings. For this, we will have a SettingsDecryptor class that implements an ISettingsDecrypt interface.

This is registered with the DI container services ready to be used by the bridge class. The Decrypt method in the example takes a plain text version of the dictionary key then

  • hashes it with the method exposed by the injected decryptor,
  • looks up the hashed key (that is in the appsettings.json)
  • then decrypts the value for that key.

So with these pieces in place, we have the components for decrypting the settings in the bridge class.

Before looking at the bridge, we will look at injecting functionality to validate the settings.

Validating Settings

By injecting a settings validator into the bridge class, we have the ability to catch any problems before the rest of our code tries to use the values (encrypted or not).

The class implements an interface that validates the settings and returns a boolean to indicate success or failure. If validation has failed, an AggregateException instance is available that holds one or more validation exceptions.

There are more elegant ways in which this can be approached, but I used this approach for simplicity to illustrate the principle of injecting a validator into the bridge.

Once the validator is registered as a DI service, we are now ready to register the bridge class that takes both the decryptor and validator.

Take It To The Bridge

The example above is fairly self-explanatory at a high level. The constructor takes the IOptionsShapshot<MyAppSettings> to get the settings class that has been constructed by the DI service using the Options pattern. This gives us access to the bound object.

We then have a decryptor which is stored as an instance field for use by the property getters to decrypt values read from the settings object.

We then have a validator instance which we call immediately to validate the settings and throw an exception if there is a problem.

The three properties exposed are proxies to the underlying settings class, with decryption taking place where necessary.

Interface Segregation

The bridge class implements the IAppSettingsResolved interface which is an aggregate of three other interfaces.

This has been done to illustrate that the settings can be registered as multiple interfaces to allow for interface segregation as part of the SOLID approach. E.g. if a controller is only interested in the SQL Server connection string, it can just ask for that rather than the full  IAppSettingsResolved. This makes it easier to implement just the required functionality in any mocks when unit testing or any other implementation you may want to register.

The registration above uses the service for IAppSettingsResolved to resolve the three other interfaces.

Conclusion

Having described the working parts above, we can come back to the ConfigureSevices method to tie it together.

Here we have done the following

  • Registered an IOptionsSnapshot<MyAppSettings> using the Configure method to bind the “MyAppSettings” configuration section to an object instance
  • Registered a decryption algorithm
  • Registered a class instance to decrypt the key/value pairs in the Secrets dictionary using the algorithm
  • Registered a class instance to validate the settingsop
  • Registered a class instance to act as the bridge/proxy to the IOptionsSnapshot<MyAppSettings> and decrypt the values
  • Registered the resolved bridge class using its multiple exposed interfaces for finer grained use

With the last of these in place, our controllers and any other dependant classes can choose whether to get the settings as a whole via IAppSettingsResolved or one of the finer grained interfaces

  • IAppSettings – the non-encrypted values
  • ISqlConnectionString – the decrypted SQL connection string
  • IOracleConnectionString – the decrypted Oracle connection string

Taking Things Further

That wraps up this series of posts for now, but you may want to take things further now that the settings class can have functionality injected into it.  Possibilities include

  • Using connection string builders to create connection strings using multiple properties (some encrypted, some not) from the bound object
  • Using the ICryptoFactory instead of ICryptoAlgorithm to use multiple algorithms for different properties

Don’t forget, the full source code including a WPF app to encrypt the settings dictionary can the downloaded from the Github repo at https://github.com/configureappio/ConfiguarationBridgeCrypto

Thanks for reading.

Creating a Bridge to your ASP.Net Core Configuration from your Controller or Razor Page (Part 3)

This is the third in my series of posts looking at how to remove the need for controllers and razor pages to have knowledge of the options pattern in ASP.Net Core.

If you have arrived here from a search engine or link, I would recommend reading the previous two posts which set the background before coming back to this post.

Creating a Bridge to your ASP.Net Core Configuration from your Controller or Razor Page (Part 1)

Creating a Bridge to your ASP.Net Core Configuration from your Controller or Razor Page (Part 2)

In Part 2, I looked at how a lambda expression could be used to act as a bridge between the IOptionsSnapshot and T by creating an additional DI service:

If all you are concerned with is ensuring that your controller or razor page does not need to refer to the options pattern, then this is a suitable solution.

However, you may have the need to do something more exotic with the configuration settings such as perform some transformation such as decrypting some data or would like to perform  a validation of the settings before invalid values get injected into a class. This is achievable using the lambda, but it becomes somewhat messy.

Instead, the approach I will describe in this post will be to split the MyAppSettings class from the previous post out into three parts:

  • An interface that defines the properties as read only values IMyAppSettings
  • A settings reader class, MyAppSettingsReader,  that implements the interface but also implements setters so that the values can be mapped into an instance by the Configure extension method
  • A bridge class, MyAppSettingsBridge, that takes the IOptionsSnapshot in the constructor and then presents itself as the interface by using the Value method to get the value that has been read from configuration.

The last part of the jigsaw is then to register the bridge as a transient service with the DI container. From therein, the controllers, razor pages and any other class that needs the settings will just need a parameter of type IMyAppSettings.

Splitting the Class Up

In the previous post, the MyAppSettings class looked like this:

We will now divide it up, starting with the interface:

Note that the properties have been defined as read-only. Given that the configuration source(s) are read only as far as the code is concerned ( JSON files, XML files, environmental variables etc.), it is unlikely you will have code that would change the values.

Then comes the two implementation of the interface:

The MyAppSettingsReader is a simple DTO that the Configure extension method can map the configuration settings to – and therefore does need setters as well as the getters.

This class does not strictly need to implement the interface as it is there simply to map settings from the configuration into an object. You could include other properties that may be components that will be combined as a value returned in a property exposed in the interface. E.g. say you have a property that is a database connection string.

The second class is the bridge itself, MyAppSettingsBridge which must implement the interface as it will be used in the DI container. Note the constructor takes IOptionsSnapshot as the parameter and then acts as the go-between for the properties for the interface.

In the example class above, the class is effectively doing the same as the lambda expression from the previous post by calling the Value property on the options object.

However, by using a class, you can add more functionality. Taking the database connection example again, you could have individual properties for the server name, the database name, etc. in the reader class which can be then passed as parameters into a connection string builder inside the bridge class whose result is then exposed as a ConnectionString property in the interface.

In my plans for another post, I will be showing how values could be encrypted in the configuration settings source and then decrypted by the bridge class.

Wiring It All Up

Now we have the interface, reader and bridge, it is time to wire it all up.

Firstly, we will set up the DI container to read the configuration into the reader. This will automatically create a DI service for IOptionsSnapshot. Next we register the bridge as a transient instance of the IMyAppSettings interface.

Why register as Transient?It is up to you really. If registered as a transient, then every call will get the latest version of the configuration injected into it from IOptionsSnapshot. 

If registered as a singleton, the IOptionsSnapshot does not reload the configuration on each request.

If you are not worried about having the ability to read the configuration without restarting the application, use the singleton and also change IOptionsSnapshot to IOptions so that the configuration monitor is not required.

Now we are all set with the DI container, so we can change our controller to take IMyAppSettings as the parameter to the controller.

The code for this post and the previous post is available on GitHub at https://github.com/configureappio/configurebridgedemo1

In the next post, I will look at injecting more functionality into the bridge to decrypt some settings before they get injected into the controller.

Creating a Bridge to your ASP.Net Core Configuration from your Controller or Razor Page (Part 2)

In my previous blog post, I set the scene to give some background as to the relative merits of using the Options pattern vs binding a configuration object to a singleton in the DI container.

To summarise

  • IOptionsSnaphot allows the configuration to be changed without having to restart the application in order for new requests to make use of the changed values. However, the DI container will present the configuration as IOptions or IOptionsSnaphot which means that any assembly that makes using of the configuration will need to refer to the Microsoft.Extensions.Options Nuget package. Some feel that this is an overhead and is harder to test than a reference to just T (where T is the bound configuration object)
  • Binding the configuration object T in the service setup and then storing it in a singleton allows method signatures to just use T as a parameter which is cleaner, but loses the ability to dynamically react to changes in the configuration (E.g. the appsettings.json being changed)

For this post, I originally planned to launch into a full blown discussion of creating a bridging class between the controller and the configuration setting object to mask the use of IOptions.

However, before doing that, I noticed a comment at the bottom of Rick Strahl’s blog post from Todd Menier pointing out that a very simple bridge could be created by using an anonymous function, which seemed a good way to start to describe the basics of creating a bridge between the controller and the configuration before going into the more complex implementation of using abstractions and classes.

Before getting into creating the bridge, a quick recap of why the bridge is required.

An Example of Out-of-the-Box IOptions implementation

Say you have a class called MyAppSettings that you want to bind configuration data to, you will have a class that looks like this:

Which in turn you add to your appsettings.json as this:

In order to wire up the two, you will have a Startup.cs class that looks like this:

The first important line is services.AddOptions() which is required for IOptions to work.

The second is services.Configure(Configuration.GetSection(“MyAppSettings”)) which takes the MyAppSettings section that has been read from the configuration sources (in our case appsettings.json, but this may have been overridden by other sources) and binds it to an instance of MyAppSettings.

Looking at the source code for Configure(IConfiguration config) shows that behind the scenes, two singletons are registered: ConfigurationChangeTokenSource and NamedConfigureFromConfigurationOptions.

The first of these monitors the configuration for changes and is used when a parameter is defined as being of type IOptionsSnaphot. This allows changes to be sent to the controller without the need to restart the application. The second handles binding the object that holds the configuration values as the class instance.

With the binding set up, in your controller, you can have a constructor that looks like this:

Note, that in order to receive the MyAppSettings instance, the constructor parameter must be either of type IOptions or IOptionsSnapshot. The difference being that the former only gets the settings as they were when the application started, but the latter reads the latest version of the settings even if they changed after the application has started.

It is this extra bit of orchestration that some people don’t like as it means that any unit tests must mock the IOptions interface and provide an implementation of the Value method to get to the MyAppSettings object that is actually of interest.

Now, if it was purely just about MVC controllers and razor pages, I am personally not too hung up on this. However, it becomes a bit more complicated if the configuration settings are required for the constructor of a class in some other assembly as it then means that the other assembly needs to have a reference to the Microsoft.Extensions.Options Nuget package. This in turn could start a dependency sprawl across multiple assemblies.

Intercepting with a Bridge

To avoid this, a bridge is required that can accept IOptions or IOptionsSnapshot as a parameter, but presents itself as MyAppSettings (or an abstraction of it).

In my next post, I intend to present a solution using a bridging class that can accept other parameters to do more than just abstract away from the options pattern, by adding functionality such as decryption.

In the meantime, a way of simply getting the options pattern out of the way is to add another service to the DI container, this time a transient that uses an anonymous lambda function to get the value out of the options pattern and present that (as suggested by Todd).

This then allows the controller (or any other class that needs access to the configuration settings) to just need the MyAppSettings class as the parameter:

This lets the DI container deal with the bridging and keeps the clients ignorant of the options pattern.

In the next post in the series, I look at moving from using the lambda to using a full bridging pattern

Creating a Bridge to your ASP.Net Core Configuration from your Controller or Razor Page (Part 1)

This post is intended to set the stage for a later post (though may become a series) I have planned in which I look at using the Bridge design pattern to break the immediate dependency on the Options pattern in a .Net Core application.

The pattern is used to bind configuration settings to an object rather than pollute code with references to the configuration directly. For those not familiar with the pattern, I recommend following the link above.

This topic became of interest to me after reading a couple of Rick Strahl’s blog posts in which he discusses binding configuration settings to objects.

In these posts, Rick looks at approaches to binding configuration sections to objects, initially by binding directly to a class and storing that class as a singleton which can then be injected into the controller (or Razor page), and then later comparing that approach to using IOptions<T> or IOptionsSnapshot<T> as advocated by the .Net team.

A comparison between the two is also looked at by Filip W in the post Strongly typed configuration in ASP.NET Core without IOptions.

In both these posts, the authors appreciate the benefits of IOptions allowing the configuration to be changed on-the-fly without having to restart the application (when using IOptionsSnapshot<T>) whereas, an application restart is necessary with a singleton, but are not keen on having to drag along the Microsoft.Extensions.Options package everywhere that the configuration object will be used and having to use IOptions in parameter definitions when what is really of interest is the object T. This is a theme that runs through several blog posts and StackOverflow questions.

Rather than repeat the same discussions here, the links above are provided as background reading to give a foundation for the content I intend to provide in my next post where I will look at creating an intermediary (or bridge) so that projects that have in interest in the configuration object do not have to have a direct dependency on taking an IOptions<T> parameter, but instead take an abstraction representing the configuration.

By bringing an intermediary into the mix, it will also allow for adding functionality such as early validation of the configuration before it is used and decryption of sensitive data.

This will be the focus of the next post.