Building a Terraform template to securely push application telemetry to App insights workspace bypassing local authentication

Azure Application Insights workspace is a cloud-based service for monitoring and analyzing the performance of applications. It provides real-time insights into the application's behavior, such as request and response times, user behavior, and error rates. 

In the past, Azure Application Insights was primarily used programmatically through its web APIs or various SDKs by providing an instrumentation key. This instrumentation key was required to interact with the platform and extract insights about the application's performance or query the data stored in it. However, this experience was limited because it lacked native identity authentication, making it challenging to secure the instrumentation key. Developers had to take extra precautions to secure the key and store it, which added an additional overhead to the development process. This absence of native identity identification made the workplace open to possible security breaches and unauthorized access to data.

Recently, Microsoft has made significant changes to Azure Application Insights Workspace to support Azure Active Directory (Azure AD) authentication. This has enabled developers to opt-out of local authentication and use Managed Identities instead. 

By using Managed Identities, telemetry data can be exclusively authenticated using Azure AD, providing a more secure and streamlined way of interacting with the platform. With this change, developers no longer need to worry about managing and storing the instrumentation key securely, as the authentication is handled by Azure AD. This improves the security of the telemetry data and reduces the overhead associated with managing authentication credentials. 

This blog post assumes that the reader has a basic understanding of the Azure Active Directory integration enablement for Azure Application Insights Workspace. If not, it will be recommended that you do the reading on MS learn and know details of it and also take a look at feature pre-requisites. 

The focus of this blog post is on how to configure Azure AD integration using a Terraform template and validate it using a sample .NET web API that talks to the Application Insights Workspace securely using its managed identity when deployed on an Azure Web App. 

Let's take a look at what a terraform template looks like that is responsible for deploying below resources

• Resource group.
• App service plan.
• Web app with it's system assigned managed identity.
• Log analytics workspace along with app insight resource.
• Role assignment to grant required permission to the web app's managed identity on the app insights resource.

There are a few key points that need to be focused on.  Firstly, the flag "local_authentication_disabled" must be set to "true" in the Application Insights configuration. This disables local authentication and enables the use of Azure AD for authentication. Secondly, the Azure resource role "Monitoring Metrics Publisher" is a pre-requisite for communication between the telemetry publisher and the Application Insights Workspace. This role must be assigned to the managed identity of the web app resource in order for it to be able to communicate with the Application Insights resource. 

Focusing on these two points will ensure that the Terraform template is set up correctly and the web app is able to communicate with the Application Insights securely using Azure AD authentication.

Now that the Terraform template for configuring Azure AD integration has been discussed, it's time to focus on verifying the setup. The easiest way to do this is to write a sample web API code and deploy it to the Azure Web App resource that was provisioned in the previous step. This will allow us to see if the telemetry data starts flowing to the Application Insights Workspace. 

For this post, a .NET 6 web API project with VS 2022 will be created with minimal code that configures the connectivity between the web app resource and the Application Insights. This project will be deployed to the web app and the telemetry data will be monitored in the Application Insights Workspace to confirm that the integration has been set up correctly.

Here is how the Program.cs of web api could look like - hard-coding in it for brevity

Also, note that in order to integrate the AAD-based authenitcation in your source code, it is important to refer to the correct version of SDKs and for that reason, you might need to install the Application Insights .NET SDK starting with version 2.18-Beta3

Two important points to mention from the sample code above i.e. Firstly, the use of the "ManagedIdentityCredential" provider to perform authentication using the managed identity. This allows the web API to communicate with the Application Insights Workspace securely using Azure AD authentication. Secondly, the connection string contains the instrumentation key and ingestion endpoint.

At this point, it may seem counterintuitive that the instrumentation key is still being used despite the goal being to not specify it. However, the instrumentation key is still required for configuring the connection between the web API and the Application Insights Workspace. The reason the instrumentation key is still used is because it acts as a identifier for the Application Insights Workspace, and allows the "ManagedIdentityCredential" provider to reach the correct resource. The provider uses the instrumentation key to establish the connection between the web API and the Application Insights Workspace.

It is important to note that, since local authentication is disabled in the Application Insights, only Azure AD objects such as managed identities can successfully authenticate to it and the "Monitoring Metrics Publisher" role must be granted to the managed identity in order to allow it to communicate with the Application Insights Workspace.

With this setup in place, you should be ready to start seeing telemetry data from your application in the Application Insights Workspace.

In summary, when working with local authentication disabled in Application Insights Workspace, it is essential to use the "Monitoring Metrics Publisher" role in addition to the instrumentation key in order to publish telemetry data and by following this setup, you can ensure that your telemetry data is securely sent to the correct Application Insights Workspace, while taking advantage of the enhanced security and ease of use provided by Azure AD authentication and managed identities.