Building Responsible AI Agents with Copilot Studio and Azure Application Insights

In the fast-evolving world of AI agent development, it’s easy to get caught up in the excitement of what’s possible. But as we scale our solutions—especially in enterprise environments—questions around monitoring, sustainability, and responsible development quickly surface.

This post dives into how we can go beyond just building agents in Copilot Studio, and start thinking about how to track, understand, and support them in production. The key? Azure Application Insights.

Why Monitoring Matters

When IT teams start pushing back on AI implementations, it’s rarely about the tech itself. It’s about the lack of visibility. How do we know what the agent is doing? How do we detect when something goes wrong? How do we ensure the system is maintainable?

These aren’t just enterprise concerns—they’re responsible development concerns. And they’re critical if we want our solutions to be trusted and scalable.

What Copilot Studio Offers Out-of-the-Box

Copilot Studio provides a user-friendly interface for building agents, and it does offer some basic monitoring capabilities. You can see which flows are triggered, what data is accessed, and how users interact with the agent.

But for deeper insights—especially across multiple agents or hybrid architectures—you need something more robust.

Enter Azure Application Insights

Azure Application Insights is a powerful telemetry platform that allows you to instrument your applications and agents. It’s widely used in the pro-code world, but it’s just as valuable for low-code solutions.

Here’s what it enables:

• Structured logging: Capture detailed traces of agent activity.
• User behaviour tracking: See which buttons are clicked, which flows are triggered, and how users interact.
• Centralised monitoring: Aggregate telemetry from multiple agents and applications into a single dashboard.

And the best part? It’s cheap to run and easy to set up.

Setting It Up

To integrate Application Insights with your Copilot Studio agent:

1. Create an Application Insights instance in Azure.
2. Grab the connection string from the instance.
3. In your agent settings, go to Advanced and paste the connection string.

Now, every time your agent takes an action—like calling a flow or writing to a database—you’ll see a trace in Application Insights.

Real-World Example: NCCD Support Assistant

In the video, we walk through an agent built for the education space: the NCCD Support Assistant. It interacts with several flows, including:

• Accessing the NCCD public website
• Adjusting data tables
• Registering entries in a database

By connecting this agent to Application Insights, we can monitor every interaction, track performance, and ensure the system behaves as expected.

Unified Telemetry Across Architectures

One of the most powerful aspects of Application Insights is its ability to unify telemetry across different types of applications. Whether you’re running:

• Low-code agents in Copilot Studio
• Azure Functions
• Bespoke pro-code applications

You can surface all telemetry in one place. This makes it easier to manage, monitor, and support your entire AI strategy.

Final Thoughts

Building AI agents is just the beginning. To make them truly enterprise-ready, we need to think about observability, sustainability, and supportability.

Azure Application Insights gives us the tools to do just that—without adding complexity or cost.