This is a problem I've run into with several clients at my current job, as AI Engineer. The manager wants to know if their AI agent is actually helping customers, and how much to charge for it.
So to start from scratch - how do you build AI agents today? 2 options:
1. Hope that senior management approves the development budget
2. Try to estimate how much it will cost and whether anyone will actually want the agent — with no clear idea of how much revenue it'll bring or how many people will use it.
So I built a dashboard that lets managers track this process. It shows exactly what a manager wants to see - how much a conversation cost and how it contributed to the user. No, not like LangFuse or LangsSmith.
There's also AI-agent-guided wizard on how to define the "value" the agent provides to the customer, and then verify whether the customer actually received it.
This is a problem I've run into with several clients at my current job, as AI Engineer. The manager wants to know if their AI agent is actually helping customers, and how much to charge for it. So to start from scratch - how do you build AI agents today? 2 options:
1. Hope that senior management approves the development budget 2. Try to estimate how much it will cost and whether anyone will actually want the agent — with no clear idea of how much revenue it'll bring or how many people will use it.
So I built a dashboard that lets managers track this process. It shows exactly what a manager wants to see - how much a conversation cost and how it contributed to the user. No, not like LangFuse or LangsSmith. There's also AI-agent-guided wizard on how to define the "value" the agent provides to the customer, and then verify whether the customer actually received it.
[dead]