Really cool to see the difference in token usage across different harnesses.
These kind of independently-verified data points will be super important for companies (and individuals!) trying to measure RoI on token spend as they recover from letting people go wild on token usage in the past year.
I have started using this to inspect some of my heavier sessions and it has helped uncover some of the parts of my workflow and my project's build pipeline that were really slowing me down.
I also had no idea how many cache misses were happening when I stepped away for an hour or more at times.
I can highly recommend https://agentsview.io/ for a way to dig into your LLM usage. It reads your local conversation files and provides a ton of visualizations as well as the ability to dig into each conversation and see all the tool calls/etc. It's incredibly powerful.
Looks interesting. One of the angles that I approached while building this was focusing on cache misses. Not sure if this shows much info about this category. My tool is also local only + detecting changes and syncing into a sqlite db
yeah i tried to do even deeper and classify the types of misses and their monetary cost. for example, I was using Fable at work, and a single TTL miss (responding an hour and half later), cost $6 for one message
Really cool to see the difference in token usage across different harnesses.
These kind of independently-verified data points will be super important for companies (and individuals!) trying to measure RoI on token spend as they recover from letting people go wild on token usage in the past year.
I have started using this to inspect some of my heavier sessions and it has helped uncover some of the parts of my workflow and my project's build pipeline that were really slowing me down.
I also had no idea how many cache misses were happening when I stepped away for an hour or more at times.
Nice.I have been using codeburn but it only tells you spend not messages and everything else.
Is there a way i can connect it to my profile and get an idea of my token use?
There's a link to the repo at the bottom. It runs locally on your machine
dug around in there briefly and the per-session explorer is the best part imo
I can highly recommend https://agentsview.io/ for a way to dig into your LLM usage. It reads your local conversation files and provides a ton of visualizations as well as the ability to dig into each conversation and see all the tool calls/etc. It's incredibly powerful.
Looks interesting. One of the angles that I approached while building this was focusing on cache misses. Not sure if this shows much info about this category. My tool is also local only + detecting changes and syncing into a sqlite db
Yes, it does show cached vs not:
- https://cs.joshstrange.com/jzwC4Z9y
- https://cs.joshstrange.com/nynGc5V9
yeah i tried to do even deeper and classify the types of misses and their monetary cost. for example, I was using Fable at work, and a single TTL miss (responding an hour and half later), cost $6 for one message