Every morning, Rohan reviewed every call

Every morning, Rohan would spend his first 2 to 3 hours reviewing cold calls.

Our reps called prospects in North America through the night, India time, and his evenings were full of meetings, so the start of the day was the only time left for reviews. Rohan spent those hours listening to calls, recording feedback, rewriting the script's wording and fixing whatever had broken overnight, which meant the most important hours of his workday were gone before anything else could start.

The problem was that the process was manual at both ends. Every time a rep made a call, they had to record it on their phone, upload the recording to Notion, post it in their Slack channel, and then type in everything else by hand: dials, people not interested, voicemails. Then Rohan would get up, scroll through all of the Slack messages, listen to every single recording and leave his feedback under each one as another recording. The reps would listen to that and reply with questions, and then nobody would reply.

It was a mess...

The mess grew with every rep we added. One morning the queue was 50 calls and about 30 long practice sessions, and Rohan lost 10 to 15 minutes per rep just finding and replaying recordings before he could start coaching. Recordings came in with names like "call1234". He gave the same feedback again and again, because nothing tracked whether anyone had acted on it, and when he got tired of repeating himself, he started recording Looms on the principles behind the script instead.

So we built our own coaching platform.

Most obvious thing given our unparalleled expertise in dev. Few know this...

The first working version was live with our reps, pulling calls straight from our dialer, about 36 hours after we started. By day 3, every call was being reviewed on it, and Rohan was doing his morning reviews there instead of in Slack. Getting it to the point where everyone trusted it took about 7 days, and the rest of this post is about the decisions we made in those 7 days and the fine-tuning in the weeks after.

Why didn't we just buy a sales CRM?

The first thing we did was look at what we could buy. AI sales coaching tools that sit on top of a CRM start at about $300 per seat per month. We had 5 cold callers and were hiring more, so before anyone made a single call, we were looking at $1,500 to $2,000 a month for the reps alone. Admin seats for Rohan and the managers would come on top of that, the dialer would come on top of that, and every new hire would push the bill higher.

That might have been fine for a sales team that knew what it was doing. We didn't, not yet. This was our first time running cold calling, and cold callers are hard to judge before they start. You get a rough read in the interview, and then you wait and watch it play out over days of calls. Paying per seat meant paying the same for a rep who would turn out great and a rep who would be gone in 2 weeks.

One early hire cost us over $2,000 before we had heard enough of his calls to let him go.

And we had something no vendor could sell us. Rohan had been doing the job by hand for weeks, so we already knew exactly what the software had to do. His voice notes, his Looms and every version of his script were sitting in Slack. Buying a tool would have meant bending that process to fit someone else's idea of sales coaching, so we decided to build one around the process we already had.

Map business problems to features before writing code

Before anyone opened an editor, we spent our time on one question: what does Rohan actually do when he reviews a call? The core job was easy to say, make call review and learning faster. But a reviewer that doesn't know the script, the product or Rohan's standards gives generic feedback, and reps stop reading generic feedback. So the core job only works if a ring of supporting features sits around it, and working out that ring is most of what a forward deployed engineer does.

We worked it out by watching Rohan. We went through his screen recordings and his Slack replies and wrote down every habit as a problem the software had to solve.

He kept repeating points that were already written in our SOPs, so the platform imports the SOPs and the reviews quote them for him. He kept checking calls against the script, so it imports the script for each offer. He kept correcting words reps should never say, so there's an anti-script the reviews flag. And he kept re-explaining the product, so the product-knowledge training reps learn from goes into every review, because a rep who can't explain the product can't follow the script either.

The rest of the map came out the same way:

• Repeating points already written in our SOPs → Import the SOPs, so reviews refer to them
• Checking each call against the script → Import the script for each offer
• Correcting words reps should never use → An anti-script that reviews flag
• Explaining the product again and again → Product-knowledge training inside every review
• Waiting for reps to record and upload calls → Pull every call from the dialer automatically
• Hearing a rep, a receptionist and a transfer on one call → Separate the speakers in the transcript
• Coaching across Slack, voice notes and Looms → Manager notes on the transcript, ranked above the AI
• Guessing why a rep's numbers dropped → Dials and connects per rep, by the hour

Only once that map existed did we open Claude Code, and we kept that order on purpose. When code is this cheap to produce, the expensive mistake is building the wrong thing quickly, and the map was our defence against it. It also gave us a filter for every request that came later: which problem on the map does this solve?

How we built it in one day with Claude Code

With the map done, the build itself took about a day. We had a working mock in front of Rohan 2 to 3 hours after we started, a rebuilt version by the end of that afternoon, and the platform live with reps and connected to the dialer the next night. That speed came from deciding almost everything before the first prompt, and then letting Claude Code do the building.

One premise, with every feature built around it

We started from one premise: make call review and learning faster for Rohan and for the reps. Every feature on the map was bolted onto that premise, and the design work was making all of them behave as one system.

That meant deciding the data model before anything else. Companies contain offers, reps are assigned to offers, and each offer has its own script, SOPs and playbook, because the same rep can call for several offers in a week. Every call is stored with its outcome and the reason for it, like "blocked at reception (owner not in)", so a review, a dashboard and a coaching note all read from the same record and never disagree.

One large prompt, and letting Claude ask the questions

Then I wrote one massive prompt with everything in it: the problem-to-feature map, the data model, the integrations, and how every screen should behave. I gave it to Claude, and instead of building straight away, it played back its own understanding of the system. It filled the holes it could, and for the rest it asked me open questions about how the parts should work together.

Most of them were product decisions, the kind we would normally make in an internal meeting: how granular the daily report should be and what it should include, or whether reps should be able to see each other's calls and scores. I answered them, it asked a few more, and after 2 or 3 rounds of that, I let it go off and build the full platform. We ran Claude Opus 5.5 with extended reasoning throughout.

For the interface, we added the open-source Taste skill, which steers coding agents away from generic, template-looking screens. It's why the first version already looked like something the team would want to open every morning.

One warning from this part: that loop of questions can go on forever. Claude can always find another question, every answer opens up two more, and it's easy to mistake planning for progress. Many of those questions get answered faster by users than by the agent, so once the plan held together, we stopped refining and built a version.

The stack is simple. The backend runs on Hono, a lightweight web framework, and transcription goes through AssemblyAI. We chose AssemblyAI because it is fast and accurate, and because it separates speakers. A single cold call can have a receptionist, a transfer and an owner on one recording, and every review depends on knowing who said what.

A mock in front of Rohan the same afternoon

We didn't wait for the platform to be finished before showing it to Rohan. About 2 hours after his kickoff videos, I posted 2 short screen recordings of a first version running on sample data, with a calls overview, the main pages and a playbook. Nothing was connected to the dialer yet. It was already about 80% of what the team needed, and that was the reason to show it then, because Rohan could see the missing 20% the moment it was on his screen.

His review came back as a video 21 minutes later. He said he liked seeing what we were building before I built all of it in one shot, and then he listed what was missing.

Playbooks had to be per offer, because the same reps would call for several offers. Cold calling runs in daily cadences, so performance had to be read by the day, not by the call. Practice sessions had to count as a KPI beside live calls. His own comments on a call had to sit above the AI's, with a dismiss button so the reviewer would stop suggesting what he had already rejected.

None of those were bugs in the mock. They were things Rohan knew about running a sales floor, and he could only say them once he saw something to react to. I added the company and offer structure that afternoon. By 5:30 PM, a rebuilt version showed dials, connects, meetings booked and follow ups for each rep, on a calendar by the day.

[VIDEO GOES HERE: clip-first-review.mp4]

Rohan's first review of the mock, 21 minutes after I shared it: think about days, not calls. Cut from an 8-minute recording, with the screen blurred.

The same thinking decided where each feature showed up. Managers spent their day in Slack, so we gave them a Slack bot that lets them send a rep feedback straight from a message. Reps got the simplest view we could make. Some of them were using Slack for the first time, so when Rohan watched the onboarding video, he asked me to hide the bot from reps and record a second, simpler video showing only what they had to do.

The next evening, the first practice sessions showed up on Rohan's screen. His reply was "literally insane", and 36 minutes later he asked me to roll the platform out to the reps, send them their logins and announce it to the team.

That first afternoon also produced our only argument. I had built the backend on Hono, a lightweight framework, because our server was small. Rohan called me, asking why we weren't using Express, the standard choice. I calmed him down by pointing out that the code was the least of our worries: Claude was writing it, and switching frameworks later would take 1 prompt. I promised him we would move to Express once development was done. 3 to 4 weeks later, while writing this post, I realized I never did. It never mattered, and I never had to look at the code. When code costs 1 prompt to rewrite, most technical choices stop deserving an argument.

How Rohan's reviews fine-tuned the product in its first week

The morning after the rollout, Rohan did his reviews on the platform instead of in Slack, and at 7:17 AM he posted 9 short videos about using it.

His message said the platform was already at 7 out of 10, and that these fixes would take it to 9. He also wrote that it was one of the least stressful days he'd had giving feedback, and the most actionable first day.

Most of the fixes were small. Calls had to open in a new tab. The Back button had to keep his place. The call title had to show the caller's name instead of "Unknown", and he needed to change a call's status by hand. In a spec, these read like polish. For someone reviewing 50 calls before the reps start dialing, each one decided whether those calls fit into his morning at all.

One video changed how I thought about the product. Rohan said his Looms would always come before the platform, because the script keeps changing and his feedback changes with it. So we stopped treating his reviews as comments on the platform and started treating them as input to it. A Loom on a call, or on a few days of calls, becomes a list of focus items for that rep, and every new call that rep makes is checked against them.

‍

Rohan, the morning after rollout, on why his Looms would always come before the platform. Screen blurred.

The cycle also broke once, and that taught me the most. Rohan sent a batch of fixes on a Saturday so they would be live by Monday evening, when he next sat down to review. At 8:23 on Tuesday morning, he still couldn't. Back didn't work, the page kept jumping to the top, and there was no pause button while he listened. He had 50 calls waiting, plus about 30 long practice sessions from 2 reps.

The fixes were deployed that afternoon, but I hadn't told him, because I was still testing them. His message said it plainly: "An update on the timeline would have been helpful."

From then on, every big release came with a changelog written in Rohan's terms, listing the changes he would notice in review. The first one opened with calls opening in a new tab, Back keeping your place, Space to play and pause, and every outcome showing its reason in brackets.

A feedback cycle has 2 halves. Taking the feedback in is the half engineers think about. Telling the person what happened to it, on the timeline they need, is the half that keeps them sending it.

The principles behind every feature

Looking back, the code was the smallest part of this project, because Claude wrote almost all of it. The time went into understanding how Rohan coached, what the reps needed between his reviews, and which problems needed a feature at all. 3 principles came out of that work, and they decided more of the product than any framework choice did.

Build for your problem, whatever the category says

Sales CRMs track calls. They have no place for a rep practicing a cold call with an AI the night before a shift, so if we had bought one, practice would have stayed in Slack threads and phone recordings. We didn't care what sales CRMs are for. Our reps were new to cold calling, practice was how they improved between Rohan's reviews, and so practice became a first-class part of the platform.

A practice session is scored against the same script stages as a live call, and the 2 are charted side by side on each rep's page. A rep with no live calls yet gets scores from practice alone, and Rohan set a target of 35 practice sessions a week. His feedback turns into focus items for that rep, and the focus items are written into the practice prompt the rep copies into Claude. A weakness Rohan spots on Monday's calls becomes the scenario that rep practices next.

Take every manual step away from the reps

On the first day, I offered to build drag-and-drop uploads for call recordings. Rohan told me to use the dialer's API instead, because reps would not upload anything by hand, and the same rule decided everything after that. Calls come in from the dialer on their own. Dials that never turned into a conversation still show up, so a rep's effort is visible even on a day with no connects.

Practice went the same way. At first, reps recorded practice sessions themselves and uploaded the files, and one rep's recordings had pauses in them. Those pauses hid how fast the rep answered, and response time is one of the things practice is meant to test. So recording moved into the platform. The rep puts a phone running Claude next to the laptop, presses record, and the session is logged when they finish. We later removed uploads altogether, so practice is always recorded live.

Let the human outrank the AI

Rohan asked for this in his very first review. An AI reviewer that keeps making the same wrong suggestion loses the reps' trust, and once they stop believing it, they stop following any of it. So Rohan can leave a note on the exact line of a transcript where a rep went wrong, or select the exact words. His note shows above the AI review and regrades the call, with the scores, stages and outcome following his judgement.

[VIDEO GOES HERE: clip-manager-feedback.mp4]

Rohan, the morning after rollout, asking again for his feedback to sit on top and regrade the call. Screen blurred.

When the AI flags something he disagrees with, he can dismiss it for that one call, or tell it to stop flagging that point everywhere. The second option adds a rule to the Brain, the part of the platform that keeps the script and his rules for grading against it. Every dismissal teaches the reviewer something, so over time its judgement moves closer to his.

Code bugs are about Choices, not Code

When Claude writes the code, the bugs engineers used to chase mostly go away. Syntax errors, broken loops and off-by-one mistakes are rare. The bugs that remain are choices: a default someone picked, an assumption about how another system behaves, or a limit nobody questioned. We hit 3 of them in the first week of daily use, and no test caught any of them.

The first one started with Rohan's gut.

On a Tuesday evening, he sent me a voice note saying one rep seemed to have made more calls than her page showed, and that a good call he had reviewed the Friday before was gone. He was right. Her page fetched only 25 calls under a label that said "Last 60 days", and she had 31, so 6 calls were hidden, including the one he remembered. I found the cause 38 minutes later. The page size was a sensible default on the day it was written. Once reps were dialing every day, it turned into a decision to hide their work, and nobody had made that decision on purpose.

The second one cost us bookings.

2 days later, at 7:20 in the morning, Rohan asked why some calls weren't in the platform. A rep was saying he had booked meetings, and none of those calls showed up. The dialer marks a call as "Completed", with a length of 0 seconds, while it is still in progress. Our sync waited 3 minutes, saved whatever the dialer reported, and marked the call as seen so it would never fetch it again. Short calls came through fine. Long calls, which are exactly the ones where meetings get booked, were saved as empty records and never updated.

The fix took a few minutes: a call that is still going now waits until it ends before it is saved. Every missing call came back with its recording, and the AI reviews followed within 20 minutes. Rohan wrote back that he could "now understand how an AI Employee can truly do software development." The 3-minute wait was a reasonable choice. The assumption under it, that "Completed" meant the call had ended, was wrong.

The third was a billing choice.

We paid for the AI models with prepaid credits on OpenRouter, and that same evening, they ran out. Reviews failed until I topped the credits up, and the next morning Rohan asked me to rerun every review that had failed. Prepaid credits were a choice too, and we had made it without any alert for when they ran low.

No test would have caught these, because in each case the code did what it was written to do. What catches them is a habit. When an agent writes your code, write down the choices it made for you: every limit, every wait, every status you trust from another system, and every bill that can run out. Then ask the person using the product whether the numbers look right.

Fine-tuning the scores after 10 days of calls

Some fine-tuning only becomes possible after people have used the product for a while. About 10 days into daily use, with every review already running on the platform, Rohan sent me a 3-minute voice note marked "Feature request". The score felt arbitrary to him. He would listen to a call, read his own feedback and look at the score, and the number didn't match what he had heard. Some of the weakest calls scored high, and a rep who was booking meetings scored low. He couldn't say what the grader was doing wrong, only that he couldn't look at a call and agree that it was a 9.

That kind of problem can't show up on day 1. A single score looks fine on its own. The pattern only appears once the same person has reviewed dozens of calls from the same reps and can compare the numbers with what he knows about each of them. No test can make that comparison, because a test doesn't know which rep is good. Rohan knew, because he had listened to every one of their calls and had practiced on the platform himself.

The grading had already changed once. A few days after rollout, we loaded version 5 of Rohan's script into the Brain and regraded every call made since he wrote it, 117 in all, 62 live and 55 practice, in the same afternoon. After the voice note, the rubric itself stayed almost the same. Rohan kept updating the script and I kept loading each version into the Brain. The one change that made a difference was the model doing the judging.

Rohan's guess was that we needed a bigger model. Until then, every call was reviewed by GLM, a cheap open model, to keep the cost of each review low. Claude Opus did the expensive thinking around it, writing the prompts and checking the cheap model's work.

The objection search showed where that split broke. Its first run read 96 calls and found 213 objections in 30 groups, and Rohan asked whether the model was big enough to catch them all. Rerunning it on Opus would cost about $10. His reply came within a minute: run it, because this was the most important step and we would rather not second-guess it.

That changed how I thought about cost. I had been saving money on the step Rohan cared about most. He priced the rerun by what a missed objection would cost the sales team, and $10 was nothing next to that.

The same week, the math changed for every call. Anthropic now gives Claude Max subscribers monthly API credits, and we already paid for Max, so today Opus 5.5 reviews every call out of credits we already had. Rohan has been more satisfied with its judgement since then.

From $2,000 a month to the cost of a small server

On the first day, the cheapest option we priced was $1,500 to $2,000 a month, once we counted seats for 5 callers, the admins and the cold-calling software. That bill would have grown with every rep we hired, and we were hiring.

Today the platform runs on a $12 to $20 a month server at Hetzner, which also runs our screen recorder and several other internal tools. Transcription goes through AssemblyAI, speaker separation included, for about $20 to $30 a month. Reviews run on Claude Opus 5.5, paid for with the monthly API credits that come with Claude Max plans: $100 a month on Max 5x and $200 on Max 20x. We have several Max plans, and all the reviews and grading use at most about $50 of those credits a month. This is a general offer that anyone on those plans gets, and Team plans get a pooled version. The details are in Anthropic's help article on monthly API credits.

The credits come with a catch. They are capped, and when they run out, API calls stop until the next month. We had already seen what a silent stop looks like when our prepaid credits ran out one evening, so we keep our usage well under the cap.

The bigger change is how the cost grows. A per-seat tool costs more with every rep, while our cost grows with the number of calls, and at our volume that is close to nothing. All in, we pay about $32 to $50 a month in cash for the server and transcription, and the reviews come out of Max credits we were already paying for.

What our routine looks like now

A rep's day is now just calls. Every call comes in from the dialer with its recording, transcript and review, so nobody records anything on a phone or posts it in a channel. Reps open the platform once a day, or every couple of days, to read the reviews of their calls, check their focus items and copy their practice prompt into Claude.

Rohan's mornings start from calls that are already reviewed. The AI has graded each one against the script, so his time goes into what only he can do: a comment on the exact sentence, or the exact word, where a rep lost the call. His note regrades the call, and the rep sees it at the top the next time they open it. Calls he hasn't opened show a blue dot, like unread email, so he always knows where he stopped.

His reviews now take 7 to 12 minutes per rep at most, and the quality went up even more than the time went down. With the basics already covered by the AI review, every minute goes into the finer points of a call, and his feedback is sharper now than it ever was when much of his morning went into finding recordings.

The feature that surprised us most was the heat map: each rep's dials and connects, hour by hour. When a rep isn't able to book meetings, the heat map can show why. Their dials might land in hours when nobody picks up, or a lead list might be mostly voicemail. In those cases, the fix is on our side: a better lead list, a different calling window, or a plan for voicemail. We went from coaching one rep at a time to seeing where each step of the funnel leaks, hour by hour.

What I'd tell another forward deployed engineer

If I started this project again tomorrow, these are the 10 rules I would follow.

1. Map business problems to features before you open the editor. Every feature should answer a problem you watched happen.
2. Watch the user work. Their screen recordings, voice notes and messages make a better spec than anything they would write for you.
3. Build for the problem in front of you, whatever your product category says.
4. Take every manual step away from the people using it. A step that depends on someone remembering to do it will be skipped on a busy day.
5. Plan with the agent, but ship a version within hours. An agent can ask questions forever, and users answer many of them faster.
6. Let the human outrank the AI, and let every correction teach it.
7. Treat every bug as a choice. Write down the assumption behind each default, so you know what to revisit.
8. Treat the feedback loop as the work. Ship, watch someone use it, fix what they hit, and tell them what changed in their own terms.
9. Judge the AI only after a week or more of daily use. Before that, there is nothing to calibrate against.
10. Plan your exit from day 3. In his first review, Rohan said the platform would go to the development team once it was stable. Build so that someone else can own it.

Built for one team first

Today the platform is an internal tool. We built it for one sales team: its script, its SOPs, its product knowledge and the way Rohan coaches. That focus is why it worked so quickly, and it is also what no per-seat tool could have given us, because a vendor has to build for every team at once.

It saves us easily 3 to 4 hours a day. The reviews, the reps' follow-up questions, the manual logging and the CRM upkeep no longer land on anyone, and those hours compound into work we never had time for before. We plan to let other teams use it later.

When software can be built in a day, writing the code becomes the small part of the job. The work that is left is what a forward deployed engineer is there for: sitting close to the people with the problem, understanding it well enough to map it to features, and staying in the feedback loop until they trust what you built. The essence of software development is no longer in the code. It is in understanding the problem.