Managers often struggle to review enough calls to properly coach their team. AI Call Scoring solves this by automatically evaluating every call using your chosen framework.
With AI Call Scoring you can:
Assess calls at scale without listening to every recording
Identify coaching opportunities by spotting low-scoring calls
Highlight best-practice calls that can be shared with the team
Track rep performance over time through scoring reports
How AI Call Scoring works
Create a scorecard
Admins or Managers define the criteria that should be evaluated during calls.
AI analyzes the call
Once a call is done, Jiminny analyzes the transcript and evaluates the conversation against the scorecard criteria.
Each section receives a score
Each scoring section receives a score from 1 to 5, based on how well the defined criteria were met.
AI provides explanations and timestamps
Each score includes a short explanation and timestamps linking to the relevant moment in the call, so managers can quickly review the context behind the score.
This allows managers to understand call quality without listening to every conversation.
Setting up AI Call Scoring
Only Admins and Managers can create and manage scorecards.
To create a scorecard:
Go to Organisation Setting > AI Automation > AI Call Scoring
Create a new scorecard - you can choose from a pre-defined template or create one on your own. We advise you to start off with a template that best suits your needs. You can then tweak it based on your specific needs.
Choose when you want to scorecard to be applied - to which team, call type, deal stage or activity type.
Add sections describing what should be evaluated
In each section define the scoring criteria - provide guidance on what a top score looks like for you
Test Your Scorecard
Before rolling out a scorecard across your team:
Test it on a few recent calls
Review the AI explanations and timestamps
Adjust the wording if the scoring results are unclear
Iterating on scorecards helps ensure the AI evaluation matches your team’s expectations.
Viewing call scores
Once a call is ready, the AI scoring results appear in the call page (under the player).
Each scoring section includes:
A score from 1–5
An explanation for the score
Timestamps linking to the relevant part of the call
Scorecard templates
To help teams get started quickly, Jiminny provides templates based on the needs of every team - BDR/SDR, Sales, Customer Success, Implementation.
Once you select the one you prefer, you can customize it to match your team’s process.
Best practices for writing scoring questions
The accuracy of AI Call Scoring depends largely on how the scoring criteria are written. Clear and specific criteria help the AI evaluate calls more reliably.
1. Focus on observable behavior
Write criteria based on what actually happens in the conversation, not subjective judgments.
Good example - Did the salesperson confirm the customer’s timeline for making a decision?
Less effective - Was the salesperson good at understanding the customer?
Observable behaviors are easier for AI to detect reliably.
2. Be specific about what success looks like
Clearly describe what the AI should look for in the call.
Good example - Did the salesperson summarize the next steps and confirm them with the customer?
Less effective - Did the salesperson close the conversation well?
The more concrete the behavior, the easier it is for the AI to evaluate.
3. Break complex goals into separate questions
Avoid combining multiple requirements into one scoring section.
Less effective - Did the salesperson identify the decision maker and discuss budget?
Better approach - Create separate sections:
Did the salesperson identify the decision maker?
Did the salesperson confirm the budget?
This improves scoring accuracy and makes coaching insights clearer.
4. Align scorecards with the call type
Different types of calls require different evaluation criteria.
For example:
Discovery calls may focus on identifying problems and business impact
Demo calls may focus on product positioning and handling objections
Applying the right scorecard to the right call type improves scoring accuracy.
Tip: use clear and structured criteria
AI Call Scoring works best when scoring criteria are written in a clear and structured way.
When creating criteria:
Describe what the rep should do
Specify signals the AI should look for
Avoid vague or abstract wording
Structured criteria help the AI evaluate conversations more consistently and provide clearer scoring results.
How to use AI Call Scoring
Here are some practical ways to use AI Call Scoring in your day-to-day workflow.
1. Identify coaching opportunities quickly
Use Team Insights or OnDemand filters to find calls with the lowest scores.
For example, you can filter calls by:
Activity type
Team or rep
Score range (e.g. calls scoring below 3)
These calls are often the best starting point for coaching conversations. Instead of reviewing random calls, you can focus on the ones where the AI detected gaps in the conversation.
When using Team Insights you will be able to track the progress over time. For example, you might review:
Average scores by rep
Improvements after coaching sessions
Which criteria consistently score lowest
2. Set up alerts for weak calls
You can create Saved Searches and Nudges to automatically surface low-scoring calls.
For example:
Trigger a notification when a call scores below a certain threshold
Monitor specific activity types such as discovery or qualification calls
This allows you to step in early when important conversations did not go well, rather than discovering issues later in the sales process.
3. Build a library of strong calls
AI Call Scoring also helps identify high-performing calls.
You can add these calls to Playlists and organize them by topics such as:
Strong discovery conversations
Excellent objection handling
Effective next-step agreements
These playlists can then be shared with new hires or reps working to improve specific skills.
4. Investigate calls faster
When reviewing a low-scoring call, you can use Ask Jiminny on the call to quickly understand what happened.
For example, you might ask:
“Did the rep confirm the customer’s timeline?”
“Did the customer explain their main challenge?”
“Were next steps clearly agreed?”
This helps you quickly identify the root cause of a low score and prepare more focused coaching feedback.
5. Identify patterns across the team
Over time, AI Call Scoring can highlight patterns across your organization.
Managers can combine scoring data with Ask Jiminny Panorama insights to understand trends such as:
Which parts of the sales process are most often missed
Common objections raised by customers
Topics that frequently appear in successful calls
These insights can help teams refine their playbooks, training materials, and sales process.
FAQ
How are calls scored?
Each section of the scorecard receives a score between 1 and 5, depending on how well the call meets the defined criteria. Then the overall score is the average from the section scores.
What happens if multiple scorecards apply to one call?
This can happen for example when you have set a generic score card and another one that is for a specific activity type. In such cases we will take the scorecard that has the more detailed filters.
Can AI score calls in different languages?
Yes. The transcript can be in any supported language, while the scoring criteria can be written in English or a language that you specify in the prompt.
Can scorecards be updated later?
Yes. Scorecards can be edited at any time to refine the scoring criteria or adapt them to changes in your sales process. Keep in mind this will not update the scores of historical call.