📊 Instructions for Using the Scorecard in Oversai
The Scorecard is an evaluation tool that allows you to assess agent performance based on a defined score goal.
If the evaluated person meets or exceeds this goal → they are marked as "Passed" ✅
If they don't reach it → they are marked as "Failed" ❌
If the Scorecard has no score goal, the evaluation shows a score but no Passed / Failed result.
🧩 SCORECARD STRUCTURE
1. Sections and questions
The Scorecard is organized into sections.
Each section is broken down into evaluation questions.
2. Section weights
Each section has a percentage weight.
The total of all section weights must add up to 100%.
The weight of each section is defined manually.
3. Questions and scoring
There are two kinds of questions:
Multiple choice → each option has its own points. The question is worth the points of its highest-scoring option, and the agent earns the points of the option selected.
Rating → the question is worth the top of its scale (for example 5 on a 1–5 scale), and the agent earns the rating given.
The maximum score of a section is the sum of the maximum scores of its questions.
4. Question option types
Default → Adds only the indicated points.
Fail section → If selected, the entire section is scored with 0 points, regardless of the other answers.
Fail all → If selected, the overall Scorecard score is automatically set to 0, regardless of the other answers.
N/A (Not Applicable) → An option without points. It excludes the question from the calculation: its points are removed from the total available, and the section score is calculated with the applicable questions only.
🧮 FINAL SCORE CALCULATION
Each section gets a score from 0% to 100%:
Section score = (points earned / points available) × 100
The final score is the sum of each section score multiplied by its weight.
🔹 Example 1: Step-by-step
A section has a maximum of 10 points.
The agent earns 5 points.
Section calculation: (5 / 10) × 100 = 50% 3. If that section is weighted at 40%, then: 50% × 40% = 20 points 4. Repeat the calculation for each section. 5. The final Scorecard score is the sum of all section results.
🔹 Example 2: Step-by-step (with some N/A within a section)
A section has a maximum of 20 points, distributed across 2 questions (10 points each).
One question is marked as N/A, and the other is answered correctly.
Adjusted calculation: - Question 1: N/A → excluded from the calculation - Question 2: 10 / 10 points
Adjusted total possible points: 10 3. Section calculation: (10 / 10) × 100 = 100% 4. If that section is weighted at 25%, then: 100% × 25% = 25 points 5. This means the section contributes 25 points to the final Scorecard score.
Important: When a question is marked as N/A, its points are removed from the total available points, and the section score is recalculated proportionally using only the applicable questions.
🔹 Example 3: Step-by-step (with all questions marked as N/A within a section)
When every question in a section is marked as N/A, there is nothing to evaluate in that section. The section has no score and is left out of the final score: its weight is shared among the other sections, in proportion to their own weights.
A Scorecard has three sections:
Section
Weight
Result
A
40%
5 / 10 points → 50%
B
40%
10 / 10 points → 100%
C
20%
All questions N/A → no score
Section C is left out, so the applicable sections (A and B) now add up to 80% of weight. Each one's weight is scaled up so they add up to 100% again:
Section A: 40% / 80% → counts as 50%
Section B: 40% / 80% → counts as 50%
Section results:
Section A: 50% × 50% = 25 points
Section B: 100% × 50% = 50 points
Final score: 25 + 50 = 75%
Important: A section where every question is N/A does not count as 100% — it does not count at all. The final score reflects only the sections that applied to the interaction.
🔹 Example 4: Fail section and Fail all
Fail section: in Example 3, if a Fail section option had been selected in Section B, Section B would score 0% even with every other answer correct. Final score: 25 + 0 = 25%.
Fail all: if a Fail all option is selected in any section, the final score is 0%, whatever the other answers are.
🔹 When every question is N/A
An evaluation must have at least one question that applies. If every question of every section is marked as N/A, the evaluation cannot be completed — answer at least one question, or skip the interaction.
🔹 Non-scored evaluations
A non-scored evaluation records the answers and feedback, but gives no score to the questions, the sections or the evaluation, and no Passed / Failed result.
🏆 EXAMPLE OUTCOME
Agent's final score: 90%
Defined score goal: 80%
Since the agent meets or exceeds the goal → Result: "Passed" ✅
✨ BENEFITS OF USING SCORECARDS IN OVERSAI
Objective and consistent evaluations.
Flexibility to adapt the structure and weights according to your policies.
Ability to include critical criteria (Fail section, Fail all).
Fair scores when parts of the Scorecard don't apply: N/A questions and sections never inflate or lower the result.
Clear visibility of agent performance with simple, transparent metrics.