EBT grading data: how to turn grades into training needs

EBT grading is the recording of a numeric grade for each pilot competency after an observed assessment in evidence-based training (EBT). The grade is data, not the verdict. EASA’s EBT Manual says grading “serves only documentary and training quality assurance purposes” (quoted). Turning grades into training needs means reading them at three levels: the pilot, the group and the grading itself.

This guide is for heads of training, EBT program managers and quality managers at airline training departments, and for training organizations that prepare EBT instructors. It uses EASA and EU texts and published operator presentations as of November 2026, and it separates what the regulation requires from guidance and from your own decisions. It is not legal or airworthiness advice. Rules differ by authority, so check your competent authority before you change an approved program. FAA programs were not reviewed.

The post adds a triage table for reading grade patterns, a worked example with invented numbers that shows why an average hides a training need, and a trigger template you can adapt. For the definitions of competencies and observable behaviors, see our CBTA guide to competencies and grading.

Key takeaways

• ORO.FC.231 requires a feedback process that measures EBT system performance, a grading system, and regular checks of that system’s accuracy. It does not say how to read grades, so triggers and thresholds are your decisions.
• Read each competency three ways: the individual pilot against your competency standard, the group by fleet or rank, and instructor by instructor. The same low grades can mean a pilot need, a program need or a grading difference.
• An average can hide a training need. In the worked example two competencies share a mean of 3.08, yet 4% of pilots score 2 or below in one and 31% in the other.
• Check instructor concordance before you change training. In one vendor-presented set of 2,662 ratings, 67.7% of instructors identified a training need where the operator standard defined one.
• Set trigger rules and minimum group sizes before you pull data, keep named grades inside a nonjeopardy culture, and keep the behavior-level evidence behind each grade.

What does EASA require an operator to do with EBT grading data?

Regulation (EU) 2020/2036 added ORO.FC.231, which requires an operator to measure and evaluate EBT system performance through a feedback process, to use a grading system for pilot competencies, and to verify that system’s accuracy at regular intervals. It does not say how to read the grades. That reading is guidance and your own design.

The table separates the layers as of November 2026. The regulation wording comes from the EUR-Lex page for Regulation (EU) 2020/2036. The oversight guidance and the EBT Manual are non-binding EASA material.

Two cautions apply. The ORO.FC.231 wording was read on the EUR-Lex page for the 2020 regulation, not in the current consolidated text, so confirm it there before you quote it in a manual. The EBT Manual and the oversight guidance describe themselves as non-binding, so they show good practice and not an obligation.

Which EBT grading data do you need to keep to answer training questions?

Keep four layers, because each answers a different question. The EBT Manual recommends a grade from 1 to 5 for each competency in the evaluation and scenario-based training phases, and an overall competent or not competent result. Behavior-level markers and comments explain why a grade was given. Without them, a grade shows a gap but not its cause.

The table lists the layers. The first, second and fourth rows follow the EBT Manual’s data management section. The behavior-marker row and the task-monitoring label come from one operator’s published slides.

Operator presentations show how the layers are used. Lufthansa’s slides, shown at an EASA EBT webinar in 2021, use behavior markers for root-cause analysis in the debrief and for scenario design. Finnair’s 2022 presentation says company-level and fleet and position-level competency data decide which competencies are highlighted in EBT modules, while individual data drives customized training and possible remedial training.

How do you tell a pilot need from a program need or a grading problem?

Look at where the low grades sit. Low grades for one pilot point to an individual training need. Low grades for one competency across a fleet or rank point to a program need. Low grades concentrated with one instructor point first to a grading difference, so check that before acting on the other two.

The EBT Manual supports the first reading. It calls the evaluation phase “the first look assessment of competencies to identify individual training needs” (quoted), and says areas that do not meet the minimum competency standard “will become the focus of subsequent training” (quoted). The table adds the group and grading readings. It is this post’s triage suggestion, not a regulatory form.

The fourth row follows the EBT Manual’s Section I, 9. It notes that application of procedures and application of knowledge “normally have the lowest grading” (quoted), lists instructors identifying training needs wrongly among the possible causes, and says the EBT manager should decide whether the pattern is genuine.

Why does an average grade hide a training need?

An average squeezes a spread of grades into one number, so two competencies can look equally healthy while one has a long tail of weak performances. In the invented example below, both competencies average 3.08, yet 4% of pilots score 2 or below in one and 31% in the other.

EASA’s oversight guidance makes a related point about instructor concordance data: “A simple average does not provide enough insights in the data to enable a proper analysis” (quoted). The same logic applies to pilot grades. The table uses invented data for 48 pilots in one evaluation cycle. It is not from any operator, and “low” here means grade 2 or below, a choice made for this example.

Both rows read as roughly adequate on average. Only the distribution shows that Competency B deserves a closer look. Report the count at each grade and the share at or below your low-grade line, and give the mean only as a side figure.

Is a low grade a training need or an instructor difference?

It can be either, and the grades alone cannot tell you. Split the low grades by instructor, then test the pattern on shared reference performances. If one instructor accounts for most low grades, treat it as a grading question before it becomes a training question.

The next table splits Competency B from the example by instructor. Each instructor graded eight of the 48 pilots. The data is invented.

Instructor 4 gave 7 of the 15 low grades. That could mean a stricter standard or a harder set of crews, because pilots are rarely assigned to instructors at random. Only shared reference performances, such as recorded sessions that every instructor grades, can separate the two. Even on the other five instructors’ grades the share is 8 of 40, or 20%, so a program signal probably remains.

The EBT Manual says concordance metrics “must drive specific interventions in instructor training” and that the requirements “do not specify statistical thresholds” (both quoted), so you set the target agreement. Our guide to CRM assessment shows a rater check against expert reference ratings, and the guide to examiner standardization covers the wider method.

Why check at all? One data set shows the scale of the issue. As reported by Halldale in November 2025, Frank Steiner of conavitra, which runs the ICAP concordance system, presented 2,662 instructor ratings of standardized videos with predetermined operator standards. Where the standard defined a training need, 67.7% of instructors identified it, so about one in three missed it. Another 28.2% flagged a need for pilots the standard rated as proficient. This is a single vendor-linked data set, so read it as an illustration, not an industry rate.

Pro tip: Read Behavior Markers Before Choosing A Topic. When a competency flags, list the behavior markers marked as not met and count which ones recur. Lufthansa’s slides focus group-level behavior analysis on the “-” markers. A flag on a broad competency becomes a usable scenario or ground-training topic only at the behavior level.

How do you set triggers that turn grade data into decisions?

Write the rules before you look at the data: what counts as a low grade, how many low grades in a group trigger a review, how many ratings you need before reporting a group, and who owns the action. The sources reviewed set none of these numbers, so the figures you choose need a reason recorded in your own documents.

The table is a template. Every trigger figure in it is invented for illustration and must be replaced by values you can defend.

The reason for the last row is practical. A cell of three pilots is noisy and can identify individuals. It also keeps group reports within the de-identified use described in the next section.

Pro tip: Test Your Triggers On Past Cycles. Before you adopt a trigger, run it on your last two cycles of grades and count the alerts it would have raised. A trigger that fires for every group gets ignored, and one that never fires is not doing a job.

Who should see individual EBT grades?

Fewer people than want them. The EBT Manual says “a strong nonjeopardy culture must be implemented” for data reliability (quoted), and that grading serves only documentary and training quality assurance purposes. Use de-identified data for group trends, and keep named data for the pilot, the instructors and the standardization lead.

That split is a suggestion based on one operator’s practice. Lufthansa’s slides pair de-identified data for group trends with personalized data for a pilot’s own history, early recognition of performance drops and inter-rater reliability, and they state that disciplinary superiors have no access. That is a company rule, not a regulatory one.

Authorities have a defined interest. EASA’s AMC and GM to Part-ARO say the authority may access pilot grading results to verify “the suitability of the operator’s grading and assessment scheme” (quoted). Grades tied to a named pilot are personal data in most settings, so involve your data protection lead before you design reports. This is not legal advice.

What can written exam data add to EBT grading data?

Written and computer-based exams add item-level knowledge evidence that observed grades cannot give. Tag each question to the same competency or topic keys your EBT program uses, and a weak application-of-knowledge grade can be checked against what the knowledge exam shows. EBT grades still come from observed sessions, not from exams.

Our guide to aviation exam analytics covers topic-level and item-level analysis for a cohort. The SimpliAssess features page lists Item Banking, Tag Management and Analytics and Reporting as feature labels without descriptions, and its published pages do not describe EBT grading, rubric scoring or observation capture. Ask any vendor, SimpliAssess included, to show how exam results and observed grades would be joined before you plan around it.

What should you do next?

Pick one competency and your last two EBT cycles. Build the distribution table by fleet or rank and by instructor, using counts at each grade instead of a mean, and mark any cell too small to report.

Then have your instructors grade the same recorded performance, compare the results, and decide whether the pattern is a pilot, program or grading need. Write your triggers down, and make sure the feedback process described in your management system matches what you actually do. Ask your competent authority if you are unsure what it expects to see.

SimpliTrain is a separate training platform from the same company. Its aviation industry page lists competency based training model paths and training analytics, but it does not mention EBT or explain how grades are captured, so ask for a walkthrough of grade capture and reporting before you assume it fits.

Frequently asked questions

What is EBT grading?

EBT grading is the recording of a grade for each pilot competency after an observed assessment in evidence-based training. EASA’s EBT Manual recommends a grade from 1 to 5 per competency for the evaluation and scenario-based training phases. Grading is not the competency assessment itself. It documents performance and supports training quality assurance.

Is there a minimum passing grade in EBT?

The regulation names no single grade. ORO.FC.231(d)(1) requires a performance criterion and scale for each competency, with a minimum acceptable level the operator defines. In EASA’s five-point table, grade 2 is described as “minimally acceptable.” Your own competency standard decides which grade counts as meeting it, so write it into your documents.

How often must an operator verify its grading system?

ORO.FC.231(d)(2) says the operator must verify the accuracy of the grading system “at regular intervals” against a criterion-referenced system. The text read for this post gives no number of months. Choose an interval you can defend, such as after each instructor intake or program change, and record the method and result.

What is instructor concordance in EBT?

Instructor concordance is how closely instructors give similar grades to the same performance. ORO.FC.231(a)(4) requires an EBT instructor standardisation and concordance assurance programme, and the operator must demonstrate sufficient concordance. EASA’s manual lists agreement statistics such as Cohen’s kappa but sets no threshold, so the operator sets its own minimum.

Can you use average grades to track EBT performance?

You can report them, but not alone. EASA’s guidance lists numerical competency grades as data for statistical trend analysis, yet an average can hide a tail of weak performances, as the example above shows. Report the count at each grade and the share at or below your low-grade line, split by fleet, rank and instructor.

Who is allowed to see individual EBT grades?

The sources reviewed do not list who may see named grades, except EASA’s Part-ARO guidance that the authority may access pilot grading results to verify the operator’s grading scheme. Operators set internal access rules, and one airline’s published slides keep disciplinary superiors out of the data. Ask your data protection lead. This is not legal advice.