CRM training assessment: measuring non-technical skills

CRM assessment in aviation is the observation of how crews use non-technical skills, such as communication, leadership, situation awareness and decision-making, followed by a debrief. EASA’s guidance describes it as observing, recording, interpreting and debriefing crew performance “using an accepted methodology.” A written test can show what crew members know about crew resource management (CRM). Only observed performance shows what they do.

This guide is for heads of training, chief instructors, CRM facilitators and quality managers at airline training departments and approved training organizations, and for small companies that sell CRM courses. It uses EASA and FAA texts as of October 2026 and separates requirements, guidance and your own decisions. It is not legal or airworthiness advice, so check your competent authority before you change an approved course.

The post adds a map of the sources, a split of what a test and an observer can each show, a rating sheet design and a worked check of a rater against expert reference ratings. For the wider method, see our CBTA guide to competencies and grading.

Key takeaways

• EASA’s guidance places CRM skill assessment in the operational environment and limits it to observable behaviors, while the FAA’s advisory circular is voluntary and says its behavioral markers are not a checklist for evaluating individual crew members.
• A computer-based test can check CRM knowledge and judgment in a described situation, but speaking up, sharing a plan and managing workload must be observed in a crew.
• NOTECHS has four categories, the operator chooses the rating scale, and a good rating sheet has one behavior per line and a separate “not observed” option.
• Check each rater against expert reference ratings and look at the direction of errors, not only the overall agreement figure.
• The texts reviewed set no CRM pass mark and no agreement threshold, so those are your decisions inside the method your authority accepts.

What do EASA and the FAA say about CRM assessment in aviation?

EASA’s guidance says CRM skills should be assessed in the operational environment, on observable behaviors only, and then debriefed, using a method the operator describes in its operations manual. The FAA’s circular is voluntary guidance that stresses assessing the training program and calibrating evaluators. The texts reviewed set no pass mark for CRM.

The table lists each source, what it says and what kind of text it is, as of October 2026.

Two cautions apply. The wording of paragraph (h), GM6 and the line-check point comes from EASA’s ED Decision 2015/022/R. EASA has amended the Part-ORO AMC and GM since, including in Amendment 17, and its Easy Access Rules for Air Operations stands at Revision 24 (March 2026). The current consolidated text of those paragraphs could not be opened for this post, so confirm it before you quote it in a manual.

Also, the assessment methods in these sources describe flight crew. Cabin crew and dispatcher rules, including EASA’s cabin crew provisions, were not reviewed here.

Do EASA and FAA texts agree on rating individual crew members?

Not fully. EASA’s guidance names NOTECHS as a validated method for assessing individual flight crew CRM skills. The FAA’s circular says its behavioral markers are not presented as a checklist for evaluating individual crew members. Both can hold, because the texts answer different questions for different readers.

EASA’s guidance is written for operators who must describe their method and standards in the operations manual before assessment starts. The FAA circular says it presents “one way, but not necessarily the only way” to address CRM training (quoted), and that compliance is not mandatory.

We suggest this resolution. Under an EASA operator or approved organization, rate individuals with the method written into your approved procedures. For a US-focused CRM course, use the FAA markers for feedback and program evaluation, and rate individuals only against a standard the certificate holder or client has approved.

What does NOTECHS measure, and how do you write the behaviors?

NOTECHS rates four categories: cooperation, leadership and managerial skills, situation awareness and decision-making. Each category has elements, and EASA’s guidance lists example behaviors for each. You write the exact behaviors, and the guidance leaves the scale to the operator.

The table shows the categories and elements in GM6 ORO.FC.115, with one example marker in EASA’s wording for each category.

If you run evidence-based training (EBT), the guidance advises using the EBT competencies and grading system instead of a separate scheme.

Write each marker as one behavior with an observable verb. A 1998 UK CAA report (CAA Paper 98005) cites researchers who recommend one behavior per marker, with the example that “made a decision” cannot be seen, while “communicates a decision” can. If a line holds two behaviors, such as “briefs the plan and checks understanding,” split it, or raters will score different halves.

What can an online test show about CRM, and what needs an observer?

A computer-based test can show whether crew members know CRM concepts and can pick a sound action in a described situation. It cannot show whether they speak up, share a plan or manage workload in a real crew. EASA’s AMC says computer-based CRM training should not stand alone, and its assessment paragraph points to the operational environment.

That AMC sentence is about training, not testing, so it is context rather than a ban on CRM knowledge checks. The table splits the evidence by NOTECHS category and is this post’s suggestion, not a requirement.

The item below is original, written for this post. It is not from any exam or question bank and does not represent an official test.

Stem: On a briefed approach, the first officer sees that the aircraft is above the planned flight path. The captain, who is flying, has not commented. Which action best fits your operator’s CRM standard?

• A. Wait, because the captain is in command and is probably aware.
• B. State what you see in plain words and ask what the captain intends. (Keyed answer)
• C. Take the controls without saying anything.
• D. Raise it at the debrief.

The keyed answer depends on your standard operating procedures and callouts, not on this post. The item shows that a learner knows the preferred response, not that the learner would say it to a tired captain at 3 a.m. For writing items of this kind, see our guide to case-based assessment for soft skills and our overview of online assessment in aviation training.

How do you design a CRM rating sheet that raters can use?

Build the sheet from the standards in your operations manual: one observable behavior per line, four rating options, and a space for what was said or done. Raters record evidence first and rate second. Add fields for unsafe and positive behavior, since EASA’s guidance covers both.

The four options are seen, partly seen, not seen and not observed. “Not seen” means the scenario gave a chance and the behavior did not happen. “Not observed” means the scenario gave no chance. The table shows three invented rows.

The separate “not observed” option matters. In an evaluation of the NOTECHS method by O’Connor and colleagues (2002), raters tended to score behaviors that the experts judged were not present, instead of using that option.

Pro tip: Plan an Inject for Each Marker. Before the session, list the scenario event that gives each marker a chance to appear, such as a late runway change or a conflicting instruction. A marker with no inject will come back “not observed” for every crew, and you will have nothing to grade.

How do you train and check the people who do CRM assessment in aviation?

Train raters before live use, then check each one against expert reference ratings. The FAA’s circular says instructors, supervisors and check pilots need special training to calibrate and standardize their own skills. EASA’s guidance expects examiners to be trained in CRM concepts and in assessing CRM skills.

The NOTECHS evaluation shows why. Its raters had about three hours of training, which the authors judged insufficient for ambiguous scenarios. Category-level agreement ran roughly from 0.64 to 0.87 by scenario, and pass/fail decisions matched the expert reference about 83 to 84 percent of the time. Situation awareness and decision-making were the hardest categories to rate. The study is from 2002, so use it as a warning, not a benchmark.

EASA’s EBT Manual (version 2.4, 3Q2026) lists agreement statistics for instructors and says the requirements specify no statistical thresholds, so the operator sets its own minimum. For the percent agreement and kappa method, see our competency checklist guide.

What does a rater check against reference ratings look like?

Have an expert panel agree a reference rating for each marker on recorded crew performances, then have the rater score the same recordings alone. The invented example uses 8 recordings with 5 markers each, so 40 judgments. The panel found the behavior present (seen or partly seen) in 24 and absent in 16.

Overall agreement is (21 + 10) / 40, or 77.5 percent, and Cohen’s kappa is about 0.52. Both numbers hide the pattern. The rater missed 3 of 24 present behaviors (12.5 percent) but credited 6 of 16 absent behaviors (37.5 percent). This rater leans generous, which is the same direction the NOTECHS study found.

The fix is targeted. Add recordings where a behavior is clearly missing, review the six credited cases with the panel, and re-rate later. Set your own target for false credit, since no text reviewed gives one.

Pro tip: Report Direction, Not Only Agreement. Track false credit and misses separately for each rater and each category. Two raters with the same 77.5 percent can need opposite coaching.

How should you record and use CRM assessment results?

Use CRM assessment results to give feedback to crews, to decide on additional training and to improve the CRM training system. EASA’s guidance describes de-identified summaries for that last use. Keep the raw evidence notes with each rating.

Build in three limits. EASA’s guidance says CRM assessment alone should not fail a line check, unless the behavior could unacceptably reduce the safety margin. EASA’s EBT Manual says grading serves documentary and quality assurance purposes and should be secondary for the trainee, and recommends a competent or not competent result for the overall session. And your operations manual should describe the methodology, standards and terms before assessment starts.

Store the marker, rating, evidence note, scenario version, rater, date and any follow-up training. Keep knowledge test scores apart from observed ratings, so a high test score never hides a low observed one.

What should you do next?

Pick one course and one marker set, either the four NOTECHS categories or your EBT competencies. Write eight to twelve markers with one behavior each, build the rating sheet with a “not observed” option, and describe it in your operations manual or course specification. Ask your competent authority whether that is what it expects.

Then run the calibration check before the first live cohort, report false credit and misses for each rater, retrain where needed and repeat.

On tools, the SimpliAssess features page lists item banking, test management, tag management and analytics and reporting, which suit the knowledge side. Its published pages do not describe observation rating sheets or rater workflows, so keep observed ratings in a controlled form. SimpliTrain is a separate training platform from the same company. Its aviation page lists “CRM refreshers” among the recurrent training items it tracks, but does not describe how CRM observations are rated or stored, so ask for a walkthrough of that first.

Frequently asked questions

What is CRM assessment in aviation?

CRM assessment is the observation, recording and debriefing of how crews use non-technical skills, using a method the operator has defined. EASA’s guidance limits it to observable behaviors and places it in the operational environment. A knowledge test can show what crew members know about CRM, but not what they do.

Is CRM assessment pass or fail?

The texts reviewed set no CRM pass mark. EASA’s guidance says CRM assessment alone should not fail a line check unless the behavior could unacceptably reduce the safety margin. The FAA’s circular uses no pass or fail terms. Your operator or course owner sets the standard, so write it into your assessment guide and check your authority.

What is NOTECHS in CRM assessment?

NOTECHS is a method for rating non-technical skills. EASA’s guidance names it as a validated method for assessing flight crew CRM skills and groups behaviors into four categories: cooperation, leadership and managerial skills, situation awareness and decision-making. The operator chooses the rating scale. Operators using EBT are advised to use the EBT competencies and grading instead.

Can you assess CRM with an online test?

Only in part. An online test or scenario item can check knowledge of CRM concepts and the choice of a sound action. Behaviors such as speaking up or sharing a plan must be observed in a crew, in the operational environment under EASA’s guidance.

How much training do CRM raters need?

The texts reviewed give no number of hours. The FAA’s circular calls for special training so that evaluators calibrate and standardize their own skills. In the 2002 NOTECHS evaluation, about three hours was judged insufficient for ambiguous scenarios. Check each rater against expert reference ratings.

What is the difference between CRM training and CRM assessment?

CRM training teaches the concepts and practices, and EASA’s guidance allows classroom and computer-based methods plus the operational environment. CRM assessment observes and rates performance, and the same guidance places it in the operational environment, not during non-operational CRM training. Trainers may still give feedback in class.