How to Set Pass Marks (Cut Scores) for Professional Training Courses

A cut score is the minimum test score that separates pass from fail. For a professional training course, it should come from a documented panel judgment, not from a default of 70% or 80%. The most practical method for a small provider is a modified Angoff study, in which subject matter experts rate each item and the ratings are averaged into a pass mark.

This guide includes a worked example with ten invented items and six invented judges, with the arithmetic checked in Python. It also shows what measurement error does to a pass mark and gives a table for choosing between six standard-setting methods.

It is written for course owners and training providers, not for large licensure programs, which need a psychometrician.

Key takeaways

• A cut score is a judgment, not a measurement. ETS says it is impossible to prove a cut score correct, so the defense is a documented, sensible process.
• For a new course test with no learner data, a modified Angoff panel is the most practical method. Contrasting groups and Bookmark need far more learners than most single courses have.
• In the worked example, six judges and ten items gave a cut score of 7.04 items (70.4%) with a standard error of 0.26 items. The test’s own measurement error, 1.3 items, was about five times larger.
• The Standards for Educational and Psychological Testing ask for clear documentation of the rationale and procedures behind a cut score, so keep a record of the panel, the ratings and the result.

What is a cut score, and how does it relate to a pass mark?

A cut score is a chosen point on a test’s score scale that decides whether a performance is good enough for a stated purpose. In a training course, the pass mark is that point as learners see it, such as 7 of 10 items or 70%. Most people use the two terms interchangeably.

The number itself cannot be proven right. The ETS Cut Scores Primer says no purely objective method exists and that cut scores reflect the combined judgments of the people involved. What you can show is that the process was sensible.

That is what the Standards for Educational and Psychological Testing (AERA, APA and NCME, 2014) ask for. Standard 5.21 says the rationale and procedures used to establish cut scores should be documented clearly. Standard 5.22 asks that judgment-based procedures let the judges apply their knowledge and experience in a reasonable way.

Why is a flat 70% or 80% pass mark hard to defend?

A flat percentage ignores how hard the test is. Seven items correct on an easy test and seven correct on a hard test mean different things. A default is defensible only when you can show where the number came from, such as a regulator, an accreditor or a client contract.

The choice matters. In the invented 24-learner cohort used later in this guide, moving the pass mark from 7 to 8 items out of 10 drops the number of passing learners from 17 to 12.

If an outside body sets the number, record who set it, where it is written and when you last checked it.

Which cut score method should a small training provider use?

For a new course test with a panel of five or more subject matter experts and no learner data, use a modified Angoff study. A fixed percentage fits only when someone else sets the number. The other methods need learner scores, and the last two need more learners than most single courses have.

The table compares six methods by what each needs and when each is defensible for a small provider. The requirements come from the ETS primer. The fit column is a judgment for providers with small cohorts.

How does a modified Angoff study work, step by step?

In a modified Angoff study, each expert estimates, item by item, the percentage of borderline learners who would answer correctly. The estimates are averaged for each item and summed across items to give the cut score. Discussion and, when available, learner data help the experts calibrate.

Step 1: Describe the borderline learner. Write a short description of a learner with just enough knowledge and skill to pass, tied to the course objectives. ETS calls this person the borderline test taker. Agree on the description before anyone sees the items.

Step 2: Pick the panel. Choose experts who know the content and the learners. ETS says research suggests 10 to 15 judges. A small provider may manage five or six, which limits precision, as the example below shows.

Step 3: Rate the first items together. ETS recommends that judges make preliminary judgments on the first few items, discuss them, then judge the rest.

Step 4: Rate every item independently. Each judge gives the percentage of 100 borderline learners who would answer the item correctly. ETS suggests this framing for judges who dislike probabilities. For multiple-choice items, ETS says the estimate should ordinarily be at least the chance of guessing, which is 25% with four options.

Step 5: Discuss and revise. Show where judges disagree. Spend any second round on data, such as item difficulty and the pass rate the cut would give. A 2016 simulation by Shulruf and colleagues found a second round without extra test data was redundant.

Step 6: Average and sum. Average the ratings for each item, add the item averages, and convert the total to items and a percentage.

Pro tip: Anchor Judges on Real Learners. Before rating, ask each judge to jot down initials for two or three past learners they would have passed only just. Rating each item with those people in mind keeps the discussion on the borderline learner instead of the best learner.

What does a worked modified Angoff example look like?

Six judges rated ten items. Averaging the item ratings and summing them gave a cut score of 7.04 out of 10 items, or 70.4%. The course sets its pass mark at 7 of 10. All numbers here are invented to show the arithmetic, not to suggest what any real course should require.

The table shows each judge’s rating, as the percentage of 100 borderline learners expected to answer the item correctly, and the average for each item.

The ten item means add up to about 704 percentage points. Divide by 100 and a borderline learner is expected to score 7.04 items, or 70.4%. A learner cannot score 7.04, so the course rounds to the nearest whole item and sets the pass mark at 7 of 10.

Judge A is the strictest rater (6.15 items) and Judge D the most lenient (8.05). Averaging is meant to cancel out that stringency. Items 5 and 9 had the widest spread, 25 points from lowest to highest, so discuss them first.

How precise is a panel’s cut score?

The standard error of the cut score shows how much the result would move with another panel of the same kind. It is the standard deviation of the judges’ totals divided by the square root of the panel size. Here it is 0.64 divided by 2.45, or 0.26 items.

As a rough guide, read the result as “about 7 items, give or take half an item.” With the same spread among judges, the standard error would be 0.37 items for three judges, 0.20 for ten and 0.16 for fifteen.

Published advice on panel size varies. ETS says research suggests 10 to 15 judges. A guide from the Royal Australasian College of Physicians’ rehabilitation medicine faculty sets a minimum of five panel members for its exams. A 2016 simulation study by Shulruf and colleagues suggested about 15 judges for tests of 80 items or more, no fewer than 20 for shorter tests and about 30 for a 10-item test. The authors recommend confirming this in practice.

Six judges on ten items is therefore thin by research standards. Say so in your documentation and check the result after the first cohort.

What is the standard error of measurement, and should it change the pass mark?

The standard error of measurement (SEM) estimates how far a learner’s observed score can differ from their true standing by chance. The RACP guide gives it as the standard deviation of total scores times the square root of one minus reliability. Standard 2.14 asks that SEMs be reported near each cut score.

The example uses an invented cohort of 24 learners. Their scores averaged 7.1 items with a standard deviation of 2.17, and the KR-20 reliability estimate was 0.65. The SEM is 2.17 times the square root of 0.35, or 1.28 items. That is 12.8 percentage points on a ten-item test.

A learner who scored 7 therefore has a band of roughly 5.7 to 8.3 items. A borderline learner, by definition near the cut score, can pass one day and fail the next. A 7 and a 6 on this test do not reliably tell you which learner is stronger.

The RACP guide notes that setting the pass mark one SEM below the standard reduces false negatives, and one SEM above reduces false positives. The table shows what each choice does to the invented cohort.

Moving up to 8 fails five more learners, and moving down to 6 passes two more. Where a false pass is costly, such as a safety-critical skill, a provider may choose the higher mark. That is a policy decision, not a statistical one, so record who made it and why.

In most courses, the better fix is a longer test. Doubling to 20 items that behave like the existing ten would cut the SEM from about 12.8 to about 9.0 percentage points. That is my calculation under classical test theory, and it assumes the new items match the old ones.

Pro tip: Check Impact, Then Leave the Number Alone. Before announcing a pass mark, apply it to your last cohort’s scores. If the pass rate surprises you, review the items and the borderline description first. For credentialing tests, Standard 11.16 says the passing level should not be adjusted to control how many people pass.

What should you document so the cut score holds up?

Document enough that someone could reconstruct the study after everyone has forgotten the details. ETS advises treating the study as if someone will later have to reconstruct and defend it. For a training course, that means a short record kept with the assessment.

Keep these items in that record:

• The purpose of the test and what a pass is meant to show
• The borderline-learner description
• The panelists, their expertise and any training they received
• The method and the number of rounds
• Every rating, by item, judge and round
• The cut score, its standard error and the SEM
• Any adjustment, who approved it and why
• The date and what will trigger a review

When an item changes, its rating is stale. Keeping each rating with the item record, as described in building an item bank for a training company, lets you re-rate only what changed. After the first cohort, item analysis for trainers shows whether an item the panel rated easy performed hard.

When is a cut score study not enough for a training provider?

A panel study like this one is enough for a course-level pass mark, but not for high-stakes credentialing. Licensure and certification programs use larger panels, psychometric support and independent governance. If learners will use your pass mark to claim a certification, read the credential guidance first.

The post on certificate of completion vs certification explains what evidence each credential needs. If you plan to sell an exam, selling certification exams as a training provider covers the launch checklist and the conflict-of-interest question.

What should you do next?

Pick one course assessment and run a small study. Write the borderline-learner description, recruit at least five subject matter experts, collect ratings independently and record the result. Then compare the pass mark with your next cohort’s scores, and revisit it whenever the items change.

If your items live in an assessment platform, check that it keeps item-level data and results for each assessment. The SimpliAssess features page lists item banking and analytics and reporting among its capabilities.

If you also deliver the course itself, SimpliTrain is the training platform from the same company. Its page for training companies lists assessments and exams, question banks, and certifications and grading in its navigation.

Frequently asked questions

What is a good passing score for a training course?

There is no universal number. A good passing score is one you can justify from the stakes, the content and a documented method. Round numbers like 70% or 80% are easy to pick, but a panel study may land higher or lower on the same test. For compliance training, check whether a regulator or client already sets the figure.

How many judges do you need for an Angoff panel?

The ETS primer says research suggests 10 to 15 judges. One medical college guide sets five as its minimum. A 2016 simulation study suggested about 15 judges for tests of 80 items or more, and more for short tests. A small provider can run five or six judges, but should document the limit.

What is the difference between a cut score and a pass mark?

In everyday use, nothing. Both mean the score that separates pass from fail. Strictly, the cut score is the point a standard-setting method produces, and the pass mark is the rule learners see. The two can differ if policymakers adjust the recommended number, which is why any adjustment should be documented.

What is a borderline learner?

A borderline learner has knowledge and skills just on the line between passing and failing. ETS uses the term borderline test taker, and many guides say minimally competent. Judges rate each item for this person, not for the average or best learner, so write the description and agree on it before rating.

Can you set a cut score without a panel of experts?

Yes, but it is weaker. A fixed percentage required by a regulator or client is defensible if you document its source. A single instructor’s judgment is a starting point, not a standard. Add at least a second expert, write down the borderline-learner description and review the result against your first cohort’s scores.

Should you lower the cut score if too many learners fail?

Not just to change the pass rate. First check the items for errors, the borderline description and whether the course prepared learners for the test. If the study was sound, a low pass rate may be a teaching result. The Standards say credentialing pass levels should not be adjusted to control pass numbers.