How to Build a Situational Judgement Test (SJT) Step by Step

To build a situational judgement test, pick one job and its three to five key competencies, collect real critical incidents from experienced staff, and write short scenarios with one-action response options. Then choose a response instruction and format, let a separate expert panel build the scoring key, and pilot the test before you use any score. This guide is for training managers, HR teams and assessment authors who write their own SJTs. It is not a candidate prep page.

Most builder guides give a five-step checklist and stop before the hard parts: choosing a scoring rule and reading pilot data. This guide covers both, with one item scored four ways and a blueprint grid you can copy.

Key takeaways

• Plan with a blueprint grid of competencies and situation types before you write a single scenario.
• For most uses, pair “should do” instructions with a rate-each-option format, because published sources favor that mix for reliability and high-stakes use.
• The same answer earns very different scores under different scoring rules, so test your rule with dummy answer sheets before launch.
• Pilot an overlength form, drop weak items, and check group differences before any hiring use.

What is a situational judgement test, and when should you use one?

A situational judgement test (SJT) presents a realistic work problem and asks the test taker to judge how well each possible response would work. It measures judgment in context, not recall of facts. Use one when the job turns on choosing between imperfect actions, such as handling a complaint. Many US sources spell it “judgment.”

A scenario-based multiple-choice question, covered in our guide to writing scenario-based questions for training courses, usually has one correct answer tied to a taught rule. An SJT usually has several defensible actions, and the key reflects expert agreement about which are better. If your content has one right answer, write the multiple-choice item instead.

The evidence is moderate. A review chapter by Lievens and De Soete (2015) reports meta-analytic validity for job performance of r = .34 (McDaniel and colleagues, 2001) and r = .26 (McDaniel and colleagues, 2007). Christian, Edwards and Bradley’s 2010 meta-analysis of 84 studies found r = .38 for teamwork SJTs and .25 for interpersonal skills SJTs. HumRRO’s summary of a 2020 review by Whetzel, Sullivan and McCloy adds that SJTs add little incremental validity over cognitive ability and personality tests, while Lievens and De Soete report 6 to 7 percent over personality for knowledge-based instructions. Treat an SJT as one input beside other evidence.

How do you plan a situational judgement test before writing scenarios?

Plan with a blueprint grid: list the competencies the job needs, the situation types where each shows up, and how many items each cell gets. Writing scenarios first tends to over-sample whatever was easiest to imagine.

Start from a job analysis: interviews, observation or a review of role documents with people who do the work. Keep to three to five competencies. The grid below is an illustrative example for a customer service team lead. The item counts are examples, not recommendations.

Write extra items beyond the 15 in the grid. HumRRO’s summary advises building overlength forms and scoring only the items that function properly.

How do you gather critical incidents and turn them into scenarios?

Run a short workshop with experienced employees and ask for real situations that were hard, what people did, and how it turned out. These critical incidents become your scenarios. HumRRO’s summary says basing scenarios on critical incidents improves realism, and that scenarios should be brief and specific.

Tiffin and colleagues (2020) describe the traditional route: experts write critical incidents, experts and novices supply possible responses, and a separate group of experts builds the scoring key. Remove names, products and anything sensitive from each incident.

Then edit each incident into four parts: who the test taker is, what has happened, the complication, and the question. TestGorilla’s help center suggests scenarios of 80 words or fewer. That is one vendor’s house guideline, not a standard.

Pro tip: Mine Weak Options From Novices. Ask new hires or recent learners what they would do in each incident, and use their real mistakes as your weaker options. Invented weak options often sound artificial.

How do you write response options that separate good judgment from poor?

A workable default is four options per scenario. Each option is one action the test taker could take, worded in the same style, with a spread from clearly effective to clearly poor. HumRRO’s summary advises one action per option, a distinction between active wrongdoing and passive inaction, and a check for loaded wording.

Here is a template item, using the same customer service lead. We use it again in the scoring section.

Role: You lead a customer service shift.
Situation: A customer has called three times about the same late refund.
The agent on the line says the refund team has not replied.
Your shift ends in 20 minutes.
Question: How effective is each action?
A. Call the refund team yourself, get a date, and tell the customer today.
B. Tell the agent to ask the customer to call back tomorrow.
C. Promise the customer the refund will arrive tomorrow.
D. Email the refund team and tell the customer you have escalated it.

Option B is passive inaction and option C is an unchecked promise. Suppose a panel rates each option from 1 (very ineffective) to 5 (very effective). The illustrative means are A 4.6, D 3.8, B 2.4 and C 1.6.

Which response instruction and format should you choose?

For most training and hiring uses, choose “should do” instructions and ask test takers to rate every option. Published sources agree this mix gives the most scoring information and holds up better when stakes are high. Choose “would do” instructions only when you want typical behavior, which is closer to personality.

Instructions change what you measure. Lievens and De Soete describe “should do” instructions as measures of maximal performance and “would do” instructions as measures of typical performance. Should-do scores correlate more with cognitive ability. Would-do scores correlate more with personality: Agreeableness r = .37 versus .19, Conscientiousness .34 versus .24 and Emotional Stability .35 versus .12.

On faking, HumRRO’s summary says should-do instructions appear to reduce it better. Tiffin and colleagues note that in high-stakes settings candidates tend to adopt a should-do frame whatever you ask. For hiring, expect should-do answers either way.

If you use the single-response format, check reliability in your own pilot rather than trusting either source.

How do you build the scoring key?

Ask a separate group of experts to rate every option independently, then use their mean ratings as the key. HumRRO’s summary recommends 10 to 12 raters with varied perspectives, because outliers skew results with fewer. Tiffin and colleagues note that most developers use 12 to 30.

Rational keys, built from expert judgment, are the most common approach, and HumRRO’s summary says empirical keys, built from candidate response data, perform similarly. For a small pilot sample, a rational key is the realistic start, because an empirical key needs enough respondents to be stable (our editorial judgment).

Check where experts disagree. Our post on case-based soft skills assessment shows how to use the standard deviation of panel ratings to flag options to rewrite.

How does the same answer score under different scoring rules?

The scoring rule changes both the score and what it rewards. The table scores the item above, using its illustrative panel means, for four examinees under four rules. All numbers are illustrative, and the rules are examples we chose, not published standards.

• Best/worst: 1 point for picking the keyed best option (A) and 1 for the keyed worst (C), shown as a percent of 2.
• Ranking: 1 minus the total rank distance from the key order (A, D, B, C), divided by 8, the largest possible total for four options.
• Rating distance: 1 minus the mean gap between the examinee’s ratings and the panel means, divided by 4, the largest possible gap on a 1 to 5 scale.
• Single pick: the picked option’s key mean minus 1, divided by the best key mean minus 1.

Three things stand out. Rating distance compresses scores: Y scores 88.8 against X’s 90.0, while best/worst separates them at 50 and 100. Examinee H earns 72.5 without making a single judgment. And scores from different rules are not interchangeable, so set any pass mark only after piloting. Our guide to setting cut scores for professional training courses covers that step.

Pro tip: Score a Hedger Before Launch. Make two dummy answer sheets, one that rates every option at the scale midpoint and one that always picks the first option, and score both with your rule. If either lands near your planned pass mark, change the rule before any real test taker sees it.

How do you pilot and trim the test?

Build more items than you need, give the overlength form to a pilot group, and keep only items that function. HumRRO’s summary warns that coefficient alpha is not appropriate for multidimensional SJTs. It suggests split-half reliability with a Spearman-Brown correction when content is balanced across the halves.

Do not expect a high alpha. Tiffin and colleagues report a pooled Cronbach’s alpha of about 0.46 for traditional SJTs in one meta-analysis, so balance items across your blueprint cells and do not use the SJT alone. Our guide to item analysis for trainers covers difficulty and discrimination, which you can apply to each SJT item.

The default actions are editorial, not published rules.

How do you use SJT scores fairly, especially for hiring?

Use SJT scores as one input, check score gaps between groups, and keep evidence that the content matches the job. This is general information, not legal advice. Check with counsel before using any test in employment decisions.

HumRRO’s summary reports smaller racial group differences on SJTs than on cognitive ability tests, and slightly higher average scores for women than men. Your instruction and format choices also affect group gaps, as the format table above shows, so read your own pilot data.

In the US, the Uniform Guidelines on Employee Selection Procedures include the four-fifths rule: a selection rate “less than four-fifths (4/5) (or eighty percent)” of another group’s is generally regarded by federal enforcement agencies as evidence of adverse impact. As of November 2026, SHRM reported on October 30, 2026 that the EEOC is pursuing rulemaking to eliminate the guidelines’ interpretive portions. An attorney quoted there said this does not change employers’ Title VII obligations. Confirm the current position with counsel.

What should you do next?

Start small. Pick one role, fill in the blueprint grid, run one critical incident workshop and draft six to eight scenarios. Have ten experts rate the options and score the two dummy answer sheets before you write more.

If you plan to deliver the test online, check what your platform supports before you pick a format. SimpliAssess’s published FAQ lists multiple choice, true/false, fill-in-the-blank, essay and open-ended questions and says question formats can be customized. Its published pages do not describe rating-scale or ranking question types, partial-credit scoring or branching, so you can ask about those in a SimpliAssess demo before you build.

Frequently asked questions

What is a situational judgement test?

A situational judgement test shows realistic work scenarios and asks test takers to judge how effective each possible action would be, or to pick the best and worst. Answers are scored against a key built from expert ratings. It measures judgment in context rather than factual recall, and it is used in hiring and promotion decisions.

How many scenarios should an SJT have?

The sources we read give no fixed number. Build more items than you plan to use, pilot the longer form, and keep the ones that work, as HumRRO’s summary advises. Balance the final set across your blueprint cells so that no competency rests on only one or two items.

Should I use “should do” or “would do” instructions?

Use “should do” when you want to measure judgment about best practice, and for high-stakes decisions, which HumRRO’s summary advises. Use “would do” for typical behavior, which links more closely to personality. In high-stakes settings, candidates tend to answer as if asked what they should do, so expect that frame anyway.

Can I build an SJT with only a few experts?

You can start, but treat the key as provisional. HumRRO’s summary recommends 10 to 12 raters because outliers distort results with fewer. With a smaller panel, check how much experts disagree on each option, rewrite contested ones, and rebuild the key once you have more raters.

Are situational judgement tests fair?

They often show smaller group differences than cognitive ability tests, but not zero. HumRRO’s summary reports smaller racial differences and slightly higher average scores for women. Differences vary with instruction and format, so check your pilot data and take legal advice before using scores in hiring. This is general information only.

Can candidates fake an SJT or be coached for it?

To some degree. HumRRO’s summary says faking can change rank order and hiring outcomes, though SJTs are generally less vulnerable than traditional personality measures. It lists key stretching and within-person standardization as ways to reduce coaching effects. Should-do instructions appear to limit faking, but not reliably in high-stakes settings.