Exam Randomization and Versions for Regulated Aviation Exams
Exam randomization, also spelled exam randomisation, means the system changes what each candidate sees: the order of questions, the order of answer options, or which items are drawn from a bank. Exam versions (parallel forms) are separate, pre-built papers meant to be equally hard. Randomization protects against copying and memorization. Equivalent versions protect fairness. Aviation exams need both.
This guide is for exam officers, quality managers and chief instructors at training organizations that set or deliver aviation theory exams. It covers what the rules we could read say about question selection, which settings are safe, how papers overlap and drift in difficulty (with a worked example), and what to record. Rules are dated as of November 2026, and your competent authority has the final word.
Key takeaways
- Randomization and equivalence are different jobs: shuffling and random draws hide answers, but only blueprint and difficulty control make two papers fair to compare.
- The regulator texts we read set limits on where questions come from and how learning questions are kept apart from exam questions. None of them mandated shuffling option order, so your settings are your own policy.
- In our worked example, one candidate’s pass chance ranged from 42% to 75% across purely random papers, but only 58% to 59% when each paper drew equally from five difficulty tiers.
- Store the items, versions and option order each candidate actually saw. A random seed alone is not enough to rebuild a paper.
What is the difference between exam randomization and exam versions?
Randomization changes the presentation or selection of items per attempt. Exam versions are fixed papers assembled in advance to a blueprint. Randomization reduces answer sharing; versions give you control over content and difficulty. A random draw can create thousands of unique papers, but nobody has reviewed any single one.
The table lists each technique, what it protects, and what can go wrong. Systems often bundle them under one checkbox, but they carry different risks.
| Technique | What changes per candidate | Protects against | What can go wrong |
|---|---|---|---|
| Option order shuffle | Position of A, B, C | Copying a neighbor’s letter pattern | Breaks items that refer to other options or letters |
| Question order shuffle | Sequence of items | Copying by position; sharing “question 7” | Splits scenarios and shared figures from their items |
| Simple random draw | Which items appear | Memorizing a single paper | Uneven topic coverage and uneven difficulty |
| Blueprint-constrained draw | Items, within fixed counts per topic and tier | Memorization, with coverage kept constant | Needs enough tagged items in every cell |
| Fixed parallel forms | Which of several reviewed papers | Exposure of one paper; lets you review every paper | Fewer variants; each form’s exposure grows with use |
What do the aviation rules say about question selection and versions?
The sources we could read regulate where questions come from and how they are protected, and leave shuffling and version counts to the organization. For flight crew exams drawn from the European Central Question Bank, EASA’s rules assign question selection to the competent authority, not the training provider. Check your authority’s current instructions.
The table gives the exact reference and what each source leaves open. We fetched EASA’s Aircrew Easy Access Rules, EUR-Lex, the FAA order and the ITC/ATP guidelines in November 2026. The Part-66 guidance material is the Issue 2 text dated 17 December 2015, so confirm that nothing has changed.
| Source and reference | What it says | What it leaves open |
|---|---|---|
| EASA Part-FCL, ARA.FCL.300(b)(2) | “Questions for an examination shall be selected by the competent authority, according to a common method.” (quoted) | The text we read does not describe the selection method, shuffling or form counts |
| Regulation (EU) 2023/989, Part-66 Appendix II, point 1.14 | “the questions used as part of the MBT learning programme shall not be used in examinations.” (quoted) | Whether and how you randomize; how many papers you hold |
| EASA Part-66 AMC/GM, GM 66.B.200 paragraph 5 | Questions “should be of the multiple choice type with three alternative answers” and “should have only one true answer.” (quoted) | Option order; whether the same item may appear in several papers |
| FAA Order 8080.6H, chapter 3, paragraph 14a | Knowledge test item banks “must be retained within a secure test maintenance and delivery application/network,” separate from test preparation programs. (quoted) | How test forms are generated from the bank; the text we read did not say |
| ITC/ATP Guidelines for Technology-Based Assessment (2022), guideline 2.2 | “If multiple linear forms are being administered, they should be parallel.” (quoted) This is a professional guideline, not an aviation rule. | Aviation-specific thresholds for form equivalence |
Three roles follow. The regulation decides which questions may reach an exam and how learning material is separated. The authority or approved organization decides how papers are built and delivered. You decide presentation settings and, where the authority leaves it open, form strategy. Our Part-66 secure delivery guide covers the first two. No regulator approving a tool or setting is implied.
Which randomization settings are safe for an aviation exam?
Shuffle answer options and stand-alone questions by default, and pin anything whose meaning depends on position. Items that point to a figure, a manual extract, a scenario stem or another option must keep that relationship. A shuffle that breaks an item creates a defect that is invisible until a candidate complains.
Which answer options should stay in a fixed order?
Pin options that carry a natural order (ascending torque values, procedure steps) or that refer to each other (“both of the above”). Better still, rewrite such items.
Should question order change too?
Shuffle order only across independent items, and keep each scenario or shared figure with its questions. The ITC/ATP guidelines say a delivery system “may randomize item ordering and answer choice order to enhance FFT security” (quoted, fixed-form testing background), which is permission, not a requirement. Our exam security settings checklist lists the other settings that sit beside it.
Pro tip: Test the Key After the Shuffle. Generate a paper, shuffle it, and confirm each marked answer still points to the same option text, not the same letter. An answer key tied to letter positions instead of option text would mis-score candidates whenever options move.
How much do two random papers overlap, and does difficulty drift?
With a simple random draw, two candidates share a predictable number of items, and the difficulty of each paper varies by chance. Drawing 40 items from a 120-item bank, two papers share 13.3 items on average, and some candidates get noticeably easier papers. Constraining the draw by topic and difficulty tier removes the drift and leaves the overlap unchanged.
The following is our own illustrative calculation, not data from any exam. The bank has 120 items in five difficulty tiers of 24, with item facility (proportion answering correctly) from 0.50 to 0.92 and a mean of 0.72. Each paper has 40 items and a pass mark of 30 correct (75%), chosen only to make the arithmetic clear. The candidate has a 75% average chance of answering a bank item correctly, so sits near the cut. We simulated 4,000 papers per method.
| Draw method | Average items shared by two papers | Spread of paper mean facility (5th to 95th percentile) | Candidate’s pass chance (5th to 95th percentile) |
|---|---|---|---|
| Simple random draw of 40 | 13.3 | 0.694 to 0.755 | 42% to 75% |
| 8 items from each of 5 tiers | 13.3 | 0.722 to 0.725 | 58% to 59% |
The same candidate, with the same knowledge, passes 42 times in 100 on an unlucky paper and 75 on a lucky one when the draw is only random. That is a fairness problem and an appeal risk. The overlap figure is 40 × 40 ÷ 120, and the chance of two papers sharing 20 or more items is under 1%. Read our item analysis guide for how to estimate facility values from real sittings.
Pro tip: Stratify by Difficulty Tier, Not Only by Topic. Topic counts give you coverage, but without a difficulty tier the draw can still stack hard items on one paper. Tag each item with a tier from live item statistics, and refresh the tiers after each exam window.
When should you use fixed parallel forms instead of a random draw?
Use fixed parallel forms when you need to review every paper before use, when stakes are high, or when the authority or an auditor may ask to see exactly what a candidate sat. Use constrained random draws when the bank is large and tagged.
Guideline 2.6 says “multiple linear fixed forms should be developed when possible and as needed to manage content exposure” (quoted). The comments to guideline 2.2 define parallel forms as “equivalent in psychometric properties, including content coverage, cognitive complexity, and difficulty” (quoted) and note that random forms “may be sufficient for certain low-stakes scenarios (e.g., practice).” For a graded aviation knowledge exam, that points to constrained draws or fixed forms.
Whichever you pick, test equivalence before release:
- Same count of items per syllabus topic and per cognitive level on every form.
- Mean item facility within a narrow band you set in advance, for example two percentage points.
- No item shared across forms beyond a cap you have written down, including across adjacent exam windows, which guideline 2.7’s comments raise for security.
If your pass mark is fixed by regulation or your authority, you cannot lower it to compensate for a hard form, so equivalence must come from how the paper is assembled. Check what applies to your exam.
What should you record so a randomized paper can be rebuilt later?
Record the exact items, item versions, option order and form identifier shown to each candidate, plus the draw-rule version and timestamps. This lets you answer an appeal, scope a leak to the candidates who saw an item, and run item analysis. Retention periods are set by your competent authority, so check them.
| Record field | Why it matters |
|---|---|
| Attempt ID, candidate, exam, date | Ties the paper to one sitting and one identity check |
| Draw-rule version | Shows which blueprint and tier counts applied |
| Item IDs and item versions, in order shown | Rebuilds the paper even after an item is edited |
| Option order per item | Lets a reviewer re-score a disputed answer |
| Form ID (fixed forms) | Scopes a leak or an item error to a form |
| Answers, scores by part, times | Supports appeals and item statistics |
Our post on building an item bank covers item IDs and status fields, which this record depends on.
How do you test a randomization setup before going live?
Generate at least ten papers and inspect them against your blueprint, then confirm scoring, figures and accommodations behave. Repeat after every change to the bank or draw rules.
- Check counts per topic and tier on all ten papers.
- Read three papers end to end, looking for orphaned figures, split scenarios and options that refer to a moved option.
- Re-score a shuffled paper by hand against the key.
- Confirm no learning-programme item appears in the exam pool, which Appendix II point 1.14 forbids for Part-66 module exams.
- Confirm that accommodations such as extra time do not change the paper’s content.
What should you ask a vendor about randomization?
Ask whether the vendor can constrain draws by topic and difficulty, pin options, keep scenarios together, and export what each candidate saw. Treat any answer as a vendor statement to verify in a trial. No regulator approves exam software, so evaluation falls to you.
SimpliAssess’s published features page lists item banking, test management, analytics and reporting, among others, but does not describe randomization or versions, so ask those questions directly. SimpliTrain’s question bank page says it randomizes question selection and answer order for every attempt within admin-set draw rules by topic and difficulty tier, and that question edits are version controlled.
What should you do next?
Pick one live exam and audit it against this page: list which of the five techniques it uses, run the ten-paper rehearsal, and check the record fields for one past sitting.
If your draws are only random, add tier counts first. In the example above, that removed most of the pass-chance spread. Then agree with your quality manager what your authority expects for form counts and records.
To see how a training platform describes draw rules and item versions, read the SimpliTrain question bank page, then test any claim in a trial before you rely on it.
Frequently asked questions
Does EASA require exam randomization?
Not in the texts we read. ARA.FCL.300(b)(2) says the competent authority selects questions for an examination “according to a common method,” but the text we read does not mention shuffling. For exams your own organization runs, randomization is a policy choice unless your competent authority says otherwise, so ask them.
Should I shuffle answer options in a Part-66 style exam?
Usually yes, except for options with a natural order or that refer to other options. The Part-66 guidance material we read says questions should have one true answer among three alternatives. Shuffling is a delivery choice, not a stated rule. Test the answer key after shuffling, because a key tied to positions would mis-score moved options.
Is randomizing questions the same as having multiple exam versions?
No. Randomization changes presentation or selection per attempt, while versions are pre-built papers you can review in advance. A random draw can create papers nobody has checked, so if you randomize, constrain the draw by topic and difficulty to keep every paper on the same blueprint.
How many questions should a bank hold for random draws?
No source we read sets a number. As arithmetic, drawing n items from a bank of N gives two candidates n × n ÷ N shared items on average, so a larger bank reduces overlap. The right size depends on your exposure tolerance, so agree a target with your quality manager.
Can I reuse learning-check questions in the final exam?
For Part-66 module exams, Appendix II point 1.14 says questions used in the learning programme “shall not be used in examinations” (quoted). Testing the same subject is different from reusing identical questions. For other exam types, such as an organization’s own tests, ask your competent authority what applies before you reuse items.
What should I keep to prove which questions a candidate saw?
Keep the attempt ID, the item IDs and versions in the order shown, the option order, the draw-rule version, answers, scores and timestamps. A random seed alone is risky, since software changes can alter what a seed produces. Retention periods are set by your competent authority.
