Building an Item Bank for a Training Company: Tagging, Versioning and Retiring Questions
An item bank is a managed pool of test questions where every question carries tags, a status and a history. To build one, give each item a record with the same fields, set clear rules for when an edit creates a new version, and retire items when review or statistics say they no longer work.
Most training companies start with a folder of quiz questions. That works until a course changes, a client asks for a different exam, or nobody knows which question is the current one. This guide gives you a record template, a versioning rule set and a retirement checklist you can adopt this week.
It covers bank management only. How to write a good question is a separate skill and a separate post.
Key takeaways
- Every item needs the same small set of fields: objective, blueprint area, cognitive level, difficulty, status, version, usage count, last reviewed and source.
- A change to what the item asks or what counts as correct makes a new version. A typo fix does not.
- Statistical cutoffs for retiring items are rules of thumb from teaching and testing guidance, not laws, and they need enough test takers to mean anything.
- Retire items by changing their status and archiving them, never by deleting them.
What is an item bank, and when does a training company need one?
An item bank is a structured collection of test items with tags, statistics and a record of changes, used to build many tests from one pool. A training company needs one once the same questions feed more than one course, client or exam form.
Signs you have outgrown a folder include near-duplicate questions, no way to tell which version a learner saw, and forms that over-test one topic and skip another. A bank makes every question findable, traceable and reviewable.
If you are still deciding what kind of system you need, our overview of what an online assessment platform is explains where item banking sits next to authoring, delivery and reporting.
What fields should every item bank record carry?
Every item record should carry at least nine fields: learning objective, blueprint area, cognitive level, difficulty, status, version, usage count, last reviewed date, and source or reference. Add an item ID as a permanent name. These fields let you build balanced forms and prove where each question came from.
The table below is a starting schema. It is a template built from common item banking practice, so adjust the lists of allowed values to fit your courses.
| Field | What to record | Example value | Why it matters |
|---|---|---|---|
| Item ID | A permanent, unique code that never changes | FS-0412 | Lets you trace one question across forms and versions |
| Learning objective | The exact objective the item measures, copied from the course | FS-3.2: Identify safe food holding temperatures | Links the item to what you taught |
| Blueprint area | The section of the test plan the item fills | Temperature control | Keeps each form balanced across topics |
| Cognitive level | A Bloom’s taxonomy level from a fixed list | Apply | Stops a bank from filling up with recall items |
| Difficulty | An author estimate at first, then the observed p-value once data exists | Est. medium; observed 0.68 | Lets you balance forms and spot items that behave oddly |
| Status | One value from a fixed list | Draft, In review, Pretest, Active, Under review, Retired | Controls which items can appear on live tests |
| Version | A version number plus a link to the previous version | v3 (replaces v2) | Keeps statistics tied to the wording learners saw |
| Usage count | How many times the item was scored on live tests | 214 scored responses | Shows exposure and whether statistics are trustworthy |
| Last reviewed | Date and reviewer of the last content check | 2026-03-14, J. Rivera | Finds items nobody has looked at in a long time |
| Source / reference | The manual, regulation or course page that backs the right answer | Course manual, p. 41 | Makes a challenge from a learner or client quick to settle |
Tying each item to an objective is what makes the bank useful for before-and-after testing. Our guide to pre and post course assessments for instructor-led training shows why both tests need the same objectives at the same difficulty, and an objective tag is how you check that.
How do you keep tags consistent across authors?
Use fixed lists, not free text. Define allowed values for blueprint area, cognitive level and status, and give each value one written definition. A tag that authors can type freely will turn into five spellings of the same idea within a month.
Keep the list short, because a handful of values is easier to apply than thirty. Assign one person to approve new tag values, and write down each tag change so older items can be updated to match.
Pro tip: Tag The Objective, Not The Topic. “Food safety” is a topic and tells you little. “Identify safe food holding temperatures” is an objective, and it tells you whether an item belongs on a form. If you can only fill the topic field, the course objectives probably need tightening first.
When does an edit create a new version of an item?
An edit creates a new version when it could change how a learner answers or what counts as correct. A fix that changes only spelling, punctuation or layout stays on the same version. When in doubt, ask whether a learner who knew the old item could answer the new one differently.
The rules below are our suggested working policy, not an industry standard. They exist so that statistics always describe the exact wording learners saw.
| Type of edit | New version? | What to do |
|---|---|---|
| Change the correct answer or answer key | Yes | Create a new version, retire the old one, reset statistics |
| Change the meaning of the question stem | Yes | Create a new version and relink it to the objective |
| Add, remove or replace an answer option | Yes | Create a new version, because distractor data no longer applies |
| Change the objective, blueprint area or cognitive level | Yes | Create a new version and review the item against the new tag |
| Add, remove or swap a stimulus such as an image, chart or scenario | Yes | Create a new version if the content differs in any way that affects the answer |
| Update the source because the manual or regulation changed | Yes, if the answer changes | If the answer stays the same, log it as a minor fix and update the source field |
| Fix a typo, punctuation or formatting | No | Edit in place and add a note to the change log |
| Fix a broken image link or add alt text | No | Edit in place, confirm the content is identical, log it |
Three habits keep this working. Each new version stores a link to the one it replaces. The old version moves to Retired but stays in the bank. Statistics stay with the version they came from, so a rewritten item starts with a clean record and goes back to Pretest before it counts toward scores.
Who approves a version change matters too. Let authors make minor fixes, but require a second reviewer for any new version. That is the same separation of duties a content team uses when it works from a centralized test content hub.
When should you retire a question from the item bank?
Retire a question when it is wrong, outdated, overexposed, or statistically broken in a way that review cannot fix. Statistics alone should flag an item for review, not retire it. A human checks the answer key and the content first, because a bad statistic often points to a bad key.
The statistical triggers below are rules of thumb. Each comes from published teaching and testing guidance, listed in the source column, and none is a universal standard.
| Trigger | Rule of thumb | Source | First action |
|---|---|---|---|
| Item is too easy or too hard | Difficulty (proportion correct) should fall between about 0.25 and 0.75 for most items | University of Florida teaching module | Flag for review |
| Mastery item looks easy | A difficulty of 0.80 to 1.00 is acceptable for mastery items; 0.30 to 0.70 for items meant to separate learners | University of Arizona College of Medicine, Phoenix | Check the item’s purpose before acting |
| Weak discrimination | The lowest recommended discrimination index is 0.20 | University of Florida teaching module | Flag for review |
| Very weak point-biserial | 0.0 to 0.10 is marginal and should probably be revised or replaced; below 0.0 is a likely key error | Assessment Systems, classical test theory item statistics | Check the answer key first |
| Top group does worse than bottom group | Fewer high scorers than low scorers answering correctly is a red flag for an erroneous item | University of Arizona College of Medicine, Phoenix | Check the answer key first |
Those sources use slightly different ranges, which is normal. Use the stricter reading for items meant to rank learners and the looser one for pass or fail mastery checks. Whichever you choose, write the cutoffs into your bank policy so reviewers apply them the same way.
These numbers also assume enough responses. One testing vendor warns that its ranges “assume a decent sample size” and that with few test takers many items get flagged. A class of twelve will not give stable statistics, so combine data across sessions before you act on it.
Pro tip: Add A Sample Gate To Every Statistical Flag. Only let an item trigger an automatic flag after it has a minimum number of scored responses that your team sets and writes down. Until then, show the statistic as provisional so nobody retires a good item because of one odd class.
What does a retirement checklist look like?
A retirement checklist is a short list of yes or no checks run on any flagged item before its status changes. It separates real problems from noise and records why each decision was made. The list below mixes the sourced triggers above with practice rules we suggest.
- Does the answer key match the source or reference field? If not, fix the key and create a new version.
- Is the content still true? Compare it to the current manual, regulation or course page.
- Do the statistics breach a trigger in the table, and are there enough scored responses to trust them?
- Has the item been seen by more learners than your exposure cap allows? Set the cap yourself, and record it in your policy.
- Has a leak, a learner challenge or a client complaint been reported?
- Does the bank still have enough items in this blueprint area if this one goes?
- Has a reviewer chosen an outcome and recorded the reason and date?
For the last step, one certification testing provider describes subject matter experts marking each item as Retain, Edit, Rework or Delete during a content review, according to PSI’s guidance on test content reviews. The same four outcomes work well for a training company. Change “Delete” to “Retire” so the record stays in your archive.
What should you do next?
Start small. Pick one course, list its current questions, and add the nine fields from the schema table to each one. Mark every item Active, Under review or Retired, and sort the Under review pile first.
Then write a one-page bank policy that holds your allowed tag values, your versioning table, your statistical triggers and your exposure cap. The checklist to kick-start your own assessment platform can help you decide what else the system around the bank needs. For the statistical side, read the Standards for Educational and Psychological Testing from AERA, APA and NCME, which set out professional expectations for testing practice.
If you would rather not run this by hand, the SimpliAssess features page lists, as of October 2026, item banking, tag management and bulk import. If you want to see how assessments fit a wider training operation, you can look at the SimpliTrain product page and request a demo.
Frequently asked questions
What is the difference between an item bank and a question bank?
Many vendors use the two terms to mean the same thing. In this guide, an item bank is a question pool where every record carries tags, a status, a version history and statistics. A plain question bank may be just a list of questions with no tracking behind it.
How many items should an item bank have?
There is no fixed number. Start from your blueprint: count the items one test form needs in each area, then multiply by how many different forms you want to rotate. Add a buffer so you can retire items without leaving a gap in any area.
How often should you review items in the bank?
PSI says comprehensive test reviews should happen at regular intervals but does not give a timeframe. A practical policy is to review an item whenever its course changes and at least once a year. Use the last reviewed field to find items that have gone too long.
Should you delete retired items?
No. Change the status to Retired and archive the record. You keep the history if a learner or client challenges an old result, and you stop anyone from reusing a bad item by accident. Deleting also removes the statistics you may need to explain why it was pulled.
Can you manage an item bank in a spreadsheet?
Yes, for one course with a few authors, using the schema table as your columns. A spreadsheet struggles with locked versions, enforced tag lists and several people editing at once. When those problems start costing you time, that is the point to move to a purpose-built system.
What is a good item difficulty?
It depends on the item’s job. One university teaching module suggests most items fall between 0.25 and 0.75, while another medical school says 0.30 to 0.70 for items meant to discriminate and 0.80 to 1.00 for mastery items. These are rules of thumb, so judge each item against its purpose.
