How StoryAudit is validated
For a tool whose whole job is telling you what's actually inside a course, the accuracy of that reading is the entire product. Here's how it's tested, and what it won't guess.
Tested against real courses, not sample data
Every StoryAudit release runs an automated regression suite before it ships. That suite doesn't validate against synthetic or hand-written test files. It runs the actual parsing and analysis code against real .story packages: real slide XML, real trigger logic, real quiz structures, real media and caption files, pulled from actual Storyline courses.
Snapshot as of July 22, 2026. This suite grows with every release; see the changelog for what's shipped since.
When we find a case the parser gets wrong, it becomes permanent
The test suite isn't written once and left alone. When a real course exposes a case StoryAudit handles incorrectly (a trigger structure it misses, a quiz type it doesn't recognize, an edge case in how Storyline encodes something), that exact file, or a version of it, gets fixed and then locked in as a regression test. The bug can't come back silently, because the test that would have caught it the first time now runs on every future release.
That's the actual mechanism behind the parser's accuracy: not a one-time certification, but a growing, permanent library of real, previously-wrong cases that now have to stay right.
Storyline version support
StoryAudit reads the .story project package format used across Storyline 2, Storyline 3, and Storyline 360. Storyline 3 and Storyline 360 share that structure directly (Articulate kept them on the same underlying build lineage until Storyline 3's final update in 2023), and Storyline 2 uses the same package format one generation earlier. The regression suite above runs primarily on Storyline 360-era courses, since that's where day-to-day development happens. Backward compatibility with Storyline 2 and Storyline 3 has been separately confirmed by running real client course files from each (not synthetic files built to match the spec) through the same parser, with correct results. If a file uses a genuinely different package structure StoryAudit can't read, it says so immediately on drop rather than guessing.
What gets checked
The regression suite covers the parts of a course where being wrong actually matters to a reviewer or auditor:
- Quiz answer keys across every supported question type (Multiple Choice, Multiple Response, True/False, Matching Drag-and-Drop, Freeform Drag-and-Drop), including cases where the correct answer changed between two versions of a course.
- Trigger and branching logic: jump targets, variable conditions, and course-completion paths, including a shortest-path walk that catches courses a learner could "complete" without seeing the content.
- Accessibility data: alt text and closed captions, matched against the actual asset a slide uses, not just counted as present or absent.
- Media asset identity: files are matched between course versions by their actual byte content (an MD5 check against Storyline's own declared checksums), not by filename, so a renamed-but-identical file isn't flagged as changed, and a swapped image with the same filename is.
- Course structure: multi-scene slide numbering, slide masters and layouts, course settings, and project-level configuration.
What StoryAudit won't guess
Just as important as what gets checked is what StoryAudit refuses to quietly estimate:
A quiz question type StoryAudit doesn't fully support gets an explicit on-screen notice to verify it by hand in Storyline, rather than a silently wrong or missing answer key entry.
Image content search is disclosed as byte-for-byte matching only. A resized or re-compressed copy of an image won't register as the same file, and the app says so rather than implying broader visual matching it doesn't do.
Course duration is labeled an estimate, because it is one: Storyline doesn't record actual seat time, so any duration figure is a labeled approximation, never presented as measured fact.
The goal isn't a tool that always has an answer. It's a tool that's honest about which answers it's actually sure of.
Questions about a specific check, or want more detail for a technical review? Email support@storyaudit.app. For data handling and infrastructure, see the security overview and data processing addendum.