Tobacco packaging sold in Canada must satisfy over a hundred distinct regulatory requirements — where specific text sits, how far elements are from package edges and from each other, and what colours warning components must be. Inspectors verified all of it by hand, taking up to three hours per package.
We won a competitive RFP through the AI Source List to build a production assistive system that performs a first pass over inspection photographs and returns prioritized findings, so inspectors examine the areas most likely to be non-compliant first. Review time fell roughly tenfold, to around twenty minutes per package.
The system does not determine compliance. Inspectors do.
How this work was awarded
This engagement came through open federal procurement, and the record is public.
We were qualified on Canada’s AI Source List — the pre-vetted supplier list departments draw from for artificial intelligence work — and won the contract through competitive RFP, as prime contractor rather than a subcontractor to a larger integrator. The deliverable was a production system used by federal inspectors in their statutory work, not a pilot or a proof of concept.
The department subsequently awarded further work under the same solicitation.
Both records are published by Public Services and Procurement Canada and can be verified independently:
The problem
Verifying a tobacco package against the regulations means checking each element’s position, its distance from edges and neighbours to millimetre tolerances, and the colour values of warning components — across both the English and French faces, and across more than a hundred separate requirements.
Done manually, this took up to three hours per package. It was also unevenly applied: inspection photographs varied widely in quality and framing because inspectors photographed packages differently, which made consistent measurement difficult before any analysis began.
The requirement was a first-pass system precise enough to be useful at millimetre tolerances and colour-accurate enough to be trusted, that presented findings clearly enough for inspectors to act on quickly — while leaving the compliance determination entirely with them.
What we built
Normalizing the photograph. Inspector photographs arrive with varying rotation, skew, framing, glare, and illumination, sometimes with similarly coloured objects behind the package. A custom-trained YOLO-based model locates the package corners; classical computer vision techniques — edge and contour detection, perspective warp, affine transformation — refine that into a tightly cropped, axis-aligned, background-removed image of each package face. Accompanying reference photographs receive identical treatment so measurements can be related to real-world units.
We addressed input quality from both ends: photo-taking guides to improve consistency at capture, and training data that deliberately included degraded real-world cases — glare, poor illumination, heavy skew, cluttered backgrounds — so the models handled what inspectors actually submit rather than what we wished they submitted.
Segmenting the package face. Each face divides into its defined regulatory components, whose relative positions and sizes are specified. The system locates them, verifies their arrangement and proportions fall within expected bounds, and confirms warning imagery matches the pre-approved set. Where a tax stamp is present it is redacted before any downstream processing, so its content never enters the analysis.
Reading and measuring. Text areas are read using detection and recognition models (DBNet and RobustScanner) fine-tuned for this specific setting. The foundation models were trained for broad generalization across arbitrary text; our problem was the opposite — a narrow, highly structured set of text in fixed positions, bilingual. We fine-tuned on data close to what appears on these packages, including a new approved design not yet in circulation, trading generality for accuracy where it mattered.
Recognized text is checked against permitted terms. Positional measurements and colour values are compared against the regulatory requirements.
Prioritizing findings. Every measurement passes through an evaluation step comparing it against regulatory thresholds and reviewer best practice, producing a deviation score and category per check — within bounds, slightly outside, significantly outside, or unable to assess. Those scores rank what the inspector should look at first and roll up to a package-level indication.
The “unable to assess” category matters as much as the others. A system that silently skips a check it cannot perform is worse than useless in a regulatory setting.
Where the human stays in the loop
The system produces prioritized indications of where non-compliance is likely. The inspector verifies visually, then with instruments where warranted, and makes the official determination of non-compliant versus no evidence of non-compliance.
This division was a design requirement from the outset, and it also changed who does what: claim submission can be handled in volume by junior staff, so senior inspectors spend their time on the determinations only they can make.
Results
Deployed February 2024, ahead of schedule and within the allocated budget, with a testing window before go-live so inspectors and supervisors could use the system and give feedback first.
| Before | After | |
|---|---|---|
| Review time per package | Up to 180 minutes | ~20 minutes |
| Automated processing | — | Under 5 minutes per submission (contracted SLA) |
| Checks performed | 100+, manually | 100+, screened automatically, prioritized for review |
Roughly a tenfold reduction in review time.
What it doesn’t do
- It doesn’t determine compliance. It indicates where to look. The regulatory determination is the inspector’s, always.
- It reports when it cannot assess a check rather than passing it silently.
- It depends on photograph quality. Guides and robustness training raise the floor; they don’t eliminate it.
- It is specific to this regulatory regime. The checks encode one jurisdiction’s requirements.
Stack
YOLO-based detection locates the regions that carry regulated text. Classical CV in OpenCV handles geometric normalization — deskew, perspective correction, scale. Text detection and recognition use DBNet and RobustScanner fine-tuned for bilingual English and French copy; both ship in MMOCR, which is the natural toolbox for that pairing. Built in PyTorch. Deployed as a containerized service reached over a REST API, so the inspector’s existing review tool calls it rather than being replaced.