Product quiz field guide

# Product quiz match-quality tests: a 16-case QA pack

A reproducible test pack for checking product quiz eligibility, recommendation stability, catalog changes, explanations and fallback behavior.

13 September 2026·Product Quiz Guide

Answer: test a product quiz with named input cases and expected results before measuring conversion. Cover ordinary buyers, hard exclusions, tied scores, boundary values, missing catalog data, inventory changes and no-match fallbacks. A quiz is not ready merely because every path reaches a results page.

## What is product quiz match-quality testing?

Match-quality testing checks whether a fixed set of answers produces an eligible, explainable and stable recommendation under documented rules. It differs from interface testing, which checks whether a visitor can complete the flow, and conversion analysis, which measures what visitors do afterward.

## Start with the Recommendation invariants ledger

Write the properties that must always be true before writing individual cases. These invariants turn a subjective review into a repeatable pass or fail check.

InvariantPass conditionFailure exampleEligibilityNo excluded item can winFragranced product shown after fragrance exclusionAvailabilityResult respects the stated stock policyUnavailable size recommended without warningExplanationEach reason maps to a real answer or ruleResult cites sensitivity that was never selectedStabilityIrrelevant answers do not change the winnerNewsletter preference changes mattress firmnessFallbackEmpty sets produce a useful routeBlank result after conflicting constraintsVersioningResult records rule and catalog versionOld and new scoring appear in one report

## Use this 16-case test pack

- Most common valid profile.
- Second common valid profile.
- Minimum eligible budget.
- Maximum supported requirement.
- One hard exclusion.
- Multiple compatible exclusions.
- Conflicting exclusions that leave no eligible result.
- Exact tie between two eligible results.
- One point on either side of a score boundary.
- Missing optional answer.
- Missing required answer blocked by the interface.
- Missing catalog attribute.
- Recommended variant unavailable.
- Entire recommended product unavailable.
- Deep-link return or result reload.
- Logic version changed while catalog inputs stay fixed.

## Run a Boundary twin test

Create two cases that differ by only one boundary value. If a mattress finder sends sleepers up to 90 kg to medium firmness and sleepers above 90 kg to firm, test 90 and the smallest accepted value above 90. Every other answer must match. The expected change should be explicit and the explanation should name the factor.

Boundary twins reveal off-by-one errors, unit-conversion mistakes and hidden score interactions that ordinary paths miss.

## Worked skincare test

The Northstar brief contains 24 products with skin type, sensitivity, concern, fragrance, price, compatibility, inventory and market attributes. One case selects sensitive, dry skin, a barrier concern, fragrance-free products and a midrange budget.

Expected result: every recommended item is marked suitable for sensitivity, fragrance-free, compatible as a routine, available in the visitor's market and inside the budget rule. Each explanation cites at least one selected need. If no complete routine survives, the quiz explains the constraint and offers a safe partial route instead of silently relaxing sensitivity.

## Worked B2B plan test

The Atlas brief has four plans. A case requiring SSO, an audit log and a stated usage volume must first exclude plans missing either security feature. Weighted preferences then rank the survivors. A lower-priced ineligible plan can never win because preference scores do not override hard requirements.

## Test catalog changes separately

Shopify currently documents product status, markets, variants, inventory, pricing, media, categories and metafields as distinct product data. A catalog-driven finder may depend on several of them. Change one input at a time: set a variant unavailable, remove a required metafield, change a market, archive a product and alter a price boundary. Verify both the result and its explanation after each change.

Use the [catalog-readiness checklist](https://bestproductquiz.com/blog/product-finder-catalog-readiness) to identify every field that participates in matching before building these cases.

## Instrument failures without leaking answers

Record controlled quiz, logic, result and catalog version IDs with recommendation events. Also record a controlled no-result reason. Keep raw personal answers and contact details out of analytics parameters. Google Analytics documents recommended ecommerce and lead events, but the quiz-specific recommendation event and its safe parameters need a written implementation contract.

The [product quiz analytics map](https://bestproductquiz.com/blog/product-quiz-analytics) explains how to keep completion, recommendation and post-result denominators separate.

## Make failure behavior accessible

If a required answer is missing or the rules return no valid match, identify the issue in text and make it perceivable to assistive technology. Do not communicate the state only through color. W3C guidance says notifications should be concise and clear, with understandable correction instructions. Move focus deliberately when the visitor must repair an earlier answer.

Repeat the complete failure path with the [product quiz accessibility checklist](https://bestproductquiz.com/blog/product-quiz-accessibility).

## Limitations

Sixteen cases are a reusable minimum, not exhaustive proof. Large catalogs, variant combinations, regional availability and regulated requirements can create many more meaningful states. Pair this deterministic pack with production monitoring and sampled human review of recommendation explanations. The worked cases are design examples, not a match-accuracy benchmark.

## Sources

- [Shopify, product details](https://help.shopify.com/en/manual/products/details/product-details-page)
- [W3C, user notification](https://www.w3.org/WAI/tutorials/forms/notifications/)
- [Google Analytics, recommended events](https://support.google.com/analytics/answer/9267735)

Method: an original deterministic QA matrix applied to the Northstar skincare and Atlas software briefs. No hands-on vendor result or match-accuracy benchmark is claimed. Sources and examples were verified on 13 September 2026.

Corrections: send the page URL, the exact statement and a current source through the [contact form](https://bestproductquiz.com/contact).

[Compare the eight product recommendation quiz builders](https://bestproductquiz.com/#comparison) or read the [full testing protocol](https://bestproductquiz.com/how-we-test).

---

Canonical: https://bestproductquiz.com/blog/product-quiz-match-quality-tests
