Product quiz field guide # Product quiz resilience testing: a failure-mode QA plan Test how a product recommendation quiz recovers from catalog, inventory, saved-state, CRM, email and analytics failures. 18 October 2026·Product Quiz Guide Answer: test a product quiz by deliberately breaking every dependency that can affect eligibility, ranking, explanation or continuation. Define when to retry, degrade, block a recommendation or hand off to a person, then verify that no failure silently produces an invalid match. ## What is product quiz resilience? Product quiz resilience is the ability to preserve decision integrity and provide a usable recovery path when network requests, catalog data, integrations, saved state or downstream actions fail. Graceful recovery can mean continuing with a disclosed limitation, but it can also mean stopping before an unverified recommendation is shown. This article owns dependency and failure-state testing. The [match-quality QA pack](https://bestproductquiz.com/blog/product-quiz-match-quality-tests) covers deterministic decision correctness, the [no-match decision tree](https://bestproductquiz.com/blog/product-quiz-no-match) covers valid empty-result states, and the [launch runbook](https://bestproductquiz.com/blog/product-quiz-launch-checklist) covers coordinated release and rollback. ## Failure-mode register Register each dependency by the decision job it can affect. A display-only failure can permit a result that an eligibility-data failure must block. DependencyInjected failureDecision riskSafe responseCatalog APITimeoutStale or missing eligibilityRetry, then block the current recommendationInventoryPartial responseUnavailable item recommendedMark status unknown and withhold the cart actionPricingCurrency mismatchMisleading result valueShow a verified source currency or defer the priceRule serviceVersion unavailableUntraceable calculationStop and preserve the answer recordSaved stateCorrupt or expiredWrong answers restoredExplain the problem and offer an explicit restartCRM writeTimeout or uncertain responseLost or duplicated handoffRetry the same operation identity and show statusEmail workflowEnrollment failureFollow-up promise brokenPreserve the result and separate the failed next stepAnalyticsBlocked requestMissing measurementKeep the visitor path functional ## Resilience acceptance matrix Failure severityRecommendation allowed?Required messageRequired recordCosmetic asset missingYes, if meaning remains clearNone or non-blockingClient errorOptional enrichment unavailableYes, with disclosureWhat is temporarily absentDependency and timePrice or inventory uncertainConditionalCurrent uncertainty and next stepProduct, market and source stateEligibility data incompleteNoA safe match cannot be verifiedAnswers, rule version and failed dependencyRule version unknownNoThe result cannot be recalculatedDecision trace and recovery tokenDownstream handoff failedPreserve the resultThe result is available; the next step needs retryOperation identity and handoff status ## Failure-injection procedure - Inventory dependencies. List every request, data source and state transition from entry through the post-result action. - Label the decision effect. Mark whether each dependency affects eligibility, ranking, explanation, display or handoff. - Define the permitted outcome. State whether the quiz may continue, disclose a limitation, preserve state or stop. - Inject distinct failures. Test timeout, error, empty, partial, stale and malformed responses separately. - Exercise recovery. Confirm retries are bounded, repeated actions do not create duplicates and terminal failure is visible. - Test the interface. Verify keyboard focus, status announcements and preservation of valid answers. - Reconcile evidence. Compare the visible outcome with analytics, dependency logs and the stored decision record. - Keep regressions. Turn every production incident into a permanent deterministic case. ## Worked Northstar outage drill Northstar injects an inventory response containing 18 of 24 SKUs. Because inventory is a hard cart constraint, the quiz does not label the omitted products unavailable; their status is unknown. It calculates a recommendation only when the complete eligible set can be verified. If the bounded retry fails, it preserves the answers, explains that live availability cannot be checked and offers a later retry or alert. The failure does not weaken sensitivity or fragrance constraints to fill the page. Shopify currently separates product status, variants and multiple inventory quantities—including available, committed and unavailable units—so the test records which exact field and market state the recommendation consumed. ## Worked Atlas failure drill Atlas calculates an eligible plan, but the CRM handoff times out. The visitor keeps the plan result and explanation because the failure occurred after the recommendation. The write is retried with the same operation identity, following the [recommendation-context handoff contract](https://bestproductquiz.com/blog/product-quiz-crm-handoff), so an uncertain response cannot silently create duplicate contacts or follow-up sequences. If retries exhaust, the interface says the recommendation remains available but the requested specialist handoff is not confirmed. It supplies a manual route with the result reference instead of claiming that a representative will follow up. ## Test performance and recovery separately Measure ordinary performance and failure recovery as related but distinct properties. Google's current Core Web Vitals guidance covers loading, interactivity and visual stability and distinguishes simulated lab measurement from real-user field evidence. A fast error page is not a valid recommendation, while a correct result that never becomes interactive is not usable. Keep retries bounded, reserve space for status messages to avoid layout movement and ensure that failure dialogs or banners do not trap keyboard focus. W3C guidance says success and error notifications should be concise, understandable and provide a correction path; dynamic error summaries also need to be exposed to assistive technology. ## Instrument failures without redefining conversions Record the dependency, failure class, attempt number, terminal state, affected decision job and whether the result was preserved. Analytics failure must not block the user path. Google Analytics recommends events such as generate_lead for actual form or information-request submissions and provides DebugView and Realtime tools for verification; dependency failures and recovery attempts can use documented custom events rather than being mislabeled as leads. RecordMinimum contextDo not inferdependency_faileddependency, failure_class, decision_jobThe entire quiz failedrecovery_startedoperation_id, attempt, preserved_stateThe retry will succeedrecovery_succeededoperation_id, attempts, resulting_stateThe earlier result was invalidrecovery_exhaustedterminal_state, user_path, manual_routeThe visitor abandonedrecommendation_blockedmissing_evidence, rule_version, answer_recordA no-match result was calculated ## Limitations This plan is not a claim of uptime, disaster-recovery certification or universal reliability. It does not replace load, security, privacy, accessibility-conformance or vendor-contract testing. Exact thresholds depend on catalog volatility, consequence and architecture, and synthetic fault injection cannot prove that every production failure has been covered. ## Sources - [web.dev, Web Vitals](https://web.dev/articles/vitals) - [W3C WAI, User notifications](https://www.w3.org/WAI/tutorials/forms/notifications/) - [Shopify Help Center, Product details page](https://help.shopify.com/en/manual/products/details/product-details-page) - [Google Analytics, Recommended events](https://support.google.com/analytics/answer/9267735) Method: the original Failure-mode register and Resilience acceptance matrix were applied to Northstar skincare and Atlas B2B outage drills. All four primary sources were rechecked on 18 October 2026. No uptime or customer-performance result is claimed. Corrections can be submitted through the site's [corrections process](https://bestproductquiz.com/corrections). Corrections: send the page URL, the exact statement and a current source through the [contact form](https://bestproductquiz.com/contact). [Compare the eight product recommendation quiz builders](https://bestproductquiz.com/#comparison) or read the [full testing protocol](https://bestproductquiz.com/how-we-test). --- Canonical: https://bestproductquiz.com/blog/product-quiz-resilience