Skip to content

feat: add deterministic visual-change detection for PRs. - #7742

Draft
BillLeoutsakosvl346 wants to merge 23 commits into
stagingfrom
codex/design-diff-engine
Draft

feat: add deterministic visual-change detection for PRs.#7742
BillLeoutsakosvl346 wants to merge 23 commits into
stagingfrom
codex/design-diff-engine

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Add an advisory deterministic designer-notification engine for PRs targeting staging. It reads immutable Git objects and flags supported appearance changes: shared styles, local colour/padding, typography, layout and custom styling on new controls. Copy, routine documentation, media, extra options, unchanged shared-component reuse and uncertainty alone are exempt.

Results are bounded, deterministic JSON artifacts. No PR comments, labels, annotations, Slack, screenshots or AI are added. Schema 3.0.0, engine 0.5.2, policy 5.0.0 use binary flag/exempt; operational failures exit nonzero with flagged: null.

The trusted production workflow activates after reaching main. Its absence on this draft PR is not a passing cloud analysis.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

The notification policy is narrower than earlier engine revisions. Consumers must check schema and policy versions.

Testing

Bun 1.4.1: 554 tests / 35 suites passed. Engine type check, Biome, API-validation audit, test discovery, frozen dependency install and workflow validation passed. Current head b3b25a54ecb33fcc45715aa187383d77555e5456: CI is running; local validation is complete.

Cloud evaluation ran the same frozen 180 comparisons against 3c49d5bdb0817ac030ca8a56be779af323fc57d5:

Sample Flagged Rate
Original 21/120 17.5%
Holdout 10/60 16.7%
Total 31/180 17.2%

Previously: 112/178 (62.9%), with two failures. All 180 now completed. All 17 previously identified backend false positives are clean. Review of current notifications found 25 qualifying, five apparent false positives and one borderline case. Original broad-content labels remain unchanged; this is source review, not pixel ground truth.

Median comparison: 55.7s; maximum: 670.3s. Maximum peak memory: 2.31 GiB. Largest report: 1.59 MiB. Cloud wall time: 21 minutes across 30 workers; no application builds.

Subsequent consolidation removes 763 net lines across two cleanup commits. The final pass removes unused SVG movement machinery and ignored comparison arguments, and validates current documentation inputs without comparing exempt content. 23 synthetic before/after reports matched byte-for-byte, and file/symbol renaming regressions pass. The earlier cleanup also matched 695 extractor comparisons. The cleanups have not had a separate full historical replay.

No new PR-specific exceptions were added. Existing repository conventions and notification policy remain fixed. Both historical cohorts have informed development; the next generalization check must use a fresh independently labeled sample without tuning during evaluation.

Remaining limitations: media wrappers, OG-only fonts, removed copy wrappers and unused DOM measurement can still flag; unknown runtime styling can be missed. Matching and consumer counts are partial. The benchmark does not establish complete recall.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

Not applicable; repository automation only. Output is a versioned JSON artifact.

@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 12, 2026 8:54pm UTC

Request Review

…engine

# Conflicts:
#	package.json
#	scripts/check-script-test-coverage.ts
#	vitest.scripts.config.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant