Skip to content

feat(importer): read-only Item migration inventory report (#1148) - #1168

Merged
JOY (JOY) merged 2 commits into
devfrom
data/1148-item-inventory
Oct 28, 2026
Merged

JOY (JOY) merged 2 commits into
devfrom
data/1148-item-inventory

Conversation

@JOY

@JOY JOY (JOY) commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1148

What

apps/importer/src/shopee/jobs/item-inventory-report.ts: the M0 inventory of the Item / variant / listing design (#1129, Docs/Proposals/2026-10-27-item-variant-listing-model.md section 7.3), as a re-runnable, read-only report. It is meant to be run again after M2 and M3.

  • Reads categories, master_products and products with the anon key (SUPABASE_URL + SUPABASE_ANON_KEY, or the NEXT_PUBLIC_ pair). It does SELECTs only, has no write path and never reads the secret key.
  • Mirrors the 7.3 SQL checks: shape, khc:, unknown and status counts, Item brand/category conflicts, sized earbuds, and listings without a master.
  • Slug checks follow the 7.1 helper regexes:
    • two Items on one Item slug;
    • an Item whose masters strip to different slugs;
    • a slug that is still sized after stripping.
  • Runs today's ProductNormalizer on every seed, master name and listing title. That is how seed key drift shows up, which SQL cannot do. Listing drift is bucketed as item, variant or size_not_stated.
  • Prints one JSON document to stdout. The target host goes to stderr.
pnpm --filter @repo/crawlers exec tsx src/shopee/jobs/item-inventory-report.ts

The run results and the data-fix list are posted on #1148 and #1150.

Tests

  • New src/shopee/__tests__/item-inventory-report.test.ts, wired into the importer test script. It covers every report section, the NULL-brand and TB cases, and the split_part key semantics.
  • pnpm typecheck, eslint and prettier pass on the changed files.

Review gate

  • A fresh-context subagent found no blockers. Its 4 should-fix items are fixed in the second commit: NULL handling, TB and suffixed variant keys, the missing slug cases, and test coverage.
  • Codex review: no issues found.

🤖 Generated with Claude Code


📌 TL;DR

This PR introduces a read-only inventory audit tool (item-inventory-report) to validate data integrity for the Item/Variant migration (#1148), checking for key shape violations, slug collisions, and normalization drift. It also adds a comprehensive test suite to verify the report generation logic against edge cases.

🎯 Type of Change

  • 🚀 New feature
  • 🐛 Bugfix
  • 🧹 Refactor
  • ⚡ Performance
  • 📚 Documentation
  • ⚙️ CI / Configuration

🔍 Changes Walkthrough

File Summary of Changes
apps/importer/package.json Updated the test script to include the new item-inventory-report.test.ts in the test execution chain.
apps/importer/src/shopee/jobs/item-inventory-report.ts New File: Implements the core audit logic. It defines interfaces for inventory data, utility functions for key/slug normalization, and buildInventoryReport which analyzes categories, masters, and listings for conflicts, drift, and shape errors. Includes a CLI entry point to fetch data from Supabase (read-only) and output a JSON report.
apps/importer/src/shopee/__tests__/item-inventory-report.test.ts New File: Unit tests for the inventory report. Validates helper functions (itemKeyOf, itemSlugOf, etc.) and asserts the correctness of the report output for complex scenarios involving legacy keys, slug collisions, storage mismatches, and normalization drift.

📊 Architectural Flow

flowchart TD
    subgraph CLI_Entry
        A[Direct Execution Check] --> B[Load Env Vars]
        B --> C[Create Supabase Client<br/>Anon Key Only]
    end

    subgraph Data_Fetch
        C --> D[loadInventory]
        D --> E[Fetch Categories]
        D --> F[Fetch Master Products]
        D --> G[Fetch Listings]
    end

    subgraph Analysis
        E & F & G --> H[buildInventoryReport]
        H --> I[Normalize Keys/Slugs]
        H --> J[Check Shape & Legacy]
        H --> K[Detect Conflicts & Drift]
        H --> L[Identify Slug Collisions]
    end

    subgraph Output
        I & J & K & L --> M[Generate JSON Report]
        M --> N[Write to stdout]
    end
Loading

JOY (JOY) and others added 2 commits October 28, 2026 00:32
Runs the M0 checks of the Item / variant / listing design (section 7.3)
over the anon-readable tables, and runs today's ProductNormalizer on each
seed, master name and listing title to find key drift, which SQL cannot.
SELECTs only, with the anon key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NULL brand or category is not an Item conflict, as in SQL count(DISTINCT).
TB and suffixed variant keys count as sized. Slug checks now also flag an
Item whose masters strip to different slugs and a slug still sized after
stripping. Listings with no stated size get their own drift bucket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rate Ready Ready Preview Oct 27, 2026 5:36pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a4640917-c289-4d7a-bfad-4b032e40ae06
Commits

Reviewing files that changed from the base of the PR and between 645c155 and 1ff875c.

Files selected for processing: 3
  • apps/importer/package.json
  • apps/importer/src/shopee/__tests__/item-inventory-report.test.ts
  • apps/importer/src/shopee/jobs/item-inventory-report.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

dos[bot]
dos Bot approved these changes Oct 27, 2026

@dos dos Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏱️ Code Review completed (3 eligible files · 20,525 chars · 2 model call(s) · 1 PR unit(s))

Whole diff with bounded context at base 645c155 / head 1ff875c.

✅ CLEAN_PASS: Complete diff reviewed

Context investigation was bounded to the explicitly requested snapshot paths.

@JOY
JOY (JOY) merged commit 7780e34 into dev Oct 28, 2026
19 checks passed
@JOY
JOY (JOY) deleted the data/1148-item-inventory branch October 28, 2026 04:00
@JOY JOY (JOY) mentioned this pull request Oct 28, 2026
3 of 6 tasks

This branch was successfully deployed

1 active deployment
Preview — 1ff875c6 Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

data: read-only inventory for the Item migration (seed key drift, legacy keys, slug collisions)

1 participant