Product variant detection is the process of identifying different versions of the same product\u2014such as size, color, storage, capacity, material, or configuration\u2014and correctly associating each variant with its own identifiers, availability, and other attributes.<\/p>\n\n\n\n
This sounds straightforward until you begin collecting product data at scale.<\/p>\n\n\n\n
A single e-commerce product page may represent dozens or even hundreds of possible combinations. Some variants have their own SKU or GTIN. Others share a parent product identifier. Certain combinations may be unavailable, while others may not exist at all. On modern JavaScript-based websites, selecting a variant can also change the product URL, price, images, availability, or data returned by background requests.<\/p>\n\n\n\n
If these relationships are extracted incorrectly, downstream datasets can become unreliable. Prices may be assigned to the wrong size, inventory may be attributed to the wrong color, and product matching systems may compare different configurations as if they were identical.<\/p>\n\n\n\n
This guide explains how to detect product variants reliably and model them as structured data for e-commerce analysis.<\/p>\n\n\n\n
A product variant is a specific configuration of a broader product.<\/p>\n\n\n\n
Consider a smartphone sold with three storage capacities and four colors:<\/p>\n\n\n\n
128GB \u00d7 Black\n128GB \u00d7 White\n128GB \u00d7 Blue\n128GB \u00d7 Green\n\n256GB \u00d7 Black\n256GB \u00d7 White\n...\n\n512GB \u00d7 Green<\/code><\/pre>\n\n\n\nThe product family may be the same, but each valid combination can represent a different sellable item.<\/p>\n\n\n\n
A useful model is:<\/p>\n\n\n\n
Product Family\n \u2193\nVariant Dimensions\n \u2193\nSpecific Variant\n \u2193\nRetailer Offer<\/code><\/pre>\n\n\n\nFor example:<\/p>\n\n\n\n
Example Phone X\n \u2193\nStorage + Color\n \u2193\n256GB \/ Black\n \u2193\nSKU: PX-256-BLK\nGTIN: 0123456789012\n \u2193\nPrice \/ Availability \/ Seller<\/code><\/pre>\n\n\n\nProduct variant detection therefore involves more than extracting options from a dropdown. The objective is to reconstruct the relationship between the parent product, its variant dimensions, valid combinations, identifiers, and commercial information.<\/p>\n\n\n\n
Product Family vs Product Variant<\/a> vs Offer<\/h2>\n\n\n\nBefore building a variant extraction system, separate three concepts that are frequently mixed together.<\/p>\n\n\n\n
A product family<\/strong> represents the broader product model.<\/p>\n\n\n\nA product variant<\/strong> represents a particular configuration of that product.<\/p>\n\n\n\nAn offer<\/strong> represents the commercial conditions under which a retailer or seller is currently selling that variant.<\/p>\n\n\n\nFor example:<\/p>\n\n\n\n
Running Shoe Model X\n \u2193\nBlack \/ EU 42\n \u2193\nRetailer A\n$129\nIn Stock<\/code><\/pre>\n\n\n\nPrice and availability belong primarily to the offer observation. Size and color belong to the variant. The model name belongs to the broader product identity.<\/p>\n\n\n\n
Keeping these layers separate makes later price comparison and inventory analysis considerably easier.<\/p>\n\n\n\n
Why Product Variant Detection Is Difficult<\/h2>\n\n\n\n
The visible page does not necessarily represent the complete product structure.<\/p>\n\n\n\n
Some websites render every available option into the HTML. Others expose only the currently selected variant. JavaScript may retrieve new information after a user changes a color or size.<\/p>\n\n\n\n
There are also dependencies between options.<\/p>\n\n\n\n
Suppose a shoe is available in:<\/p>\n\n\n\n
Black:\n38, 39, 40, 41, 42\n\nWhite:\n39, 40, 41<\/code><\/pre>\n\n\n\nA scraper that independently extracts:<\/p>\n\n\n\n
Colors = [Black, White]\nSizes = [38, 39, 40, 41, 42]<\/code><\/pre>\n\n\n\nmight incorrectly generate:<\/p>\n\n\n\n
White \/ 38\nWhite \/ 42<\/code><\/pre>\n\n\n\neven though those combinations do not exist.<\/p>\n\n\n\n
The challenge is therefore not simply discovering option values. It is discovering valid relationships between option values<\/strong>.<\/p>\n\n\n\nStart by Identifying Variant Dimensions<\/h2>\n\n\n\n
Variant dimensions describe the properties that distinguish one variant from another.<\/p>\n\n\n\n
Common dimensions include:<\/p>\n\n\n\n
Color\nSize\nStorage\nMemory\nCapacity\nMaterial\nStyle\nLength\nWidth\nVoltage\nRegion\nPack quantity<\/code><\/pre>\n\n\n\nThe dimensions depend heavily on the product category.<\/p>\n\n\n\n
For apparel:<\/p>\n\n\n\n
Color + Size<\/code><\/pre>\n\n\n\nFor smartphones:<\/p>\n\n\n\n
Color + Storage<\/code><\/pre>\n\n\n\nFor laptops:<\/p>\n\n\n\n
Processor + RAM + Storage + Display<\/code><\/pre>\n\n\n\nFor furniture:<\/p>\n\n\n\n
Material + Color + Size<\/code><\/pre>\n\n\n\nFor industrial products:<\/p>\n\n\n\n
Voltage + Capacity + Connector + Revision<\/code><\/pre>\n\n\n\nAvoid assuming that every option shown on a page represents a variant dimension. Options such as delivery method, warranty extension, gift wrapping, or financing may modify the purchase experience without creating a different underlying product variant.<\/p>\n\n\n\n
Where Variant Information Can Be Found<\/h2>\n\n\n\n
A robust extraction system should inspect several possible sources.<\/p>\n\n\n\n
Variant Selectors in the DOM<\/h3>\n\n\n\n
The most obvious source is the product interface.<\/p>\n\n\n\n
A page might contain:<\/p>\n\n\n\n
<select name=\"size\">\n <option value=\"S\">Small<\/option>\n <option value=\"M\">Medium<\/option>\n <option value=\"L\">Large<\/option>\n<\/select><\/code><\/pre>\n\n\n\nColors may instead appear as buttons:<\/p>\n\n\n\n
<button\n data-color=\"black\"\n data-sku=\"TSHIRT-BLK\">\n Black\n<\/button><\/code><\/pre>\n\n\n\nThese elements can reveal available option values and sometimes identifiers.<\/p>\n\n\n\n
However, the DOM may represent only the current state rather than the complete variant matrix.<\/p>\n\n\n\n
Embedded Product Data<\/h2>\n\n\n\n
Many modern e-commerce websites include serialized product data somewhere in the page.<\/p>\n\n\n\n
A simplified object might look like:<\/p>\n\n\n\n
{\n \"product_id\": \"TSHIRT-001\",\n \"variants\": [\n {\n \"sku\": \"TSHIRT-BLK-S\",\n \"color\": \"Black\",\n \"size\": \"S\",\n \"available\": true\n },\n {\n \"sku\": \"TSHIRT-BLK-M\",\n \"color\": \"Black\",\n \"size\": \"M\",\n \"available\": false\n },\n {\n \"sku\": \"TSHIRT-WHT-M\",\n \"color\": \"White\",\n \"size\": \"M\",\n \"available\": true\n }\n ]\n}<\/code><\/pre>\n\n\n\nWhen available and appropriate to process, this representation can be much more useful than independently reading buttons from the interface because the relationship between attributes and variants is already explicit.<\/p>\n\n\n\n
The exact location and structure vary by website, so extraction logic should be source-aware.<\/p>\n\n\n\n
JSON-LD Product Data<\/h2>\n\n\n\n
Some websites expose product information using structured data.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
{\n \"@context\": \"https:\/\/schema.org\",\n \"@type\": \"Product\",\n \"name\": \"Example Running Shoe\",\n \"sku\": \"SHOE-BLK-42\",\n \"color\": \"Black\",\n \"size\": \"42\",\n \"offers\": {\n \"@type\": \"Offer\",\n \"price\": \"129.00\",\n \"priceCurrency\": \"USD\",\n \"availability\": \"https:\/\/schema.org\/InStock\"\n }\n}<\/code><\/pre>\n\n\n\nJSON-LD is useful because fields have clearer semantic meaning than arbitrary CSS classes.<\/p>\n\n\n\n
But it should not automatically be assumed to contain every variant.<\/p>\n\n\n\n
Some pages expose structured data only for the currently selected variant. Others describe the parent product rather than individual configurations.<\/p>\n\n\n\n
Treat JSON-LD as one evidence source rather than the complete variant model unless the source has been verified.<\/p>\n\n\n\n
Browser Network Responses<\/h2>\n\n\n\n
On JavaScript-heavy websites, changing an option may trigger an XHR or fetch<\/code> request.<\/p>\n\n\n\nThe interaction might look like:<\/p>\n\n\n\n
Select Black\n \u2193\nSelect Size 42\n \u2193\nBrowser Request\n \u2193\nVariant API\n \u2193\nSKU \/ Price \/ Availability \/ Images<\/code><\/pre>\n\n\n\nFor browser automation workflows, Playwright can observe responses produced during normal page interactions:<\/p>\n\n\n\n
page.on('response', async (response) => {\n const contentType =\n response.headers()['content-type'] || '';\n\n if (!contentType.includes('application\/json')) {\n return;\n }\n\n try {\n const data = await response.json();\n\n \/\/ Inspect relevant product or variant responses.\n } catch {\n \/\/ Ignore responses that are not valid JSON.\n }\n});<\/code><\/pre>\n\n\n\nProduction implementations should filter for known relevant endpoints or expected response schemas rather than storing every JSON response.<\/p>\n\n\n\n
Network observations can be particularly valuable when variant state is not fully represented in the initial HTML.<\/p>\n\n\n\n
Do Not Generate a Cartesian Product Blindly<\/h2>\n\n\n\n
Suppose a page contains:<\/p>\n\n\n\n
Colors:\nBlack\nWhite\nBlue\n\nSizes:\nS\nM\nL\nXL<\/code><\/pre>\n\n\n\nA naive system may calculate:<\/p>\n\n\n\n
3 \u00d7 4 = 12 variants<\/code><\/pre>\n\n\n\nBut perhaps the actual catalog contains only:<\/p>\n\n\n\n
Black: S, M, L, XL\nWhite: S, M, L\nBlue: M, L<\/code><\/pre>\n\n\n\nThere are nine valid variants, not twelve.<\/p>\n\n\n\n
This distinction is critical.<\/p>\n\n\n\n
Instead of generating every possible combination, build the variant matrix from evidence supplied by the website.<\/p>\n\n\n\n
A structured representation could be:<\/p>\n\n\n\n
{\n \"Black\": [\"S\", \"M\", \"L\", \"XL\"],\n \"White\": [\"S\", \"M\", \"L\"],\n \"Blue\": [\"M\", \"L\"]\n}<\/code><\/pre>\n\n\n\nOr preferably store each actual variant independently.<\/p>\n\n\n\n
Model Every Variant as Its Own Record<\/h2>\n\n\n\n
For downstream analytics, a flat variant record is often easier to work with than a deeply nested product object.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
{\n \"parent_product_id\": \"TSHIRT-001\",\n \"variant_id\": \"TSHIRT-BLK-M\",\n \"attributes\": {\n \"color\": \"Black\",\n \"size\": \"M\"\n },\n \"sku\": \"TSHIRT-BLK-M\",\n \"gtin\": \"0123456789012\",\n \"availability\": \"out_of_stock\",\n \"price\": 29.99,\n \"currency\": \"USD\"\n}<\/code><\/pre>\n\n\n\nAnother variant becomes another record.<\/p>\n\n\n\n
This structure makes it easier to answer questions such as:<\/p>\n\n\n\n
\n- Which sizes are available?<\/li>\n\n\n\n
- Which color has the highest price?<\/li>\n\n\n\n
- Which variants disappeared from the catalog?<\/li>\n\n\n\n
- Which exact configuration matches another retailer?<\/li>\n\n\n\n
- Which SKU changed availability?<\/li>\n<\/ul>\n\n\n\n
The parent-child relationship remains available through parent_product_id<\/code>.<\/p>\n\n\n\nAssociate SKU and GTIN With the Correct Variant<\/h2>\n\n\n\n
This is one of the most important parts of the entire process.<\/p>\n\n\n\n
A product page may contain:<\/p>\n\n\n\n
Parent SKU\nCurrent Variant SKU\nAlternative Variant SKUs\nRecommended Product SKUs\nRelated Product IDs<\/code><\/pre>\n\n\n\nSimply searching the HTML for sku<\/code> can return several valid values.<\/p>\n\n\n\nEach identifier should therefore be stored together with evidence about its scope.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
{\n \"identifier_type\": \"sku\",\n \"value\": \"TSHIRT-BLK-M\",\n \"scope\": \"variant\",\n \"variant\": {\n \"color\": \"Black\",\n \"size\": \"M\"\n },\n \"source\": \"variant_data\"\n}<\/code><\/pre>\n\n\n\nThis prevents a valid identifier from being attached to the wrong configuration.<\/p>\n\n\n\n
The same principle applies to GTIN, UPC, EAN, MPN, and retailer-specific product IDs.<\/p>\n\n\n\n
Distinguish Unavailable Variants From Nonexistent Variants<\/h2>\n\n\n\n
These states are not equivalent.<\/p>\n\n\n\n
Consider:<\/p>\n\n\n\n
Black \/ XL \u2192 Out of stock\nWhite \/ XL \u2192 Does not exist<\/code><\/pre>\n\n\n\nBoth may be impossible to purchase today, but they carry very different information.<\/p>\n\n\n\n
An out-of-stock variant exists in the retailer’s catalog and may become available again.<\/p>\n\n\n\n
A nonexistent combination may never have been offered.<\/p>\n\n\n\n
A useful status model could include:<\/p>\n\n\n\n
in_stock\nout_of_stock\nunavailable\nnot_offered\nunknown<\/code><\/pre>\n\n\n\nAvoid converting all disabled options into out_of_stock<\/code>.<\/p>\n\n\n\nThe website may disable an option because it is incompatible with another selection rather than because inventory is zero.<\/p>\n\n\n\n
Dynamic Variant Selection Requires State Management<\/h2>\n\n\n\n
Some websites do not expose every variant until the user interacts with the page.<\/p>\n\n\n\n
Selecting one attribute may change the available values of another.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
Step 1: Select Color = Black\nStep 2: Available sizes become 38, 39, 40, 42\nStep 3: Select Size = 42\nStep 4: Page updates SKU and price<\/code><\/pre>\n\n\n\nBrowser automation can reproduce these interactions where appropriate, but the collector needs to understand page state.<\/p>\n\n\n\n
After each selection, verify that the expected change has occurred before recording the result.<\/p>\n\n\n\n
Avoid relying solely on arbitrary waits such as:<\/p>\n\n\n\n
await page.waitForTimeout(3000);<\/code><\/pre>\n\n\n\nA more reliable strategy is to wait for a meaningful state change, such as a relevant response, changed SKU, updated product element, or confirmed selection state.<\/p>\n\n\n\n
Variant URLs Can Be Valuable Identifiers<\/h2>\n\n\n\n
Some retailers create a distinct URL for each variant.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
\/product\/shirt?color=black&size=m<\/code><\/pre>\n\n\n\nOthers use:<\/p>\n\n\n\n
\/product\/shirt\/variant\/847291<\/code><\/pre>\n\n\n\nAnd some update only the browser state without changing the URL.<\/p>\n\n\n\n
When a stable variant-specific URL exists, preserve it.<\/p>\n\n\n\n
A useful record might include:<\/p>\n\n\n\n
{\n \"parent_url\": \"https:\/\/example.com\/product\/shirt\",\n \"variant_url\":\n \"https:\/\/example.com\/product\/shirt?color=black&size=m\"\n}<\/code><\/pre>\n\n\n\nThis improves traceability and makes it easier to revisit the exact configuration later.<\/p>\n\n\n\n
Do not assume query parameters always define variants, though. Some parameters represent tracking, sorting, referral information, or session state.<\/p>\n\n\n\n
Normalize Variant Attributes Carefully<\/h2>\n\n\n\n
Retailers may represent equivalent attributes differently.<\/p>\n\n\n\n
Sizes might appear as:<\/p>\n\n\n\n
Large\nL\nLG<\/code><\/pre>\n\n\n\nStorage:<\/p>\n\n\n\n
1 TB\n1TB\n1000GB<\/code><\/pre>\n\n\n\nColors:<\/p>\n\n\n\n
Midnight\nMidnight Black\nBlack<\/code><\/pre>\n\n\n\nNormalization can make cross-retailer analysis easier, but it should not destroy meaningful distinctions.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
Black\nSpace Black\nMidnight<\/code><\/pre>\n\n\n\nshould not automatically be normalized into one color merely because they appear visually similar.<\/p>\n\n\n\n
A safer structure preserves both:<\/p>\n\n\n\n
{\n \"raw_color\": \"Midnight\",\n \"normalized_color_family\": \"Black\",\n \"manufacturer_color\": \"Midnight\"\n}<\/code><\/pre>\n\n\n\nThis allows broad color analysis without losing the manufacturer’s actual variant definition.<\/p>\n\n\n\n
Pack Size Is Also a Variant Attribute<\/h2>\n\n\n\n
Quantity is frequently overlooked.<\/p>\n\n\n\n
Consider:<\/p>\n\n\n\n
Protein Bar 60g\nProtein Bar 60g \u00d7 6\nProtein Bar 60g \u00d7 12<\/code><\/pre>\n\n\n\nThese are not equivalent commercial products.<\/p>\n\n\n\n
A structured record should distinguish:<\/p>\n\n\n\n
{\n \"unit_size\": 60,\n \"unit\": \"g\",\n \"pack_quantity\": 12,\n \"total_quantity\": 720\n}<\/code><\/pre>\n\n\n\nWithout pack normalization, price comparison systems can incorrectly conclude that a 12-pack is dramatically more expensive than a single unit.<\/p>\n\n\n\n
This is one reason variant detection and price comparison should be connected but remain separate stages.<\/p>\n\n\n\n
Detect Parent Products and Child Variants<\/h2>\n\n\n\n
Many e-commerce platforms internally use a parent-child catalog model.<\/p>\n\n\n\n
The parent might contain:<\/p>\n\n\n\n
Title\nDescription\nBrand\nProduct family\nShared images<\/code><\/pre>\n\n\n\nwhile children contain:<\/p>\n\n\n\n
SKU\nGTIN\nColor\nSize\nPrice\nAvailability<\/code><\/pre>\n\n\n\nYour dataset can mirror this relationship.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
{\n \"parent\": {\n \"id\": \"SHIRT-100\",\n \"name\": \"Performance T-Shirt\",\n \"brand\": \"Example\"\n },\n \"variants\": [\n {\n \"id\": \"SHIRT-100-BLK-M\",\n \"color\": \"Black\",\n \"size\": \"M\"\n }\n ]\n}<\/code><\/pre>\n\n\n\nThis reduces duplication while preserving variant-specific information.<\/p>\n\n\n\n
How Product Variant Detection Improves Product Matching<\/h2>\n\n\n\n
Variant detection becomes particularly important when data is collected from multiple retailers.<\/p>\n\n\n\n
Suppose Retailer A lists:<\/p>\n\n\n\n
Camera X \u2014 Black \u2014 Body Only<\/code><\/pre>\n\n\n\nwhile Retailer B lists:<\/p>\n\n\n\n
Camera X Black + 18-55mm Lens<\/code><\/pre>\n\n\n\nAt the family level, both may belong to Camera X.<\/p>\n\n\n\n
At the sellable-product level, they are different.<\/p>\n\n\n\n
A robust workflow therefore looks like:<\/p>\n\n\n\n
Product Collection\n \u2193\nIdentifier Extraction\n \u2193\nProduct Family Detection\n \u2193\nProduct Variant Detection\n \u2193\nVariant Normalization\n \u2193\nCross-Retailer Product Matching<\/code><\/pre>\n\n\n\nThis reduces the risk of comparing prices or availability across incompatible configurations.<\/p>\n\n\n\n
Track Variant Changes Over Time<\/h2>\n\n\n\n
Product variant data is not static.<\/p>\n\n\n\n
Retailers may:<\/p>\n\n\n\n
\n- Add new colors.<\/li>\n\n\n\n
- Remove discontinued sizes.<\/li>\n\n\n\n
- Introduce new storage configurations.<\/li>\n\n\n\n
- Restock existing variants.<\/li>\n\n\n\n
- Replace an old SKU.<\/li>\n\n\n\n
- Change the GTIN associated with a listing.<\/li>\n\n\n\n
- Convert a bundle into a standalone listing.<\/li>\n<\/ul>\n\n\n\n
Instead of overwriting the previous record, consider storing observations over time.<\/p>\n\n\n\n
For example:<\/p>\n\n\n\n
{\n \"variant_id\": \"SHIRT-BLK-M\",\n \"observed_at\": \"2026-09-24T10:00:00Z\",\n \"availability\": \"in_stock\",\n \"price\": 29.99\n}<\/code><\/pre>\n\n\n\nHistorical observations make it possible to distinguish a genuine catalog change from a temporary collection failure.<\/p>\n\n\n\n
Validate Product Variant Detection at Scale<\/h2>\n\n\n\n
A variant extractor should have measurable quality indicators.<\/p>\n\n\n\n
Useful metrics include:<\/p>\n\n\n\n
Average variants per product\nProducts with zero detected variants\nVariants missing SKU\nVariants missing key attributes\nDuplicate variant combinations\nInvalid attribute combinations\nVariant extraction failure rate\nUnexpected changes in variant count<\/code><\/pre>\n\n\n\nSuppose a retailer normally has an average of eight variants per apparel product and the extracted average suddenly falls to 1.2.<\/p>\n\n\n\n
That could indicate a real catalog change, but it could also mean the extractor is now seeing only the default selected variant.<\/p>\n\n\n\n
Monitoring distribution changes can reveal problems that request-success metrics alone will miss.<\/p>\n\n\n\n
Common Product Variant Detection Mistakes<\/h2>\n\n\n\n
One common mistake is assuming that every selectable option represents a separate variant. Warranty, shipping method, gift packaging, and subscription settings may not define product identity.<\/p>\n\n\n\n
Another is creating all possible combinations from independently extracted option lists. This produces nonexistent variants when attributes depend on one another.<\/p>\n\n\n\n
A third mistake is attaching the first SKU or GTIN found on the page to every variant.<\/p>\n\n\n\n