Standardizing Unit Pricing Across Marketplaces #

A 12 fl oz bottle on one storefront, a 1 L six-pack on another, and a “family size” tub on a third are the same product to a shopper but three incomparable rows in a scraped feed. Until every observation is reduced to a single canonical price-per-base-unit — price per gram, per millilitre, or per count — any cross-marketplace comparison silently rewards whoever ships the largest container, and your outlier detector flags honest prices because it is comparing a single can against a case. This guide is a focused, runnable recipe for parsing the quantity and unit out of marketplace listings and collapsing them onto one comparable axis. It sits under the parent guide on statistical outlier detection for price data, and assumes the monetary axis is already clean: currencies aligned per converting multi-currency prices to a base currency and tax/shipping stripped per the tax and shipping cost normalization rules stage. This step only has to answer one question: how much product am I getting for that already-clean price?

The unit price for one observation is normalized_price / (pack_count × unit_quantity_in_base). The hard part is never the division — it is recovering pack_count, unit_quantity, and a trustworthy unit from inconsistent titles, attribute fields, and structured payloads, then refusing to compare a price-per-gram against a price-per-millilitre.

Prerequisites & Input Contract #

Each record must arrive already matched to a canonical sku_id and already reduced to a single base currency with tax and shipping removed, so normalized_price reflects the money paid for the goods alone. This stage adds the denominator; it must never touch the numerator. Genuine multi-buy mechanics (BOGO, “3 for $10”) are resolved earlier by parsing complex promotional discount structures, so by the time a row reaches here, normalized_price is the effective price for the physical pack described by its size fields.

# Input contract: one matched, money-normalized observation per (sku_id, marketplace_id).
record = {
    "sku_id": "p-44182",            # canonical product id (post catalog matching)
    "marketplace_id": "amazon-us",  # source storefront
    "normalized_price": 7.49,       # base currency, tax + shipping already stripped
    "title": "Acme Olive Oil Extra Virgin 2 x 16.9 fl oz",  # raw listing title
    "size_attr": None,              # structured size string if the API exposed one, else None
}

Library versions used throughout: Python 3.11+, pandas>=2.1, and numpy>=1.26. Install with pip install "pandas>=2.1" "numpy>=1.26". No third-party unit library is required — a curated mapping is both faster and auditable, and avoids pint-style packages silently coercing across physical dimensions (mass ↔ volume) when a title is ambiguous.

The output contract adds three fields and never overwrites the input:

# Output: input record plus a comparable axis.
{
    "base_unit": "ml",            # canonical unit for this product's dimension
    "quantity_base": 999.79,      # total base-unit quantity in the pack (2 × 16.9 fl oz → ml)
    "unit_price": 0.007491,       # normalized_price / quantity_base, in base currency per base unit
}

Step-by-Step Implementation #

The pipeline parses a (count, quantity, unit) triple, resolves the unit to a canonical base within its physical dimension, expands multi-packs, and only then divides. Each step is independently testable.

Step 1 — Define the canonical unit map per dimension #

Group every unit you encounter by physical dimension and pick one base unit per dimension. Conversions are exact constants, kept as Decimal-friendly floats with enough precision that price-per-gram does not drift across thousands of SKUs.

# Each unit maps to (base_unit, factor) where 1 unit = factor base_units.
# Dimensions never mix: a mass query can never resolve to a volume base.
UNIT_TABLE = {
    # mass -> grams
    "g":     ("g", 1.0),      "kg": ("g", 1000.0),
    "oz":    ("g", 28.349523), "lb": ("g", 453.59237),
    # volume -> millilitres
    "ml":    ("ml", 1.0),     "l":  ("ml", 1000.0),
    "fl oz": ("ml", 29.573530), "gal": ("ml", 3785.411784),
    "pt":    ("ml", 473.176473), "qt": ("ml", 946.352946),
    # count -> unit
    "ct":    ("unit", 1.0),   "count": ("unit", 1.0),
    "pk":    ("unit", 1.0),   "pack":  ("unit", 1.0),
    "ea":    ("unit", 1.0),   "unit":  ("unit", 1.0),
}

Step 2 — Parse count, quantity and unit from the title #

Titles encode size as N x Q unit (“2 x 16.9 fl oz”), Q unit (“750 ml”), or fractions (“1/2 lb”). A single regex with named groups handles all three; the multiplier defaults to 1 when absent.

import re
from fractions import Fraction

# Longest unit aliases first so "fl oz" wins over "oz".
_UNIT_ALT = "|".join(sorted(UNIT_TABLE, key=len, reverse=True)).replace(" ", r"\s")
_SIZE_RE = re.compile(
    rf"(?:(?P<count>\d+)\s*[x×]\s*)?"          # optional "2 x"
    rf"(?P<qty>\d+(?:[./]\d+)?(?:\.\d+)?)\s*"  # 16.9 or 1/2
    rf"(?P<unit>{_UNIT_ALT})\b",
    re.IGNORECASE,
)

def parse_size(text: str):
    """Return (count, quantity, raw_unit) or None if no size is present."""
    if not text:
        return None
    m = _SIZE_RE.search(text)
    if not m:
        return None
    count = int(m.group("count")) if m.group("count") else 1
    raw_qty = m.group("qty")
    qty = float(Fraction(raw_qty)) if "/" in raw_qty else float(raw_qty)
    return count, qty, m.group("unit").lower().strip()

Step 3 — Resolve to the canonical base and expand the pack #

Reject unknown units rather than guessing — a silent fallthrough is how a price-per-gram ends up averaged with a price-per-count. Prefer the structured size_attr when present; fall back to the title only when it is missing.

def to_base(record: dict):
    """Attach base_unit and quantity_base, or mark the record unresolved."""
    parsed = parse_size(record.get("size_attr")) or parse_size(record.get("title"))
    if parsed is None:
        return {**record, "base_unit": None, "quantity_base": None,
                "unit_price": None, "uom_status": "no_size_found"}
    count, qty, unit = parsed
    if unit not in UNIT_TABLE:
        return {**record, "base_unit": None, "quantity_base": None,
                "unit_price": None, "uom_status": f"unknown_unit:{unit}"}
    base_unit, factor = UNIT_TABLE[unit]
    quantity_base = count * qty * factor          # total base units in the pack
    return {**record, "base_unit": base_unit, "quantity_base": quantity_base,
            "unit_price": None, "uom_status": "ok"}

Step 4 — Compute the comparable unit price #

Division happens last, guarding against a zero or missing denominator so a malformed size never produces inf.

def attach_unit_price(record: dict) -> dict:
    q = record.get("quantity_base")
    if not q or q <= 0:
        return {**record, "unit_price": None,
                "uom_status": record.get("uom_status", "bad_quantity")}
    return {**record, "unit_price": round(record["normalized_price"] / q, 6)}

# Worked example
row = {"sku_id": "p-44182", "marketplace_id": "amazon-us",
       "normalized_price": 7.49,
       "title": "Acme Olive Oil Extra Virgin 2 x 16.9 fl oz", "size_attr": None}
print(attach_unit_price(to_base(row)))
# {... 'base_unit': 'ml', 'quantity_base': 999.7853..., 'unit_price': 0.007491,
#  'uom_status': 'ok'}

Step 5 — Compare only within a dimension and a readable scale #

A raw price-per-millilitre is unreadable on a dashboard, and two products are only comparable when their base_unit agrees. Group by (sku_id, base_unit) before ranking, and present a per-100 scale.

import pandas as pd

DISPLAY_SCALE = {"g": 100, "ml": 100, "unit": 1}  # price per 100 g / 100 ml / each

def cheapest_per_sku(rows: list[dict]) -> pd.DataFrame:
    df = pd.DataFrame(rows)
    df = df[df["uom_status"].eq("ok") & df["unit_price"].notna()].copy()
    df["display_price"] = df.apply(
        lambda r: round(r["unit_price"] * DISPLAY_SCALE[r["base_unit"]], 4), axis=1)
    # Rank within each product AND its base unit, never across dimensions.
    df["rank"] = df.groupby(["sku_id", "base_unit"])["unit_price"].rank(method="min")
    return df.sort_values(["sku_id", "rank"])

Recommended parameters by category #

Quantity-parsing tolerances differ by category. Tighten where titles are clean and structured; relax where merchandising language dominates.

Verification & Testing #

Unit-price logic fails silently, so assert against hand-computed values and against an independent physical constant. These run under pytest with no fixtures.

import math

def test_multipack_volume():
    row = {"normalized_price": 7.49, "size_attr": None,
           "title": "Acme Olive Oil 2 x 16.9 fl oz"}
    out = attach_unit_price(to_base(row))
    assert out["base_unit"] == "ml"
    assert math.isclose(out["quantity_base"], 2 * 16.9 * 29.573530, rel_tol=1e-9)

def test_fraction_mass():
    out = to_base({"title": "Deli Ham 1/2 lb", "size_attr": None})
    assert math.isclose(out["quantity_base"], 0.5 * 453.59237, rel_tol=1e-9)

def test_known_constant_gallon():
    # Independent check: 1 US gallon must round-trip to 3785.41 ml.
    out = to_base({"title": "Spring Water 1 gal", "size_attr": None})
    assert round(out["quantity_base"], 2) == 3785.41

def test_unknown_unit_is_quarantined_not_guessed():
    out = to_base({"title": "Mystery Bundle family size", "size_attr": None})
    assert out["uom_status"] == "no_size_found"
    assert out["unit_price"] is None

def test_dimensions_never_cross():
    rows = [attach_unit_price(to_base(r)) for r in [
        {"normalized_price": 3.0, "title": "Juice 500 ml", "size_attr": None},
        {"normalized_price": 3.0, "title": "Sugar 500 g", "size_attr": None}]]
    df = cheapest_per_sku([{**r, "sku_id": "x", "marketplace_id": m}
                           for r, m in zip(rows, ["a", "b"])])
    # Two different base units => no single rank-1 across dimensions.
    assert set(df["base_unit"]) == {"ml", "g"}

Run with pytest -q; all five must pass before the stage feeds a dashboard. Treat any change to UNIT_TABLE as a schema change and re-run test_known_constant_gallon to catch precision regressions.

Edge Cases & Gotchas #

• Ambiguous “oz” (mass vs. fluid). “16 oz” of nuts is mass; “16 oz” of soda is volume, but the title may omit “fl”. Resolve using the product category from catalog matching, not the title alone — default a beverage/liquid category’s bare oz to fl oz, and log the assumption so an audit can find it.
• “Family size” and other non-quantitative sizes. Merchandising words carry no number. parse_size correctly returns None; route these to a no_size_found quarantine for manual sizing rather than imputing a guess that would corrupt the per-unit average.
• Pack-of-pack nesting. “Case of 4, 6 x 330 ml” implies 24 cans. The single-regex parser captures only the first N x Q; detect a second multiplier (case of N, N-pack) and multiply it into count before Step 3, or the per-unit price reads 4× too high.
• Serving size vs. container size for supplements. A bottle may list “60 ct” and “30 servings”. Always denominate on the physical count (ct), never the serving claim — servings are a marketing-defined unit and break cross-marketplace comparability.

Performance Notes #

Parsing is O(n) in rows and dominated by the regex; on commodity hardware parse_size clears ~150–250k titles/second single-threaded, so a million-row feed normalizes in seconds and the stage is never the bottleneck — the upstream scrape is. Keep UNIT_TABLE as a module-level dict so the compiled _SIZE_RE and the alias list build once at import, not per call. For very large feeds, vectorize Step 5 with groupby (already O(n log n) on the sort) rather than per-row Python loops, and push the (sku_id, base_unit) grouping into the same pass that computes statistical outlier detection for price data so the comparable axis and the z-score share one scan. When titles get genuinely adversarial — free-text bundles, localized unit words — a regex stops scaling and the next step is a small trained sequence tagger, but that is rarely justified before the regex covers ~98% of a category cleanly.

Frequently Asked Questions #

Why divide by a canonical base unit instead of just comparing shelf prices? Because shelf price conflates price and pack size. A larger container almost always shows a higher absolute price yet a lower unit price; without the per-base-unit axis, a competitor-intelligence feed systematically mistakes “bigger pack” for “more expensive” and fires wrong repricing signals.

Should I ever convert between mass and volume to compare two listings? No. Mass and volume are different physical dimensions and the conversion depends on a product-specific density you do not reliably have. Keep them in separate base units and only rank within a dimension. Crossing dimensions is the most common way unit-price tables produce nonsense.

What do I do with rows where no size can be parsed? Tag them no_size_found and exclude them from per-unit aggregation — never impute. An unresolved size is a data-quality task (improve the parser or source a structured size_attr), not a value to invent, and silently guessing poisons every average it touches.

Related #

• Statistical Outlier Detection for Price Data — the parent guide; the comparable unit price produced here is the value its robust estimators score.
• Filtering Fake Sale Prices Using Historical Averages — a sibling check that baselines the same normalized_price over time once it is unit-comparable.
• Tax and Shipping Cost Normalization Rules — the upstream stage that strips fees so the numerator reflects goods alone before this division.
• Converting Multi-Currency Prices to a Base Currency — aligns the currency axis so unit prices are comparable across regional marketplaces.
• Data Normalization & Promo Parsing Pipelines — the end-to-end normalization sequence this UOM stage plugs into.