Price Monitoring Competitor Intelligence Workflows

Estimating Price Elasticity from Sales History

You have a few years of weekly price and units for one product and a single question: how much do sales move when the price moves? This is a focused, runnable recipe for estimating own-price elasticity from that history in Python — building a clean panel, fitting a log-log ordinary least squares model with statsmodels, reading the elasticity coefficient and its confidence interval, and adding the two controls that keep the estimate honest. It sits under price elasticity of demand modeling, which covers the wider theory, endogeneity, and how the number turns into a price. The competitor series used as a control here comes from building a competitive price index.

Prerequisites & Input Contract

The estimator consumes a per-period sales panel for one product. Prices must already be in a single base currency and cleaned of fake-sale artifacts upstream; this recipe only fits a model. Library versions: Python 3.11+, pandas>=2.1, numpy>=1.26, and statsmodels>=0.14. Install with pip install pandas numpy statsmodels.

# One row per period for a single product.
# columns (all required except competitor_price, which may be None):
#   period            ISO week label, the time index
#   units             int, units sold in the period (> 0)
#   price             float, own net price in base currency
#   competitor_price  float | None, matched rival price
#   on_promo          bool, any active promotion in the period
sample = {
    "period": "2025-W07",
    "units": 318,
    "price": 24.99,
    "competitor_price": 26.49,
    "on_promo": False,
}

You need roughly 30+ periods in which the price genuinely varied across at least a 10–15% range. A price that never moved carries zero information about elasticity, and the model below will return a wide, useless interval if you feed it a flat price history.

Step-by-Step Implementation

Step 1 — Build and sanity-check the panel

Load the rows, drop zero-unit and stockout periods before any log transform (a log of zero is undefined and a stockout is not weak demand), and confirm there is real price variation to exploit.

import numpy as np
import pandas as pd

def build_panel(rows: list[dict]) -> pd.DataFrame:
    df = pd.DataFrame(rows)
    df = df[df["units"] > 0].copy()                 # log(units) needs units > 0
    # Neutral-fill a missing competitor price with own price so the control
    # contributes nothing rather than dropping the row.
    df["competitor_price"] = df["competitor_price"].fillna(df["price"])
    # Guard: elasticity is unidentified without price movement.
    spread = df["price"].max() / df["price"].min() - 1
    if spread < 0.10:
        raise ValueError(f"insufficient price variation: {spread:.1%} range")
    return df

panel = build_panel(rows)          # rows = your list of period dicts
print(len(panel), "usable periods;",
      f"price range {panel.price.min():.2f}-{panel.price.max():.2f}")
# 84 usable periods; price range 19.99-29.99

Step 2 — Fit the log-log model with statsmodels

Regress log(units) on log(price). Because both sides are logged, the price coefficient is the elasticity — no rescaling. Add a promotion dummy and the logged competitor price as controls, and use heteroskedasticity-robust standard errors so noisy weeks do not distort the interval.

import statsmodels.formula.api as smf

model = smf.ols(
    "np.log(units) ~ np.log(price) + np.log(competitor_price) + C(on_promo)",
    data=panel,
).fit(cov_type="HC3")   # HC3 = robust SEs, safe default for small panels

Step 3 — Read the elasticity coefficient and its confidence interval

Pull the log(price) coefficient — that is own-price elasticity — together with its 95% interval. Report them as a pair, never the point estimate alone.

beta = model.params["np.log(price)"]
ci_low, ci_high = model.conf_int().loc["np.log(price)"]
cross = model.params["np.log(competitor_price)"]   # cross-price elasticity

print(f"own-price elasticity : {beta:.2f}  95% CI [{ci_low:.2f}, {ci_high:.2f}]")
print(f"cross-price (rival)  : {cross:+.2f}")
print(f"R-squared            : {model.rsquared:.3f}")
# own-price elasticity : -1.68  95% CI [-1.97, -1.39]
# cross-price (rival)  : +0.54
# R-squared            : 0.712

Interpretation: -1.68 means a 1% price rise costs about 1.68% of units — the product is elastic, so cutting price grows revenue. The +0.54 cross term has the expected sign for a substitute: when the rival raises price, your units rise. The interval [-1.97, -1.39] is comfortably negative and narrow enough to act on.

Step 4 — Wrap it into a single reusable function 

def estimate_elasticity(rows: list[dict]) -> dict:
    panel = build_panel(rows)
    m = smf.ols(
        "np.log(units) ~ np.log(price) + np.log(competitor_price) + C(on_promo)",
        data=panel,
    ).fit(cov_type="HC3")
    lo, hi = m.conf_int().loc["np.log(price)"]
    return {
        "elasticity": round(m.params["np.log(price)"], 3),
        "ci_95": [round(lo, 3), round(hi, 3)],
        "cross_price": round(m.params["np.log(competitor_price)"], 3),
        "r_squared": round(m.rsquared, 3),
        "n_periods": int(m.nobs),
    }

print(estimate_elasticity(rows))
# {'elasticity': -1.68, 'ci_95': [-1.97, -1.39],
#  'cross_price': 0.54, 'r_squared': 0.712, 'n_periods': 84}

The relationship the fit captures is a straight line in log space, and its slope is the elasticity you just read:

Log-log scatter of units against price with a downward-sloping fitted elasticity lineA scatter plot with the natural log of price on the horizontal axis and the natural log of units on the vertical axis. Nine observation points fall along a downward-sloping band from upper left, high units at low price, to lower right, low units at high price. A straight fitted regression line runs through the cloud; its slope is the elasticity, labelled beta-hat equals minus 1.68. Because both axes are logged, the constant slope of the line is the constant elasticity of demand.ln(price) →ln(units) →slope = elasticityβ̂ = −1.68

Verification & Testing

Four checks separate a usable estimate from a spurious one:

def verify(result: dict) -> None:
    e, (lo, hi) = result["elasticity"], result["ci_95"]
    assert e < 0, f"positive elasticity {e}: confounder uncontrolled"      # sign
    assert hi < 0, f"CI crosses zero {result['ci_95']}: too noisy to use"  # significance
    assert (hi - lo) < 0.8, "CI too wide: need more price variation"       # precision
    assert result["r_squared"] > 0.3, "weak fit: missing controls?"       # fit
    print(f"elasticity {e} is usable")

verify(estimate_elasticity(rows))
# elasticity -1.68 is usable

Beyond the asserts, do two magnitude sanity checks by eye. First, the sign must be negative — a positive coefficient is almost always a stockout or missed promotion, not a real Veblen effect. Second, the magnitude should land inside the plausible range for the category (grocery staples near −0.5, consumer electronics nearer −2.0); a fitted -6.3 is a data problem, not a demand curve. An $R^2$ below about 0.3 usually means a control is missing rather than that demand is unpredictable.

Edge Cases & Gotchas

  • Too little price variation. If the price barely moved, the log(price) coefficient is estimated off almost nothing and the interval explodes. The build_panel guard raises early on a <10% range; when it fires, pool the product with its category rather than trusting a single-SKU fit.
  • Promo confounding. If the price only ever dropped during promotions, log(price) and the promo dummy are collinear and both coefficients swing wildly between refits. Confirm there is non-promotional price movement; if not, the promo effect and price effect are genuinely inseparable in this data.
  • Stockouts read as low demand. A period where the item was out of stock shows depressed units at whatever price was listed, biasing elasticity toward inelastic. Filter stockout periods with an availability flag before Step 1, not after.
  • Endogenous pricing. If a manager raised price precisely in high-demand weeks, OLS biases elasticity toward zero. Treat a single-equation estimate as a lower bound on sensitivity and read the parent guide on price elasticity of demand modeling for instruments and price tests.

Performance Notes

Fitting one product is trivial — a single OLS on a few hundred rows returns in low single-digit milliseconds. The cost appears at catalog scale, when you fit thousands of products on a schedule. Keep each fit independent and embarrassingly parallel: dispatch products across a process pool or a task queue rather than looping in one interpreter, since statsmodels releases little that helps a threaded loop. Cache the fitted coefficient and CI keyed by a hash of the input panel so an unchanged history is never refit, and only re-estimate on a rolling window (for example the trailing 78 weeks) when new periods arrive. That turns a nightly all-catalog refit into work proportional to the products that actually sold that week.

Frequently Asked Questions

Why use a log-log model instead of dividing percentage change in units by percentage change in price? The naive ratio attributes every demand swing to price, including swings caused by promotions, seasonality, or a competitor’s move. The log-log regression holds those controls fixed, so the price coefficient measures price alone, and it reports a confidence interval that tells you whether the estimate is trustworthy at all.

What if my elasticity comes out positive? A positive own-price elasticity almost never means a genuine luxury effect. It signals an uncontrolled confounder, usually a stockout period or a promotion your flag missed, where high price and low units coincide for reasons unrelated to demand response. Reject the estimate, filter those periods, and refit rather than repricing on it.

How many periods of history do I need? Around 30 or more periods in which the price actually varied across at least a 10 to 15 percent range. Fewer periods, or a price that barely moved, produce a wide confidence interval that the verification step will correctly reject; pool sparse products into a category-level model instead.