The Fundamental Question

Gelman, Hill, and Vehtari (2020) put it bluntly:

“Before fitting a model… it is a good idea to understand where your numbers are coming from.”

This is not preliminary work : it is the analysis.

Why Data Understanding Matters

In finance, data problems lead to:

• Real losses: Backtests on biased data overestimate returns
• Regulatory failures: Risk models trained on crisis-free periods underestimate tails
• Misallocated capital: Algorithmic systems that leak future information fail in production

The most sophisticated model built on flawed data produces flawed results : often with false confidence.

What Is a DGP?

Every dataset is the output of some data generating process : the mechanism that determines:

• What gets recorded
• When it gets recorded
• How it gets recorded

Understanding the DGP reveals assumptions embedded in your data before you add modelling assumptions.

Stock Prices: Not Just “The Market”

The prices you observe are the result of:

Each choice affects downstream analysis and embeds theoretical assumptions.

The Trader’s Dilemma: Make or Take?

Every time a trader wants to buy or sell, they face a strategic choice:

Make liquidity (patience) : post a limit order and wait

• “I’ll buy at £99.95” → your order joins the book as a bid
• You get a better price, but you might not get filled
• You are a maker : you add liquidity to the market

Take liquidity (urgency) : hit an existing order now

• “I’ll buy at whatever the cheapest seller is offering” → you pay the ask (£100.05)
• You trade immediately, but you pay the spread as your cost
• You are a taker : you remove liquidity from the market

The spread is the price of this choice. Patient traders earn it; urgent traders pay it. Every “price” in your dataset is the result of someone choosing to take.

First Principles: How Prices Form

Every price you observe in financial data is the outcome of this make-or-take negotiation. The makers set the terms:

• A buyer wants to pay as little as possible → posts a bid: “I’ll buy at most £99.95”
• A seller wants to receive as much as possible → posts an ask: “I’ll sell at no less than £100.05”

These limit orders accumulate in the order book : a live record of supply and demand:

• Bids stack below the current price (demand waiting to be filled)
• Asks stack above the current price (supply waiting to be taken)
• The spread (£100.05 − £99.95 = £0.10) is the gap where no one yet agrees

A taker closes the gap: a buyer pays the ask price, or a seller accepts the bid price. That crossing is the “price” in your dataset.

The Order Book: A Concrete Example

Consider a stock trading near £100. The book at one instant:

Reading this table: Bids are sorted highest-first (most aggressive buyer on top). Asks are sorted lowest-first (most aggressive seller on top). They converge towards the spread : the frontier of disagreement.

Why it matters for data: The “price” you see depends on which price : last trade, bid-ask midpoint, closing auction, VWAP : and each carries different information.

What Traders Actually See: The DOM Ladder

On a trading terminal, the order book appears as a Depth of Market (DOM) ladder : a compact vertical strip with bids on the left and asks on the right, converging at the spread.

The same data as the table : green depth bars grow from the centre showing bid volume, red bars show ask volume. The best bid and best ask are highlighted where they meet the spread.

The Depth Chart: Visualising the Order Book

The depth chart is this same order book rendered visually : demand (bids) extending left, supply (asks) extending right, just like the supply-demand diagrams from economics.

Reading the chart: Bids (green, left) are demand : buyers waiting below the price. Asks (red, right) are supply : sellers waiting above. The deeper the bars, the more volume it takes to move the price through that level.

• Liquidity: Deeper book = lower price impact for large orders
• Spread: The cost of immediacy : to trade now, you must cross it
• Imbalance: More demand than supply (or vice versa) → price pressure

Professional Data Infrastructure

Key principle: Use curated databases, not ad-hoc API calls.

Why this matters:

• Consistency: Same data across all analyses (reproducibility)
• Quality: Pre-validated, survivorship-bias-aware, adjusted for corporate actions
• Efficiency: Load once, analyse many times (vs repeated API calls)

In this course: Bloomberg database contains ~30 securities, 2015-2024, daily adjusted prices.

In labs, you’ll load from this database using a fallback cascade (local file → Colab URL → synthetic).

DGP Example: Adjusted vs Unadjusted Prices

Critical distinction: adjusted prices account for corporate actions (splits, dividends).

The DGP Affects What Questions You Can Answer

Bloomberg database structure:

• Daily close prices (adjusted for splits/dividends)
• Index constituents at point-in-time
• Survivorship-bias-aware (includes delisted securities)

This DGP enables: Long-term return analysis, portfolio backtests, survivorship studies

This DGP prevents: Intraday patterns, microstructure analysis, order flow studies

Measurement: Observed vs Latent

We rarely observe what we truly want to study:

This is the measurement problem in statistical science.

Modern example: Sentiment analysis counts words (“bullish”, “bearish”) but misses context and irony. Transformer models (BERT, GPT) help by capturing semantic meaning.

The Fox Paradox: When Ratios Mislead

Problem: Accounting ratios (ROA, ROE) treat all firms as homogeneous, ignoring scale.

Example: Two firms, both with 10% ROA:

The paradox: Simple average ROA = 10%, but portfolio ROA = 10% (same here by construction). The issue arises when ROAs differ: small firms get equal weight in the average despite contributing trivially to total profit.

Implication for measurement: Ratios ignore economic significance. Better: use size-weighted metrics or separate analysis by scale.

Classical Measurement Error Model

When proxy \(x\) measures true variable \(x^*\) with noise:

\[x = x^* + u\]

where \(u \perp x^*\) (classical error assumption).

Consequence in regression (\(y = \alpha + \beta x^* + \varepsilon\)):

\[\hat{\beta}_{OLS} = \beta \cdot \underbrace{\frac{\text{Var}(x^*)}{\text{Var}(x^*) + \text{Var}(u)}}_{\text{signal-to-total ratio}}\]

Since \(0 < \text{Var}(x^*) / [\text{Var}(x^*) + \text{Var}(u)] < 1\), we have \(|\hat{\beta}_{OLS}| < |\beta|\).

This is attenuation bias : measurement error shrinks coefficients toward zero.

Attenuation Bias: Simulation

Validity vs Reliability

Validity: Does your measure capture the construct you intend to study?

Reliability: Are your measurements consistent and reproducible?

Both matter, but validity > reliability : a reliable but invalid measure is consistently wrong.

Example: High-frequency volatility estimates are highly reliable (consistent) but may not be valid measures of “risk” (depends on your risk concept).

Reliability in Financial Data

Reliability question: Are findings robust to reasonable variations in measurement?

Example metric: Sharpe ratio = (Return - Risk-free rate) / Volatility

This measures risk-adjusted performance : higher is better. But is it reliable across time periods?

The HDI Lesson: Composite Measures Can Mislead

Human Development Index (HDI): UN measure combining life expectancy, education, and income.

Sounds comprehensive, but Gelman, Hill, and Vehtari (2020) show:

“The map is pretty much a map of state income with a mysterious transformation and a catchy name.”

Lesson: Most HDI variation comes from income alone : other components add little information.

Finance parallels:

• “Smart beta” funds may be mostly value or momentum tilts with marketing labels
• “Alternative data” signals may proxy publicly available information
• “Risk-adjusted returns” depend entirely on how you define risk (volatility? downside? beta?)

Always ask: What is my variable actually measuring?

Selection Bias: The Silent Killer

Definition: Your sample differs systematically from the population you want to study.

Four common forms in finance:

Survivorship Bias

Databases of currently listed stocks exclude companies that:

• Failed (bankruptcy, liquidation)
• Were acquired (merger, takeover)
• Delisted (voluntary or involuntary)

Result: The worst performers are removed → upward bias in measured returns.

Magnitude from research: Academic studies find survivorship bias of ~0.5-1.5% per year in US mutual funds; crisis periods show much larger biases (5-10%+ in UK banking 2008).

Survivorship Bias: Simulation

Simulation setup: 100 funds, 5 years monthly returns (mean 0.5%, volatility 4% monthly). Funds with cumulative loss > 50% “fail” and exit the database.

Look-Ahead Bias: The Insidious Form

The most dangerous form in backtesting : using information not available at the time.

Why each type creates bias:

• Point-in-time failures: Restated earnings incorporate information revealed after trading date → you “know” future information when making past decisions
• Index membership: Announcement of S&P 500 addition causes price jump → using post-announcement prices for pre-announcement period inflates returns
• Universe selection: “I’ll study high-momentum stocks in 2020” → selecting on outcome creates cherry-picking → phantom predictability

All three share common flaw: Decision at time t uses information from time t+1 or later.

Look-Ahead Bias: Example

Simulation: 100 stocks, 10 years monthly (120 periods), each with random returns (mean 0.05%, vol 2% monthly). We select top 10 performers.

Wrong approach: Select top 10 over full 10 years, then evaluate performance in second half → uses future information.

Right approach: Select top 10 over first 5 years only, then evaluate in second half → only uses past information.

Connection to Statistical Science: Generalisation

Selection bias violates Gelman’s Challenge 1 (Sample → Population):

• Your sample: survivors, available data, reported outcomes
• Your population: all entities that existed, including failures

Consequence: Inferences don’t generalise from sample to population.

Solution: Survivorship-bias-free databases, point-in-time datasets, temporal validation.

Professional tools: Bloomberg maintains point-in-time databases that record:

• Which securities existed at each date (including now-delisted)
• Index memberships as of specific dates (S&P 500 constituents in 2010, not today’s)
• As-originally-reported financials (not restated)

This enables look-ahead-free backtesting.

Step 1: First Look at Your Data

Always start with basic inspection:

Step 2: Check for Missing Data

Why check: Missing data patterns reveal systematic issues (delistings, thin trading, data vendor problems).

Step 3: Calculate and Examine Returns

Time-Varying Correlations: The Crisis Effect

Static correlations hide non-stationarity: Correlations increase during crises (when you need diversification most).

Rolling correlation: Calculate correlation over moving window (e.g., 60 trading days).

Stylised fact: Crisis correlations : correlations spike during market downturns.

Portfolio implication: Using historical average correlation for risk management underestimates tail risk.

Checking Fact 1: Weak Return Autocorrelation

ACF (Autocorrelation Function): Measures correlation between a series and its lagged values.

• Blue shaded region: 95% confidence band for “no correlation”
• Bars within band: No significant autocorrelation at that lag
• Bars outside band: Significant autocorrelation (predictability)

For returns: Most lags should be within band (unpredictable).

Checking Fact 2: Volatility Clustering

Key insight: Returns unpredictable, but volatility (squared returns) shows persistence.

Checking Fact 3: Fat Tails

Checking Fact 4: Negative Skewness

Negative skewness: Left tail (losses) extends further than right tail (gains).

Interpretation: Extreme losses are more likely than extreme gains (crashes vs rallies).

Why Stylised Facts Matter for Modelling

These facts constrain sensible model specifications:

Simple linear models assuming i.i.d. normal errors will be severely misspecified.

Next weeks: We’ll cover AR models (Week 3) and GARCH models (Week 4) that properly handle these properties.

Common Data Quality Issues

Even Bloomberg data can have issues:

Price errors:

• Decimal point mistakes (rare in Bloomberg, common in free sources)
• Stale prices (unchanged for many days during thin trading)
• Missing corporate action adjustments

Timing errors:

• Timestamp misalignments across sources
• Weekend/holiday prices (should be excluded)
• Look-ahead bias from restated data

Validation Workflow in Practice

Missing Data Patterns: MCAR, MAR, MNAR

Not all missing data is equal:

In finance, missing data is rarely MCAR:

• Failed funds stop reporting (MNAR: failure → missing)
• Illiquid assets have gaps (MNAR: low liquidity → missing)
• Voluntary disclosure is strategic (MNAR: poor performance → missing)

Key question: Why is it missing? The mechanism matters.

Data Problems Are Uncertainty Problems

Recall from Week 1: statistical science is the study of variation and uncertainty.

Data quality issues introduce systematic uncertainty:

• Survivorship bias → overestimate expected returns (upward bias)
• Look-ahead bias → overestimate predictability (spurious patterns)
• Measurement error → underestimate relationships (attenuation bias)
• Missing data (MNAR) → biased sample statistics

These aren’t random noise : they’re systematic biases that invalidate inference.

The Three Challenges: Data Quality Edition

Gelman, Hill, and Vehtari (2020)’s three fundamental challenges apply directly to data work:

Challenge 1: Generalisation (Sample → Population)

• Question: Is your sample representative of the population?
• Data threat: Survivorship bias, availability bias
• Example: Studying “bank performance” using only current survivors misses failures
• Solution: Survivorship-bias-free databases, careful sample selection

Challenge 2: Comparison (Treatment → Control)

Challenge 2: Causal Inference

• Question: Can you attribute effects to specific causes?
• Data threat: Look-ahead bias, confounding from DGP
• Example: Using restated earnings to predict stock returns (data includes post-announcement info)
• Solution: Point-in-time datasets, temporal validation

Look-ahead bias is a form of post-treatment bias: using information generated after the “treatment” (investment decision) to evaluate outcomes.

Challenge 3: Measurement (Proxy → Construct)

Challenge 3: Validity

• Question: Does your measurement capture what you intend to study?
• Data threat: Proxy variables, latent constructs
• Example: Using historical volatility to measure “risk” (backward-looking, regime-dependent)
• Solution: Multiple measures, validation studies, acknowledge limitations

Attenuation bias: measurement error systematically underestimates true relationships.

Quantifying Data Processing Uncertainty

Question: How sensitive are conclusions to data cleaning choices?

Sensitivity analysis: Test multiple reasonable outlier treatments, compare results.

Connection to Assessment

In your coursework, you will:

• Load and validate financial data (CW1, CW2)
• Document data provenance and quality checks (all assignments)
• Handle missing data and outliers (all assignments)
• Avoid survivorship and look-ahead bias (CW2 backtests)
• Justify measurement choices (all assignments)

Quality of data work directly affects quality of inference.

Lab Structure: Dual-Track Approach

Track 1: Colab/Home (Lab 02 APIs)

• Practice API calls with fallback strategies
• Implement data validation pipeline
• Explore stylised facts with free data
• Build reproducible workflows

Track 2: Bloomberg Terminal (Lab 02 Survivorship Bias)

• Extract data from Bloomberg using Excel formulas
• Quantify survivorship bias with real UK banking crisis data
• Compare survivors vs full population returns
• Professional data workflow experience

Directed Learning

Core readings (course textbook):

• Chapter 02: Data and Measurement in Finance : covers DGP, selection bias, validity
  • https://quinfer.github.io/financial-data-science/chapters/02_data_measurement.html
• Complete Lab 02 APIs (homework version) : hands-on validation pipeline
  • https://quinfer.github.io/financial-data-science/labs/lab02_apis.html
• Prepare for Bloomberg session (if applicable) : review Excel formulas

Extension (recommended):

• Chapter 03: Time Series & Volatility : extends stylised facts coverage
  • https://quinfer.github.io/financial-data-science/chapters/03_volatility_modelling.html
• Write brief reflection: “What data quality issues affect my coursework area?”
• Implement validation pipeline for your own data

Further reading (if interested): Gelman & Hill (2020) Ch 2; Tsay (2010) Ch 1