# Metrics Every formula the engine computes, with its exact convention. Runbook section 5.5. Two rules hold everywhere in this document: - **Annualization uses 252 trading days.** There is no calendar-day variant anywhere. - **Every dispersion figure uses the sample standard deviation with denominator n-1.** Sharpe, Sortino, volatility, and tracking error all sit on the same denominator, so they stay comparable to each other. Each formula exists in exactly one place in the codebase. The independent check lives in `engine/test/golden/verification.xlsx`, whose cells recompute these figures with live Excel formulas beside the engine's own answers, and in `datapack/tests/test_verification.py`, which recomputes the same figures in Python. Both run against the committed golden fixture. --- ## 1. The return series The backtest produces one daily return for every trading day in the period **after** the first. The first day has no prior NAV, so there is no return for it; a fabricated zero on day one would dilute every ratio below by one observation. ``` r_t = NAV_t / NAV_(t-1) - 1 ``` NAV is an index starting at 1. Every cost and cap in the model is a proportion, so the starting level does not affect any figure. **A bankrupt strategy's series is padded with zeros to the period end, never truncated.** Once NAV reaches zero, every position is closed at that day's close and each remaining trading day carries a return of exactly 0. This is a grading requirement rather than a cosmetic one: under the length-weighted Sharpe of section 8 below, truncating the series would shrink the weight of precisely the period the strategy blew up in and make an early bankruptcy partially self-forgiving. `returns.csv` carries `date, portfolio_return, benchmark_return, risk_free_rate`, one row per return. --- ## 2. Return metrics | Metric | Formula | Notes | |---|---|---| | Cumulative return | `prod(1 + r_t) - 1` | Geometric, over the whole series | | Annualized return | `prod(1 + r_t)^(252/n) - 1` | Geometric. A total loss returns exactly -1 rather than a complex number | | Annualized volatility | `stdev_sample(r) * sqrt(252)` | Denominator n-1 | | Hit rate | `count(r_t > 0) / n` | A day of exactly zero is not a win | | Best and worst rolling 63-day | `max` and `min` of `prod(1 + r) - 1` over every window of 63 consecutive days | Reported as a pair | --- ## 3. The Sharpe ratio **This is the figure that carries grading weight, so it is stated exactly.** ``` excess_t = r_t - rf_t rf_t is the pack risk-free rate for that date Sharpe = mean(excess) / stdev_sample(excess) * sqrt(252) ``` - `mean` is the arithmetic mean, not the geometric one. - `stdev_sample` uses the **n-1** denominator. - `rf_t` is the **daily** risk-free rate from the pack's factors file, which is the daily RF column of the Ken French Data Library. It is a daily rate already and is never de-annualized. - The sample is the exact period requested, using dates present in the pack calendar. - For a bankrupt strategy the input is the zero-padded series, so the day count is the full period whether the strategy survived or not. - A series whose excess returns have no dispersion has no Sharpe ratio. The engine reports it as not-a-number rather than an infinity. One implementation, in `engine/src/metrics.ts`, used by the student app and the audit runner alike. A second implementation anywhere, in any language, is a defect. Using one risk-free series for both the Sharpe ratio and the factor regressions is what makes the two internally consistent. --- ## 4. Risk metrics ### Sortino ratio ``` downside = sqrt( sum( min(r_t, 0)^2 ) / (n - 1) ) Sortino = mean(r) / downside * sqrt(252) ``` The target is **zero daily return**, not the risk-free rate. The numerator is therefore the plain mean of daily returns, not the excess mean. A series with no down days has no downside deviation and no Sortino ratio. ### Maximum drawdown Walked along the NAV path: ``` peak_t = max(NAV_0 .. NAV_t) drawdown_t = NAV_t / peak_t - 1 maxDD = min over t of drawdown_t ``` Reported with the **peak date and the trough date** that produced it, which is what makes it interpretable. The value is negative or zero. ### Calmar ratio ``` Calmar = annualized return / |maximum drawdown| ``` Not a number when there was no drawdown. --- ## 5. Benchmark-relative metrics The benchmark is **SPY adjusted close (total return)**, inherited from the course configuration. It is never a student choice. The beta and correlation *signals* carry their own `reference` parameter, which is a signal input and not the grading benchmark. | Metric | Formula | |---|---| | Beta | Slope of the daily OLS of `r_t` on `b_t` | | Alpha (annualized) | Intercept of that same regression, times 252 | | Tracking error | `stdev_sample(r_t - b_t) * sqrt(252)` | | Information ratio | `mean(r_t - b_t) / stdev_sample(r_t - b_t) * sqrt(252)` | Days on which the benchmark has no observation are dropped from both sides rather than filled. --- ## 6. Turnover and exposure **One-way turnover** at a rebalance is the sum of absolute weight changes over 2: ``` turnover = sum over names of |w_new - w_old| / 2 ``` A book that sells everything and buys a completely different book has a one-way turnover of 1. Weights are NAV fractions measured immediately before and after the execution. **Annualized turnover** scales the realized total by the sample's own length: ``` annualized = sum(turnover at each rebalance) * 252 / n ``` So one full one-way turnover across 126 trading days annualizes to 2. **Exposure series**, recorded daily as fractions of NAV: long, short, cash, and margin usage. Margin usage is the maintenance requirement over NAV. Average long, short, cash and gross exposure are the means of those series; gross is long plus short. Between rebalances, weights **drift with returns and are never silently re-normalized**. --- ## 7. Factor regressions OLS of **daily** excess portfolio returns on the pack's factor series, in two specifications: - **Fama-French three factor**: `mktRf`, `smb`, `hml` - **Carhart four factor**: those three plus `umd` ``` r_t - rf_t = alpha + b1*mktRf_t + b2*smb_t + b3*hml_t [+ b4*umd_t] + e_t ``` Reported: coefficients, **Newey-West standard errors at lag 5**, t-statistics, R-squared, and annualized alpha (daily alpha times 252). The Newey-West covariance is the sandwich ``` Var(b) = (X'X)^-1 * Omega * (X'X)^-1 Omega = S_0 + sum over l = 1..L of w_l * (S_l + S_l') w_l = 1 - l / (L + 1) S_l = sum over t of e_t * e_(t-l) * x_t * x_(t-l)' ``` with `L = 5`. At lag 0 this reduces to the White heteroskedasticity-consistent covariance, which is checked directly in the test suite. The linear algebra is implemented in the engine rather than taken from a statistics library, which is a deliberate constraint: at four regressors plus an intercept the normal equations are well conditioned, and a dependency would add a second place for a convention to live. **Monthly regressions are not offered.** The periods are too short for them to say anything. --- ## 8. Fund metrics (Deliverable 2) A fund combines N component strategies over one period, rebalancing the component weights **monthly** and letting them drift in between. **`lengthWeightedSharpe` is the graded performance figure and exists in exactly one place in the codebase.** ``` lengthWeightedSharpe = sum over periods of (Sharpe_p * tradingDays_p) / sum over periods of tradingDays_p ``` across design, test1 and test2. Length weighting is deliberate and stays. It gives the design period roughly three quarters of the graded figure at the example dates, which is a known and accepted property: the instructor controls the balance between in-sample and out-of-sample by choosing the period lengths at pack build time, not by changing this function. Both graded components use it: baseline quality (equal allocation) and allocation improvement (chosen weights minus baseline). The code reports the **raw difference** and does not floor it, so a team that allocated worse than equal weighting sees that. **A bankrupt component stays in the fund at its allocated weight and carries its real loss.** Its padded return series is already defined to the period end, so the fund arithmetic needs no special case: the component contributes its -100 percent and then zeros, and each monthly rebalance funds it back to its target weight where it earns nothing. It is not dropped, its weight is not redistributed, and it is not excluded from the equal-allocation baseline. Component weights must sum to 1 within 1e-9, sit within [0.05, 0.40], carry no negatives, and cover every team member. The 0.40 ceiling implies teams of three or more. --- ## 9. The cost model **`spreadBps` is the full quoted bid-ask spread, and crossing it costs half of it per side.** At every execution, which means every rebalance trade, every trigger fill, every margin liquidation, every forced exit, and the initial establishment of the book: ``` cost = (commissionBps + spreadBps / 2) / 10000 * |dollar value traded| ``` deducted from cash. **Worked example at the shipped defaults.** Commission 2 bps, quoted spread 8 bps: ``` per execution = 2 + 8/2 = 6 bps round trip = 12 bps ``` Buying $10,000 of a name costs `0.0006 * 10000 = $6`. Selling it later costs another $6. Charging the full spread on each side, as an earlier draft did, doubles the assumed friction and misreads the field label a student is looking at. The form labels the field "quoted bid-ask spread (bps), half charged per side" and shows the resulting per-execution cost live. --- ## 10. Margin Margin classes are read per name at each check from the **split-adjusted close**: | Condition | Initial | Maintenance | |---|---|---| | Close under $10 | 100 percent both sides | 100 percent both sides | | Otherwise, long | 50 percent | 25 percent | | Otherwise, short | 50 percent | 30 percent | - **At each rebalance execution**: the initial requirement on the target book must be at most NAV. If it is not, the long and short books are scaled down proportionally until it binds, and the event is flagged. - **Daily between rebalances**: the maintenance requirement must be at most NAV. On a breach, positions are liquidated proportionally at the **next trading day's close** until the **initial** requirement is restored, costs are charged on the liquidation, and the event is flagged with its date and amount. Restoring only the maintenance level would leave the book one tick from another call. - Positive cash earns the daily risk-free rate. Negative cash pays the risk-free rate plus 150 bps annualized, compounded daily on an actual/252 basis over trading days. - If NAV reaches zero on any day, every position is closed at that day's close and the strategy is marked bankrupt. --- ## 11. Price bases, and which figure uses which | Series | Adjustment | Used by | |---|---|---| | `adj_close` | splits and dividends | All return computation, all return-based signals, position values, NAV | | `close`, `open`, `high`, `low` | splits only | Trade-rule levels and fills, margin classification, market cap, display | | `volume` | splits | Dollar volume | Positions are carried as a number of **adjusted shares**, so their value is that count times `adj_close` and total return accrues automatically, dividends included, with no separate dividend series. A trade-rule fill happens at a quoted split-adjusted level, so the level is converted onto the adjusted basis by that day's own `adj_close / close` factor before it touches a position. Mixing the two bases directly would leak the accumulated dividend adjustment into every trigger fill. **Market cap uses split-restated shares times the split-adjusted close, never `adj_close`.** `adj_close` is reduced by the accumulated dividend adjustment, and that reduction differs across names by how much each has paid out, so using it would understate market cap unevenly across dividend payers and distort every price ratio and every size screen. **The drawdown signal is computed on the split-adjusted close**, not on `adj_close`. Section 5.2 of the runbook names `adj_close` explicitly where it means it (the moving average signal does) and says "price" here, which section 4.3 defines as the split-adjusted series. --- ## 12. Derived fundamental ratios Computed in the engine from the pack's base metrics plus the snapshot-date split-adjusted close. Never stored in the pack, so their definitions live in one place. | Ratio | Definition | Null when | |---|---|---| | market cap | restated shares times split-adjusted close | shares or price missing | | P/E | market cap over net income TTM | net income <= 0 | | P/B | market cap over book equity | book equity <= 0 | | P/S | market cap over revenue TTM | revenue is 0 | | P/CF | market cap over operating cash flow TTM | OCF <= 0 | | ROE | net income TTM over book equity | book equity <= 0 | | ROA | net income TTM over total assets | total assets is 0 | | net margin | net income TTM over revenue TTM | revenue is 0 | | leverage | (total assets minus book equity) over book equity | book equity <= 0 | | revenue growth | revenue TTM over the year-ago revenue TTM, minus 1 | either figure missing, or the year-ago figure <= 0 | A zero denominator produces a null rather than an infinity. That is not a substituted value: it is declining to report a number that does not exist. **Sector is filter-only.** It is categorical, so it has no direction to rank in and no z-score to weight, and the schema rejects it as a ranking criterion. --- ## 13. Ranking and combination **`weightedRank`** converts each signal to fractional ranks in [0,1] with the best at 1, then weights and sums. Ranks are ordinal positions under the ordering (value ascending, then ticker ascending), so a tie takes adjacent ranks in ticker order. **`weightedZScore`** converts each signal to cross-sectional z-scores **winsorized at plus and minus 3**, signs them by direction so a higher score is always better, then weights and sums. A signal with no cross-sectional dispersion contributes zero to every name rather than removing every name from the composite. Signal weights are normalized to sum to 1 at run time; the form shows the normalized values. **A null on any declared signal removes the ticker from the composite for that rebalance.** There is no partial scoring and no renormalization across the surviving signals. A ticker scored on two of three signals would be ranked head to head against fully scored names, which pushes the thinnest-data names systematically toward the extremes of the composite and straight into the long or short book. **Ties break by ticker ascending, plain ASCII, everywhere.** This is a correctness requirement: the student and the instructor must select the same names from the same data. No locale-aware comparison appears anywhere in a data path. --- ## 14. Stated omissions These are echoed into every results export. They all make a backtest look slightly better than reality. - No stock borrow fees on short positions, and no short rebate spread. - No market impact beyond the spread assumption. Fills do not move the price. - No taxes. - No intraday path beyond daily open, high, low and close. - Margin is modeled at the captured schedule of section 10, not against live broker rules.