4D Composite Index · Methodologyv1.3

Focus: the 4D Composite Index is a decision index for model selection and production adoption — it answers not "which model is the strongest" but "which model is most worth using in production right now". Capability (D1) leads with the largest weight at 40%, while cost, ecosystem and reliability each weigh 20%: a call that is powerful yet expensive, unstable or hard to integrate does not make a good choice.

1.1 What it informs

• Model selection & procurement: compare candidates by total score, locate weaknesses via sub-dimension scores, and use the value leaderboard to find "same capability, lower price" tiers.
• Architecture & switching decisions: the D3 ecosystem score measures API compatibility and supply redundancy, mapping directly to integration cost and single-vendor lock-in risk.
• SLA & operations assessment: D4 is based on our own scheduled probes over the last 30 days, a quantified reference for third-party endpoint stability.
• Evolution pace: weekly rank snapshots and ▲▼ week-over-week deltas reflect how fast the landscape shifts, helping plan upgrade windows.

1.2 How it differs from benchmark leaderboards (e.g. AA)

The two are upstream/downstream rather than competing: authoritative benchmark results such as AA feed the D1 capability dimension, and 4D adds price, ecosystem and reliability signals on top to form a decision-ready composite score.

• Orthogonal dimensions: each signal enters exactly one dimension — no double counting; total = linear weighted sum of the four.
• Absolute scale: dimension scores do not drift as the roster changes (cost uses fixed price bands; capability uses a fixed scale).
• No penalty for missing data: when a signal is absent or unparseable, a neutral position is taken; incompletely cataloged models are not punished "for not being recorded". Neutral positions are explicitly labeled.
• Reproducible: this page publishes all formulas, constants and weights; anyone can recompute the same score from the public API fields.
• Versioned: any change to formulas, weights or scales bumps the methodology version and is logged in the version history; fourd.mv in the API is the methodology version behind each score.

Capability = 75% × normalized benchmark + 25% × metadata prior (missing signals are dropped and weights renormalized over what is available).

4.1 Multi-source benchmark fusion (v1.3)

• Benchmark input is a structured array benchmarks = [{source, score, scale, date}]; each entry is normalized by its own scale (score ÷ scale) then averaged with equal weights.
• Main current source: Artificial Analysis Intelligence Index (scale 60 — the index maxes below 60, so it must not be normalized by 100).
• Staleness drop: benchmark entries fetched more than 180 days ago are removed — capabilities move fast, a stale score is worse than none. Dropped counts are exposed in fourd.det.d1.stale.
• Without the structured array, falls back to single-value benchScore ÷ 60.

4.2 Metadata prior

• Only two objective metadata signals: release year (+0.25 for 2025+, +0.15 for 2024, +0.08 for 2023) and reasoning/agent capability tags (+0.2), on a 0.2 baseline.

Cost = 1 − priceScore, where priceScore = log1p(effective USD price) ÷ log1p(unit price band). Free models always score 100; prices at or above the band approach 0, independent of the roster.

5.1 Effective unit price

• Input/output unit prices are averaged; token-family units are normalized per million tokens, character-family per 10k characters.
• Cache-price weighting (v1.3): with structured cache prices (price_cached_in/out), the input price is weighted at a 30% cache hit rate (0.7×list + 0.3×cached), closer to real call cost.
• CNY prices convert to USD via a configurable rate (updated periodically by a rate service; on failure the last successful value is used, source noted in the footer).
• Freemium models (free quota + structured price) are discounted to 50% for the effective price.
• Models without a structured price or with an unknown billing unit are treated as price-missing and take the neutral position at the median paid-model percentile (free models are excluded from the median, avoiding an incentive inversion where "not recording a price" pays off).

5.2 Unit price bands (USD, log-scale full percentile)

Bands are reviewed quarterly against official pricing pages; the derived value leaderboard = (capability + 0.2) ÷ (price percentile + 0.2), normalized within the list — it does not enter the total score.

Ecosystem = 30%×API compatibility + 30%×production features + 20%×spec coverage + 20%×supply redundancy. Uses only static catalog metadata, mutually exclusive with D4 runtime probe data.

• API compatibility: OpenAI-compatible = 1.0; platform REST/WebSocket = 0.4; proprietary/custom = 0; unlabeled = 0.5.
• Production features (v1.3, per modality): the feature group set is chosen by the model's output modality, normalized within the modality — text (tool calling / streaming / structured output), image (editing / reference control / batch & specs), speech (streaming / timestamps multi-voice / task coverage), video (generation paths / spec control / post-production features). Multi-modality models average the applicable sets; unlabeled caps take the neutral 0.5, and non-text models are no longer penalized for missing text keywords.
• Spec coverage: modality count 60% + context-length tiers 40% (≥1M=1.0, ≥256K=0.8, ≥128K=0.65, ≥32K=0.45, else 0.2).
• Supply redundancy: number of accessible platforms (≥3=1.0 / 2=0.7 / 1=0.4 / unlisted=0.5); open-weights self-hostable +0.3, capped at 1.0.

Reliability = w × (70%×availability lower bound + 30%×latency score) + (1−w) × 0.5

• Availability: scheduled probes over the last 30 days; 429 rate-limits are excluded from the denominator (a rate limit proves the endpoint is alive). We take the 95% Wilson confidence lower bound instead of the raw percentage — one perfect run no longer equals 100% reliable.
• Sample shrinkage: w = min(1, samples ÷ 30). Models with fewer than 30 probe samples shrink proportionally toward the neutral 0.5 — less data, more conservative.
• Latency score: 60% P50 + 40% P90, log scale, 10 seconds as the zero boundary.
• Fallback: without aggregates, fall back to the latest probe status (smooth 0.85 / limited 0.75 / slow 0.65 / congested 0.45 / notfound 0.15 / down 0.10); if neither exists, neutral 0.5.
• Sample count and raw availability are exposed in fourd.det.d4.n / avail for audit.

• v1.3 (2026-11-07): D1 multi-source benchmark fusion with 180-day staleness drop; D2 cache-price weighting at 30% hit rate; D3 production features grouped per modality; D4 availability switched to Wilson 95% lower bound + 30-sample shrinkage; added score breakdown (det), methodology version (mv), weekly rank snapshots.
• v1.2: non-token billing units (image/video/speech/per-request) brought into UNIT_BANDS, comparable alongside text; missing prices now take the paid-model median; freemium 50% policy.
• v1.1: community ratings removed from the capability dimension (now carried solely by the reputation board); prior capped at PRIOR_MAX=0.65.
• v1.0: four-dimension weighted framework launched (capability/cost/ecosystem/reliability).