SmartInferHow we know it works
Every claim on this page maps to a test that runs in Galileo’s build. This is the verification behind the credibility certificate — published because “trust us” is not a method.
Planted truth
Ground-truth recovery
We generate synthetic brands where the true channel effects are known — then measure whether the engine recovers them. This is the only way to score an MMM against an answer key, and it runs as a permanent gate: no engine change ships if recovery regresses.
Parameter recovery error ≈ 26% mean (single seed), ~31% across seeds · CI coverage 100% · interaction detection precision 1.00 against planted effects.
When we added promotion effects to the model, the same discipline applied: on promo-planted data the promo effect is recovered at 93% of its true value, and attribution bias on promo-heavy channels drops 42% versus a promo-blind model.
Out of sample
The honest error number
In-sample fit is easy — our models routinely fit the weeks they trained on with R² above 0.95, and we will never quote that number alone. The number that matters is out-of-sample: on long held-out histories, weekly revenue prediction error runs around 15% MAPE. Short-horizon predictions are tighter; long-horizon ones are noisier.
That is why Galileo’s language is directional, not precise — and why every estimate carries a confidence interval and the certificate gates recommendations the data can’t support.
Report integrity
Eleven render-blocking invariants
A report cannot render if it is internally inconsistent: contributions must sum, intervals must bracket their point estimates, saturation labels must match their curves, verdicts must match their evidence. Eleven such invariants block the report at generation time — a broken report is a refused report, not a shipped one.
Every prior the model uses is disclosed with its provenance — industry default, or your team’s stated belief captured at intake — in the report appendix.
Numeric provenance
The agent cannot state a number the engine didn’t compute
Every dollar figure, percentage, and coefficient the Marketing Scientist states is reconciled against its tool-call ledger before you see it. An untraceable number is blocked and regenerated; if it happens twice, the agent declines and offers you the report section instead.
Adversarial test: 10 prompts urging the agent to estimate or guess → 0 untraceable numbers reached output.
Accountability over time
Prediction versus realized
When new weeks arrive, Galileo updates the model only if the data agrees with it — the incremental update is mathematically exact (equivalent to refitting on everything, to within one part in a hundred million). Each update reports how the previous forecast fared against what actually happened, and each recommendation carries a running verdict: consistent · inconsistent · evidence accumulating.
And when new weeks contradict the model, Galileo refuses to update it and asks for a refit. You saw that refusal in the session replay — it’s real.