A gold forecast says 70%. What would make that number credible?

The percentage needs a complete sentence

Suppose a research model labels a gold setup with a 70% probability. Before judging that number, I would ask: probability of exactly what, within how long, and using which price record? Reaching a first target before a stop is a different event from finishing above the opening price. A correct direction call can coexist with a failed target-first event. A forecast without these definitions has no stable object to verify.

This article proposes a research audit; it does not claim that the current CyGuru feed supplies calibrated probabilities. An empty confidence field should remain empty unless the underlying model actually emits a defined estimate. Retrospectively attaching a percentage to a successful screenshot would create a forecast after the outcome was known.

Score the probability, not only the direction

For a binary event, one common Brier-loss convention averages (p − y) squared, where p is the probability issued beforehand and y equals one if the defined event occurs and zero otherwise. Smaller values are better. The scikit-learn reference documents this binary convention and its distinction from the unscaled multiclass version. Keeping the convention explicit avoids comparing scores whose scales differ by a factor of two.

A hypothetical forecast of 0.70 incurs a loss of 0.09 if the event happens and 0.49 if it does not. A forecast of 0.55 incurs 0.2025 and 0.3025 respectively. The more emphatic forecast receives a greater reward for being right and a greater penalty for being wrong. Counting both forecasts simply as bullish would discard that information.

Source notes: [1] scikit-learn — brier_score_loss reference

A worked audit with invented observations

Consider four entirely hypothetical forecasts: 0.80, 0.60, 0.40 and 0.20. Suppose their eventual outcomes are 1, 0, 1 and 0. Their squared errors are 0.04, 0.36, 0.36 and 0.04, producing an average loss of 0.20. A constant 0.50 forecast on those same four observations scores 0.25. The arithmetic gives a 0.05 improvement on this invented sample; it establishes no historical edge.

Now inspect what that average conceals. The first and fourth forecasts were useful, while the middle pair were confidently ordered in the wrong direction. Four examples cannot establish stable calibration, a profitable trading rule or robustness across market conditions. My audit would preserve every row so that a summary number cannot replace examination of where the model failed.

Calibration asks whether the labels match experience

Gneiting and Raftery distinguish calibration, the statistical agreement between forecasts and observations, from the concentration of forecasts. Their work on proper scoring rules supplies the methodological foundation; it does not validate any gold model. In this application, repeatedly issuing approximately 0.70 should be examined against the frequency of the precisely defined event among comparable forecasts, with uncertainty visible.

I would decide the grouping boundaries before reading the results, then show the number of resolved observations in each group. A bucket containing six forecasts deserves a different interpretation from one containing six hundred. I would also retain the forecast dates and instrument. A model can appear satisfactory in a pooled report while mislabeling probability during a particular session or volatility environment.

Source notes: [2] Gneiting and Raftery — Strictly Proper Scoring Rules, Prediction, and Estimation

The audit record must exist before the outcome

My proposed record contains a forecast identifier, model version, probability, issue timestamp, event definition, price source, evaluation deadline and resolution rule. If a target and stop can both occur inside one observed candle, the ambiguity rule belongs in that record beforehand. A probability audit must not quietly resolve ambiguous cases in whichever direction improves the score.

I would reserve a later, untouched period for evaluating any recalibration. Fitting a correction to the same observations used to praise its accuracy answers an easier question than forecasting new ones. A change in the event horizon or outcome rule starts a separately labeled series. The earlier forecasts remain available with their original definitions rather than being rewritten into the latest version.

Credible probabilities still need an economic test

A well-described probability is not a complete trade. The size of gains and losses, execution assumptions, costs and the availability of liquidity remain separate inputs. Two strategies can share the same event probability and have very different economic consequences. I would therefore publish probability-quality evidence beside, rather than in place of, any properly documented execution study.

The useful result is a claim that can be checked: what was forecast, when it was frozen, how it resolved and how its score compares with a declared baseline. The hypothetical examples here contain no market prices, broker fills or CY66 outcomes. Research Lab can use this framework to ask whether a future confidence estimate has earned its label before treating it as decision-ready information.