The gold signal mix changed. Did the model weaken—or did the sample change?

Start with three competing explanations

Suppose a gold research dashboard begins to look weaker. The convenient explanation is that the market has entered a new regime. That may be worth investigating, but the same display can arise from several different mechanisms. The model might face a different mixture of familiar setups; the relationship between a setup and its outcome might have changed; or the feed and classification machinery might be recording a different object.

I would keep those explanations separate until the evidence distinguishes them. A software change calls for an engineering investigation. A changed population calls for a comparison that controls for composition. A changed outcome relationship may justify reviewing the model itself. Treating all three as ‘the market changed’ prevents the diagnosis from guiding an appropriate response.

A new mixture can change the aggregate

River’s introduction distinguishes changes in the distribution of input features from changes in the relationship between inputs and targets. It also describes change that arrives abruptly or gradually. For gold research, useful input descriptors might include session, timeframe, direction, engine version and a volatility measure available at signal creation. These are proposed monitoring fields, not evidence of a current regime shift.

Consider a purely hypothetical archive that once contained mostly overnight setups but now contains mostly New York setups. Even if each group’s behavior were unchanged, an aggregate statistic could move because the group weights changed. I would first compare the population shares, then examine like-for-like groups using a consistent outcome definition. This is a diagnostic decomposition, not permission to discard the less flattering group.

Source notes: [1] River — Concept drift

Keep the reference sample still enough to see change

A monitoring system needs a reference period and a comparison period. If both are continuously replaced without a retained history, a slow shift can become difficult to recognize: yesterday’s abnormal conditions become today’s baseline. My preferred record would retain one fixed reference alongside a rolling recent window, with their dates, record counts and missing fields visible.

The fixed reference answers whether the present resembles the population for which the research claim was originally made. The rolling window answers whether the latest observations differ from more recent conditions. Neither comparison proves causation. A holiday, an engine rollout or a data outage can alter the observed mix without identifying a durable change in the economics of gold.

A detector is a question, not a regime label

River’s ADWIN documentation describes an adaptive window divided into subwindows whose averages are compared. Its settings include how often the check runs and the minimum subwindow length; the documentation notes tradeoffs involving detection delay and false positives. An alert from such a procedure concerns the monitored numerical stream under its configuration. It does not identify the economic cause.

I would declare that stream before reviewing the results. Monitoring a volatility input, a missing-data indicator and a forecast-error series asks three different questions. Correlated signals and repeated checks complicate interpretation, so a dashboard should retain the detector version, settings, observation count and alert time. I would not describe a default sensitivity setting as a demonstrated false-alarm rate for this particular gold feed.

Source notes: [2] River — ADWIN

Audit the receiver before blaming the market

Check whether the symbol mapping, timezone conversion, aggregation interval or indicator version changed around the alert. Compare the rate of missing fields and the delay between source and receiver timestamps. A shortened delay can make outcomes appear to resolve faster even if the underlying price process is unchanged. A newly included timeframe can change the sample composition immediately.

Outcome monitoring also needs patience. River’s evaluation design can delay the revelation of labels to reflect their actual availability. In a gold event stream, recently opened cases may still be unresolved while older cases have complete histories. Comparing their apparent completion proportions without accounting for time available to resolve is not evidence of model decay. Preserve unresolved and ambiguous classifications rather than silently converting them into failures or successes.

Source notes: [3] River — progressive validation implementation

Predefine what an alert permits

My response protocol would distinguish investigation, a documented temporary pause, and model replacement. The first may require only a data-quality alert. The last should require a separately evaluated candidate and a dated decision. If a change is approved, keep the original record and start a new version; do not rewrite previous forecasts to make the replacement appear to have been operating all along.

Gold Terminal makes the identity and event trail easier to inspect, while Research Lab supports the wider audit. Selected favorable cases remain selected examples and cannot establish an overall success rate or prove that a regime is stable. This article reports no observed drift or performance deterioration in CY66 or Hazem. It proposes an investigation method; broker execution, costs and P&L remain unverified. Its image illustrates sample composition, not market history.