The gold test looked strongest on Tuesday. Why stop the record there?

A live dashboard changes the temptation

A gold research dashboard may refresh after every new observation. That is useful for checking whether the data pipeline works. It also offers repeated opportunities to declare a model successful at whichever moment its record looks best. The stopping date can become an unreported choice even when the strategy code itself never changes.

My concern is the evidence claim attached to that endpoint. A researcher who planned to evaluate a full month and then announces victory after a favorable Tuesday has changed the procedure. The remaining days are no longer allowed to influence the headline. This article examines that choice; it reports no CyGuru strategy result and proposes no target sample size for a real trading system.

Why repeated inspection is not a neutral detail

Johari, Pekelis and Walsh study continuous monitoring in the context of A/B experiments. Their research shows why choosing sample size from repeatedly inspected conventional p-values can invalidate the intended inference, and develops methods designed for sequential use. A gold strategy study is a different application, so the paper's guarantees cannot simply be transferred to dependent market returns.

The relevant lesson is procedural. The researcher must state not only what statistic is calculated, but when it may justify a conclusion. Reading a dashboard is not itself forbidden. Treating every refresh as another unadjusted opportunity to reject a null hypothesis is the problem. A visually impressive moment and a predeclared evaluation endpoint are different pieces of evidence.

Source notes: [1] Johari, Pekelis and Walsh — Always Valid Inference

A hypothetical endpoint-selection exercise

Imagine an invented research record with cumulative average net results of +0.30 units after 20 observations, +0.08 after 40 and −0.02 after 60. These numbers are illustrative, not gold returns. A report ending at 20 could sound persuasive; a report ending at 60 would describe a different picture. Neither number becomes false merely because another endpoint exists. The issue is whether the endpoint was selected after seeing the path.

I would retain all three dated snapshots and the original plan. If the plan specified 60 resolved observations, the first two snapshots are interim descriptions. If an operational incident forced an early stop, the incident and its timing belong in the report. The author should not retrofit a claim that the first milestone was always the intended test.

A workable separation between monitoring and judgment

For a simple evaluation, I would write down a collection window, eligibility rules, outcome deadline, main metric and reporting date before collecting the prospective record. During collection, a separate monitor would check missing data, duplicate records, clock errors and changes in the feed. These checks protect the integrity of the experiment without turning each favorable metric into a publication trigger.

Any suspension for operational reasons should be recorded with the information available at the time. Afterward, I would distinguish the completed planned analysis from an interrupted study. Extending the window solely because the result disappoints is another change requiring disclosure. A fixed endpoint does not solve every research problem, but it makes one important discretionary choice visible.

Sequential methods are an alternative with assumptions

Howard, Ramdas, McAuliffe and Sekhon develop confidence sequences: interval procedures with coverage designed to hold across a time horizon under specified conditions. This is a formal alternative to treating an ordinary fixed-sample interval as valid at every inspection. It does not mean that any interval drawn on a live dashboard automatically has that property.

For market research, I would require the implementation to identify its assumptions about dependence, tail behavior, outcome bounds and the quantity being estimated. Costs may vary, signals can overlap, and the market environment may change. Those features matter to whether the chosen theorem applies. A sophisticated name is insufficient; the code, assumptions and validation record must describe the same procedure.

Source notes: [2] Howard, Ramdas, McAuliffe and Sekhon — Time-uniform confidence sequences

Publish the decision path with the conclusion

My proposed publication record includes the original stopping rule, every authorized amendment, the reason for each amendment and the complete endpoint history. An exploratory finding can remain useful if labeled exploratory. A later prospective evaluation can then test the idea under a fresh plan. That separation gives a promising observation somewhere to go without promoting it prematurely into established evidence.

For a public terminal, frequent updates and disciplined research can coexist. The live view shows what has been received; the research report explains what the accumulated record permits us to conclude. A selected screenshot cannot supply the missing stopping rule or the unseen continuation. Before trusting the strongest-looking moment, ask whether the same publication would have appeared if the next observations had been unfavorable.