The gold filter improved its hit rate. What did it leave out?

A better percentage can describe fewer opportunities

A filter that keeps only a small fraction of gold setups can display a higher target-touch percentage while retaining fewer target-reaching examples overall. That is not automatically a failure. It is a trade-off that should be made visible. The relevant question is what the filter was designed to improve and what it gave up to obtain that improvement.

The research literature on selective classification separates accuracy on accepted cases from the proportion of cases accepted. Geifman and El-Yaniv studied this trade-off in image-classification experiments. I borrow the distinction as a reporting framework, not their statistical guarantees or experimental results. Financial signal histories have different dependencies, labels and operating conditions.

Source notes: [1] Geifman and El-Yaniv — Selective Classification for Deep Neural Networks

A deliberately invented baseline

Suppose a hypothetical, fully resolved dataset contains one hundred eligible setups. Under a fixed nominal OHLC rule, sixty touch TP1 before SL and forty touch SL first. A prespecified filter accepts forty setups: thirty TP1-first and ten SL-first. Its accepted-set TP1-first fraction is 75%, compared with 60% for the baseline. Its coverage is 40%.

The rejected sixty contain thirty TP1-first and thirty SL-first observations. The filter retained half of the baseline’s TP1-first examples and excluded three quarters of its SL-first examples. These are different statistics with different denominators. Reporting only 75% conceals the scale of the exclusion. All counts here are invented to illustrate arithmetic; they are not CyGuru performance, an account win rate or a return calculation.

Keep the rejected group in the audit

I would retain one row per eligible setup with its original identifier, eligibility reason, filter decision and outcome status. The baseline must be defined before the filter is applied. Otherwise a researcher can quietly remove difficult observations from both the numerator and denominator and describe the cleaner sample as the original population.

The excluded group should remain inspectable. Does the rule reject an entire time zone, an indicator family or nearly every high-volatility observation? Such a pattern could be intentional, but it changes the scope of the resulting claim. A filter that works only by concentrating on one narrow setting should be described as conditional on that setting, rather than presented as a general improvement.

Unresolved evidence needs a separate count

The invented example assumes all outcomes are resolved under the same rule. A real audit may contain missing candles, an ambiguous bar touching both exits, or setups still open at cutoff. Those observations must not automatically become failures or disappear. Show eligible, accepted, rejected and unresolved counts, then specify which resolved subset supports each fraction.

I would also compare unresolved proportions across accepted and rejected groups. If accepted setups have cleaner data coverage, a better reported fraction might partly reflect different observability. That is an investigation to perform, not an accusation to presume. Preserving the unknown states makes the investigation possible and prevents a confidence score from concealing a data-quality difference.

Choose the threshold before judging it

Scikit-learn’s official guidance warns against using test data to choose a model or fit preprocessing. Applied here, that means choosing the filter threshold on development data and then evaluating the frozen rule on a later, untouched interval. Trying many cutoffs on the reporting sample and publishing only the best accepted-set percentage measures the selection process as well as the filter.

My preferred comparison would publish several predetermined operating points with coverage beside each outcome fraction, while keeping the final evaluation interval fixed. If the filter changes position sizing, exit levels or cooldown state, it is no longer merely selecting from an unchanged baseline. That broader policy needs a separate replay; its outcomes cannot be reconstructed by deleting rows from the old result table.

Source notes: [2] scikit-learn — Common pitfalls and recommended practices

Define the improvement that matters

A research objective might prioritize rejecting uncertain setups, preserving a minimum number of observations, or making an audit easier to reproduce. Those are legitimate but different objectives. None can be reduced automatically to the highest target-touch percentage. Write the objective first and identify a result that would count against the proposed filter.

For CyGuru, this framework belongs in independent Research Lab work alongside the event record visible in the terminal. A nominal OHLC target observation is not a verified fill. Commissions, spread, slippage, financing and P&L are absent from this comparison. No verified production dataset was evaluated here, and no performance improvement is claimed. The useful deliverable is an audit that shows both the selected evidence and the evidence it leaves outside.