1. Define the population before selecting examples
Record the system version, symbol, timeframe, eligible signal types and evaluation dates. Every eligible record enters the denominator, including losses, expirations and unresolved cases. Selected case studies can illustrate a question but cannot substitute for the complete population.
2. Audit the dataset
Keep the original file, feed, timezone, bar timestamp convention and retrieval date. Test for duplicate timestamps, non-monotonic order, invalid prices and missing sessions. Distinguish expected market closures from unexplained gaps. Never describe an imported date range as complete three-year coverage without a coverage audit.
3. Align information with decision time
A signal must exist at the time the simulated decision is made. Indicators that use a candle close cannot assume an earlier fill inside that same candle without a justified execution model. Revisions to economic releases and later news must not be introduced into the earlier information set.
4. Specify entry, exit and expiry
Define when a signal becomes actionable, what quote side triggers entry and stop orders, how targets are evaluated and when a proposal expires. A reference price on a dashboard is not evidence of broker execution. Archive updates as well as the initial proposal.
5. Resolve intrabar ambiguity
When a candle contains both stop and target, OHLC alone cannot reveal which occurred first. Use a finer verified record or mark the case unresolved. A conservative ordering can be a separately labeled sensitivity scenario; silently choosing the profitable ordering biases the result.
6. Model costs in the correct units
State spread, slippage, commission, financing and account-currency conversion assumptions. Name the product’s contract size and tick or pip convention. A gold price change per ounce is not an account profit until size and costs are known. Report gross and net results separately.
7. Separate development from evaluation
Choose a time-based holdout that is not used to select parameters. If it influences tuning, it is no longer untouched. Keep the number of tried variants visible; selecting the best of many historical rules can manufacture an apparent edge. Test stability across costs, subperiods and reasonable parameter changes.
8. Report the full outcome distribution
Publish the sample size, reporting period, system version, outcomes, unresolved count, costs and evaluation rules. Define the denominator used for any win rate. Include drawdowns, losing streaks and the distribution of returns; a positive average can conceal materially different risks. Do not pool incomparable versions without disclosure.
9. Distinguish three levels of evidence
A screenshot observation documents what is visible in a capture. A rule-evaluated outcome requires a reproducible price path and fixed rules. Broker execution requires actual order and fill records. These levels should never be exchanged merely because the displayed price later moved favorably.
10. Preserve a review trail
Retain original inputs and the exact rules used for each published result. Record corrections with a date and explanation. Compare later findings with the initial hypothesis, including contradictions. An independently timestamped record is stronger than a browser-local plan, which the user can alter.
What is available in this Research Lab?
The existing candle tool calculates descriptive price statistics from user-imported CSV files and supports an event-calendar comparison. The protocol notebook records plans locally. Neither feature is a complete historical CY66 execution simulator. Automated outcome evaluation awaits timestamped signals, complete price coverage and explicit contract and execution rules.
Source discipline
Use original market and release sources, record their observation times and respect usage rights. An external live chart is useful for current context; it does not preserve a historical test dataset.