Why We Keep an Append-Only Trading Research Log
A research process becomes much easier to fool when yesterday’s failed test can quietly become today’s successful story.
An append-only trading research log gives every important experiment a permanent place in the record. We preserve what was proposed, what data was available, what passed, what failed, and what we concluded at the time. Later evidence can challenge an earlier interpretation, but it cannot erase it.
We now apply this discipline to our computer-vision-inspired pivot research, feedback-loop trading research, and other automated-system experiments. It is less tidy than a highlight reel. It is also much more useful.
Evidence classification: This article describes our research-governance process and uses a sealed detector-ranking result as an example. It does not present evidence of trading profitability.
Precommitments make the rules visible before the result
A precommitment records the test before it is run. Depending on the experiment, that can include the hypothesis, data windows, model comparison, metrics, pass/fail threshold, negative controls, and the next action allowed by each outcome.
This matters because a flexible rule is easy to satisfy after seeing the numbers. A researcher can change a threshold, select a friendlier window, or emphasize a secondary metric that happened to look good. Each decision may sound reasonable on its own, but together they turn a test into a retrospective explanation.
Precommitments do not make a hypothesis correct. They make it clear whether the evidence met the rule that existed before the evidence was known.
Chronological splits and blind labels protect the future
Trading data has an order. A model should learn from permitted history and then face later data it did not use for fitting, calibration, threshold selection, or strategy selection. Randomly mixing past and future observations can hide regime changes and create leakage that would not be available in live operation.
For manually labeled chart events, we add another separation: labels for a blind package are created without inspecting the model’s predictions. The model and its evaluation protocol are frozen first. Once predictions or labels influence a design choice, that window becomes consumed development evidence and cannot honestly be called untouched again.
We also use negative controls and causal checks. They help answer whether a result reflects genuine information available at decision time or an accidental relationship created by labeling, timing, or implementation.
A failed gate stays failed
One of our exact-one-bar pivot-ranking models produced a normalized average-precision lift of 9.8008% against a frozen 10.0000% gate. The formal result was a failure. We did not round it up, lower the old threshold, or rewrite the test after seeing the answer.
We separately classified the result as a bounded yellow-light near miss. That classification allowed one narrower, development-only follow-up under a new precommitment. It did not turn the original result into a pass, establish that the 10% cutoff was economically meaningful, or authorize candidate emission, lifecycle testing, P&L claims, or trading.
This distinction is important. A pass/fail gate protects the integrity of the original test. A carefully limited near-miss policy can still permit learning when uncertainty around a knife-edge threshold warrants one more controlled question.
Later evidence is appended, not backfilled
Research conclusions change. When later evidence confirms, narrows, contradicts, or supersedes an earlier interpretation, we add a dated entry that says so. If an earlier statement contains a factual error, we preserve it and append an erratum explaining the correction.
The same rule applies to failures. Rejected early-turn detectors, weak trading families, contradictory windows, and unproductive model additions remain part of the canonical history. That record helps us avoid repeating old experiments and makes current decisions easier to audit.
Public communication does not need to expose live thresholds, proprietary feature lists, or operational architecture. It should disclose enough to distinguish a development diagnostic from a blind out-of-sample result—and either of those from executable trading evidence.
What the log is designed to prevent
- Hindsight bias: changing the story after seeing the result.
- Data reuse: describing a tuned window as fresh evidence.
- Metric substitution: replacing a failed primary metric with a favorable secondary one.
- Category errors: presenting prediction accuracy as proof of profitable execution.
- Failure amnesia: forgetting rejected ideas and repeating the same search.
An append-only log does not guarantee good research. It makes weak reasoning harder to hide—including from ourselves. For an automated trading program, that is one of the most valuable controls we can build.
Follow future research notes on the Evolve Quant blog or join our newsletter.
Research status as of August 5, 2026. Later evidence may narrow or supersede this interpretation.
Important: This article is for educational and research purposes only. It is not investment advice, an offer to manage money, or a promise of future trading results.