Practical guides · Different markets

Backtesting with AI: how to design an honest test

The AI can help you document and review a test. The results have to come from reproducible data and rules, not from an invented prediction.

Markets with AI · Materials prepared with support from AI · Updated 3 October 2026 · Method and limits.

Start with a rule that can be run

Before you ask for results, write about what data triggers the condition, when it's available and at what later time the entry simulates. A confirmed closure was not confirmed at the start of that candle. Using it retroactively to run at the best price changes the test.

The AI can turn a description into a list of rules and place ambiguities. You have to decide the criteria of the experiment and provide sufficient information. If you have been without prices, costs or timing of implementation, there is no basis for presenting an estimated profitability.

A product-appropriate dataset

FieldWhy ask
Date and time zoneIt allows to order comments and to separate periods.
OHLC and frequencyIt allows to check defined conditions about candles.
Product and unitsDistinguish cash units and multipliers.
Costs and executionIt includes what separates prices and can be achieved.
Adjustments, maturities or contract changesAvoid treating different series as a single without documenting them.
Absentee data ruleImpand complete gaps with built prices.

The data documentation should say what includes the series and what excludes them. A future of a raw material may require to identify maturities and an action may have corporate events. Don't figure the supplier's treated those elements the way you need them.

Propt to design the test

Requests protocol before curve

It sets an educational test of this fictitious hypothesis: to compare a closure with a maximum of 12 earlier candles and to evaluate a later condition as defined by the pupil. No real prices are provided. Reforms: exact rule to complete, required data, time of later entry, output, quantity, costs, treatment of ambiguous candles, timing of design and later validation. Rejection criteria have to be established before results have been found. Do not generate profitability and a curve without data. Use only information provided. Separates fact, assumption and missing data. Don't make up prices, sources and results. Do not recommend buying or selling or assign probabilities without evidence.

That's what I'm talking about. That's what I mean. That page doesn't check a model automatically.

Why separate design and validation

Use the design section to define the hypothesis. Keep a later portion unused for examination. If you look at the back result and change a rule, that section already had an impact on the design. You can document the setting, but don't continue to call it an independent validation.

Also register the essays you ruled out. Test many combinations and have only the best hidden selection process. Requests the AI to audit dates, variables and changes and then check these points against the registration of the test.

A small numerical check

Supports four net fictitious results: + 12, − 8, + 12 and − 8 €. The sum is + 8 €, there are two challenges of four and an average of + 2 €per operation. That invented list allows us to practice accounts and that's not a market backtest.

If you add 3 €of missed costs to each operation, the list goes to + 9, − 11, + 9 and − 11 €. The sum is − 4 €. The same sequence changes sign. Ask for a table to preserve both scenarios and explain the assumption that changed.

Exercise: The AI presents an actual Bitcoin curve as you only gave them these four invented results. Is that valid?

No. The list contains no real Bitcoin data and no evidence of executable inputs. It should label the exercise as a fictitious and withdraw the assignment to a real market.

What to do with the result

Check operations, costs, periods and capital fall. Examines how results change by changing costs or execution with previously written test rules. If data are missing or the evidence does not allow a conclusion, it retains that uncertainty.

A retrospective test and a later simulation answer different questions. The goal is to have a process that you can reproduce and question. In the medium term you will practice how to prevent an analysis from using future weeks to decide in an earlier week.

Practice the complete process

Solves tasks with data, prompts and independent controls at free course.

Go to related course

Continue with another task

Sources to check

The examples are computed with fictitious data. The sources do not support a sign or a performance result of these exercises.

Practical questions about backtesting with ia: how to design an honest test

What should an AI backtest include?

Data dated, a rule defined before testing, explicit execution, costs and a time-bound separation between design and validation. Save your operations record to recalculate the summary.

What's the difference between backtesting and paper trading?

A backtest goes through historic data with simulation rules. The paper trading records decisions as new data come without investing real money. None of them guarantee that actual execution will replicate the results.

Can the AI come up with operations to complete a test?

It should report what rows are missing or what execution cannot be determined. Do not accept operations filled with intuition: each result has to be traced back to data and an enforcement rule.

Fold with a lab

A backtesting course: data and validation: check the program and its sample to conduct this task with exercises and checks.