How To Optimize A Backtest
First, let’s define what optimization in trading really is:
Merriam-Webster defines optimization like this:
….as an act, process, or methodology of making something (such as a design, system, or decision) as fully perfect, functional, or effective as possible specifically : the mathematical procedures (such as finding the maximum of a function) involved in this.
Optimization is no different when it comes to trading. You look for the best variables or parameters and then search and optimize for the best settings for those variables. With today’s computing power, a software program can easily be programmed to find the best optimization for a certain strategy. It’s literally done in seconds and the software needed is cheap and good.
Is strategy optimization good in trading?
Yes, optimization is good when it’s done correctly and you know what you are doing. It’s good because you get a better understanding of what is making the strategy perform well, and you get to understand how the variables of the strategy influence the end result.
Optimizing lets you easily understand if your backtest is based on luck or randomness.
For example, if you have a trading strategy that goes long when the price breaks above the 25-day moving average, you can optimize it by changing the number of days in the moving average. If you find out that the results are much worse close to 25 days, you get a pretty strong indication that the original strategy most likely is down to chance and randomness. Hence it’s unlikely to be very successful in the future.
Instead of being ignorant of optimization, you should optimize every backtest you do. An optimization is, in reality, a test of how robust your strategy is. When you examine the variables, you get a much better understanding of your strategy: is it due to chance/luck or might it be something worth trading?
Example: An RSI strategy optimization in the S&P 500
The RSI indicator is a widely popular indicator and has performed well as a mean reversion strategy on the S&P 500 over the last 30 years. But what works best – a short or long RSI? And which thresholds are best – low or high?
Let’s test to find out. For such a test we need three optimization parameters:
- For the number of days in the RSI calculation (from 2 to 5 days with intervals of 1)
- For the threshold of when to buy (from 10 to 35 with intervals of 5)
- For the threshold of when to sell (from 35 to 90 with intervals of 5)
We test on SPY, the ETF that tracks the S&P 500, from inception in 1993 until October 2021.
The optimization involves 144 tests (4 x 6 x 6). This is a lot of tests and it might be difficult to find out where the sweet spot is. We can set some minimum variables, for example, that we should have at least 250 trades over the period in order to have a significant number of trades/observations. This reduces the number of tests down to 53. We then sort on the profit factor.
We copied the results from Amibroker into OpenOffice:
Clearly, an RSI based on two or three days seems to give the best result. The balance of where to set the buy and sell thresholds might be less obvious from the results, however. To judge that you might want to have a look at the columns for CAR/CAGR, exposure (time spent in the market), and max. system drawdown.
What did we learn from the RSI optimization example?
We get a better understanding if our trading strategy is due to chance or randomness. The more profitable backtests of the different settings, the less likely a positive result is due to chance.
Thus, strategy optimization is very good for getting a better understanding of the strategy, parameters, and settings.