Why Was my Trend Model Underperforming? - Research Article #53
An exploration of how I fixed the underperformance of a trend model
👋 Hey there, Pedma here! Welcome to the 🔒 exclusive subscriber edition 🔒 of Trading Research Hub’s Newsletter. Each week, I release a new research article with a trading strategy, its code, and much more.
If you’re not a subscriber, here’s what you missed this past month so far:
If you’re not yet a part of our community, subscribe to stay updated with these more of these posts, and to access all our content.
Have you ever wondered why your model is making less money than a simple buy and hold?
Well… I am building a bunch of models to launch for the public portfolio, and strangely enough, one of them is underperforming its benchmark.
So today we are going to explore solutions to that problem.
By the end, you’ll get an idea of how I think about these problems, keeping in mind that we don’t want to do “dumb stuff”, like tinkering with rules and risk an overfit.
This is what we were able to achieve by the end of this article:
Anyone who's spent time developing systematic trading strategies knows this feeling - that moment when you realize something isn't quite right.
I was building this trend-following model and the simulation looked quite good on its own…
(the strategy rules are described within the article)
If you’re an active reader of this newsletter, you know that I like to look at a few benchmarks to classify a model as successful:
What’s the economical driver of the model’s returns?
What’s the risk adjusted returns comparison to benchmarks?
What’s the degree of optimization utilized for the rules?
This model ticked all of the boxes.
We are exploring a known effect, quite well documented, the optimization used was null, and the risk adjusted returns beat the benchmarks almost on every metric:
Despite all of that, something caught my eye.
The strategy has underperformed the benchmarks quite severely for the past 2 years.
You see, in theory, a well-designed systematic strategy should at least match the benchmark’s risk adjusted returns.
After all, if we’re not going to match it, we better off just buy and holding, and hope for the best.
Yet here I was, watching my "sophisticated" approach get beaten by the simplest strategy possible - buy and holding.
My initial thesis was that the problem would be found around these 3 variables:
Was my position sizing too conservative?
Was I filtering out the wrong assets?
Was my ranking system failing to capture the best opportunities?
Whether you're a systematic trader or just curious about the process, you'll see how methodically breaking down a problem can reveal surprising insights about your model’s behavior.
Spoiler alert, we did get to a solution.
Let’s get into it!
Index
Introduction
Index
Thinking Through Problems with Models
Position Sizing
Market Volatility
Asset Selection
Simulation Results
Parameter Settings Overview
Simulation Performance Analysis
Thinking Through Problems with Models
When we don’t understand a problem, the first thing we don’t want to do, is to start tinkering with the features and parameters.
This tends to lead to over-optimization problems, that will destroy any chances that your model will work as expected once deployed.
In the past I thought that this only happened to extreme cases of data mining, but that’s not the case!
Even small adjustments, poorly thought through, can potentially destroy any edge you have.
So let’s be sensible and think about what really affects the performance of our model, excluding the signal for now.
Let’s assume that the signal is perfect, but the mechanics behind how to maximize the returns of that signal, are poorly designed.
There’s a few components of why I believe this should be happening:
Position sizing
Asset filtering
Ranking selection
We need to think carefully through each of these variables, before we start making judgements about signal quality.
After all, we’ve already explored this signal in quite some depth here:
Now all we have to do, is to investigate each of those possible sources of problems, until we find a solution that justifies the underperformance of the model.
Position Sizing
If we look at historical sizing, it seems pretty normal at first.
However, I am curious about what happened in late 2023 to early 2024, since the market was pretty hot, but the model reduced exposure.
