Choosing between linear and tree-based models

A student fits a Logistic Regression and a Random Forest to the same five-day-ahead direction label, on the same features, on the same chronological split. The Random Forest scores a higher ROC AUC on the training fold and a noticeably lower one out of sample. The Logistic Regression scores lower on both, but the gap between them is small. Which model is “better” depends entirely on what the researcher is trying to learn, and conflating the two questions — which model fits the data best, and which model tells the truth about what it learned — is where this comparison usually goes wrong.

What a linear model assumes

A linear model, whether it is Linear Regression, Ridge, Lasso, Elastic Net or Logistic Regression at its core, assumes the outcome is a weighted sum of the input features:

$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_p x_p$

Every feature’s contribution is constant and additive — a one-unit increase in $x_1$ moves the prediction by $\beta_1$ regardless of what the other features are doing. That assumption is restrictive, and it is also the model’s main virtue: each coefficient is directly interpretable, the fit is deterministic given the data and the regularisation setting, and there is no risk of the model inventing a decision boundary so contorted it only makes sense for the exact rows it was trained on.

Ridge and Lasso exist because raw Linear Regression has no defence against overfitting when features are numerous or correlated — which financial features usually are, since a 20-day moving average and a 50-day moving average of the same price series share most of their information. Ridge adds an $L_2$ penalty on the coefficients:

$\min_\beta \; \sum_i (y_i - \hat{y}_i)^2 + \lambda \sum_j \beta_j^2$

which shrinks every coefficient toward zero without necessarily eliminating any of them. Lasso uses an $L_1$ penalty instead, which can drive weak coefficients to exactly zero — a built-in feature-selection mechanism, useful when a researcher suspects that only a handful of the candidate features carry real signal but does not know which ones in advance. Huber Regressor keeps the linear form but changes the loss function itself, so that large residuals — the crash days discussed in model families in finance — influence the fit proportionally to their size rather than to their square.

What a tree-based model assumes instead

A decision tree makes no assumption about additivity at all. It splits the data recursively on the feature and threshold that most reduces impurity — for classification, typically Gini impurity or entropy; for regression, variance. A single split might read: “if 10-day realised volatility exceeds 1.8%, go left; otherwise go right,” and within each branch the tree can split again on a completely different feature. This lets a tree capture interactions a linear model cannot express without the researcher hand-engineering them — for instance, a momentum feature that only predicts direction when volatility is low, and says nothing when volatility is high.

Random Forest fits many such trees on bootstrapped samples of the training rows and averages their predictions, which controls the single tree’s tendency to memorise its training data. XGBoost Classifier and XGBoost Regressor, along with Gradient Boosting Classifier/Regressor and Hist Gradient Boosting Classifier/Regressor, build trees sequentially instead of independently, with each new tree fit to the residual error the previous trees left behind. Extra Trees Classifier and Extra Trees Regressor add extra randomisation to the split-selection process itself, typically trading a small amount of fit quality for a further reduction in variance. AdaBoost Classifier and AdaBoost Regressor take a different sequential approach, reweighting misclassified or poorly fit observations each round rather than fitting to residuals directly.

Method

Gradient boosting and random forest both average many decision trees, but they average different things: a random forest averages many independent trees fit to different bootstrap samples of the same problem, while a boosted ensemble is a single sequence of trees, each one correcting the errors of everything fit before it. That difference is why boosted models are typically more accurate on a fixed dataset and typically more prone to overfitting a specific training window without hyperparameter discipline.

A worked comparison

Consider ten trading days of two features — five-day momentum and ten-day realised volatility — used to predict whether the next day’s return is positive.

Day	Momentum	Volatility	Next-day up?
1	+1.2%	0.8%	Yes
2	+0.9%	0.9%	Yes
3	−0.4%	2.6%	No
4	+0.3%	2.4%	No
5	+1.5%	1.1%	Yes
6	−0.8%	0.7%	Yes
7	−1.1%	2.9%	No
8	+0.6%	2.5%	No
9	+1.0%	1.0%	Yes
10	−0.5%	0.9%	Yes

A linear boundary struggles here: momentum alone is positive on six of the six “up” rows but also positive on two of the four “down” rows (days 4 and 8), so a single linear combination of momentum and volatility has to find a diagonal cut that partially works for both features at once. A tree finds the actual structure in two splits: first, “is volatility above roughly 2.0%?” — every row above that line is a “down” day regardless of momentum. Second, within the low-volatility rows, momentum plays no further role at all; every low-volatility day in this sample was an “up” day. The true relationship here is a threshold effect in volatility, not a linear combination, and the tree finds it because it is not constrained to additivity.

This is a manufactured example built to make the point starkly — real financial data rarely separates this cleanly — but the mechanism is genuine: momentum-style features frequently interact with volatility regimes in exactly this threshold-driven way, and it is the kind of structure linear models cannot represent without a researcher manually adding an interaction term or a volatility-regime dummy in advance.

Where the trade-off actually bites

The cost of the tree’s flexibility is that it can find structure that does not generalise. A sufficiently deep, unconstrained decision tree can memorise every training row perfectly and still describe nothing true about the underlying process — the classic overfitting failure. Random Forest and the boosted variants control this with ensemble averaging and regularisation parameters, covered in detail in hyperparameters that actually matter, but the risk does not disappear; it moves from “does the model fit the shape of the data” to “does the model’s complexity match how much data actually supports it.” A linear model with five coefficients cannot overfit fifty thousand rows in any dramatic way. A thousand-tree boosted ensemble with no depth limit can overfit five thousand rows quite easily.

The other genuine trade-off is interpretability. A Ridge coefficient answers a direct question: holding everything else fixed, how does the prediction move when this feature moves? A tree ensemble’s prediction is the output of hundreds of nested if/then rules, and answering the same question requires a post-hoc attribution method — permutation importance or partial dependence, covered in the workspace’s Explainability page — rather than reading a number off the fitted model directly.

A practical starting point

Fit the linear model first, always. Ridge or Logistic Regression on a well-constructed feature set establishes the floor: if a Random Forest cannot clear it by a meaningful margin on genuinely out-of-sample folds, the additional complexity is not earning its keep, and the honest conclusion is that the relationship in this dataset is close to linear, or that the sample is too small for a flexible model’s extra capacity to pay off. If the tree-based model does clear the floor by a real margin, and that margin survives walk-forward validation rather than a single lucky split, it is telling the researcher something concrete: that there is genuine non-linear or interaction structure in the features worth understanding, at which point permutation importance and partial dependence become the tools for finding out what that structure actually is.

Key takeaways

Linear models assume the outcome is an additive, constant-weight combination of the features; tree-based models make no such assumption and can capture threshold effects and interactions directly.
Ridge, Lasso and Elastic Net differ in how they penalise coefficients — an $L_2$ penalty shrinks everything, an $L_1$ penalty can zero out weak features entirely.
Random Forest averages independent trees to reduce variance; gradient boosting and XGBoost fit trees sequentially to reduce bias, which raises accuracy and overfitting risk together.
A higher training-fold score from a tree ensemble is not evidence of a better model; only the out-of-sample gap and its stability across folds are.
Interpretability is a real cost of tree-based flexibility: reading a linear coefficient is direct, while explaining a tree ensemble’s prediction requires a separate attribution method.