Predicting the prices of crude oil and precious metals has long been one of the most challenging tasks in financial forecasting. These commodities shape global economic policy, influence geopolitical relations, and guide investment strategies for millions. Yet their prices move in unpredictable ways, driven by complex factors ranging from supply shocks to political crises. Now, a comprehensive study has shown that advanced deep learning models, particularly a type called temporal convolutional networks, can forecast these prices with remarkable accuracy even during periods of extreme volatility.

The research tested 16 different machine learning and deep learning models on daily price data for West Texas Intermediate crude oil, Brent crude oil, gold, and silver. The goal was to predict the next day's price using only historical price information. The models were evaluated across different input window lengths, ranging from 5 to 90 days of past data, and their performance was measured during a test period from early 2020 to March 2022, a span that included both the COVID-19 market crash and the sharp price swings triggered by the Russia-Ukraine conflict.

Among all the models tested, temporal convolutional networks emerged as the clear winner. For WTI crude oil, the model achieved a mean absolute error of just 1.444 dollars per barrel using 60 days of historical data. For Brent crude, the error was 1.295 dollars using only 5 days of data. Silver prices were predicted with an error of 0.346 dollars per troy ounce, again using 60 days of input. Gold proved slightly more challenging, with the best performance coming from a bidirectional gated recurrent unit model that achieved an error of 15.188 dollars using 30 days of data.

These numbers represent a meaningful improvement over traditional forecasting methods. More importantly, the models maintained their accuracy even during the April 2020 crude oil price collapse, when WTI briefly fell into negative territory for the first time in history, and during the sharp rally in commodity prices following Russia's invasion of Ukraine in February 2022. This resilience under stress is critical for real world applications, where forecasts are most valuable precisely when markets are most unstable.

Why Commodity Price Forecasting Matters

Crude oil is the lifeblood of the global economy. Its price affects everything from transportation costs to inflation rates, and sudden swings can destabilize entire nations. Governments use oil price forecasts to plan fiscal policy, allocate budgets, and negotiate trade agreements. Energy companies rely on accurate predictions to hedge against price fluctuations, secure favorable contracts, and optimize production schedules. For investors, timely forecasts can mean the difference between profit and loss in a market where billions of dollars change hands daily.

Precious metals, particularly gold and silver, play a different but equally important role. Gold is a traditional safe haven asset, often rising in value when stock markets fall. It forms a significant portion of national reserves for major economies. Silver, used extensively in electronics and solar panels, bridges the gap between investment commodity and industrial input. Accurate forecasts help mining companies plan operations, investors diversify portfolios, and policymakers manage reserves.

Yet forecasting these prices remains extraordinarily difficult. Traditional statistical models like ARIMA assume that price data follows predictable patterns and distributions. In reality, commodity markets are nonlinear, nonstationary, and subject to sudden shocks from geopolitical events, natural disasters, and shifts in supply and demand. Classical models struggle to capture these dynamics.

The Deep Learning Advantage

Deep learning models excel at finding patterns in complex, nonlinear data without requiring explicit mathematical assumptions about how the data should behave. The study tested several types of neural networks, each with different strengths.

Recurrent neural networks, including long short term memory models and gated recurrent units, are designed to process sequential data. They maintain an internal memory that allows them to learn from patterns across time. Bidirectional versions of these models process data in both forward and backward directions, capturing relationships that unidirectional models might miss. The results showed that bidirectional models consistently outperformed their unidirectional counterparts across all four commodities.

Convolutional neural networks, originally developed for image recognition, can also be applied to time series data. They excel at detecting local patterns and features. However, in this study, standalone CNN models and hybrid CNN-recurrent models generally performed poorly compared to recurrent architectures. This suggests that the temporal dependencies in commodity prices are more important than local spatial features.

Temporal convolutional networks represent a newer approach. Unlike recurrent models that process data one step at a time, TCN models use dilated convolutions to efficiently capture information from many past time steps simultaneously. This allows them to learn long range dependencies with fewer layers and faster training times. The dilated convolution architecture expands the receptive field, the range of past data points that influence each prediction, without dramatically increasing computational cost.

The study also incorporated Time2Vector embedding, a technique that encodes temporal patterns as learnable vectors with both periodic and nonperiodic components. This embedding improved forecasting performance, but only when using longer input sequences of 60 or 90 days. For shorter sequences, it offered little benefit and sometimes degraded performance.

Baseline Comparisons and Robustness

To validate the deep learning results, the researchers compared them against four baseline machine learning models: random forest, LightGBM gradient boosting, support vector regression, and k-nearest neighbors.

LightGBM emerged as the best performing traditional machine learning approach. For crude oil forecasting, its accuracy nearly matched that of the temporal convolutional networks, and it proved relatively insensitive to changes in input sequence length. However, LightGBM struggled more than TCN to capture sharp price changes during periods of extreme volatility, such as the April 2020 oil crash. This difference highlights a key advantage of deep learning: its ability to generalize from historical patterns to unprecedented market conditions.

Support vector regression and k-nearest neighbors both deteriorated significantly as input sequence length increased. With longer sequences, these models faced challenges identifying relevant patterns in high dimensional data. This curse of dimensionality is a well known limitation of traditional machine learning methods.

An important finding was that different commodities responded differently to input sequence length. For WTI and Brent crude oil, a 60 day window generally provided the best results across most models. Gold prices were more sensitive to sequence length, with performance varying substantially depending on the model and window size. Silver predictions remained relatively stable across different input lengths, particularly for the temporal convolutional network.

The coefficient of variation, a measure of how much forecast error changes with different input lengths, revealed that gold was the most challenging market to forecast consistently. Its error varied widely depending on the input window. In contrast, crude oil and silver prices could be predicted with more stable accuracy across different sequence lengths, especially when using TCN or LightGBM models.

Why Temporal Convolutional Networks Excel

Several factors explain the superior performance of temporal convolutional networks in this application.

First, TCN models can efficiently process the entire input sequence in parallel, rather than sequentially. This makes training much faster than recurrent models while still capturing long range dependencies through dilated convolutions. Each layer sees a progressively wider window of past data, allowing the network to learn patterns at multiple time scales.

Second, TCN models avoid the vanishing gradient problem that can plague recurrent networks during training. By using residual connections, where outputs from one layer are added directly to inputs of later layers, the network maintains strong gradient flow even through many layers. This architectural choice allows for deeper networks that can learn more complex patterns.

Third, the causal convolution structure ensures that predictions depend only on past data, never on future values. This prevents information leakage, a common pitfall in time series forecasting that can create artificially good results during testing but fails in real world deployment.

Fourth, TCN models proved remarkably robust to changes in input sequence length. While many models showed declining performance or instability when sequence length varied, the temporal convolutional network maintained consistent accuracy across all tested window sizes. This stability is valuable for practical applications, where the optimal input length may not be known in advance or may change over time.

Practical Implications

The findings carry significant implications for multiple stakeholders.

For governments, improved commodity price forecasts enable better fiscal planning and policy decisions. Knowing with greater confidence how oil prices will evolve helps in budget allocation, tax policy, and international trade negotiations. During times of economic stress, when forecasts are most valuable, the temporal convolutional network's ability to maintain accuracy provides crucial guidance.

Energy sector managers can use these forecasts to optimize operations and manage risk more effectively. Accurate predictions help companies decide when to hedge exposure, how to price long term contracts, and where to allocate capital investment. During the extreme volatility of 2020 and 2022, such tools would have been invaluable for navigating unprecedented market conditions.

Investors in commodity markets, whether trading futures contracts or managing diversified portfolios, benefit from any improvement in forecast accuracy. Gold and silver, in particular, serve important roles in portfolio diversification and as hedges against stock market downturns. Better price predictions enhance risk management and strategic asset allocation.

Mining companies, which must plan operations years in advance, can use improved forecasts to make more informed decisions about exploration, production, and capacity expansion. Given the long lead times in mining operations, even modest improvements in forecast accuracy can translate into substantial economic gains.

Limitations and Future Directions

Despite these advances, several limitations remain. The models in this study used only historical price data. In reality, commodity prices are influenced by a wide range of external factors including macroeconomic indicators, supply and demand dynamics, production rates, geopolitical events, and market sentiment. Future research could incorporate these additional features to further improve forecast accuracy.

Sentiment analysis from news articles and social media platforms represents another promising avenue. Recent work has shown that incorporating investor sentiment can enhance gold price predictions. Similarly, Google Trends data and natural language processing of news headlines have improved crude oil forecasts in other studies. Combining these information sources with advanced deep learning architectures could yield even better results.

The volatile and nonlinear nature of commodity markets poses inherent challenges. No model can perfectly predict sudden shocks from natural disasters, political crises, or unexpected supply disruptions. Acknowledging this uncertainty and implementing appropriate risk management strategies remains essential, even with improved forecasting tools.

Finally, while the study covered a particularly turbulent period from 2020 to 2022, testing these models across longer time spans and different market conditions would provide additional validation. The behavior of commodity markets continues to evolve, and models must be regularly retrained and evaluated to maintain their effectiveness.

The research demonstrates that modern deep learning techniques, particularly temporal convolutional networks, can substantially improve commodity price forecasting. As these methods continue to develop and incorporate richer data sources, they promise to become indispensable tools for navigating the complex world of global commodity markets.

Credit & Disclaimer: This article is a popular science summary written to make peer-reviewed research accessible to a broad audience. All scientific facts, findings, and conclusions presented here are drawn directly and accurately from the original research paper. Readers are strongly encouraged to consult the full research article for complete data, methodologies, and scientific detail. The article can be accessed through https://doi.org/10.1186/s40854-024-00637-z