2.1 XGBoost (eXtreme Gradient Boosting)

XGBoost is the most advanced version of the gradient boosting decision tree algorithm (Gradient Boosted Decision Trees, GBDT), which is based on the similar principles but includes the usage of second derivatives to increase the accuracy of the loss function, regression to prevent overfitting and data batching to boost processing speed. Due to its efficiency, flexible and optimized memory usage, XGBoost has become popular in fields such as economics, data intelligence and recommendation systems.

The XGBoost algorithm is widely used for stock price prediction based on high-frequency trading data and has been considered one of the leading machine learning methods. The high accuracy of daily stock price predictions using XGBoost shows that economic management can be done effectively with modern algorithms, as long as you can avoid overfitting and under-fitting during model parameter tuning. Thus, we can say that real-time economic analysis based on complex algorithms will get more popular due to a deeper understanding of market processes and the strategy development based on them^[2].

2.2 SVM

SVMs have been the center of attention in the machine learning community for a long time. In a study by Yang et al.^[3], SVMs were used to estimate market volatility through deviations in prediction boundaries and their reduction. It was also investigated using only an asymmetric boundary for a downward slope can reduce the negative risks too often.

The proposed methodology showed the highest prediction accuracy for daily closing prices of a stock index. The use of the SVM model provided significant advantages for both investors and regulators in evolving markets such as the Indian stock market. Further research in this area could enhance this approach by expanding the model to include macroeconomic variables other than share prices that also have a significant impact, such as interest rates, exchange rates or the consumer price index (CPI)^[3].

2.3 LSTM

All previous researchers studied the effectiveness of the LSTM models in dealing with the non-stationarity of financial time series, including noise, volatility, and non-linearity, which are characteristic of the stock market. The LSTM model was trained on historical stock price data, and the prediction accuracy was estimated based on real market price movements ^[4].

The results confirmed that LSTM models work well for forecasting stock price dynamics, as they are capable of forming forecast distributions and effectively managing time dependencies while capturing long-term patterns in the data. The new LSTM model outperforms traditional models in terms of forecast accuracy and consistency, making it more useful for practical applications^[5]. Nevertheless, the authors also emphasise the inherent unpredictability of the stock market and noted that, while LSTM can substantially improve forecast accuracy, their consistent application cannot fully eliminate the inherent uncertainty in financial forecasting^[6].

2.4 GARCH (Generalized Autoregressive Conditional Heteroskedasticity)

An extensive scientific literature has been formed on GARCH models used to predict the volatility of company shares, stock indices of countries, and commodity prices. The various GARCH model options as linear and nonlinear, asymmetric and symmetric models are assessed in these markets.

The usage of artificial neural networks for volatile forecasting is a fairly new and modern research area, as these models that traditionally used for this purpose. However, artificial neural networks have showed high efficiency in forecast, classification, and anomaly detection tasks.

2.5 Random Forest and Artificial Neural Networks

The stock market is simulated with the Random Forest method, which is an assembly machine learning method that constructs a bunch of decision trees and gives the most widespread prediction among them. In modern machine learning research, stock market forecasting is considered not only an academic problem but also a cutting-edge one due to its non-linearity and volatility. These properties are better reflected by modern techniques.

Despite the fact that algorithms like artificial neural networks (ANN) and SVM have been actively studied, ensemble methods like random forest are not often used in e-commerce system. However, in a study by Xiong et al.^[7] recent studies show that integrating knowledge graphs and graph-based neural models can significantly improve the accuracy of stock market forecasting.

The results revealed that integrating a Random Forest classifier into the task allows for accuracy ranging from 85 to 95%. The model was tested on metrics such as recall, precision, accuracy specificity and the ROC curve. The work emphasises the many advantages of non-linear models over linear models in predicting stock market trends^{[7, 8]}.

3.1 Model Structure

Two types of data sets are used in this analysis: the source data set and the training data set. The source data set, representing a large data set based on an exchange-traded fund (ETF), which used for train the model. The training data set is then used to solve new problems. During training, the source data gets pre-processed and fed into the pre-trained model. The financial decision-making system architecture is designed to be scalable, modular, and adaptable^[9]. It combines historical data analysis, machine learning models, technical indicators as the RSI, moving average convergence divergence (MACD), and automated process execution using BPMN. This multi-layered design ensures that the system not only provides accurate financial forecasts but also responds intelligently to dynamic market conditions^[10].

Fig. 1. The architecture of the Machine Learning-Based Forecasting Framework.

3.2 Selecting a Model

This section shows how to predict future market prices. The proposed method is shown in Fig. 1. Figure 1 illustrates the multi-stage structure of the given stock market trends forecasting system based on machine learning methods. The process starts with the data collection stage, where historical data on the stock market, including stock prices, technical indicators, trading volumes and macroeconomic indicators, are collected^[11]. This is data pre-processing stage, which includes cleaning, normalisation, feature generation and training sample formation. At the model selection step, a suitable prediction algorithm is selected based on data characteristics and research objectives^{[12, 13]}. The models covered include Random Forest, LSTM, GARCH, SVM, and XGBoost, each of which represents various classes of statistical and intelligent analysis methods ^[14]. After selecting a model, training is performed, where the model is trained on historical data to reveal patterns in price dynamics. The model evaluation phase involves measuring its predictive accuracy using metrics such as Accuracy, RMSE, AUC(Area Under the Curve), and others. At the comparative analysis stage, the performance of different models is compared to identify the most suitable approach depending on market conditions. Then, robust testing is conducted to ensure the stability of models in stressful or unstable market situations. The process ends with the formulation of conclusions and recommendations, where the results are summarized and suggestions are made on how to use the model in applied tasks of automated decision-making in the stock market^{[15~ 17]}.

4.1 Historical Stock Market Data

The first dataset consists of multi-year historical trading data for selected equities, including daily open, close prices, high, low, trading volume and turnover. These indicators serve as foundational inputs for time series modeling and trend analysis. The data was gathered via automated extraction from verified financial data providers, ensuring consistency, accuracy, and replicability of results^[18].

4.2 Technical Indicator Series

The second dataset contains pre-calculated values of the RSI, which widely used momentum oscillator in technical analysis. RSI values were computed using a 14-day rolling window applied to the historical price data, allowing for the identification of overbought or oversold market conditions. This dataset enables the integration of momentum-based signals into the modeling framework^[19].

4.3 Temporal Scope

The combined data spans the period from November 29, 2018 to November 29, 2023, covering various market phases, including bullish expansions and bearish corrections. This five-year time frame captures diverse financial conditions, making it suitable for evaluating the stability and adaptability of forecasting models across different economic cycles^{[20~ 22]}.

The dataset contains essential variables required for a comprehensive analysis of stock dynamics. These variables are presented in Table 1.

For the purposes of this research, a structured dataset was collected containing daily trading indicators for the shares of leading American technology companies as Apple, Microsoft, NVIDIA, Alphabet, Amazon, Meta, Tesla and others^{[23, 26]}. The data was extracted from the Bloomberg platform, which guarantees its accuracy and financial reliability as illustrated in Fig. 2.

Each row of the dataset in Fig. 2 contains the trading session date, the opening price, daily maximum and minimum prices, the closing price, the total volume of trades, information about dividends paid, data on stock splits and the name of the corresponding company^{[26~ 27]}.

Fig. 2. A fragment of the original dataset from the Bloomberg platform: daily market indicators for the largest US technology companies, including prices and trading volumes.

5.1 R-squared

R-squared evaluates how well the predicted values approximate the actual data. It is defined as:

where $y_i$ = actual value, $\hat{y}_i$ = predicted value, $\bar{y}_i$ = mean of actual values, n = number of observations. An $R^2$ value closer to 1 indicates better explanatory power of the model.

5.2 Root Mean Square Error

RMSE evaluates the square root of the average squared differences between actual and predicted values, penalizing large deviations more severely:

This metric is sensitive to outliers and often used when large errors are particularly undesirable.

5.3 Mean Absolute Error

MAE measures the average of absolute differences between predicted and actual values, offering a linear and more interpretable error metric:

It is less sensitive to outliers than RMSE and is thus suitable for applications where all errors are equally weighted.

5.4 Accuracy

For classification-based models, prediction Accuracy is defined as the proportion of correct predictions among total predictions made:

$= \frac{TP + TN}{TP + TN + FP + FN},$

where TP = True Positives, TN = True Negatives, FP = False Positives, and FN = False Negatives. This metric is effective for balanced datasets but should be supplemented with precision, recall, and F1-score in imbalanced scenarios.

To clearly assess the effectiveness of machine learning models in time series forecasting, graphs comparing actual closing prices with predicted results were constructed. Below are visualisations for each of the tested models, including XGBoost, Random Forest, LSTM, and others. These graphs provide a visual illustration of the accuracy of predictions, model stability. Each visualization is followed by a brief description reflecting the behavior of the model on the test sample.

Fig. 4. Comparison of actual and predicted closing prices of shares using the XGBoost, LSTM, SVM, and Random Forest models.

Figure 4 shows a graph comparing the actual and predicted closing prices of shares using the XGBoost, LSTM, SVM, and Random Forest models. Despite the volatility, the predictions show a close approximation to the actual market trend, indicating the model's potential ability to capture the overall market movement, but with some systematic deviations and underestimation of the amplitude of fluctuations.

Figure 5 shows a comparison of the actual closing price and Random Forest model predictions on the training (green) and test (pink) samples. The orange line shows the real closing prices, where the model predictions on the training data are almost the same as the real prices, showing good matching with historical patterns. The test sample shows a slight increase in dispersion and noise, but the overall trend continues, indicating Random Forest's ability to capture nonlinear time series patterns without excessive overfitting.

Figure 6 shows the predicted 10-day forward volatility of TCS shares, estimated using the XGBoost, LSTM, SVM, Random Forest, and GARCH models. The volatility clustering typical of GARCH is clearly visible by periods of calm behaviour followed by spikes representing market shocks, after the volatility returns to its average level. This structure validates the ability of model to account for heteroskedasticity and retain memory of recent changes in dispersion, which is critical for an adequate assessment of future risk.

Fig. 5. Comparison of actual and predicted closing prices for (A) XGBoost, (B) SVM, (C) LSTM, (D) GARCH, and (E) Random Forest models.

Fig. 6. 10-day volatility forecast for TCS shares obtained using the XGBoost, LSTM, SVM, Random Forest, and GARCH models.

A summary of all evaluations for these models is provided in Table 2 for different metrics, such as RMSE, R-squared, MAE. These quality indicators could be used to determine which model is best suitable for the actual time series data.

Fig. 7. Comparison of models in terms of forecast accuracy.

Figure 7 shows a comparative analysis of models based on accuracy metrics for market signal forecasting. The LSTM model demonstrated the highest accuracy (93.54%), which reflects the ability of model to account for long-term and seasonal dependencies in time series. It is followed by GARCH (91.28%), showing strong results due to the modelling of conditional heteroscedasticity and then XGBoost (88.14%), which accurately captures non-linear patterns. SVM (80.11%) and Random Forest (77.41%) models performed less well, which may indicate their limited ability to adapt to the complex time series structure of the data or require more careful hyperparameter tuning. This distribution of accuracies provides useful guidance for choosing priority models for further integration into trading strategies and decision-making systems.

However, this implementation has some limitations. The structure assumed stationarity inside the selected time windows and did not include explicit detection of market mode changes or consideration of macroeconomic covariates, which can reduce the model's stability during structural changes and sudden shifts in market dynamics.