1. Introduction

Global warming has become a modern phenomenon as a result of the widespread use of fossil fuels to produce energy. As a result, renewable energy resources could be a feasible solution to this issue in terms of reducing carbon dioxide (CO2) emissions and preserving pollution levels ⁽¹⁾. Renewable energy sources such as solar, wind, hydro, tidal, geothermal, and biofuels are all intriguing. The ecosystem will not be harmed by these pollution-free renewable energy sources ⁽²⁾. Many scientists and researchers have looked into and determined the potential of renewable energy sources including solar, wind, and hydropower ⁽³⁾.

Because of its clean, cost-free, and plentiful energy benefits, solar energy is one of the most promising energy sources ⁽⁴⁾. As a result, solar power has a strong demand for power generation as part of efforts to address these environmental problems. However, because of various environmental factors such as ambient temperature, solar radiation, shadows, humidity, and wind speed, the output power of photovoltaic (PV) panels is unpredictable and spontaneous. These are some of the difficulties that grid operators must overcome to effectively operate the power supply system ⁽⁵⁾. To address these issues, several strategies for balancing electric power consumption and generation have been created. One of the techniques for improving the operating reliability of power systems is to forecast demand on loads in the short term ⁽⁶⁾.

Solar power forecasting is important for energy trading companies and power network dispatching centers to make accurate decisions on critical issues such as power system scheduling and operational control ⁽⁷⁾. Furthermore, accurate solar power forecasting increases the overall power system's efficiency and power quality ⁽⁸⁾. Solar power forecasting is divided into two categories: direct and indirect forecasting. Solar power data is predicted by direct forecasting as the model performance. Indirect forecasting, on the other hand, produces the predicted values of solar radiation. As a result, PV output models are using forecasted solar radiation values to calculate solar power generation ⁽⁹⁾.

With recent advancements in Artificial Intelligence (AI)/Machine Learning (ML), load forecasting can be performed considering the weather and atmospheric conditions yielding higher accuracy as compared to conventional methods ⁽¹⁰⁾. In ⁽¹¹⁾, individual forecasting methods are proved to have limited performance, low forecasting accuracy, and high error. It is, therefore, necessary to develop combined algorithms that would yield more robust performance and increase the forecasting accuracy of models. This is achieved by using ensemble learning, where individual models serve as weak or base learners and their predictions are combined in a more accurate predictive model for classification or regression problems ⁽¹²⁾. Various ensemble learning algorithms are available in the literature, there are three types of ensemble learning depending on the way base models or learners are combined: bagging, boosting, and stacking ⁽¹³⁾. Different authors have applied the three algorithms separately to predict PV power using time series data ^(14,15-16). Authors in ⁽¹⁷⁾ present a joint bagged-boosted ANN, the proposed ensemble model produces higher accuracy prediction of short-term electricity as compared to bagged ANN and boosted ANN. Separate STACK combinations were compared in ⁽¹⁸⁾, ⁽¹⁹⁾ and ⁽²⁰⁾ to forecast solar energy and more accurate models were obtained. However, to the best of our knowledge, the proposed algorithms do not consider one single Bagged- Boosted STACK with a single meta-learner to forecast future energy values.

This paper proposes a Bagged-Boosted STACK ensemble learning model with SVRL and seven base learners are used: Elastic Net Regression, Random Forest, Linear Regression, Lasso Regression, AdaBoost, GBoost (Gradient boost), and XGBoost (Extreme Gradient Boost). Corr and PCA were used to pre-process and reduce time series data variance and over-fit.

Bagging algorithms tend to decrease model variance whilst boosting focuses on reducing bias error ⁽²¹⁾,  contributions of this work aim at solving the variance-bias tradeoff using the STACK combination of both to filter out the generalizing biases and variances. To our best knowledge, our combination of base learner models as well as the high number of based learners used in our STACK has rarely been used to forecast PV output power. The model proposed in this paper is flexible and can be used with other methods and different base and super-learners, the scheme is compared with bagging, boosting and bagging-boosting algorithms to prove the superiority of the proposed model.

2. The Proposed Intelligent PV Power Forecasting Model

The proposed model is a hybrid combination of seven different weak learners into one final SVRL meta-predictor to forecast the output power of a PV system. Improved accuracy of the forecasting model is achieved by combing bagging and boosting algorithms in level 0 of the STACK, thus reducing variance and bias errors.

2.1 Data Collection

The time series solar data of Gyeongnam, South Korea from January 2017 to June 2020 are obtained from a 350KW 3rd PV power plant. The dataset is comprised of 7 features including UNIX date and time, temperature, wind direction, wind speed, rainfall, humidity and PV output power. We carefully reviewed and the missing data were filled using Bayesian Ridge Regression. In figure 1 the PV output profile is shown, (KW),from the year 2017 to 2020.  

Fig. 1. Photovoltaic energy production from 2017 to 2020.

2.2 Feature Selection

The correlation between different data features is shown in figure 2. The correlation map shows the wind speed as the feature with the highest correlation value (0.46) with the PV output power. A high regression coefficient is also depicted for temperature (0.31) and the year, the rest of the features have a negative regression coefficient, which means a lower correlation to PV energy production. Based on feature statistics in Table 1 and the correlation map, five features were selected: wind speed, temperature, year, number of days of the week, and wind direction. The pre-processed data is normalized as in (1) to the scale of [-1, 1] allowing the model to converge faster and avoiding very large weights to be assigned to features with larger scores in previous years during training. PCA feature selection was implemented to further reduce our data dimension into two principal uncorrelated components maintaining the most possible information from the previous dataset. As shown in figure 2, the wind speed and temperature has the highest positive correlation with the PV output, this means that if these features increase, the PV output power also increases, if they decrease, the PV output power also decreases.

Fig. 2. Feature correlation heat map.

Table 1. The proposed STACK model framework.

Feature	Mean	Standard Deviation	Minimum Value	Maximum Value
Temperature	13.330018	9.547295	-13.5	35.1
Wind Direction	165.690331	124.355640	0.0	360.0
Wind Speed	2.348275	1.835690	0.0	13.6
Rainfall	0.164399	1.246811	0.0	68.5
Humidity	69.847010	21.563524	1.1	99.9
Year	2018.285043	1.029817	2017	2020
Month	6.09475	3.424773	1.0	12.0
Day	15.714174	8.792300	1.0	31.0
Day of Week number	2.998434	2.001989	0.0	6.0

2.3 The Framework

The framework structure in Figure 3 shows the steps developed in this paper to implement the applied methodology. The first step was time-series raw data pre-processing and feature statistics were analyzed, followed by data split into training and test, 80% and 20% respectively. The next step was data scaling and feature selection using CORR and PCA. The next step was the training of base models in level 0 of the STACK, predictions made by bagging and boosting models were combined and used as input for the SVRL meta-learner in level 1. The last step was the evaluation of the trained Stacking model using a testing dataset and different metrics were used to compare the proposed model with bagging, boosting, and bagging- boosting models separately.

Fig. 3. The proposed STACK model framework.

The proposed model is a hybrid combination of seven different weak learners into one final SVRL meta-predictor to forecast the output power of a PV system. Improved accuracy of the forecasting model is achieved by combing bagging and boosting algorithms in level 0 of the STACK. This is achieved by averaging the predictions made by each and every weak learner, thus reducing variance and bias errors.

A support vector machine is a machine learning method that turns the problem into linear by transformations of the original space to higher-dimensional spaces employing a kernel $K\left(X_{n},\: X_{n^{1}}\right)$ $K\left(X_{n},\: X_{n^{1}}\right)$.

In regression, the error is minimized by eliminating the penalty around $\vec{\alpha}$ ± ɛ interval. So that models do not fall into over- fitting, a certain error is admitted in the data, which is marked by the hyper-parameter C.

By combining bagging and boosting, the total forecasting error will decrease significantly. The total error of a model can be decomposed as bias + variance + error. In bagging, models with very little bias but a high variance are used, adding them reduces the variance without just inflating the bias. In boosting, models with a very little variance but high bias are used, adjusting the models sequentially reduces the bias. Therefore, each of the strategies reduces a part of the total error of the STACK.

In bagging, each model is different from the rest because each one is trained with a different sample obtained by bootstrapping. In boosting, the models have adjusted sequentially and the importance (weight) of the observations changes with each iteration, leading to different adjustments.

3. Performance Evaluation

Given that $y$ is the actual value, $\widehat{y}$ is the predicted value, $n$ is the number of data samples and represents the variance, the following metrics were used to evaluate our model performance:

Explained Variance Score (EVS): is used to measure the variability between $y$ and $\widehat{y}$, the ideal value of EVS is 1:

Mean Absolute Error (MAE): measures the absolute error between the predicted value and the actual value. It shows how big the forecast error is on average.

Mean Squared Error (MSE): measures how close data points are located from the fitted line.

Root Mean Squared Error (RMSE): measures how far data points are located from the regression line.

Determination Coefficient $(R^{2})$: $R^{2}$ is the percentage of variation of the predicted value that explains its relationship with one or more predictor variables. Generally, the higher the $R^{2}$, the better the model fits the given data.

4. Results and Discussion

This section shows the comparison of performance among the proposed Bagging-Boosting STACK SVRL, bagging, boosting, and a combination of bagging-boosting models using Elastic Net Regression, Random Forest, Linear Regression, Lasso Regression, AdaBoost, Gboost, and XGBoost as base learners of the STACK, the simulations and coding were performed using Python. On the plot, the blue dotted line represents the performance of the training set made of 80% of the original data set, whilst the green line represents the model performance on the 20% testing dataset. In figure 4, the bagging model shows a reduced bias and variance as compared to boosting in figure 5. The lowest bias error is achieved by combining bagging and boosting algorithms together as shown in figure 6, whilst the proposed STACK model in figure 7 shows the lowest variance and bias errors.

Fig. 4. Bagging Model Performance Evaluation.

Fig. 5. Boosting Model Performance Evaluation.

Fig. 6. Bagging-Boosting Model Performance Evaluation

Fig. 7. Bagging-Boosting STACK Model Performance

In figure 8, the different weak learners used in level 0 of our STACK model were compared by five different metrics and observations show the most robust performance for the Random Forest model, with a value of 0.25% for both and EVS. Figure 9 shows the results of the comparison among selected algorithms. The proposed bagging-boosting STACK shows better overall performance by solving the bias-variance tradeoff. The metrics used in assessing the four models show the lowest forecasting error for the STACK in comparison with the other algorithms, hence yielding the best forecasting ability.

Fig. 8. Comparison of Error Metrics among the base learners

Fig. 9. Comparison of Error Metrics with Existing Models

5. Conclusion

In this work, we propose a bagging-boosting STACK model with different algorithms used in solar power prediction. The proposed STACK uses an SVRL as a meta-learner to forecast PV output and seven different weak learners are used to provide input prediction to the meta-learner. The proposed STACK outperforms the predictions made by bagging, boosting, and bagging-boosting models separately. By using the proposed model, the tradeoff between variance and bias is significantly reduced. Bagging reduces the variance of the weak learners whilst boosting reduces their bias, by combining both algorithms into one STACK model, the generalization and forecasting errors are reduced. The developed methodology showed that the prediction error of PV output power can be reduced, however more data is required to training the models as shown by the results. The proposed STACK model can be improved through testing different base learners and meta-learners, but also by increasing the number of layers of the STACK.