FORECASTING EXCHANGE RATES COMPARISON IN THE PRESENCE OF INSTABILITIES

Exchange rate forecasting is an inherent approach in financial risk management, yet previous forecasting models were criticized for their poor predictive ability, mainly during periods of exceptional macroeconomic weaknesses. This is attributed to their failure to identify the importance and strength of key transmission and amplification channels, especially those linked to financial markets and uncertainty. Though there is no model that can be precise, especially during periods of crises, it is important to find a model that can yield near-accurate results. The present study therefore evaluates the different forecasting models, considering how each handles instabilities. The Rossi Sekhposyan forecast rationality test results reveal that the EGARCH model under general error distribution and APARCH under normal error distribution show the strongest evidence against rationality around the year 2009, identifying the concentration of instabilities during that time. This vindicates the need to control for instabilities in forecasting. This implies that, in the presence of instabilities, the fluctuation tests are more powerful than traditional tests.


Introduction
When applying predictive ability tests to macroeconomic time series data researchers face important practical problems: economic time series data are prone to volatility (Rossi and Soupre, 2017).As stated by Rossi (2021), "instabilities are widespread in economic time series".Odendahl et al,( 2022) also stressed that "despite various predictive comparison tests that have been proposed in the literature to help predictors in model selection, no single model is usually the best overall.Predictive performance is typically prone to volatility and is sample dependent".Therefore, when researchers test a model's predictive ability, it can be important to allow its predictive ability to change over time.Indeed, traditional predictive assessment tests are unreliable in the presence of volatility and can lead to erroneous conclusions.This problem arises because traditional tests assume stationarity and the presence of instability destroys stationarity.
Most time series techniques assume that the data are stationary.Stationary processes have a constant mean, variance, and autocorrelation structure (Songhao, 2021).It is often the case, however, that real-life data don't comply with the assumption of stationarity.Stock return data, for instance, tends to fluctuate over time, whereas economic time series are typically seasonal (Politis et al.,1999).For most time series models, the model assumptions are violated when nonstationary data is used.This leads to the estimators losing properties such as asymptotic normality and sometimes even consistency.Hence, applying a model that requires a stationary series to a nonstationary series could lead to poor estimates of the parameters and therefore poor forecasts.As stated by van Greunen, et al., (2014), the stationarity of a time series can have a significant influence on its properties and forecasting behavior.
A substantial body of research has been done on testing predicted optimality.It is, nonetheless, implicitly predicated on the notion of covariance stationarity.Because of this presumption, the models were unable to forecast events correctly when there was instability.Given the ubiquity of models that do not take instability into account, even after the financial crisis, forecasting models must take instability into account.Kenny and Morgan (2011) reported the poor predictive performance of all models that were employed in their study to capture the period of exceptional macroeconomic weakness in the fourth quarter of 2008 and the first quarter of 2009.volatility more than the unexpected good news of similar magnitude.The symmetric and asymmetric (theoretical) models which form the basis of the models used in the study are therefore discussed in detail below.Engle in 1982, accounts for the difference between the unconditional and the conditional variance of a stochastic process.While conventional econometric models operate under the assumption of constant variance, the ARCH process allows the conditional variance to vary over time, leaving the unconditional variance constant (Ding, 2011).Although financial markets may experience excessive volatility from time to time, it appears that volatility will eventually settle down to a long-run level.In the case of exchange rates, this implies that exchange rate indices are mean reverting (they are stationary processes), then this implies that shocks to exchange rate indices will have a transitory effect, in that the exchange rate will return to its trend path over time (Goudarzi, 2013).

Symmetric models  ARCH (Autoregressive conditional heteroscedasticity) model ARCH model, developed by
According to Brooks (2008), under the ARCH model, the "autocorrelation in volatility" is modeled by allowing the conditional variance of the error term, , to depend on the immediately previous value of the squared error and ARCH (1) model takes the following form: Where represents the conditional variance of the error term, represents a lagged value of the squared error.
 Generalized autoregressive conditional heteroscedasticity (GARCH) Bollerslev (1986) extended the ARCH model to the GARCH, which had the same key properties as the ARCH but required far fewer parameters to adequately model the volatility process.
The GARCH (1, 1) model is therefore given by: Where all the parameters must be positive, while the sum of quantifies the persistence of shocks to volatility.The GARCH (1, 1) model generates one-step-ahead forecasts of volatility as a weighted average of the constant long-run or average variance, the previous forecast variance, and previous volatility reflecting squared "news" about the return, .As volatility forecasts are increased following a large return of either sign, the GARCH specification captures the well-known volatility clustering effect.
Though the GARCH model has greater applicability for easy computation, it has drawbacks in the application of asset pricing.Firstly, in the GARCH model, the impacts to conditional variance of the positive and negative sides are symmetrical.This means that the model cannot explain the negative correlation between the fluctuations in stock returns as it assumes that the conditional variance is a function of lagged squared residuals.

Asymmetric GARCH Models
To capture the asymmetry in return volatility (leverage effect), a new class of models was developed, termed the asymmetric GARCH models.The asymmetric GARCH models, unlike symmetric GARCH models, capture the asymmetry in financial returns data.These models are discussed in detail below.

 Exponential General Autoregressive conditional heteroscedasticity (EGARCH)
The EGARCH model proposed by Nelson (1991), specifically captures asymmetries in volatility.In this model, asymmetries are considered exponentially.This approach captures the skewness and allows the ARCH process to be asymmetrical.In the EGARCH model, the conditional variance is an asymmetric function of the lagged disturbances : The model is stated as follows: The coefficient captures the asymmetric impact of news with negative shocks having a greater impact than positive shocks of equal magnitude if <0, while the volatility clustering effect is captured by a significant .Finally, the use of the logarithm form allows the parameters to be negative without the conditional variance becoming negative.

 Threshold general autoregressive conditional heteroscedasticity (TGARCH) model
Another volatility model commonly used to handle leverage effects is the threshold GARCH (TGARCH) model introduced by Glosten, Jagannathan, and Runkle (1993) and Zakoian (1994).The TGARCH model of order can be written as: In this model, good news, > 0, and bad news, < 0 have differential effects on the conditional variance.Here, has an impact of good news while + have an impact of bad news.If γ i > 0, bad news has a greater impact of conditional variance, whereas if ≠ 0, news impact is asymmetric.Ding et al., (1993) concluded that there was no reason for the volatility to be a linear function of the squared residuals and introduce the APARCH (p, q) model, which allows the power δ of the heteroscedasticity equation to be estimated from the data (Pedro and Marcos et al, 2011).Though the APARCH model captures asymmetry in return volatility, in this model, volatility tends to increase more when returns are positive, as compared to negative returns of the same magnitude.The APARCH model also yields the long-memory property of returns.The power parameter on the standard deviation is estimated and not imposed:

Asymmetric Power autoregressive conditional heteroscedasticity (APARCH) model
Parameter δ in the equation denotes exponent of conditional standard deviation, while parameter γ describes the asymmetry effect of good and bad news on conditional volatility.A positive value of γ means that negative shocks from the previous period have a higher impact on the current level of volatility, and otherwise (Miletic, 2015).
An APARCH (p, q) model assumes that: A positive (resp.negative) value of the "s means that past negative (resp.positive) shocks have a deeper impact on current conditional volatility than past positive shocks (see Black, 1976, Laurent, 2004).Engle and Bollerslev (1986) also put forward the integrated GARCH (IGARCH), which is an extension to the GARCH model.To capture the characteristic of volatility persistence, the GARCH model features an exponential decay in the autocorrelation of conditional variances.It has been noted that squared and absolute returns of financial assets typically exhibit serial correlations that are slow to decay, like those of an integrated I (d) process.Shocks in the volatility series seem to have a long memory and lasting impact on future volatility over a long horizon.

Integrated general conditional heteroscedasticity (IGARCH) model
The IGARCH captures this effect but a shock in this model impacts upon future volatility over an infinite horizon and the conditional variances does not exist for this model (Granger and Poon, 2002).If the AR polynomial of the GARCH representation has a unit root, then we have an IGARCH model.Thus, IGARCH models are unitroot GARCH models.
The integrated GARCH (IGARCH) is specified as: The sum of coefficients is restricted to 1.The exogenous variable can be easily reflected in the various specifications of GARCH models just by the addition of and .

Specification of the models
This section presents the empirical models that are employed in this study.

Empirical models
The empirical models are stated as follows:

 ARCH (1)
The basic ARCH model consists of two equations, a conditional mean equation and a conditional variance equation.Both equations should be estimated simultaneously given that variance is a mean equation.The mean equation estimates the conditional mean of the examined variable.The variance equation estimate this process as a typical autoregressive process.Both equations form a system that is estimated together with the maximum likelihood method (Dritsaki, 2019).
ARCH (p) model is simply an AR(p) model applied to the variance of a time series.

ARCH(1)
A time-series {ϵ(t)} is given at each instance by: Where, w(t) is the white noise with zero mean and unit variance.
Where , are parameters of the model and > 0, ≥ 0 to ensure that the conditional variance is positive.σ²(t-1) is lagged square error.
Where, , It follows that should be a positive parameter.However, in estimation, it might come out negative.Many programs allow the researcher to restrict to be positive and greater than zero.The mean equation estimates the conditional mean of the variable.It is important to get the mean equation correctly specified before estimating the ARCH/GARCH model.The mean equation typically can be modelled as an AR process, AR in combination with other explanatory variables, just as a function of other explanatory variables.It is important to test for no autocorrelation in the residuals before estimating the GARCH process.
The variance equation estimates the variance processan as a type of autoregressive process.Both equations form a system that is estimated together using maximum likelihood.The mean equation is important because if this is not correctly specified the variance estimate will not be good either.The mean equation describes the expected value of the stochastic process * + The mean equation therefore can be an AR, an ARMAX or a structural econometric model, among others (Brooks, 2002).
In the case of symmetric models, there are three coefficients in the Conditional variance equation.The constant term, ARCH term and GARCH term.The arch (alpha) term explains volatility clustering, GARCH (beta) term explains persistence (Bera andHiggins, 1993, Bonga, 2019).However, in asymmetric GARCH models, there are four coefficients in the conditional variance equation, namely, the constant term, ARCH term, GARCH term and the volatility asymmetry term.The fourth term or coefficient, gamma (leverage) term explains the leverage effect.The APARCH model adds the fifth term called ( ) delta (Kisinbay, 2003, Ding, 2011, Smolović, Lipovina-Božović and Vujošević, 2017).
ARCH effect/spillover effect ( ) represents the impact of a magnitude of a shock (size).The existence of volatility spill-overs implies that one large shock increases the volatilities not only in its own asset or market but also in other assets or markets as well (Kharchenko and Tzvetkov, 2013).Therefore, negative ARCH means the spill-over is negative related (increase volatility in one country make the other country's volatility reduced) GARCH effect ( ) implies persistence of past volatility (past volatility explain current volatility).If GARCH term is positive, it shows that there is a positive relation between the past variance and the current variance in absolute value.A higher and positive GARCH coefficient indicates that volatility takes a longer period to decay, which is implies that there is long memory (Aluko et. al., 2017).
Leverage/ volatility asymmetry term (represents the impact of a sign of a shock).If it is different from zero, this implies that there is an asymmetric effect (bad news and good news of the same size have different impacts).If the asymmetry term is negative, this implies that negative shocks have a greater impact on volatility rather than the positive shocks of the same magnitude.If the leverage coefficient (Gamma or ) is negative and significant, indicating the presence of an asymmetric behavior.The significance of negative shocks persistence or the volatility asymmetry indicates that investors are more prone to negative news in comparison to positive news.This implies that the volatility spillover mechanism is asymmetric (Nelson, 1991).

GARCH (1, 1) model
The study follows Bollerslev (1986)  Where R/US$ t represents the forecast variance of Rand/US$ daily exchange rate. is the constant longrun or average variance, represents the previous forecast variance, and represents previous volatility reflecting squared "news" about the return all the parameters must be positive, while the sum of quantifies the persistence of shocks to volatility.Squared implies that volatility forecasts are increased following a large return of either sign (negative and positive news have same impact).
A priori expectation is that all the parameters should be positive (implying that there is no asymmetry).
 Exponential GARCH (EGARCH) model EGARCH (1, 1) model can be written as: Where, is the log of the conditional variance.This implies that the leverage effect is exponential, rather than quadratic, and that forecasts of the conditional variance are guaranteed to be nonnegative.The presence of financial leverage result can be tested by the theory that .The impact is asymmetric if The coefficient captures the asymmetric impact of news with negative shocks having a greater impact than positive shocks of equal magnitude if <0, while the volatility clustering effect is captured by a significant .The forecast variance of Rand/US$ daily exchange rate is in logarithm form, implying that the model allows the parameters to be negative without the conditional variance becoming negative.
A priori expectation is that the gamma ( ) parameter should be negative (implying that there is asymmetry, that is, negative news has more impact than positive news).


Threshold GARCH (TGARCH) model Another volatility model commonly used to handle leverage effects is the threshold GARCH (TGARCH) model introduced by Glosten, Jagannathan, and Runkle (1993) and Zakoian (1994).The TGARCH model of order can be written as: Where if and if .Following Ding, Granger, and Engle"s (1993) model, the APARCH model can be stated as:

 Asymmetric ARCH (APARCH) model
Where is the forecast variance.Parameter δ in the equation denotes exponent of conditional standard deviation, while parameter γ describes the asymmetry effect of good and bad news on conditional volatility.A positive value of γ means that negative shocks from the previous period have a higher impact on the current level of volatility, and otherwise Parameter γ is expected to be positive if there is asymmetric or leverage effect.A priori expectation is that and 0. Otherwise, the PARCH model will simply be a standard GARCH specification.

Integrated GARCH (IGARCH) model
The study borrows from Engle and Bollerslev"s (1986) model and rewrites the IGARCH model as follows:

Description of the type of sample to be used
A 5-day week Rand per US Dollar rates, sourced from the Federal Reserve Economic Data (FRED) online, is the sample data used for analysis with 3038 observations, covering the period 2007/01/01 to 2018/12/31.The period 2007 was chosen as the starting period to capture the 2007-2009 global financial era.Stata software package is employed as it brings the desired results wanted by the researcher.

Out-of-sample forecasting
In forecasting, it is not necessarily the model that provides the best in-sample fit that produces the best out-ofsample volatility forecast (Shamiri and Isa, 2009;Wennström, 2014).Hence it is common to use the out-of-sample forecast to aid the selection of model which is best suited for the series under study (Andersen and Bollerslev, 1998;Hansen and Lunde, 2001;Brandt and Jones, 2006;Cerqueira et al, 2020).Before the discussion on forecasting is presented, the different models that are used in comparing traditional and fluctuation tests are presented.

Evaluation forecasting criteria
To test the model"s forecasting ability, two types of tests are used, namely, tests of relative forecast comparison and tests of absolute forecasting performance.Relative forecast comparison tests determine the best model between any two competing forecasting models in terms of predicting ability.Conversely, absolute forecasting performance tests are used to evaluate whether forecasts from one specific model fulfil some minimal requirements, like being unbiased or producing forecast errors that are unpredictable using any information available at the time the forecast is made.Whilst both tests are used to forecast the models" forecast ability, the former ones are used to compare two forecasting models, whilst the latter are used to evaluate one specific forecasting model (Rossi and Soupre, 2017).There are traditional methods that have for long been applied for both relative and absolute forecast evaluation.For relative assessment, examples are Diebold & Mariano (1995), West (1996) and Clark & McCracken (2001).Examples of absolute evaluation are Mincer & Zarnowitz (1969) and West & McCracken (1998).These traditional forecast evaluation tests assume stationarity, which is often violated when applying tests of forecasting ability to macroeconomic time series data as it is well known that economic time series data are prone to instabilities.
Nominal exchange rates used in this study are macroeconomic time series; and there are reasons to assume instabilities.Notable examples of instabilities are the 2007-2009 global financial crisis.During the 2007-2009 global financial crisis, drastic changes on several macroeconomic relationships occurred.Ng and Wright (2013) found that credit spreads replaced interest rates in predicting output growth during that time.Likewise, Rossi (2013b) revealed the presence of severe instabilities in exchange rate forecasting models.
When testing the model"s forecasting ability, it is potentially important to allow their forecasting ability to change over time.As argued by Giacomini and Rossi (2010), in unstable environments, it is plausible that the relative forecasting performance of models may itself change over time.Regardless of the increasing empirical evidence suggesting instability in the forecasting performance of econometric models relative to the naïve benchmarks (for example, Stock and Watson, 2003a), existing literature for forecasting comparison did not account for this possibility.
For example, Moran and Solomon (2017) apply traditional approaches like the Diebold and Mariano (1995) test to compare the predictive accuracy (for loss criteria MSE, MAE and MAPE), which are argued to have failed in the presence of instabilities, such as the 2007-2009 global financial crisis, despite Francq and Zakoian"s (2010) caution that different GARCH models can lead to almost equivalent predictive formulas.The implication is that the forecasting success of a model relative to its competitor seems to be linked to some specific periods in time, and there are numerous situations in which there has been a reversal in the relative forecasting ability of two competing forecasting models (Giacomini and Rossi, 2010).
In environments characterized by instabilities, it is important to compare different models to find out which models perform best in such instabilities.Traditional tests of forecast evaluation are not reliable in the presence of instabilities, which may lead to incorrect inference.To address this challenge, both relative forecast comparisons and absolute forecasting performance tests which are robust to instabilities were introduced (Rossi and Soupre (2017).To compare the relative out-of-sample forecasting performance of two competing models in the presence of possible instabilities, Giacomini and Rossi"s (2010) fluctuation test will be used.Conversely, Rossi and Sekhposyan"s (2016) fluctuation rationality test will be employed for testing absolute forecasting performance robust to instabilities.The reason for comparing these tests is to investigate and evaluate the forecasting claims that fluctuation tests are more powerful than traditional ones as stated by Giacomini and Rossi (2010), Rossi and Sekhposyan (2016) and Rossi and Soupre (2017) among others

Results
The main objective of this paper is to compare the new and traditional tests in testing forecast unbiasedness/rationality and to test competing models" forecasting performance to find the model with the best out-of-sample fit.Against this backdrop, this section presents the results for both relative and absolute comparisons of fluctuation and traditional tests respectively.

Testing for out of sample fit
The out-of-sample forecast tests are conducted using Stata commands that illustrate how to test forecast unbiasedness/rationality and how to test competing models" forecasting performance, in a way that is robust to the presence of instabilities.To test competing models" forecasting performance, Giacomini and Rossi (2010) test is employed.Conversely, Rossi and Sekhposyan (2016) test is used to test forecast unbiasedness/rationality.The results for the new and old approach (that is, the Giacomini and Rossi"s (2010) fluctuation test and the Diebold and Mariano (1995) test) are compared.Tables 1 and 2, therefore present these results respectively.

Tests of Relative Forecasting Performance Robust to Instabilities
The pairwise comparison of the model that accounts for instabilities in Table 1 show that t-distribution error assumption dominates all other error distribution assumptions, implying that all the models perform well under tdistribution assumption.Therefore, if forecasters are interested in using GARCH model to forecast series (at least exchange rate) considering inherent instabilities, the best assumption distribution to make is that of t-distribution.This is so, because it has fatter tails than the normal distribution, it can also be used as a model for financial returns exhibiting excessive kurtosis, enabling a more realistic calculation of the Value at Risk (VaR) in such cases.It can skew the accuracy concerning the normal distribution (Annapoorna, 2021).However, in the case of traditional test, the t-and general error distribution assumptions dominate, with t-distribution taking the lead.According to Table 2 , the pairwise comparison of the model that accounts for instabilities shows that GARCH under normal error distribution assumption dominates all other models, implying that the model has the best forecast during periods of instabilities.This is consistent with theory which postulates that, symmetric models perform better than asymmetric models under normal distribution.They are only outperformed by asymmetric models under distributions assumptions like t-distribution and general error assumption, due to their failure to capture leverage effect (Hentschel, 1995, Islam, 2014).The results from the traditional reveals that there is no winner as three models (GARCHnor, IGARCHnor and TGARCHnor) beat APARCH model.2008)"s study that the EGARCH skewed Student-t model outperformed GARGH, GJR and APARCH models.Surprisingly, even the results from traditional tests show that, GARCHt, IGARCHt and TGARCHt dominate APARCHt.The results are consistent with those of Abdullah et al., (2017), who model the volatility dynamics of the taka-US dollar exchange rate return using GARCH, APARCH, EGARCH, TGARCH, and IGARCH models.Their findings suggest that GARCH and IGARCH models under the student-t error distribution outperform the other models.However, this is inconsistent with the findings by Schmidt (2021), which show that the symmetric GARCH (1,1) on average has the worst volatility forecasting performance when forecasting a crisis on Nordic indices, using GARCH, EGARCH, GJR and APARCH models.The superior forecasting models were found to be the GJR (1,1) and EGARCH (1,1).The results from Table 4, reveal that the pairwise comparison of the model that accounts for instabilities show that there is no model which is the overall winner under general error distribution.On the side of traditional tests, the IGARCH is the best performer in terms of forecast accuracy.GARCH dominates all models; t dominates all 3 distribution assumptions.

Tests of Absolute Forecasting Performance Robust to Instabilities
Traditional tests of forecast rationality (such as Mincer andZarnowitz, 1969, andWest andMcCracken, 1998) assume stationarity and are thus invalid in the presence of instabilities (Rossi and Soupre, 2017).However, unlike traditional tests, the fluctuation rationality test is based on the idea of instability and therefore has a lower rejection of the null hypothesis of forecast rationality (de Prince et al., 2021).
The results for Rossi -Sekhposyan test statistics reveals that the null assumption of forecast rationality is rejected by all models.So individually these models perform well, implying that they can make a good forecast.Hence they are used popularly.It is when one model has to be chosen from more than 1 pool where it matters.That is why the results of the relative comparison show some models as worse performers.The strongest evidence against the forecast rationality appears to be around 2009 for both models.This clearly supports the idea that, in the presence of instabilities, the fluctuation tests are more powerful than traditional tests and provide a visual illustration of when predictive ability appears or breaks down in the data (Rossi and Soupre, 2017).

Discussion
The out-of sample forecast tests are conducted using Giacomini and Rossi"s (2010) and Rossi and Sekhposyan"s (2016) tests.The pairwise comparison of the model that accounts for instabilities show that t-distribution error assumption dominates all other error distribution assumptions, implying that all the models perform well under tdistribution assumption.Even, in case of traditional test, the t-distribution takes the lead.
The results for Rossi -Sekhposyan test statistics reveals that, IGARCH model under t-distribution and APARCH under t-distribution show the strongest evidence against the forecast rationality, which appears to be around 2009 for all the models.This implies that the Rossi -Sekhposyan test can make accurate forecast using the IGARCH model and APARCH under t-distribution.We can therefore conclude that in the presence of instabilities, the fluctuation tests are more powerful than traditional tests and provide a visual illustration of when predictive ability ap in the presence of instabilities, it is not appropriate to test models" forecasts by using methods that are not robust to instabilities.In fact, as revealed in this study, traditional tests may be invalid in thepresence of forecast instabilities, and more powerful tests should be used.
Policy recommendations that can be made from conclusion drawn is that, fluctuation tests should be used during the periods of instabilities as they can provide visual illustration of when predictive ability appears, especially for an emerging economy like South Africa,where exchange rates are subject to high levels of fluctuations.

FigureFigure 3
Figure 3: Rossekk IGARCHt model.A simple GARCH model can be stated as follows: