Time series is nothing but a sequence of values that are indexed in time order. That simple structure has made it very easy to collect in almost all business activities. Hence, time series have become such a widespread commodity in today’s Advanced Analytics that every data-driven organization uses them to gain insight about its business practices. Saying that, the abundance and simple structure of time series shouldn’t deceive you. Modeling objectives for time series forecasting can differ drastically from one business problem to another. This blog post is going to explain major forecasting objectives and their associated use cases in the real world setups.     

Perhaps, this is the most conventional use case of time series modeling. Application of this use case spans from forecasting the specific number of daily users to high level warehouse storage capacity. It can also accommodate a broad range of horizons from short term hourly API requests to long term annual revenue forecast. The main 

 of precision forecasting is to forecast future outcomes with the 

. Almost all forecasting methodologies are initially developed to achieve this objective. The common workflow for univariate precision  forecasting includes data preprocessing and transformations in tandem with model selection, model tuning and sometimes ensembling. If it is possible to add more information to the solution and train a multivariate model, data enhancement will also be added to the workflow. 

Figure 1 : Example of a precision forecasting model

All aforementioned steps can be done either manually or by leveraging AutoML packages like 

 which can handle most of these steps automatically. However, reaching maximum accuracy still requires a dedicated business analyst/data scientist with great domain knowledge that can validate the initial outputs and revise intermediate steps to generate high quality results for each forecast. 

Due to their single dimension, time series are inherently susceptible to be modeled without enough context. Multivariate approaches try to overcome this drawback by enhancing the learning process and bringing more dimensions to the model. However, seasoned forecasters are well aware of the fact that no matter how much effort is put into precision forecasting, there will be times that things are completely out of hand. Inaccurate forecasting outcomes might happen either because of a black swan event or as a consequence of an unexpected change in a known external factor. 

Black swan events by definition are unpredictable. For instance, when Covid-19 first hit, almost all business forecasts missed it. However, right after its first appearance, it became a known factor with significant effect on the target variable. Scenario-based forecasting provides a framework for businesses to leverage this extra piece of information into their decision-making process in a practical way. 

In particular, any organization with intent to maintain a high level of availability and reliability must have a set of contingency plans in place to mitigate unexpected situations. In scenario-based forecasting, the main 

 how an external factor would affect the target timeseries

. To do that, a known time varying external factor (regressor) is being incorporated into the model. The main assumption here is to have a good intuition about the regressor's  future behavior. This setup gives the modeler a unique capability to create different what-if scenarios with various viabilities. Then, the modeler can forecast the target variable under those scenarios for a given horizon and see how changes on the regressor will affect the outcomes. 

Figure 2 : Effect of different scenarios on forecast outcomes

Although scenario-based forecasting can be considered as an extension to precision forecasting and introducing meaningful regressors can significantly improve model accuracy, only a few forecasting methodologies can handle external regressors. Similar to precision forecasting, scenario-based forecasting also needs a dedicated data analyst/scientist with great domain knowledge to build reasonable scenarios in order to reveal valuable insight for strategic planning and risk management or sensitivity analysis. 

On another note, most of the time these regressors should be only seen as driver factors for the target variable. Specifically, regressors shouldn’t be mistaken for causality, which highlights the significance of regressor selection. For instance, real estate demand might be affected by interest rates but not caused by it.

The two previous use cases are mostly about making forecasts for a few time series that are considered as high value targets for the business. However, many data-driven businesses also need a large number of daily forecasts at different levels which might range from a few hundreds to hundreds of millions! The results of these forecasts might also be used in a bottom-up aggregation for total demand calculation or in a top-down setting for resource allocation. Regardless of the application, it is evident that forecasting at this scale can not be done manually. The main 

 to generate forecasts at large scale with 

minimum acceptable accuracy at individual time series level 

. Depending upon the number of forecasts and required accuracy, there are three major approaches for batch forecasting:

Naive methods are widely being used when a large number of point approximations are required as initial numbers. In this use case, reaching a high level of accuracy is not a major concern since these point approximations are just starter values and most probably being adjusted at operation level. These approximations can be generated by applying a single or a combination of naive methods like last value, moving average, seasonal average or drift on time series. Despite their simplicity, these methods can show decent performance for estimating non-sensitive target variables at aggregate level.

For instance, a CPG company has been successfully using a seasonal naive method for reloading more than 4.5 million vending machines. In this business problem, the exact point estimate for each vending machine is not as important as the truck load estimate, which is the aggregate of multiple vending machines in the same neighborhood. Interestingly once covid hit, the company was able to easily adjust the demand at local distributed centers by delivering only a fraction of their estimate for commercial sites.

This approach basically applies one of the precision forecasting methodologies to all time series in the batch. So any increase in the number of time series directly translates to computation time increase. Although with O(N) time complexity, this approach is not considered as highly scalable but parallel implementation of it can handle hundreds to thousands of time series. The major challenge with this approach is the necessity of custom data processing and hyper parameters tuning for all generated models. As we discussed earlier, precision methods need dedicated data transformation and hyper parameter tuning, which is in contrast with batch forecasting objectives to be scalable and automated. There are few strategies to address these issues:

Running a clustering algorithm to identify time series with comparable characteristics and applying a set of similar data preprocessing and hyper parameters to them. Dynamic Time Warping (DTW) and statistical feature extraction are two major approaches that can be used in this context to cluster time series.  

For a more diverse set of time series, it is possible to use a meta learning approach. The meta learning employs ML/DL model to recommend a proper set of hyper parameters for a given forecasting method based on the time series characteristics features. Major challenge in training a useful meta learning model is to collect a rich dataset of different time series characteristics as a feature set and their associated well optimized hyper parameters as labels.

Another possibility for forecasting a diverse set of time series is to utilize optimization techniques like Genetic Algorithm or Simulated Annealing to efficiently search the hyper parameters space for best combinations. One advantage of using this approach for batch forecasting is that initial guesses for new time series optimization can be borrowed from previous rounds of optimization. In other words, optimization cycles can learn from different time series.

 is one the best implementations of this approach in python which makes use of Genetic Algorithm to find the best set of transformations and hyper parameters for batch forecasting at scale.          

3-3 Machine Learning/Deep Learning Methods:          

The forecasting approaches that have been discussed so far are suffering from limited scalability or insufficient accuracy and consistency in results. Furthermore, these systems are difficult to maintain. Most of them are incapable of cross learning from other time series or extra non time series data. They are also unable to tackle cold start problems for new input series with short or none history. To overcome these limitations and address new needs, businesses (initially big techs) started to work on ML/DL solutions as an alternative to bring all these requirements under a unified forecasting engine.  

Machine learning approaches are not inherently designed to discover sequential order embedded in time series data. Therefore, time dependency should be introduced to the model through lag features and date attributes. A basic feature vector for training a machine learning model is composed of lag values, date attributes, 

 and extra contextual variables. Contextual variables could be all information that can differentiate one data point from others. For instance, information like page views, review rating, search history can be considered as contextual data for e-commerce use cases. In theory, machine learning models are capable of picking up similar patterns from time series characteristics and contextual features. A well built and tuned machine learning model can address the above issues to a certain point. However, there are still two unresolved problems: 

First, this approach has an accuracy cap, which is hard to pass beyond it. 

Overview of different use cases and methodologies

Time series have become such a widespread commodity in today’s Advanced Analytics that every data-driven organization uses them to gain insight about its business practices. Read more to learn how.

Time series Forecasting in Advanced Analytics

This post will focus on the external data selection process when it comes to choosing regressors to include in scenario-based forecasting.

There are a few different types of external data we can use in forecasting: covariates which are series with no known future, static features which are single variables that describe your target, and future regressors where you have both past and future values that line up with your target series’ history and forecast horizon. We’ll be focused on future regressors here because of their centrality to scenario-based forecasts. 

Scenario-based forecasting is all about measuring and planning for the effect that some external variable, or set of variables, will have on a target series to forecast. This is a powerful forecasting technique because it allows for the direct entry of human intuition into a forecast. In traditional precision forecasting, the levers a data scientist has to work with to adjust their forecast are often limited to model choice and corresponding parameters. While this is important for generating a well fit model (or properly generalized model), what we get is that every well-fit model will likely produce very similar results. This is because these forecasts are generated solely based on the historical patterns of the target series.

Scenario-based forecasting allows us to move beyond this limited approach and into one where we have more ways to apply intuition to the forecast. Without it, we are limited to forecasting recognized trends in the data, which in many cases is all that is needed, but for when the future is expected to behave differently because of external factors, we use scenario-based forecasting.

In a previous post, we make the point that regressors should be seen as driving factors of the target variable, not a causality source. This is a nuanced point, but makes up the fundamentals of our understanding on how to choose a set of regressors to use in a scenario-based forecast.

​​We make the distinction between regressors that have an effect on a forecast variable and regressors that are helpful to a model used in forecasting. If you were to look at the statistical significance of the regressors, we must be aware that any procedure involving selecting regressors first will invalidate the assumptions behind the p-values. What we are interested in here is selecting predictors that are helpful when a model is used for forecasting; the methods discussed here would not hold if you wanted to study the effect of any predictor on the target variable.

We focus heavily on the regressor selection process because it really becomes the driver of the forecasting process. One could easily select regressors that move the forecast in any direction they like, so making an informed decision on which regressors to use is critically important.

Let’s look at a few ways of breaking down different types of regressors. This will give us ideas pointing us where to look for the external data we’ll use.

These series can be described as series that are an upstream component of the target time series.

Y = f(x) where Y is your target and x is your regressor, f is some potentially unknown function.

Target: total revenue, upstream regressor: number of units sold. This is the flipped version of downstream relations. The upstream regressor is an input into the equation that leads us to our target variable.

Target: price of oil, upstream regressor: # of barrels exported from Canada

Target: number of units sold, upstream regressor: $ spent on marketing

Target: airline ticket from L1 to L2, upstream regressor: price of oil

This kind of regressor can be helpful when we only have a partial understanding of all the inputs that go into a target variable. Otherwise, it simply makes more sense to just calculate it outright given we have all the inputs to that function.

These series can be described as series that are a downstream product of the target time series. 

Y = f(x) where Y is your regressor and x is your target, f is some potentially unknown function.

Target: number of units sold, downstream regressor: total revenue. Total revenue is a function of the number of units sold. If we have a better understanding of revenue projections, perhaps because of SBF done on that variable, we have the ability to “pull” the forecasted number of units sold into place.

Target: price of oil, downstream regressor: price of gasoline

These examples often don’t make the most logical sense because an analyst would usually have a similar, if not a decreased level of intuition about the future values of the regressor. Nonetheless, if downstream regressors can be forecasted with a similar process, their effect can be valuable.

These series can be described as series that have a certain correlation to the target series. However, the analyst must be careful not to blindly assume that correlation equals causation. See 

https://www.tylervigen.com/spurious-correlations

Z = ax + by, where x is your target, y is your regressor, and a & b are unknown yet 

 coefficients. Z is unknown, irrelevant, and fixed at any given point in time.

Target: number of units sold, complementary regressor: number of complement units sold

 Target: price of gold, complementary regressor: inflation rate in the US

We expect that in most cases the target and complementary/substitutional regressors are completely independent of each other. Compared to other kinds of regressors like upstream and downstream that directly influence the target, either by pushing from upstream or pulling from downstream, relations between this kind of regressors are through some kind of proxy. Proxies to think about could be things like supply & demand, etc.

Chronos has the ability to add external time series to your precision forecast experiments. These additional series are best used to help the forecasting algorithms fit the target variable data and also to inform the model about a future scenario.

 is a powerful interface that lets the user upload their own data. This data is likely to be as current as your target variable and with the addition of custom linear components the user can then design the shape of the scenario into the future. As you design the scenario, you should ensure that it expands into the future as far as the horizon of your target series will.

On the external data selection process that drives scenario-based forecasts

Future regressors are central to scenario-based forecasting and choosing them can be a challenge. We overview three categories of future regressors to help users start building a mental model.

Choosing Regressors for Scenario-based Forecasting

Time series are a string of numbers varying across time. It could be the weather, sales of tissue paper, electricity demand of a region, or countless other things. There are plenty of useful things to do with this type of data, but here, the focus is on forecasting: predicting the future values of these series for the purpose of guiding decision making.

Almost all time series forecasting begins by considering that the series in question represents two basic elements, the 

. The signal is the underlying pattern we seek to predict, and the noise are random events such as errors in the recordings of the data. 

of time series data can include many adjustments, but is largely, and for our purposes here, entirely, focused on removing the noise so that the underlying signal is easier for models to predict.

However, to really understand what is to be done, it is important to realize that there really is no such thing as “useless noise” but rather 

. For example, sales data may be primarily driven by a weekday signal (highest sales on Saturday, for instance) and a yearly signal (highest during Christmas shopping) but then also has a mixture of global events, national events, local events, promotions, competitor marketing, weather, construction near stores, and so on indefinitely which all together produce the final sales on any particular day. Which of these are the primary signals depends mostly on the availability of quality data, the amount of effort expected for the forecasting project, and the final model to be used.

Time series specific preprocessing is then usually about separating signals. A 

 is where a signal is handled by another method. For example, a detrending might use a linear regression to handle a secondary growth signal, then passing the data, without a trend, to be handled by the primary model. A 

is where signals are removed entirely. Most commonly this is a rolling mean or exponential rolling mean where each period becomes the average of that and several previous periods.

Smoothing of a series to produce a cleaner time series. Note a potential problem here, where the max values of the peaks have been lost. That maximal information may be important for cases where peak volume is a concern.

There is no single reason why these are to be used. Usually filters are used because models, especially machine learning models, will overfit, especially when too little history is available to show the signals true pattern. Oftentimes they are advantageous simply because the output forecasts are then smoother and therefore more visually appealing than they might otherwise be. Decompositions are useful to handle signals that the downstream model will not handle as well. Some things, like PCA, are both a decomposition and a filter simultaneously.

 done simply to shift the data to better fit the assumptions of the model. A standard scaler as is used for many machine learning models can be used here, but there are additional transformations as well. Perhaps most well-known is differencing, subtracting the previous value from each period, used to remove the effects of a random walk before many models such as ARMA models. 

A simple model, coupled with proper preprocessing, will almost always outperform a “more advanced” model without preprocessing.

A random walk and the differenced random walk. Note just how different these series appear at first glance.

The most important thing to remember, when using a transformation pulled in from the broader data science world, is to be very very careful about making sure it doesn’t transfer information backwards on the time series to past points from the future. This results in erroneously high accuracy, which won’t be repeated in a real-world test.

There is no one set of preprocessing steps that will work for all models and datasets. A few, however, are frequently valuable. Outlier removal and scaling (RobustScaler, MinMaxScaler, StandardScaler from scikit-learn) are sometimes helpful and usually a safe bet. I have also often found the QuantileTransformer to be useful. Differencing is helpful especially with traditional statistical methods which may require it. Finally, and often most valuable, is either rolling mean or exponential smoothing (.rolling() or .ewm() in pandas). Unlike the others, which require little or no parameter tuning, exponential smoothing may require some adjustment. Too little smoothing, and it won’t make a difference, too much smoothing and important signals may be lost.

Two example transformations are provided for the same real-world dataset. Both of these transformations are automatically-tuned, high-quality transformations. However, the output series are radically different. The data is monthly data for the Economic Policy Uncertainty Index for the United States from FRED.

Quantile Transform and Poisson Detrending

The first, using a Quantile Transformer and a Poisson detrending, has shifted the series to be around 0 and has emphasized the peaks in the dataset but generally retains the shape of the series. This was optimized for a machine learning forecasting model. 

Scaling, differencing, and linear detrending

The second transformation here uses scaling, differencing, and linear detrending to produce a zero centered forecast that looks entirely different from the starting series. Experienced users will probably not be surprised that this was optimized for a statistical forecasting technique, as indeed the final output looks more like the ARMA processes that these models were usually designed for.

There are a couple of lessons from this exercise. The first is to realize that different models have different optimums for what preprocessing should be used. The second is that it can often be difficult to simply choose the optimal preprocessing by hand, and that searching for combinations across cross validation sets is important. Many data scientists don’t consider this aspect, tuning their model parameters but not their entire pipeline as a unit. AutoML packages can address this issue, using, for example, TPOT for machine learning generally, and AutoTS for time series forecasting specifically.

Hopefully this has been a helpful introduction to preprocessing for time series. Thanks for reading.

Smoothing of a series to produce a cleaner time series

A random walk and the differenced random walk

What it is & why to do it

Preprocessing of time series data can include many adjustments but is primarily focused on removing the noise so that the underlying signal is easier for models to predict.

Latest posts