Demand forecasting with Machine Learning

Chandan Singh, Chief Product Officer, Thinkdeeply.

Short-term demand forecasting (read days, weeks, or months) is a critical need for businesses and key to efficient warehousing, inventory management, budgeting, and pricing decisions. An accurate forecast can provide massive advantages by allowing companies to maintain adequate stock of products while reducing the overheads and expenses tied to non-moving of slow-moving inventory.

Many mathematical and statistical forecasting methods have evolved over time and still remain the key techniques for forecasting tools and ERP packages. Some common methods include Moving Average, Exponential Smoothing, Auto-Regressive Integrated Moving Average (ARIMA), Holt's Trend, etc. Currently, there are at least 70 different linear and nonlinear methods for quantitative demand forecasting. These methods use historical sales performance, seasonal trends, and other relevant data to make predictions about what sales might look like in the future.

An important assumption of statistical forecasting accuracy is the stability and repeatability of patterns. We assume that history repeats itself, i.e., the situations that occurred two or three years ago will reoccur. The assumptions are far from true. Several internal and external factors can impact the demands. Change in customer preferences and behavior, new products, pricing, promotions, social media, news, weather, and many other events are some factors that can influence demand. Most statistical methods fail to take these into consideration.

But today's increased computing power and capabilities to process a vast amount of structured and unstructured data have enabled a new breed of forecasting methods based on Machine Learning (ML). These ML methods can utilize various data besides past sales numbers like marketing polls, POS data, price, promotions, web traffic, macroeconomic indicators, social media data, key events, and weather data to increase forecast accuracy. Even when using the same basic historical sales data, some ML methods are shown to perform better than the traditional statistical methods; the use of additional data points can increase the forecast accuracy by up to 15%.

The challenge with ML methods for most businesses lies in the complexity of the infrastructure, tooling, and the skills required to build and operationalize these models. While traditional statistical forecasting methods are supported by various tools, ERP platforms, and spreadsheet programs like excel that business managers can easily use, building ML models requires more effort. This requires putting together a development and inference environment for ML, setting up data pipelines, infrastructure to source the variety of data that the ML models may utilize. It also requires that the models and their outputs be easily accessible and usable by various line managers and configurable to their specific context. These extra overheads involved in setting up the ML infrastructure might not pass the ROI for many businesses. For ML models to gain greater acceptance in businesses, they need to be easily assessable and straightforward for business users to use and customize for their scenarios. Many providers are beginning to offer these new ML models as a service, and also, some ERP platforms are starting to provide these as part of their packages, but it all seems early stages.

Thinkdeeply provides an ML-based forecasting solution for use by business users. Users can upload data from spreadsheets or ERP and other legacy systems and use simple drag and drop UIs to develop custom forecasting models. The users can choose to select from various data points to include in their model and use the model evaluation feature to compare different models' performance visually.