Demand forecasting for fresh produce and groceries with ML and RNN

Consumers’ expectations when shopping for groceries and produce are straightforward; they demand the freshest of stocks at competitive prices.

Traditional local sellers know their neighborhood and have been catering to it for decades, they are driven by their wisdom and experience.

But today we live in a world of conveniences where everything is home delivered at the click of a button.

Conventional wisdom of a local grocery is not scalable for online grocery platforms since consumer preferences and demand-supply vary across regions in a city.

When applied scientifically, Cloud and ML can help not just optimize demand-supply at a hyperlocal level, but also help businesses achieve better top line and bottom line through reducing shrinkage (wastage of perishable goods)

At Divum, we developed a custom algorithm for a leading groceries, fresh fruits & vegetables e-commerce platform to meet the following business objectives:

  • Forecast demand
  • Enable optimal procurement quantity
  • Maintain healthy inventory lines
  • Reduce shrinkage

The challenges were multifold:

Forecasting demand for perishable goods like fruits and vegetables is challenging due to their low shelf life.

Meaning, inaccuracies in forecasting can cripple the platform’s promise to deliver the freshest produce direct from farm or the platform running out of stock which prompts customers to churn out, or abandon their shopping carts.

Second, there are hundreds of SKUs and demand-supply variables for each product at a hyper local level, further amplified by external factors like festivals and long weekends.

Crunching this intelligence alongside devising dynamic promotions and offers requires multidimensional forecasting, where conventional statistical models are grossly inefficient.

Divum’s Approach:

We started with first principles by understanding all the drivers of demand including: social and macro economic factors of the micro markets the platform was servicing, alongside additional factors like seasonality (1st week of the month, regional festivals.)

Stock-outs, planned promotions, discounts, assortment range and variety of SKUs were other considerations to be kept in mind.

This model was further strengthened by combining it with data points such as a customer’s past buying history, impact of promotions, relevance of buying cycles and more.

We then built a scalable demand forecasting ML model on Google Cloud Platform (GCP) using the TensorFlow ML package.

Our proprietary ML algorithm is built on principles of Time Series Forecasting using state of the art Deep Learning Recurrent Neural Network (RNN).

SSidenote: Ever wondered how Gmail gives you smart suggestions to complete your sentence when composing a new mail? They use RNN!

We further sharpened the prediction algorithm by factoring in additional regressors such as pin code-SKU preferences, weather and also corrected for outliers and missing values.

Tech Stack deployed:

The entire framework was built on Google Cloud Platform for scale and elasticity. ML was built on TensorFlow, codebase deployed on App Engine while data was handled by Firebase, Datastore and Big Query.

Deployment and Measurable Outcomes:Deployment and Measurable Outcomes:

We deployed the forecast algorithm in an iterative and incremental manner so as to create a self improving feedback loop. Post deployment of our algorithm, usual 12% perishable inventory wastage was reduced to 6% in the initial iterations.

The impact was measured across matrices, we observed 5% increase in farmers’ margins, 2X reduction in wastage, 15% savings on last mile costs, and 5X reduction in acquisition costs.