The Catalytic Role of Data science in the Retail Industry
By Krishiv Anand
Data science and analytics have been around for a long time. However in the last decade, their applicability has soared at an accelerated pace. Retailers are stuck in a world of razor-thin margins. They not only have to exhibit superior execution at the core of every function, but also have to offer a highly differentiated and personalized shopping experience to their customers. The average human’s attention span is eight seconds, compared to nine seconds for goldfish [1]. This essentially means that a customer spends under eight seconds evaluating an advertisement, promotional email, banner, and reels on social media before going down the path-of-purchase or moving on to the next distraction on the internet. In a world of limitless distractions, a retailer must make their mark in a few seconds to compel and convince the potential customer that the content they are seeing is highly personalized and tailored for them. This is where the world of data science comes into action. At its core, data science is a blend of math, programming, statistics, analytics, and artificial intelligence (AI), delivering actionable insights to an organization.
The insights and findings that data science has enabled are the predominant reason why organizations have completely transformed the way they evaluate and tackle different opportunities. From data mining to ingesting contextual data from a variety of external sources, data science has enabled organizations to modernize their teams, business processes, and infuse objectivity in every aspect of the business. This article outlines the role of data science in the world of retail.
Figure 1. Data Investigation and Modeling Process [2]
The world of retail is evolving and innovations become table stakes in a matter of weeks due to dynamically transforming customer behavior, expectations, and engagement.. Data science and advanced analytics are used in almost all business areas in retail, including (but not limited to) marketing, customer acquisition, supply chain, store signage, store expansion, product placement, supply chain management, competitor benchmarking, and credit card usage trends. The biggest hurdle for an organization is the cultural transformation to drive adoption of data science as an integrated part of the decision-making process.
While there are endless cases where data science can be a catalyst to transform various functions, in this paper, we are going to evaluate three scenarios and provide details on the problem statement, approach, and outcomes.
Scenario #1: Customer Spending Level Analysis
Problem Statement: A retailer specializing in electronics and appliances (E&A) would like to determine the customer spending in their category across all income levels and competitor base. The findings would enhance the overall promotional strategy for the organization.
Approach: While the retailer has years of historical data for their business, they need additional data to understand the customer spending trends in the E&A category across the competitive landscape. The data and analytics team might want to consider the following steps:
- Gain access to credit card spending data from companies such as Earnest Data.
- This data will enable the retailer to view customer spending, by department, across all retailers in North America.
- Once the analytics team can ingest the data into their operational reporting, they will need to normalize the data and develop key findings for their category (E&A).
- The last step would be to partner with their marketing team to conduct market-level tests to validate the acceptance rate of certain targeted promotions.
Outcome: Data science can be a key enabler for this organization to change its overall promotional strategy. Once the team has developed several linear regression models using internal and external data, they can enhance the overall performance of the organization:
- Develop a weekly model by ingesting weekly credit card spending data. This model can depict the traffic and spending trends from the previous week and forecast similar metrics for the next week.
- Operationalize this model by partnering with the finance team and the promotional team to ensure that the efficacy of the approach is validated with the broader stakeholder group.
Scenario #2: Demand Forecasting
Problem Statement: A seasonal home decorator retailer is trying to develop a highly targeted forecasting methodology to predict demand for their products.
Approach: Since the retailer is a seasonal retailer, they need to be 80% on target from a forecast perspective. If they are unable to meet this threshold, they will have to liquidate excessive products at clearance pricing, which is 75% below retail price, causing significant cash flow issues for the organization, potentially leading to bankruptcy.
The data and analytics team should embark on a mission as stated below.
Start with the base forecasting model and add the following external data sources to the model:
- New home sales for every zip code.
- Level of disposable income available in certain markets.
- Weather patterns from the last 3 years and their effect on sales.
- New competitor stores that have emerged in the last 3 years within a 3-mile radius of all their stores.
Outcome: Once the data and analytics team operationalizes their new model, they will be able to deliver the following benefits:
- The new home sale data will enable the team to adjust their forecast and allocate products at a regional level to markets where new home sales are higher than the national average.
- Based on the level of income in certain markets, the team will be able to forecast the level of promotions that would be needed to boost sell through.
- Weather patterns can also be a key factor since based on the level of precipitation and heat, the forecasting trends can change.
- Lastly, optimizing the forecasting model based on the new competitor stores within a 3-mile radius can further enhance the efficacy of the forecast.
Scenario #3: Personalized and Targeted Marketing
Problem Statement: Today’s consumers face a dilemma. They are constantly being presented with a promotion or an offer on their devices, and the traditional way of weekend offers is in the physical mailbox.
While a consumer may find most of these offers moderately compelling, they can also be overwhelming. Retailers are constantly developing new and relevant offers for their targeted customer base, hoping that these offers will convert to a sale.
The terminology that is commonly floated around is, “How can a retailer produce an product offer that is personalized just for me? Should it be cool and not creepy? I want the retailers to know me, but I want them to respect my privacy.”
Approach: The analytics team needs to build several data models to capture internal, external, and social data elements to develop a personalized campaign for their customers:
- A gamification model that allows customers to engage with the brand’s mobile app. The team can monitor the behavior and develop offers for their customers.
- Build a recommendation model that uses transaction history from millions of customers to determine if a customer bought a certain product, and what other products they would require to complete their shopping experience. For example, if a customer bought a disposable tablecloth, the model would offer disposable spoons and forks at a 10% discount..
- Lastly, implement a personalization tool to drive a personalized website experience based on an individual’s browsing history and click-through information.
Outcome: All these models can enable the retailer to increase their promotional effectiveness and drive a highly personalized shopping experience for their customers.
Based on the scenarios stated above, the application of data science can be extremely effective in all facets of the business and drive positive results across the enterprise. Whether it be driving incremental revenue or optimizing cost, data science adds a level of objective problem-solving approach to every opportunity area within the organization.
Discussion
Before an organization embarks on the data science journey, there are a few questions that should be addressed:
- Does the organization have the right set of leaders and talent pool to lead analytics and data science?
- Can the organization ingest and operationalize the insights?
- How will adoption be driven into every function. How much modeling is too much? There is always an element of science and art involved with every business function. What is the right level of mix for every function?
These questions represent a handful of topics that are being debated, but the reality is that data science is going to be at the core of every business function. This capability is only getting started, and with the infusion of machine learning and Gen AI, the models are going to get more sophisticated and deliver value to organizations at an unprecedented pace.
About the author
Krishiv Anand is a high school senior at Coppell High School in Texas. He has an extreme passion for data science and its applicability to the retail industry. Krishiv has been an active contributor to the Dallas community by founding the Care for Kids club that focuses on organizing annual fundraisers to raise money to donate school supplies to schools in impoverished areas. In addition, Krishiv has held various research positions at Southern Methodist University and the University of Texas at Dallas. He is also the Senior Member and Underclassmen Mentor at Coppell High School Youth and Government Club and a member of the National Honor Society.
By Krishiv Anand
Data science and analytics have been around for a long time. However in the last decade, their applicability has soared at an accelerated pace. Retailers are stuck in a world of razor-thin margins. They not only have to exhibit superior execution at the core of every function, but also have to offer a highly differentiated and personalized shopping experience to their customers. The average human’s attention span is eight seconds, compared to nine seconds for goldfish [1]. This essentially means that a customer spends under eight seconds evaluating an advertisement, promotional email, banner, and reels on social media before going down the path-of-purchase or moving on to the next distraction on the internet. In a world of limitless distractions, a retailer must make their mark in a few seconds to compel and convince the potential customer that the content they are seeing is highly personalized and tailored for them. This is where the world of data science comes into action. At its core, data science is a blend of math, programming, statistics, analytics, and artificial intelligence (AI), delivering actionable insights to an organization.
The insights and findings that data science has enabled are the predominant reason why organizations have completely transformed the way they evaluate and tackle different opportunities. From data mining to ingesting contextual data from a variety of external sources, data science has enabled organizations to modernize their teams, business processes, and infuse objectivity in every aspect of the business. This article outlines the role of data science in the world of retail.
The world of retail is evolving and innovations become table stakes in a matter of weeks due to dynamically transforming customer behavior, expectations, and engagement.. Data science and advanced analytics are used in almost all business areas in retail, including (but not limited to) marketing, customer acquisition, supply chain, store signage, store expansion, product placement, supply chain management, competitor benchmarking, and credit card usage trends. The biggest hurdle for an organization is the cultural transformation to drive adoption of data science as an integrated part of the decision-making process.
While there are endless cases where data science can be a catalyst to transform various functions, in this paper, we are going to evaluate three scenarios and provide details on the problem statement, approach, and outcomes.
Scenario #1: Customer Spending Level Analysis
Problem Statement: A retailer specializing in electronics and appliances (E&A) would like to determine the customer spending in their category across all income levels and competitor base. The findings would enhance the overall promotional strategy for the organization.
Approach: While the retailer has years of historical data for their business, they need additional data to understand the customer spending trends in the E&A category across the competitive landscape. The data and analytics team might want to consider the following steps:
- Gain access to credit card spending data from companies such as Earnest Data.
- This data will enable the retailer to view customer spending, by department, across all retailers in North America.
- Once the analytics team can ingest the data into their operational reporting, they will need to normalize the data and develop key findings for their category (E&A).
- The last step would be to partner with their marketing team to conduct market-level tests to validate the acceptance rate of certain targeted promotions.
Outcome: Data science can be a key enabler for this organization to change its overall promotional strategy. Once the team has developed several linear regression models using internal and external data, they can enhance the overall performance of the organization:
- Develop a weekly model by ingesting weekly credit card spending data. This model can depict the traffic and spending trends from the previous week and forecast similar metrics for the next week.
- Operationalize this model by partnering with the finance team and the promotional team to ensure that the efficacy of the approach is validated with the broader stakeholder group.
Scenario #2: Demand Forecasting
Problem Statement: A seasonal home decorator retailer is trying to develop a highly targeted forecasting methodology to predict demand for their products.
Approach: Since the retailer is a seasonal retailer, they need to be 80% on target from a forecast perspective. If they are unable to meet this threshold, they will have to liquidate excessive products at clearance pricing, which is 75% below retail price, causing significant cash flow issues for the organization, potentially leading to bankruptcy.
The data and analytics team should embark on a mission as stated below.
Start with the base forecasting model and add the following external data sources to the model:
- New home sales for every zip code.
- Level of disposable income available in certain markets.
- Weather patterns from the last 3 years and their effect on sales.
- New competitor stores that have emerged in the last 3 years within a 3-mile radius of all their stores.
Outcome: Once the data and analytics team operationalizes their new model, they will be able to deliver the following benefits:
- The new home sale data will enable the team to adjust their forecast and allocate products at a regional level to markets where new home sales are higher than the national average.
- Based on the level of income in certain markets, the team will be able to forecast the level of promotions that would be needed to boost sell through.
- Weather patterns can also be a key factor since based on the level of precipitation and heat, the forecasting trends can change.
- Lastly, optimizing the forecasting model based on the new competitor stores within a 3-mile radius can further enhance the efficacy of the forecast.
Scenario #3: Personalized and Targeted Marketing
Problem Statement: Today’s consumers face a dilemma. They are constantly being presented with a promotion or an offer on their devices, and the traditional way of weekend offers is in the physical mailbox.
While a consumer may find most of these offers moderately compelling, they can also be overwhelming. Retailers are constantly developing new and relevant offers for their targeted customer base, hoping that these offers will convert to a sale.
The terminology that is commonly floated around is, “How can a retailer produce an product offer that is personalized just for me? Should it be cool and not creepy? I want the retailers to know me, but I want them to respect my privacy.”
Approach: The analytics team needs to build several data models to capture internal, external, and social data elements to develop a personalized campaign for their customers:
- A gamification model that allows customers to engage with the brand’s mobile app. The team can monitor the behavior and develop offers for their customers.
- Build a recommendation model that uses transaction history from millions of customers to determine if a customer bought a certain product, and what other products they would require to complete their shopping experience. For example, if a customer bought a disposable tablecloth, the model would offer disposable spoons and forks at a 10% discount..
- Lastly, implement a personalization tool to drive a personalized website experience based on an individual’s browsing history and click-through information.
Outcome: All these models can enable the retailer to increase their promotional effectiveness and drive a highly personalized shopping experience for their customers.
Based on the scenarios stated above, the application of data science can be extremely effective in all facets of the business and drive positive results across the enterprise. Whether it be driving incremental revenue or optimizing cost, data science adds a level of objective problem-solving approach to every opportunity area within the organization.
Discussion
Before an organization embarks on the data science journey, there are a few questions that should be addressed:
- Does the organization have the right set of leaders and talent pool to lead analytics and data science?
- Can the organization ingest and operationalize the insights?
- How will adoption be driven into every function. How much modeling is too much? There is always an element of science and art involved with every business function. What is the right level of mix for every function?
These questions represent a handful of topics that are being debated, but the reality is that data science is going to be at the core of every business function. This capability is only getting started, and with the infusion of machine learning and Gen AI, the models are going to get more sophisticated and deliver value to organizations at an unprecedented pace.