Most successful organizations treat data-driven decision as a primary objective and pursue it with religious zeal. However, data-driven decision making, the steps leading to it and how AI is changing it are not well-defined.
In an AI-first company, data-driven decision making means
- Strategic decisions are made by a diverse group including executives that rely on a sufficiently comprehensive set of information. By definition, a bad strategic decision should be important enough to lead to failure of the company.
- Most operational decisions are handled by continuously learning machine learning models which produce explainable decisions. Operational decisions are frequent (once a week or more frequent) and not critical (a single mistake is unlikely to lead to failure of the company).
- Operational decisions which can not be automated with good accuracy are delegated to humans.
- If data is lacking, opinion based decisions are made.
- If data exists and has been analyzed, decision maker relies on analysis.
- If data exists but is not analyzed yet, cost of analysis determines whether an opinion or data based decision will be made.
Before settling on this framework for decision making for modern corporations, we need to identify how we can evaluate different decision making models. However, if you like you can directly skip to the sections that interest you:
How to assess different decision making models?
Decision making has 3 KPIs: Speed, quality, auditability. Lack of any of these can lead to criticial failures.
- Slow decision making cripples organizations and allows competitors to gain market share.
- Bad decisions lead to failure. A bad decision, no matter how well executed, is unlikely to generate value. For example, Motorola’s decision to continue investing in Iridium satellites which cost billions of dollars resulted in a big write-off for the company. Great execution could not change the outcome as mobile communication was providing a cheaper and higher quality alternative to satellite phones by the time Iridium phones were launched.
- If decisions can not be audited, they are hard to improve. Decision quality is difficult to measure at the time of the decision but can be estimated later by considering other alternatives and current market conditions. Such postmortem analysis should lead to better decisions
What are different decision making models?
Decision making models starting from the lowest level of sophistication are:
1- Opinion based
While it provides fast decisions, decision quality can be low which is dangerous for strategic decisions. However, it can be applied for fast decisions making in cases where:
- High quality data is not available
- Cost of analysis is more than the value of a good decision
2- Driven by limited amount of data
Organizations with data quality or availability issues are likely to rely on limited amounts of data to make decisions. This is potentially the most dangerous decision making model as decision makers could use limited data to support their opinions, leading to widely supported but opinion based decisions.
A significant share of consulting projects fall under this category where consultants could be incentivized to find limited data points to support opinion based hypotheses.
Decision quality can be better than opinion based decision making if decision makers are open to new information, take into account psychological biases, vary their approach to making prediction (inside view vs outside view) and look for answers rather than searching for data to support their opinion.
3- Data-driven decision making via manual analysis
For this model, data needs to be available, high quality and there needs to be consensus about the correctness of data. It can be preferable for important decisions where timing is not critical and where cost of automation is prohibitive compared to its benefits.
4- Automated data-driven decision making
Data is available and high quality and there is consensus about the correctness of data and it has been processed so decisions are made automatically. Based on outcomes, the decision making model continuously improves itself over time. However, audit is challenging when reasons for decisions are not provided.
5- Automated data-driven explainable decision making
In addition to automated data-driven decision making, explainable machine learning models are used to make decisions. Explanations help audit decisions and continuously improve the underlying model via manual interventions on top of automated continuous learning.
Why I wrote this?
I spent most of my professional career at McKinsey where data-driven decision making is one of the core principles. It was a great principle but quite challenging to achieve as a consultant without access to internal data. Given my current experience with AI, I wanted to think how we can better define data driven decision making.
How useful was this post?
Click on a star to rate it!
Average rating / 5. Vote count:
What did you like about this post?
Appreciate it if you leave your name and surname so we can publish your review as a testimonial
Thank you for your feedback!