Or why mathematical optimisation is an indispensable tool for any data scientist worthy of the name.
The often Unnoticed Frontier
In the universe of articles you can find on the Internet, which try to shed some light on the differences and overlaps between Artificial Intelligence, Machine Learning or Data Science technologies, a crucial boundary is often overlooked. A very important nuance for selecting the tools that a data scientist must handle with solvency, those that allow him to get the most out of the data available to solve real business problems.
That boundary is the one that is created between the ability to predict that aspect of reality that our models work with, and the need we have to influence that same reality, to act on it. In many projects, all the technological apparatus deployed ends up concentrating on illuminating uncertainties: What is the probability that a customer will cancel a subscription with my company? What evolution do I expect for the prices of the products I buy or sell, or the demand of my customers?
Customers, orders, resources or products are some of those realities on which our business depends, and knowing what will become of them is always crucial. What many of the solutions being invested in today forget is that we have to make decisions about these realities.
Predictions vs. Decisions
In other words, data scientists leave us facing danger alone when we have to leave the comfortable role of spectators or when we have to take action and the churns, prices or demands we have predicted have to be converted into production schedules, portfolio configurations or resource allocations in a telecommunications network (the universe of scenarios where complex decisions take place is almost infinite).
Sometimes, the decision is implicit in the prediction itself (e.g. if I have a limited budget for a retention marketing campaign, I will target those most likely to churn), but this is not the norm. On these other occasions, a reflection comes to the fore that we hope will be of interest to those who make important decisions in their companies; a reflection that also has practical relevance for those IT departments who have to service them, and who are not realising that they are being left half-heartedly in the lurch.
The Nature of your Decisions
There is no important decision, no management worthy of the name when there is no risk, nor scarcity. We always manage based on scarcity because that is how the work of the decision-maker, the one who defines objectives, assigns resources, prioritises or delays tasks, can be distinguished as excellent or catastrophic.
There is never infinite equipment, nor materials, nor clients for all competitors, nor productive capacity, and if as a manager you analyse the reality of your scarcity, and only if you begin to identify the resources that condition it, you can begin to discover the nature of the decisions that fall on your shoulders (and to sense that we have to ask more of the solutions that data scientists offer us).
As the next step in this deconstruction, some questions that only you can answer in full knowledge of the facts will be useful:
- Where in your company’s value chain are you influencing – is it a far-reaching (strategic) or a day-to-day (operational) decision?
- How do you make that decision, defencelessly or tool-supported?
- With what level of uncertainty are you making decisions?
- How many options are available to you for each of the key decisions?
- Are there constraints on the table, or relevant connections between the elements involved in your decision?
- Do you have the tools to determine precisely what the foreseeable outcome of each choice would be? That is, can you compare them accurately?
The first ones are simpler, almost a job description; the final ones are not so simple and are essential for the reflection we want to induce with this article.
Variability
At the risk of being simplistic, if the decision we have to make is how to fit together a two-piece puzzle, no matter how big the impact of what we decide, the variability is minimal. The decision-making process can even consider all the options, evaluate their implications and assess them calmly before executing.
On the contrary, in many real scenarios, the combinatorial explosion makes the range of possibilities unmanageable. How many options do we have to select a portfolio of securities that meet specific criteria, if we have a menu of 5 options? It doesn’t seem too many, but… what if you have to compose a portfolio with 10 stocks from the S&P 500 companies? Or how many locations do I have to analyse to locate cells of a 5G operator, or windmills of a wind farm, when each location chosen influences the others? It is clear that in these cases a decision-maker cannot consider them all, let alone assess the profitability or risk of each of these possibilities.
Restrictions
In the example of the puzzle, the decision-maker has a free hand to explore his tiny decision space, he can freely search for the fit of the pieces. However, real-life always incorporates constraints. Some are ironclad and internal, inherent to the nature of our business; others are imposed by regulatory bodies, or reside in organisational, labour or cultural aspects of the company itself. They all have an impact on the decision, and they can be many and very different.
Some of these restrictions may compromise the viability of the decision (e.g. if a vehicle in your fleet finishes its journey today in Huelva, it cannot be in Madrid first thing tomorrow morning), others have some consequences but do not compromise the possibility of going ahead with the decision (e.g. extending the production capacity of a line may imply extra costs, but it will be feasible if we decide to do so). There are scenarios so constrained by reality that even reaching a feasible decision (i.e. one that respects all the restrictions) can be a titanic task.
Evaluating our Decisions
If decisions are important, it is because they are related to a business KPI that we have to take care of (we get paid for it). If we plan the production of a factory, we impact actual production, revenue and profit, on-time delivery or penalties for non-compliance. So we are forced to ask which of the feasible decisions is the one that performs best. But how the decision impacts that KPI or KPIs, which sometimes is not just one (e.g. we need to save production costs, but also minimise delays in committed deliveries) is not always trivial, and that is essential to rule out one alternative or another.
Decision-maker-explorer
For all these reasons, searching for the best decision is very similar to exploring uncharted territory, and for that we need help. If we model the problem, mathematics can be our best ally: to choose which path to follow in the prospecting of the enormous space of possibilities, which options to evaluate and which not to evaluate, how to compare one with another. In short, to get closer to that optimum we are looking for and which we know is there, somewhere lost in our particular jungle.
You’ve got a Hammer, but it’s not all Nails
All of the above allows me to conclude with a call to action.
The enormous hype that Artificial Intelligence technologies are receiving today is more than justified by their ability to transform any economic sector, but there is a well-known saying that illustrates very well what is happening: to him who has only a hammer, all problems look like nails.
If it is clear that prediction is only part of the problem, why do we settle for wonderful solutions that predict future data, and nothing more? There are many scenarios in which, if we leave the decision-maker alone with our predictions, the loneliness will end up manifesting itself in incorrect decisions that can be quantified in real terms: lower sales, skyrocketing costs. Going one step further in these projects generates brutal ROIs.
Mathematical optimisation or operations research is a mature discipline that fits like a glove in every data scientist’s toolbox. There are mathematical techniques and modelling languages, experts capable of building these models and fine-tuning them for each problem, powerful solvers to execute these searches for the optimum in the appropriate time.
If you are reading this as a data scientist, do you feel comfortable when the outcome of your predictions has not alleviated the distress of your decision-makers? If you are a decision-maker, have you identified with the loneliness described above?
Numens
Numens aims to help companies bridge this gap. Our range of services and training options is aimed at both profiles, at decision-makers who see shortcomings in the support, they receive for key decisions; who sense that there are mathematical relationships between their decisions and the results, which, if properly analysed, could help them to do their job better. But it is also aimed at those IT departments, data offices, CDOs or digital transformation departments that have detected this gap and are looking for ways to solve it, either by training their staff or by seeking expert support to complement what they know how to do well.
Not everything is a nail, nor is it possible to know everything, but it is absurd not to ask for help when we know we are starting to explore uncharted territory.
Do you need our help?
