Start from the Problem, Not the Solution

A few months ago I started digging into rate limiting and all the ways I could implement it. In particular, I (re)discovered the token bucket, which I hadn't come across in a while.

Some time later, while doing some data analysis on an application still in beta, I realized there were some slowdowns in the backend. So I figured it was the right opportunity to finally apply the token bucket.

The result of this implementation was that:

• Users started running into a 429 error much more often.
• The slowdown wasn't resolved.
• As a result, user satisfaction dropped even further.

But I learned from my mistakes. The problem was never the number of requests users were voluntarily making; the problem was related to query handling.

This article was written to keep you from making the same mistake I did and to explain how to always stay focused on the problem.

How we end up focusing on the solution

Problem solving is a fascinating field. We can find many techniques to sharpen lateral thinking, generate original ideas and tackle complex problems. But the core of it all is always the problem.

Discovering a new technology, or a new method, lets us add a new problem-solving tool to our arsenal. It's natural to want to use it. But sometimes this ends up flipping the natural process of handling a problem: a problem is created starting from the solution.

How a problem can be created

But how can a problem be created? The mechanism behind it is fairly simple and very common in digital products, for which one of the most frequent questions, especially during validation, is: “What problem does it solve?”.

There are two ways to use this question:

• The correct one, where we ask ourselves whether there is an underlying problem that our digital product is able to solve.
• The incorrect one, where the question is bent into the form of: “How can we explain why this product exists?”.

In the second case, we're making up a story built after the fact to make sense of a solution we want to adopt. But solutions only hold up when they refer to a problem that stands on its own even without a remedy.

Even when you have a new feature in mind, the process can be turned upside down to accommodate its existence. The correct steps would be:

• Ask what the user needs.
• Based on the user's needs, discuss the possible solutions.
• Finally reach a conclusion by choosing the most suitable technology.

What happens instead is that, already holding the technology or feature, we look for a justification for it.

The metaphor of the hammer and the nail

But why do we do it? It's the same mechanism that kicks in when we learn a new word: we don't want to forget it, we're proud of the new addition to our vocabulary and we feel like using it everywhere, almost actively looking for sentences to slip it into.

Only once we've fully understood its meaning, its nuances, its register and the situations in which it works better than its alternatives, however, will we have truly learned it. At that point we'll no longer need to look for artificial opportunities to say it; in the right situation, it will simply be the word we need.

This behavior is well known enough to have a name of its own: it's the law of the instrument, theorized by Abraham Kaplan and made famous by Abraham Maslow. This law, put simply, can be stated as: “If all you have is a hammer, everything looks like a nail”.

In practice, when we learn to use a tool, we start reframing problems so that they can be tackled with that tool, and not necessarily because it's the best solution. So the tool ends up distorting the way we define the problem.

How to use a new tool correctly

So should we not use the new tool? Clearly that's not what I'm arguing. After all, innovation can indeed be driven by demand, but it can also be pushed by new technological possibilities.

This means that when a new tool or a new technology emerges, it's right to ask whether there are problems or needs it solves. But that's still a different process from asking how to use it no matter what, or even inventing needs to justify applying it.

How to know if you're really solving a problem

To tell a legitimate use case apart from one where the use of the tool is forced, there are some practical exercises you can apply to make sure you don't fall into Maslow's bias.

Describe the problem without the solution. For example, if the proposal is to add an AI chatbot to make information easier to access, ask yourself whether you could argue to the team that users have trouble finding information, without even mentioning the chatbot.

Pit the new solution against the most boring one. When a new solution excites us, it's easy to compare it with other sophisticated solutions. But sometimes the most sensible comparison is with taking no action at all, or with extremely mundane solutions that could still work. If, even against these solutions, our innovative solution is the best in terms of cost, maintenance and value delivered, then it makes sense to bring it into the discussion.

The right way to use a new tool

But what do we do with what we've learned? Potentially nothing. The tool enters our arsenal, and when the right time comes to use it, it will happen naturally.

Even while learning, the value of what has been learned doesn't depend on how many times you manage to use the tool, but on being able to recognize when it's the right thing to use.

If you do want to actively find a proper way to use the new technology, though, there are a few precautions you can take so you don't use it inappropriately.

Ask yourself what you've learned, not where to use it. Instead of asking yourself: “Where can we use it?”, ask: “What problems does it solve particularly well? What sets it apart from the tools I had before? When does it make no sense to use it?”. Being able to answer these questions means you've reached a full understanding of the tool.

Go back to open problems. Bring back to mind the problems you've already encountered, especially those that are unresolved or that were closed in a way that only partly satisfies you. For each problem, ask yourself whether your new solution changes the way you would approach it. But be careful: this doesn't mean choosing the new solution a priori, but letting it compete with the other options.

In conclusion, the right sequence when you have a new tool at your disposal or acquire a new skill is:

• Understand the uses and misuses of the new tool
• Recognize the type of problem it can tackle
• Think back to known problems
• Identify a possible match
• Compare all the alternatives

This way, you'll avoid finding a problem for every solution you learn. Rather, the goal will be to learn enough solutions, and in enough depth, to recognize the right one when faced with the problem.

Just like a new word, it isn't learned when you finally manage to squeeze it into a sentence. It's learned when you know exactly when it's the word you need.