“Do you know what data you have? “

“Do you know how to work with it?”

First, try to answer these questions about your systems and business. Then, repeat this exercise while looking at your information flows. After reading this article, ask your team to do the same—to ponder these questions.
Analysing my past working years, I’ve had the privilege of working with several brilliant professionals, and one of them introduced me to an interesting concept, the title of this article, “Exploring and Discovering.” This is a self-explanatory phrase and could be perfectly used in a XV century movie, but it’s highly relevant in this XXI century.
But how can I apply this concept? As you well know, there are numerous approaches, methodologies, and frameworks. If you think about it, it probably exists somewhere, used successfully by someone, some team, or some company. Simplifying it into only four well-known steps:

Analysis   Development   Testing   Production

The concept can easily fit into the analysis phase, where you should explore what exists, how it exists, and how you can use it to implement something. But the reality is that you can apply it to all these steps and more. This concept is a permanent fixture, with the purpose of fighting the growing lack of knowledge that affects many of us. By “us,” I don’t mean only people; I mean code, processes, projects, and companies as well. And why are we losing knowledge? It can be attributed to three main reasons:

DATA is constantly changing, with its volume exponentially growing. Most of us struggle to handle the sheer amount of data within an acceptable reaction time. We can compare these incoming and processing data flows to a wild river. The amount of data is so vast and diverse that it’s hard to navigate. You probably just want to keep your head above the waterline, avoid the rocks, and stay afloat. Your daily tasks are mostly reactive, and new information flows keep appearing, needing integration. Every month, you have more data that you cannot properly handle, process, and, most importantly, understand. You simply don’t have the time for that, and because of that, you become deficient in knowledge, which affects your business.
BUSINESS, the second issue, is affected by your lack of underlying data knowledge and has its own responsibility due to its constant evolution. There is no company that persists with the exact same services or products since its foundation. Consumers change, products are replaced or adapted, and new ones are created to meet demand. The business forces the data to change, and underlying systems are compelled to adapt, grow, or even change completely. Technologies or architectures need to change to support the business. What used to be daily execution now happens hourly, and your ten-year stable system needs to be migrated. You need to learn new technologies, new processes, collect data, process it, and understand it, all in a very short period, without losing part of the legacy knowledge. You cannot migrate to a new system and lose knowledge.
PEOPLE, the third factor. If the first two issues are reactive, this third one is active. Every time you lose a professional, you lose part of your knowledge, part of your company, and part of your business. You can document many things and implement a failover process; that’s very important. However, it’s flawed because not all things can be documented. The security of doing things gained through experience is something that cannot be passed onto a document. Most of the time, you are racing against a deadline. That senior person who brings focus to the teams makes all the difference when some administrative board decides to launch a product before summer, without knowing the effort needed to implement it. You can easily lose that person who knows the process, the product, the client’s demands, or all those background processes that keep your system running.

Exploring & Discovering gives a name to a need—the need to learn what you should never forget or still need to learn. This helps you mitigate the lack of knowledge. Your success depends not only on your business offer and vision but also on the ability to get there, to be able to deliver.
Falling behind in your own internal knowledge is an error. You need to keep pace with what you have implemented, migrated, and currently support, every single day. If you migrate to a new system and then need 2 weeks to respond to a client’s billing question, you might lose the client. Allow yourself and your clients some time and effort to explore and discover. Operationalize it as a regular task, share the findings, and teach your colleagues.
Do not believe you know everything. Do not believe you have everything.
Knowledge is an advantage, not a cost.

Pedro Duran
Head of Big Data