Here is what you need to think about when setting up a modern data architecture

There are four phases through which data moves in a data-driven organization. Data is collected and stored within the architecture, processed, consumed and used. Each stage involves different elements to keep everything moving in the right direction. Everything needs to be considered carefully. What exactly do you need, what will it be used for, what should it enable, etc.?

Firstly, it is important for the data to be stored in a usable form. The groundwork must already have been done so that a data engineer can use their time effectively. Beware of having too many processes or correlations; the data must be usable across the board. A strategy also needs to be drawn up to deal with data security. When classifying datasets, you determine which security measures are required at each stage of the platform. Assign data a sensitivity category, and decide which items can be shared. In the area of data security, confidence is very quickly lost, and difficult to recover. It is also important to know the origin of data, in order to remain compliant with the GDPR. A data catalog will also support you here. This is where you keep track of what data exists within the company. When someone asks for their data to be deleted, you can immediately see everything that you have about them. There are many other components that are important for good data quality and governance. Also build in the ability to analyze data. Make sure that there is a way to browse the data to search for possibilities.

Be sure to think about what you ultimately want to do with the data. Do you want to build a dashboard to monitor what is happening in a specific area of your organization and/or do you want to set up predictive or intelligent models? In addition to building such visualizations or models, the data can also be integrated into existing business applications or even made available to third parties.

Ultimately, the data can be used in all kinds of applications, after it has come a long way. Nevertheless, this remains the starting point for many companies ("we want to do something with our data"), and they forget about the rest of the puzzle, and therefore don't get the results or the value they had in mind. Building a report or AI model is not enough on its own. The insights must be used and deployed to create added value in the organization. Unless you let the data flow through to the heart of your organization, you are left with interesting but separate models, with no impact.