As companies grow, so does the complexity of their data. When information from systems such as ERPs and project management platforms is processed directly in ever-heavier Power Query processes, refresh times increase, maintenance becomes harder and the ability to evolve is limited. To address this, Visual Thinking developed a solution based on Microsoft Fabric, turning slow and complex data processes into a modern, efficient and centralised architecture.
With an architecture based on the Medallion model (Bronze, Silver and Gold), data goes through different layers of processing, validation and organisation. This gives more control over transformations and makes changes traceable. Dataflows handle the integration with the ERP, while PySpark notebooks process the data from the project management platform, so that each source is treated according to its own characteristics. The Gold layer holds the consolidated information the business needs, ready for use in Power BI and in other applications and tools. The result is a faster, better organised and more flexible data platform. It is easier to maintain, easier to develop further, and gives the organisation a solid base for its future needs.