Track 4: Dynamic Meshes

Data mesh in Action: Building Analytical Data Pipelines using PySpark on Databricks

Explore how a datamesh concept can be applied to a wider company, how to structure the data pipelines and guidelines to ensure robust data. Together we will build a data mesh using an analytical data model approach which ultimately allows us to extract information from real world data.

Learning goal

By the end of the workshop, participants will understand the data mesh concept, gain hands-on experience with Databricks, and learn to build, integrate, and analyze data models using PySpark.

Pre-requisites

Basic understanding of PySpark is favorable but not required. Bring your laptop. Access to D ONE Azure account.

  • Lorenz Eichhorn

    Lorenz received his M.Sc. in Business Analytics from BI Norwegian Business School in Oslo, Norway. He also holds a B.Sc. in Business Administration from the University St. Gallen. Lorenz joined the team in 2021.

  • Marc Bourqui

    Marc holds an M.Sc. in Computer Science from the Swiss Federal Institute of Technology in Lausanne. Before joining D ONE, he has been a full stack developer working on implementing and maintaining data products and mobile applications. Marc has been with the team since 2020.

  • Konstantinos Pittas

    Konstantinos holds an M.Sc. from the National Technical University of Athens in Chemical Engineering and an M.Sc. in Management Science and Technology from Athens University of Economics and Business. Before joining D ONE, he worked as a Senior Quantitative Consultant at Ernst & Young in Athens. Konstantinos has been with the team since 2021.