Category: Data Engineering
-
Transformative Data Leadership: Establishing a Strong Metadata Management Program
Successful metadata management ensures data quality, improves discoverability, enhances governance and enables business users and data teams to derive value from enterprise data assets efficiently. In this article I’ll lay out a plan and guidelines for building the metadata management plan and strategy to help you succeed in 2025 and going forward.
-
The Essential Guide to Distributed Locking in Modern Systems
Understanding distributed locking in modern system design and why it’s a critical component to modern distributed system architecture
-
Containers and Orchestration: Transforming Data Platform Engineering
Kubernetes orchestration combined with Docker containers have revolutionized the way data platform teams build platforms. Teams can build cloud-native data platforms with automated fault tolerance and scalability to meet changing demands for computing resources at ease. Together with cloud computing, data platforms are able to be more flexible and cost efficient than ever before.
-
Elevate Your ETL with Jinja: Templating Techniques for Smarter Data Engineering
Explore Jinja templating and how it can enhance your data engineering workflows.
-
Understanding Python Data Structures: A Practical Guide for Beginners
An introduction to python data structures as tools to organize and manage your data so you can work with it more efficiently
-
Data Engineer vs. Data Platform Engineer: Navigating the Key Differences and Managing Priorities
Two roles that are often confused, conflated, or outright merged are the Data Engineer and the Data Platform Engineer. On the surface, they might seem interchangeable, but the nuances matter, especially when you’re managing teams or straddling both roles in one job.