☁️
AZURE
Azure for Data Engineers
Data Factory, ADLS Gen2, Synapse, Key Vault, RBAC, file formats, pipelines, and everything Azure for data engineering.
37 in-depth tutorials
Cloud computing fundamentals: on-premises vs cloud, IaaS vs PaaS vs SaaS, and the shared responsibility model.
In-depth tutorial covering azure fundamentals.
Azure Data Factory from scratch: pipelines, activities, linked services, and your first data pipeline.
In-depth tutorial covering synapse workspace setup.
In-depth tutorial covering adf vs synapse.
In-depth tutorial covering db vs dw + sql pools.
In-depth tutorial covering blob storage.
ADLS Gen2 architecture: hierarchical namespaces, access tiers, and the foundation for data lakes.
In-depth tutorial covering azure sql database.
In-depth tutorial covering azure networking.
File formats compared: CSV, Parquet, Delta, Avro, ORC, and JSON with decision framework.
In-depth tutorial covering azure rbac roles.
13Azure Connections and Authentication
In-depth tutorial covering azure connections and authentication.
In-depth tutorial covering metadata-driven pipeline.
In-depth tutorial covering synapse + audit logging.
In-depth tutorial covering parameterized datasets.
In-depth tutorial covering adf expressions.
In-depth tutorial covering incremental loading.
In-depth tutorial covering unified full+incremental.
In-depth tutorial covering pipeline json guide.
In-depth tutorial covering audit logging concepts.
In-depth tutorial covering scd types (0,1,2,3,6).
In-depth tutorial covering scd type 1 full load.
In-depth tutorial covering scd type 1 hash-based.
In-depth tutorial covering scd type 2 pipeline.
In-depth tutorial covering combined scd1+scd2.
In-depth tutorial covering data flows guide.
In-depth tutorial covering data flow joins.
In-depth tutorial covering adf triggers.
In-depth tutorial covering trigger parameters.
In-depth tutorial covering ci/cd github.
In-depth tutorial covering ci/cd azure devops.
CI/CD for ADF and Synapse: ARM templates, environment promotion, and deployment patterns.
In-depth tutorial covering ir types guide.
In-depth tutorial covering on-prem pipeline + shir.
In-depth tutorial covering data lake cleanup.
Troubleshooting guide: common ADF and Synapse pipeline errors with solutions.