đź”·
MICROSOFT FABRIC
Microsoft Fabric for Data Engineers
Lakehouse, Warehouse, OneLake, Data Factory, Dataflow Gen2, Notebooks, Real-Time Analytics, KQL, and capacity management.
38 in-depth tutorials
01Microsoft Fabric for Data Engineers
Microsoft Fabric overview: what it is, what it replaces, and how it competes.
Master the building blocks of Microsoft Fabric. Capacity explained with the apartment building analogy, all F-SKU options with pricing, PAYG vs Reserved, pause/resume cost savings, the F64 threshold, workspaces and…
Understand the foundation of Fabric. OneLake vs ADLS Gen2 vs S3 comparison, the hierarchy (tenant, workspace, item, tables, files), ADLS Gen2 API compatibility (same API, different endpoint), accessing from Azure…
Master OneLake shortcuts in Fabric. Every supported source (ADLS Gen2, S3, S3-compatible, GCS, Dataverse, on-premises, Iceberg), read/write/delete behavior per source, the delete trap explained, two-layer security…
Complete Fabric connections and gateways guide. Connection types for Azure, on-premises, cross-cloud (AWS S3, GCS), and SaaS sources with authentication options. Connection vs ADF Linked Service migration.
The definitive Lakehouse vs Warehouse guide for Microsoft Fabric. Side-by-side comparison across 17 features, the SQL analytics endpoint explained (why read-only), languages and interfaces matrix (PySpark, SparkSQL…
Fabric Lakehouse: tables, files, notebooks, SQL endpoint, and practical guide.
The hands-on Fabric Warehouse guide. Creating tables with T-SQL, loading data three ways (cross-database from Lakehouse, pipeline Copy, T-SQL MERGE), SCD Type 1 and Type 2 with MERGE, views for Power BI (monthly…
Advanced Fabric Warehouse capabilities. COPY INTO for bulk loading from CSV and Parquet. CTAS for materialized summaries and snapshots. Dynamic Management Views for active queries and sessions.
10Fabric Data Factory & Pipelines
Fabric Data Factory: activities, pipelines, Dataflow Gen2, and notebook activities.
11Fabric Copy Activity Deep Dive
Complete Fabric Copy Activity deep dive covering every tab and setting. Moving company analogy for understanding Source, Destination, Mapping, and Settings tabs.
12Data Factory Expression Language
Complete Fabric Data Factory expression language guide. Where expressions work (Copy, Notebook, If Condition, ForEach, Web activities). Pipeline functions (@pipeline, @activity, @variables, @item).
13Triggers, Scheduling & Orchestration
Deep dive into Fabric scheduling and orchestration. Schedule triggers with cron syntax and time zones. Event-based triggers for file arrival and table changes. Tumbling window triggers for historical backfill.
Dataflow Gen2: Power Query basics, connecting to sources, and your first no-code ETL.
15Dataflow Gen2: Advanced Transforms
Master Dataflow Gen2 advanced transformations. Merge Queries with all 6 join types and fuzzy matching, Append Queries for UNION ALL, Group By with multiple aggregations, Pivot and Unpivot (with Unpivot Other Columns…
16Dataflow Gen2: Production Patterns
Take Dataflow Gen2 to production. Pipeline integration patterns (Copy then Dataflow then Notebook then Refresh), parameterization (create, use in filters, pass from pipeline), incremental refresh with date filters and…
17M Language (Power Query) Guide
The complete M language reference for Dataflow Gen2. The let-in structure explained.
Fabric Notebooks: Spark, environments, mssparkutils, and library management.
Open the Spark black box. Driver and executors explained, starter pools vs custom environments, key configurations (shuffle partitions with pizza analogy, AQE autopilot, auto-optimize, auto-compact, V-Order), broadcast…
20Spark Configuration & Performance Tuning
Complete Fabric Spark configuration and performance tuning guide. Where to set configurations (Environments for persistent, %%configure for session-level, spark.conf.set for runtime) with mutable vs immutable…
Complete Spark Structured Streaming guide. Streaming vs batch same API, reading from Event Hubs, Delta Lake, and files. Output modes (Append, Complete, Update). Checkpoint location for crash recovery.
The complete Delta Lake Table Properties reference. Every TBLPROPERTIES setting explained: deletedFileRetentionDuration and logRetentionDuration with VACUUM interaction, Change Data Feed (CDF) for MLVs and streaming…
23Materialized Lake Views (MLVs)
Complete Materialized Lake Views guide. What MLVs are and how they differ from regular views and tables. When to use MLVs vs notebooks. Creating MLVs with aggregations and joins.
Master Fabric Mirrored Databases. What mirroring is and how it eliminates ingestion pipelines, how it works under the hood (transaction log, initial snapshot, continuous CDC).
Power BI in Fabric: Direct Lake mode, semantic models, Import vs DirectQuery comparison.
Master Fabric Real-Time Intelligence. Eventstream for no-code streaming ingestion with 12 supported sources and in-flight transformations. Eventhouse for petabyte-scale time-series storage.
Deep dive into Fabric Real-Time Analytics. Five window types with KQL examples: tumbling, hopping, sliding, session, snapshot. Accelerated vs standard shortcuts in KQL databases.
The complete KQL reference guide. Pipe-based syntax explained with SQL comparison. Filtering with where (comparison, string operators, has vs contains performance). Selecting with project and extend.
The complete KQL window functions reference. The serialize operator and why window functions require it. prev() and next() for accessing adjacent rows. row_number() for sequential numbering.
Master Data Activator (Reflex) in Fabric. Why dashboards are not enough (security camera vs motion sensor analogy). Core concepts (objects, properties, triggers, actions).
Every Fabric security layer explained. Seven layers from workspace roles to sensitivity labels.
Master Fabric cost management. Capacity Units explained, F-SKU sizing guide (F2 to F512 with prices), CU consumption by workload type, throttling mechanics (10-min, 60-min, 24-hr windows), smoothing and burst model…
The complete guide to the Microsoft Fabric Capacity Metrics App. What Capacity Units (CUs) are and how they are consumed. How to install the app from AppSource.
34Monitoring & Troubleshooting
Complete Fabric monitoring and troubleshooting manual. Monitoring Hub for all item types. Pipeline, notebook, Dataflow Gen2, semantic model, and Eventstream monitoring. Fabric audit logs for compliance.
The consolidated Fabric optimization reference. Lakehouse: OPTIMIZE, VACUUM, Z-ORDER, V-Order, file sizing, partitioning. Pipelines: parallel execution, scheduling strategy, Copy Activity tuning.
Git integration: deployment pipelines, CI/CD, dev to test to prod promotion.
Complete Fabric REST APIs guide. Authentication three ways (Azure AD app, Azure CLI, mssparkutils).
The complete DP-700 study guide with every exam objective mapped to 38 DriveDataScience Fabric posts. Three domains: Implement and Manage, Ingest and Transform, Monitor and Optimize.