Microsoft Fabric

Power BI Deployment and Sharing: Deployment Pipelines, Lakehouse Rebinding, Semantic Model Promotion, Variable Libraries, Power BI Apps, Row-Level Security, Workspace Roles, and What Actually Happens When You Promote to Production

Leave a Comment / Data Engineering, Microsoft Fabric, Power BI

The complete Power BI deployment and sharing guide for data engineers. Fabric Deployment Pipelines from dev to test to prod. What happens during promotion item by item. Deployment rules for environment-specific values. Lakehouse rebinding with parameter rules auto-binding and Variable Libraries. Semantic model automatic rebinding. What does NOT transfer during promotion. Will reports work without attaching the target Lakehouse. Variable Libraries for environment-agnostic code. Five sharing methods with Power BI Apps recommended. Row-level security static vs dynamic with USERPRINCIPALNAME. Workspace roles. Complete end-to-end deployment workflow. Eight common mistakes and seven interview Q and As.

Power BI Deployment and Sharing: Deployment Pipelines, Lakehouse Rebinding, Semantic Model Promotion, Variable Libraries, Power BI Apps, Row-Level Security, Workspace Roles, and What Actually Happens When You Promote to Production Read More »

Legacy Power BI vs Fabric Power BI: What Changed, What Stayed, Semantic Models, Direct Lake, P SKU to F SKU, OneLake, Composite Models, Git Integration, Web Modeling, and the Complete Migration Path

Leave a Comment / Data Engineering, Microsoft Fabric, Power BI

The complete Legacy vs Fabric Power BI comparison for data engineers. What changed and what stayed the same. Datasets renamed to semantic models. Workspaces expanded for all Fabric items. OneLake as unified storage. P SKU to F SKU licensing shift with capacity sharing. Direct Lake replacing Import for Fabric Lakehouses. Import vs DirectQuery vs Direct Lake deep comparison. Default semantic model decoupling. Composite models mixing Direct Lake and Import. Web modeling in the browser. Git integration for version control. Complete migration path from legacy to Fabric. When to convert to Direct Lake and when not to. Eight common mistakes and seven interview Q and As.

Legacy Power BI vs Fabric Power BI: What Changed, What Stayed, Semantic Models, Direct Lake, P SKU to F SKU, OneLake, Composite Models, Git Integration, Web Modeling, and the Complete Migration Path Read More »

KQL Window Functions: serialize, prev, next, row_number, row_cumsum, row_rank_dense, row_rank_min, row_window_session, scan Operator, and Every Pattern for Fabric Real-Time Analytics

Leave a Comment / Azure, Data Engineering, Microsoft Fabric

The complete KQL window functions reference. The serialize operator and why window functions require it. prev() and next() for accessing adjacent rows. row_number() for sequential numbering. row_cumsum() for running totals. row_rank_dense() and row_rank_min() for ranking (dense vs gaps). row_window_session() for automatic session detection. The scan operator for stateful row processing. Partitioned windows with the restart parameter. KQL vs SQL window function comparison table. Eight real-world patterns, common mistakes, and interview Q&As.

KQL Window Functions: serialize, prev, next, row_number, row_cumsum, row_rank_dense, row_rank_min, row_window_session, scan Operator, and Every Pattern for Fabric Real-Time Analytics Read More »

Microsoft Fabric Capacity Metrics App: Every Page Explained — Health, Compute, Storage, Timepoint, Item History, Autoscale, Throttling, Smoothing, and the Chargeback App

Leave a Comment / Azure, Data Engineering, Microsoft Fabric

The complete guide to the Microsoft Fabric Capacity Metrics App. What Capacity Units (CUs) are and how they are consumed. How to install the app from AppSource. Every page explained: Health, Compute, Storage, Timepoint, Timepoint Summary, Timepoint Item Detail, Item History, Autoscale Compute for Spark. Interactive vs background operations and how smoothing works. Throttling stages (delay, rejection) and how to diagnose them. The Chargeback App for cost allocation. Real-world diagnostic scenarios, eight common mistakes, and seven interview Q&As.

Microsoft Fabric Capacity Metrics App: Every Page Explained — Health, Compute, Storage, Timepoint, Item History, Autoscale, Throttling, Smoothing, and the Chargeback App Read More »

Fabric Spark Configuration and Performance Tuning: shuffle.partitions, autoBroadcastJoinThreshold, maxPartitionBytes, AQE, Autotune, Native Execution Engine, and Every Setting Data Engineers Must Know

Leave a Comment / Data Engineering, Microsoft Fabric

Complete Fabric Spark configuration and performance tuning guide. Where to set configurations (Environments for persistent, %%configure for session-level, spark.conf.set for runtime) with mutable vs immutable properties table and the critical root-vs-conf placement rule for %%configure. The Big Three settings: spark.sql.shuffle.partitions (default 200, grocery checkout lane analogy, guidelines by data size, demonstration with empty partitions), spark.sql.autoBroadcastJoinThreshold (default 10MB, product catalog analogy, why increase to 100-256MB, manual broadcast hint, OOM warning with 20% rule, checking join strategy with explain()), and spark.sql.files.maxPartitionBytes (default 128MB, delivery truck analogy, when to increase/decrease). Adaptive Query Execution with GPS analogy (coalesce partitions, auto broadcast conversion, skew join splitting, AQE vs manual tuning). Autotune ML-based optimizer (how to enable, when it works best, checking driver log recommendations). Native Execution Engine (Velox/Gluten C++ engine, 2-4x faster, enable via %%configure or Environment, UDF fallback limitation). Memory and resources (driver vs executor memory, node sizes table, starter vs custom pools comparison, high concurrency mode). Delta Lake settings (optimize write, auto compaction, V-Order default-on, target file size). Join strategy deep dive (broadcast hash, sort-merge, shuffle hash with decision table). Four production configuration templates (small/medium/large/massive data). Spark UI diagnosis (reading jobs/stages/executors tabs, identifying shuffle bottlenecks, identifying data skew with median vs max gap). 8 common mistakes and 8 interview Q&As.

Fabric Spark Configuration and Performance Tuning: shuffle.partitions, autoBroadcastJoinThreshold, maxPartitionBytes, AQE, Autotune, Native Execution Engine, and Every Setting Data Engineers Must Know Read More »

Fabric Copy Activity Deep Dive: Every Tab, Every Setting, Fault Tolerance, Staging, Logging, Intelligent Throughput, Parallelism, and Production Patterns Every Data Engineer Must Know

Leave a Comment / Data Engineering, Microsoft Fabric

Complete Fabric Copy Activity deep dive covering every tab and setting. Moving company analogy for understanding Source, Destination, Mapping, and Settings tabs. How the Copy activity works under the hood (4-step process). General tab (naming conventions, timeout best practices with restaurant analogy, retry and retry interval, secure input/output). Source tab (table vs query vs stored procedure with room analogy, partition options with None/Physical/Dynamic Range comparison table and performance benchmarks, additional columns for audit lineage, query timeout and isolation level). Destination tab (Lakehouse vs Warehouse differences table, table action Append/Overwrite/Upsert with bookshelf analogy, key columns for upsert, pre-copy script for idempotent reloads, destination partitioning, max rows per file). Mapping tab (auto vs manual mapping, import schemas, type conversion two-stage flow, schema drift handling). Settings tab (ITO with truck-size analogy and cost impact benchmarks, Degree of Copy Parallelism with tuning guidance, how ITO and parallelism compound, fault tolerance with postal service analogy and when/when-not to enable, enable staging with loading dock analogy and Workspace vs External options, session logging with file path structure, data consistency verification, preserve metadata). Monitoring output (11-field table, reading throughput, identifying bottlenecks across queue/pre-copy/transfer/post-copy phases). Five production patterns: standard SQL-to-Lakehouse, large table with Dynamic Range partitioned read, SQL-to-Warehouse with required staging, fault-tolerant load with quarantine session logging and Teams alerts, metadata-driven copy with per-table ITO and fault tolerance settings from config table. Six cost optimization tips. Eight common mistakes. Eight interview Q&As.

Fabric Copy Activity Deep Dive: Every Tab, Every Setting, Fault Tolerance, Staging, Logging, Intelligent Throughput, Parallelism, and Production Patterns Every Data Engineer Must Know Read More »

Scroll to Top