Fabric Git Integration and Deployment Pipelines: Version Control, CI/CD, and Promoting Changes from Dev to UAT to Production

Leave a Comment / Azure, Data Engineering

Fabric Git Integration and Deployment Pipelines: Version Control, CI/CD, and Promoting Changes from Dev to UAT to Production

You have built notebooks, pipelines, dataflows, lakehouses, and warehouses in Fabric. Everything works in your development workspace. Now your manager asks: “How do we deploy this to production without breaking anything?”

Without CI/CD, the answer is terrifying: manually recreate everything in the production workspace, hope nothing is different, and pray nothing breaks. One wrong connection string, one missed notebook, one forgotten pipeline parameter — and production is down.

Git Integration and Deployment Pipelines solve this. Git Integration gives you version control — every change is tracked, reversible, and collaborative. Deployment Pipelines give you promotion — push changes from Dev to Test to Production with one click, with rules that keep environment-specific settings intact.

Think of Git Integration like Google Docs version history for your entire Fabric workspace. Every save is tracked. You can see who changed what, when, and revert to any previous version. Deployment Pipelines are like a publishing workflow — the writer drafts in a Dev document, the editor reviews in a Test document, and only approved content reaches the Published (Production) document. Same content, controlled promotion, no manual copy-paste.

Table of Contents

  • Why CI/CD Matters for Fabric
  • The Two Tools: Git Integration + Deployment Pipelines
  • Part 1: Git Integration (Source Control)
  • What Gets Tracked in Git
  • Supported Git Providers
  • Connecting a Workspace to Git
  • Step-by-Step: Connect to Azure DevOps
  • Step-by-Step: Connect to GitHub
  • The Commit and Sync Workflow
  • Committing Changes to Git
  • Syncing Changes from Git to Workspace
  • Branching Strategies
  • Feature Branch Workflow
  • Resolving Conflicts
  • Part 2: Deployment Pipelines (Environment Promotion)
  • What Is a Deployment Pipeline
  • Creating a Deployment Pipeline
  • Assigning Workspaces to Stages
  • Deploying from Dev to Test
  • Deploying from Test to Production
  • Deployment Rules (Environment-Specific Settings)
  • What Deployment Rules Handle
  • Selective Deployment (Choose What to Deploy)
  • Part 3: Putting It Together — Complete CI/CD Workflow
  • The End-to-End Flow
  • Scenario 1: Data Engineer Develops a New Pipeline
  • Scenario 2: Hotfix in Production
  • Scenario 3: Team of 5 Collaborating on a Project
  • Advanced: Automating with Fabric APIs
  • Git Integration vs Deployment Pipelines: When to Use Which
  • The Recommended Production Setup
  • Common Mistakes
  • Interview Questions
  • Wrapping Up

Why CI/CD Matters for Fabric

Without CI/CD

Developer builds in Dev workspace → works great!
Developer manually recreates in Prod workspace:
  "Did I copy all the notebooks? I think so..."
  "Wait, the connection string is different in Prod..."
  "Oops, the pipeline parameter was hardcoded for Dev..."
  → Production breaks at 2 AM
  → No way to roll back
  → No record of what changed

With CI/CD

Developer builds in Dev workspace → commits to Git
  (Every change tracked, reversible, auditable)
Deploy to Test workspace → test with production-like data
  (One click, deployment rules swap connection strings)
Deploy to Production → one click, validated, safe
  (If something breaks, roll back to previous version in Git)

The Two Tools: Git Integration + Deployment Pipelines

Tool Purpose Analogy
Git Integration Source control — track changes, branch, collaborate, revert Google Docs version history
Deployment Pipelines Environment promotion — move content Dev → Test → Prod Publishing workflow (draft → review → publish)

Fabric’s best practice: Git Integration is for source control. Deployment Pipelines are for release. Use BOTH together for a complete CI/CD workflow.

Part 1: Git Integration (Source Control)

What Gets Tracked in Git

Fabric Item Supported in Git?
Notebooks ✅ Yes
Pipelines (Data Factory) ✅ Yes
Dataflow Gen2 ✅ Yes
Semantic Models (Power BI datasets) ✅ Yes
Reports (Power BI) ✅ Yes
Lakehouse (metadata only) ✅ Yes
Warehouse (metadata only) ✅ Yes
Spark Job Definitions ✅ Yes
Spark Environments ✅ Yes
Lakehouses/Warehouse data ❌ No (only structure, not data)
Mirrored databases ❌ No
Eventstreams ❌ No

Important: Git tracks the DEFINITIONS (code, configuration, metadata) — not the DATA. Lakehouse tables, warehouse data, and uploaded files are NOT stored in Git.

Supported Git Providers

Provider Status
Azure DevOps (Azure Repos) ✅ Fully supported
GitHub ✅ Fully supported
Bitbucket, GitLab ❌ Not supported yet

Connecting a Workspace to Git

Step-by-Step: Connect to Azure DevOps

  1. Open your Fabric workspace (e.g., DataEng_Dev)
  2. Click Workspace settings (gear icon)
  3. Click Git integration
  4. Select Azure DevOps
  5. Configure:
  6. Organization: your-org (e.g., naveen-org)
  7. Project: your-project (e.g., FabricProject)
  8. Repository: your-repo (e.g., fabric-analytics)
  9. Branch: main (or dev)
  10. Folder: / or /dev-workspace (subfolder in repo)
  11. Click Connect and sync

Fabric synchronizes the workspace items with the Git branch. Each item becomes a folder in the repository with JSON/notebook definitions.

Step-by-Step: Connect to GitHub

  1. Workspace settings → Git integration → GitHub
  2. Authenticate with your GitHub account
  3. Select repository → branch → folder
  4. Click Connect and sync

The Commit and Sync Workflow

Once connected, the Source control panel appears in your workspace:

Workspace: DataEng_Dev
  ┌──────────────────────────────────────────┐
  │ SOURCE CONTROL                            │
  │                                           │
  │ Status: 3 uncommitted changes             │
  │                                           │
  │ Modified:                                 │
  │   ✏️ NB_Clean_Customers (notebook)        │
  │   ✏️ PL_Daily_ETL (pipeline)              │
  │   ➕ DF_New_Dataflow (new item)           │
  │                                           │
  │ [Commit] [Undo all]                       │
  └──────────────────────────────────────────┘

Committing Changes to Git

  1. Click Source control in the workspace toolbar
  2. Review the list of changed items
  3. Select which changes to commit (or select all)
  4. Write a commit message: “Added customer cleaning dataflow and updated ETL pipeline”
  5. Click Commit

Changes are pushed to the connected Git branch.

Git log:
  commit abc123 — "Added customer cleaning dataflow and updated ETL pipeline"
    Modified: NB_Clean_Customers.notebook
    Modified: PL_Daily_ETL.pipeline
    Added: DF_New_Dataflow.dataflow

  commit def456 — "Fixed date filter in incremental load"
    Modified: NB_Incremental_Orders.notebook

  commit ghi789 — "Initial workspace setup"
    Added: 12 items

Syncing Changes from Git to Workspace

When someone ELSE pushes changes to the branch (or you make changes from another workspace):

  1. Click Source control
  2. Status shows: “2 updates available”
  3. Click Update all → changes from Git are applied to the workspace

Branching Strategies

main (production code)
  │
  ├── feature/add-customer-pipeline (Developer A)
  │     → Create feature branch
  │     → Connect a feature workspace to this branch
  │     → Build and test
  │     → Create Pull Request to merge into main
  │     → Code review + approval
  │     → Merge → main updated → Dev workspace syncs
  │
  └── feature/fix-date-filter (Developer B)
        → Same process, independent branch

Step by Step:

  1. In Azure DevOps/GitHub, create branch feature/add-customer-pipeline from main
  2. In Fabric, create a new workspace DataEng_Feature_CustomerPipeline
  3. Connect this workspace to the feature/add-customer-pipeline branch
  4. Build and test in this workspace
  5. Commit changes to the feature branch
  6. Create a Pull Request (PR) from feature → main
  7. Team reviews code, approves, merges
  8. Delete the feature workspace

Resolving Conflicts

When two people change the same item on different branches:

Developer A changes NB_Clean_Customers on feature-A
Developer B changes NB_Clean_Customers on feature-B

Both try to merge into main:
  Developer A merges first → success
  Developer B merges → CONFLICT!

Resolution:
  Developer B pulls latest main → resolves conflict → commits → merges

Fabric shows conflicts in the Source control panel with a Conflict indicator. Resolve by choosing the correct version.

Part 2: Deployment Pipelines (Environment Promotion)

What Is a Deployment Pipeline

A Deployment Pipeline defines 2-4 stages (typically Dev → Test → Production) and lets you promote content between stages with one click:

┌──────────┐     Deploy →     ┌──────────┐     Deploy →     ┌──────────┐
│   DEV    │ ───────────────► │   TEST   │ ───────────────► │   PROD   │
│ workspace│                  │ workspace│                  │ workspace│
│          │                  │          │                  │          │
│ Build    │                  │ Validate │                  │ Serve    │
│ Iterate  │                  │ UAT test │                  │ Stable   │
│ Experiment│                 │ QA check │                  │ Governed │
└──────────┘                  └──────────┘                  └──────────┘

Creating a Deployment Pipeline

  1. In the Fabric portal, click Workspaces in the left sidebar
  2. Click Deployment pipelines (or find it under workspace settings)
  3. Click + New pipeline
  4. Name: Analytics_Deployment
  5. Number of stages: 3 (Dev, Test, Production)
  6. Click Create

Assigning Workspaces to Stages

  1. Open the deployment pipeline
  2. For each stage, click Assign a workspace:
  3. Dev stage: → DataEng_Dev workspace
  4. Test stage: → DataEng_Test workspace
  5. Prod stage: → DataEng_Prod workspace
  6. Each workspace now shows its items in the pipeline view
Analytics_Deployment Pipeline:

DEV (DataEng_Dev)          TEST (DataEng_Test)        PROD (DataEng_Prod)
├── NB_Clean_Customers     ├── NB_Clean_Customers     ├── NB_Clean_Customers
├── PL_Daily_ETL           ├── PL_Daily_ETL           ├── PL_Daily_ETL
├── DF_Clean_Orders        ├── DF_Clean_Orders        ├── DF_Clean_Orders
├── sales_lakehouse        ├── sales_lakehouse        ├── sales_lakehouse
└── sales_warehouse        └── sales_warehouse        └── sales_warehouse

Deploying from Dev to Test

  1. In the pipeline view, click Deploy between Dev and Test
  2. Review the items that will be deployed:
  3. New items (exist in Dev, not in Test) → will be CREATED
  4. Modified items (different version) → will be UPDATED
  5. Unchanged items → skipped
  6. Click Deploy
  7. Wait for deployment to complete (typically 1-5 minutes)

Deploying from Test to Production

Same process — but typically requires: – Approval from a manager/lead – Validation that Test passed QA – Deployment window (e.g., not during business hours)

Deployment Rules (Environment-Specific Settings)

The critical feature that makes deployment work: rules that change connection strings, parameters, and settings per environment.

Problem without rules:
  Dev notebook connects to: dev_lakehouse
  When deployed to Prod: notebook still connects to dev_lakehouse ← WRONG!

Problem solved with rules:
  Deployment rule: "In Prod stage, change lakehouse connection to prod_lakehouse"
  Dev → Test: lakehouse swapped to test_lakehouse
  Test → Prod: lakehouse swapped to prod_lakehouse

What Deployment Rules Handle

Rule Type Example
Data source Dev lakehouse → Prod lakehouse
Connection strings dev-sql-server → prod-sql-server
Semantic model data source Dev SQL endpoint → Prod SQL endpoint
Parameters environment = “dev” → environment = “prod”

Setting Up a Deployment Rule

  1. In the deployment pipeline, click the ⚙️ settings icon on the Prod stage
  2. Select the item to configure (e.g., PL_Daily_ETL pipeline)
  3. Add rule:
  4. Rule type: Data source
  5. Source (Dev): dev_lakehouse
  6. Target (Prod): prod_lakehouse
  7. Save

Now every deployment automatically swaps the connection.

Selective Deployment (Choose What to Deploy)

You do not have to deploy EVERYTHING:

  1. Click Deploy between stages
  2. In the review dialog, UNCHECK items you do not want to deploy
  3. Deploy only the selected items

Use case: You updated 3 notebooks and 1 pipeline, but only the pipeline is ready for production. Deploy only the pipeline. The notebooks stay as-is in Prod.

Part 3: Putting It Together — Complete CI/CD Workflow

The End-to-End Flow

CONTINUOUS INTEGRATION (Git):
  Developer creates feature branch → builds in feature workspace →
  commits to Git → creates Pull Request → code review → merge to main →
  Dev workspace syncs with main branch

CONTINUOUS DEPLOYMENT (Deployment Pipeline):
  Dev workspace (synced with main) → Deploy to Test → QA validation →
  Deploy to Production → done

Combined:
  Feature branch → PR → main → Dev workspace → Deploy → Test → Deploy → Prod

Scenario 1: Data Engineer Develops a New Pipeline

Day 1:
  1. Create branch: feature/daily-sales-pipeline
  2. Create workspace: DataEng_Feature_Sales
  3. Connect workspace to feature branch
  4. Build: notebook, pipeline, dataflow

Day 2-3:
  5. Test in feature workspace (run notebook, validate output)
  6. Commit changes to feature branch
  7. Create PR: feature/daily-sales-pipeline → main
  8. Team reviews: "Looks good, approved ✅"
  9. Merge PR → main branch updated

Day 3:
  10. Dev workspace auto-syncs from main (new pipeline appears)
  11. Test in Dev workspace with full data
  12. Deploy Dev → Test via deployment pipeline
  13. QA validates in Test workspace

Day 4:
  14. Deploy Test → Production via deployment pipeline
  15. Deployment rules swap connections to production sources
  16. Pipeline is live in production 🎉
  17. Delete feature workspace (cleanup)

Scenario 2: Hotfix in Production

Problem: Production notebook has a bug — wrong date filter

Fix process:
  1. Create branch: hotfix/fix-date-filter from main
  2. Fix the bug in a hotfix workspace connected to this branch
  3. Test the fix
  4. Create PR: hotfix/fix-date-filter → main (expedited review)
  5. Merge → Dev workspace syncs
  6. Deploy Dev → Test → Prod (fast-track deployment)
  7. Bug fixed in production within hours, not days
  8. Full audit trail in Git: who fixed what, when, why

Scenario 3: Team of 5 Collaborating on a Project

Team setup:
  Shared workspace: DataEng_Dev (connected to main branch)
  Each developer: own feature workspace + feature branch

Developer A: feature/customer-dim (building customer dimension)
Developer B: feature/product-dim (building product dimension)
Developer C: feature/sales-fact (building sales fact table)
Developer D: feature/pbi-reports (building Power BI reports)
Developer E: feature/pipeline-orchestration (building the pipeline)

Each developer:
  → Works independently in their feature workspace
  → No conflicts with other developers
  → Commits to their branch
  → Creates PR when ready
  → Merges into main after review

Main branch accumulates all approved changes.
Dev workspace reflects the latest integrated code.
Deployment pipeline promotes from Dev → Test → Prod.

Advanced: Automating with Fabric APIs

For fully automated deployments (trigger on PR merge):

# Trigger deployment programmatically via Fabric REST API
import requests

# Get auth token
token = get_fabric_token()  # Azure AD authentication

# Deploy from Dev to Test stage
url = f"https://api.fabric.microsoft.com/v1/deploymentPipelines/{pipeline_id}/deploy"
headers = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}

payload = {
    "sourceStageOrder": 0,      # 0 = Dev
    "targetStageOrder": 1,      # 1 = Test
    "isBackwardDeployment": False,
    "newWorkspace": False
}

response = requests.post(url, headers=headers, json=payload)
print(f"Deployment status: {response.status_code}")

Integrate with Azure DevOps / GitHub Actions: Trigger the Fabric API call on PR merge → fully automated Dev → Test deployment.

Git Integration vs Deployment Pipelines: When to Use Which

Scenario Tool
Track changes and revert mistakes Git Integration
Collaborate on same workspace with branches Git Integration
Code review before merging changes Git Integration (Pull Requests)
Promote content from Dev to Test to Prod Deployment Pipelines
Swap connection strings between environments Deployment Pipelines (rules)
Selective deployment of specific items Deployment Pipelines
Automate deployment on PR merge Both (Git triggers API → Pipeline deploys)
Audit who changed what and when Git Integration (commit history)

Use BOTH for production: – Git Integration = source control (track, branch, collaborate, revert) – Deployment Pipelines = release management (promote, configure, deploy)

Workspaces:
  DataEng_Dev  → Connected to Git (main branch)
  DataEng_Test → NOT connected to Git (receives content via deployment pipeline)
  DataEng_Prod → NOT connected to Git (receives content via deployment pipeline)

Feature workspaces (temporary):
  DataEng_Feature_XYZ → Connected to Git (feature/XYZ branch)
  Created per feature, deleted after merge

Git repository:
  main branch → represents Dev workspace state
  feature/* branches → one per developer/feature

Deployment pipeline:
  Dev stage → DataEng_Dev workspace
  Test stage → DataEng_Test workspace
  Prod stage → DataEng_Prod workspace
  Rules: swap data sources per environment

Workflow:
  feature branch → PR → main → Dev syncs → Deploy → Test → Deploy → Prod

Common Mistakes

  1. Not using Git at all — “I’ll just rebuild in Prod manually.” One mistake = hours of debugging. One Git commit = instant rollback. There is no excuse to skip version control.

  2. Connecting ALL workspaces to Git — only Dev and feature workspaces should be Git-connected. Test and Prod should receive content ONLY through the deployment pipeline — never direct edits.

  3. No deployment rules for environment-specific settings — deploying Dev notebooks to Prod without swapping connection strings means Prod reads Dev data. Always configure deployment rules.

  4. Skipping the Test stage — “It works in Dev, deploy straight to Prod.” This is how production breaks. Always validate in Test with production-like data.

  5. Not using feature branches — all developers committing directly to main creates chaos. Feature branches isolate work until it is ready and reviewed.

  6. Forgetting that data is NOT in Git — Git tracks definitions (notebooks, pipelines). Data stays in OneLake. Deploying from Dev to Prod does not copy data — only the item definitions. Data pipelines must run in each environment separately.

  7. Editing items directly in the Prod workspace — all changes should flow through the CI/CD process. Direct edits in Prod create drift between Git and the workspace.

Interview Questions

Q: How does CI/CD work in Microsoft Fabric? A: Fabric uses two complementary tools. Git Integration provides source control — connecting workspaces to Azure DevOps or GitHub for tracking changes, branching, and collaboration. Deployment Pipelines provide environment promotion — moving content from Dev to Test to Production with one click. Deployment rules handle environment-specific settings like connection strings. The recommended flow: feature branch → Pull Request → main → Dev workspace → Deploy → Test → Deploy → Prod.

Q: What is the difference between Git Integration and Deployment Pipelines? A: Git Integration is for source control — tracking every change, enabling branching, code review via Pull Requests, and rollback to previous versions. Deployment Pipelines are for release management — promoting validated content between environments (Dev, Test, Prod) with rules that swap connection strings and parameters. Use Git for collaboration and traceability. Use Deployment Pipelines for safe, controlled releases.

Q: What items are supported in Fabric Git Integration? A: Notebooks, Pipelines, Dataflow Gen2, Semantic Models, Reports, Lakehouse metadata, Warehouse metadata, Spark Job Definitions, and Spark Environments. Important: Git tracks item definitions and code, NOT data. Lakehouse tables, warehouse data, and uploaded files are not stored in Git.

Q: How do you handle environment-specific configurations (Dev vs Prod)? A: Using Deployment Rules in the Deployment Pipeline. For each stage (Test, Prod), you configure rules that swap data sources, connection strings, and parameters. For example, a rule that changes dev_lakehouse to prod_lakehouse when deploying to the Prod stage. This ensures items work correctly in each environment without manual changes.

Q: How do you handle hotfixes in production? A: Create a hotfix branch from main, fix the issue in a temporary workspace connected to the hotfix branch, test the fix, create an expedited Pull Request, merge into main, sync to Dev workspace, then fast-track deploy through Test to Production. The full audit trail is maintained in Git — who fixed what, when, and why.

Wrapping Up

Git Integration and Deployment Pipelines transform Fabric from a development playground into a production-grade platform. Git gives you version control — every change tracked, reversible, collaborative. Deployment Pipelines give you controlled releases — Dev to Test to Prod with environment-specific rules.

The formula is simple: track everything in Git, develop in feature branches, review with Pull Requests, merge to main, and deploy through the pipeline. No manual recreation. No “works in Dev, breaks in Prod.” No 2 AM production outages from untested changes.

Related posts: – Databricks Git Integration and CI/CD – CI/CD with ARM Templates (ADF/Synapse) – Fabric Data Factory & Pipelines – Fabric Foundations: Capacity, Workspaces, Items

Naveen Vuppula is a Senior Data Engineering Consultant and app developer based in Ontario, Canada. He writes about Python, SQL, AWS, Azure, and everything data engineering at DriveDataScience.com.

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top
Share via
Facebook
X (Twitter)
LinkedIn
Mix
Email
Print
Copy Link
Copy link
CopyCopied