Fabric Triggers, Scheduling, and Orchestration: Schedule Triggers, Event-Based Triggers, Tumbling Window Triggers, Notebook Scheduling, and Advanced Orchestration Patterns

Our Data Factory post covered pipeline basics. This post goes deep on WHEN and HOW things run: schedule triggers (run at specific times), event-based triggers (run when data arrives), tumbling window triggers (run for specific time slices), notebook scheduling, and advanced orchestration patterns.

Table of Contents

• Schedule Triggers
• Cron-Based Scheduling
• Time Zone Configuration
• Event-Based Triggers
• File Arrival Triggers
• Table Change Triggers
• Tumbling Window Triggers (Backfill Pattern)
• Notebook Scheduling
• Schedule a Notebook Directly
• Schedule via Pipeline (Recommended)
• Advanced Orchestration Patterns
• Pattern 1: Master-Child Pipeline
• Pattern 2: Conditional Execution
• Pattern 3: Retry with Backoff
• Pattern 4: Fan-Out Fan-In (Parallel Then Merge)
• Pattern 5: Dependency Chain Across Pipelines
• Dynamic Expressions for Scheduling
• Monitoring Scheduled Runs
• Common Mistakes
• Interview Questions
• Wrapping Up

Schedule Triggers

Basic Schedule

1. Open your pipeline → click Schedule in toolbar
2. Configure:
3. Start date: 2026-06-05
4. Repeat: Every 1 Day
5. Time: 06:00 AM
6. Time zone: (UTC-05:00) Eastern Time
7. End date: None (runs indefinitely)
8. Click Apply

Cron-Based Scheduling

For complex schedules:

Every day at 6 AM:             0 6 * * *
Every hour:                     0 * * * *
Every 15 minutes:               */15 * * * *
Weekdays at 6 AM:              0 6 * * 1-5
First of every month at 2 AM:  0 2 1 * *
Every 6 hours:                  0 */6 * * *

Multiple Schedules

A pipeline can have MULTIPLE schedule triggers: – Schedule 1: Daily at 6 AM (full load) – Schedule 2: Every 2 hours during business hours (incremental)

Event-Based Triggers

File Arrival Triggers

Trigger a pipeline when a new file lands in OneLake:

1. Pipeline settings → Add trigger → Event
2. Source: OneLake path (e.g., Files/incoming/)
3. File pattern: *.csv or customers_*.parquet
4. Debounce: 5 minutes (wait for file to finish writing)

File lands in Files/incoming/orders_20260605.csv
  → Trigger fires after 5-minute debounce
  → Pipeline runs with file path as parameter
  → Pipeline Copy Activity reads the specific file
  → Processed file moved to Files/archive/

Table Change Triggers

Trigger when a Delta table changes:

bronze.raw_customers gets new rows (from mirroring or another pipeline)
  → Event trigger detects the Delta table version change
  → Pipeline runs to process the new data (Bronze → Silver)

Tumbling Window Triggers (Backfill Pattern)

Process data in specific, non-overlapping time windows — perfect for historical backfill:

Tumbling window: Daily, starting 2026-01-01
  Window 1: 2026-01-01 → processes Jan 1 data
  Window 2: 2026-01-02 → processes Jan 2 data
  ...
  Window 155: 2026-06-05 → processes today's data

If Window 50 fails, it RETRIES without affecting other windows.
Each window is independent — parallel processing possible.

Use case: Backfill 6 months of historical data without one giant pipeline run

Notebook Scheduling

Schedule a Notebook Directly

1. Open your notebook
2. Click Schedule in toolbar
3. Set frequency (daily, hourly, etc.)
4. The notebook runs on schedule without a pipeline

Schedule via Pipeline (Recommended)

Pipeline: PL_Daily_ETL
  ├── Notebook Activity: NB_Bronze_to_Silver (with parameters)
  ├── Notebook Activity: NB_Silver_to_Gold
  └── Teams Activity: Notify on success/failure

Pipeline has schedule trigger: Daily at 6 AM

Why pipeline is better: error handling, sequencing, parameters, monitoring, Teams alerts — notebooks alone cannot do this.

Advanced Orchestration Patterns

Pattern 1: Master-Child Pipeline

Master Pipeline: PL_Master_Daily_ETL
  ├── Execute Pipeline: PL_Ingest_Customers (wait for completion)
  ├── Execute Pipeline: PL_Ingest_Orders (wait for completion)
  ├── Execute Pipeline: PL_Transform_Silver (depends on both above)
  └── Execute Pipeline: PL_Build_Gold (depends on silver)

Each child pipeline is independently testable and reusable.

Pattern 2: Conditional Execution

Pipeline:
  Lookup: Check if new data exists
  If Condition: @greater(activity('Lookup').output.count, 0)
    True → Run ETL activities
    False → Skip (no new data today) → Send "No data" Teams msg

Pattern 3: Retry with Backoff

Copy Activity settings:
  Retry: 3
  Retry interval: 30 seconds
  → First try fails at 6:00 AM
  → Retry 1 at 6:00:30 AM
  → Retry 2 at 6:01:00 AM
  → Retry 3 at 6:01:30 AM
  → All retries exhausted → activity fails → red path → alert

Pattern 4: Fan-Out Fan-In

ForEach Activity (parallel, batch size 5):
  Items: ["customers", "orders", "products", "inventory", "returns"]
  → 5 Copy Activities run IN PARALLEL (batch size 5)
  → All complete → next activity runs (the "fan-in")
  → Notebook: Process all 5 tables together

Pattern 5: Dependency Chain Across Pipelines

PL_Ingest (6:00 AM trigger)
  → On success → Execute Pipeline: PL_Transform
    → On success → Execute Pipeline: PL_Serve
      → On success → Semantic Model Refresh

Dynamic Expressions for Scheduling

# Today's date in pipeline parameter
@formatDateTime(utcNow(), 'yyyy-MM-dd')

# Yesterday
@formatDateTime(addDays(utcNow(), -1), 'yyyy-MM-dd')

# First day of current month
@formatDateTime(startOfMonth(utcNow()), 'yyyy-MM-dd')

# Dynamic file path with date partitioning
@concat('Files/incoming/', formatDateTime(utcNow(), 'yyyy/MM/dd'), '/')

# Conditional: full load on Monday, incremental other days
@if(equals(dayOfWeek(utcNow()), 1), 'FULL', 'INCREMENTAL')

Common Mistakes

1. Scheduling all pipelines at the same time — creates CU spikes and throttling. Stagger by 10-15 minutes.
2. Not using event triggers for file arrivals — polling a folder every 5 minutes wastes CU. Use event triggers.
3. Not setting retry on Copy activities — transient network errors are common. Always retry 2-3 times.
4. Scheduling notebooks directly instead of via pipeline — no error handling, no alerts, no sequencing.
5. No debounce on file triggers — trigger fires while file is still uploading → reads partial file. Set 5+ minute debounce.

Interview Questions

Q: What types of triggers are available in Fabric Data Factory? A: Schedule triggers (cron-based time schedules), event-based triggers (file arrival in OneLake, table changes), and tumbling window triggers (process data in specific time slices, ideal for backfill). A pipeline can have multiple triggers simultaneously.

Q: What is a tumbling window trigger and when do you use it? A: A tumbling window trigger processes data in non-overlapping, fixed-size time windows (e.g., daily). Each window is independent — if one fails, it retries without affecting others. Ideal for historical backfill (process 6 months of data as 180 independent daily windows) and for ensuring exactly-once processing per time period.

Wrapping Up

Triggers and scheduling control WHEN your data platform runs. Schedule triggers for regular cadence, event triggers for reactive processing, tumbling windows for backfill. Combine with orchestration patterns (master-child, conditional, fan-out) for production-grade data platforms.

Related posts: – Fabric Data Factory – Fabric Notebooks – Fabric Monitoring

---

Naveen Vuppula is a Senior Data Engineering Consultant based in Ontario, Canada.