Azure Data Factory Expressions Explained: The Complete Guide to @dataset(), @item(), @activity(), and More

If you have ever stared at an ADF expression like @concat(item().SchemaName, '.', item().TableName) and thought “I know it works, but I don’t really understand WHY” — this post is for you.

ADF and Synapse use a custom expression language that looks simple but has rules that trip up even experienced developers. When do you use @dataset() vs @item()? Why does @item() work inside ForEach but not outside? What is the difference between @activity() and @pipeline()?

This post is the reference I wish I had when I started.

Table of Contents

• What Are ADF Expressions?
• The @ Symbol: When to Use It and When Not To
• The 7 Core Expression Functions
• @dataset() Explained
• @item() Explained
• @activity() Explained
• @pipeline() Explained
• @trigger() and @variables() Explained
• Utility Functions
• How Values Flow Through the Pipeline
• The Scope Rules
• Common Mistakes and How to Fix Them
• Quick Reference Cheat Sheet
• Wrapping Up

What Are ADF Expressions?

ADF expressions are dynamic content — values calculated at runtime instead of being hardcoded.

Static: Source table: SalesLT.Customer — always reads Customer.

Dynamic: Source table: @concat(item().SchemaName, '.', item().TableName) — reads whatever table the ForEach iteration specifies.

Every expression starts with @ and can contain functions, property access, and operators.

The @ Symbol: When to Use It and When Not To

Rule 1: Use @ Once at the Beginning

@item().TableName                    -- CORRECT
@activity('Lookup').output.value     -- CORRECT
@concat('Hello', ' ', 'World')      -- CORRECT

Rule 2: Never Use @ Inside Another @

Once inside an expression, everything is already in expression context:

@concat(@item().SchemaName, '.', @item().TableName)     -- WRONG
@concat(item().SchemaName, '.', item().TableName)       -- CORRECT

Rule 3: Strings Use Single Quotes

@concat(item().FolderName, '/', 'data.csv')     -- CORRECT (single quotes)
@concat(item().FolderName, "/", "data.csv")     -- WRONG (double quotes)

Rule 4: Escape Single Quotes by Doubling

To produce a literal single quote, use two single quotes together. For SQL WHERE clauses, you need three quotes together which produces: end-string + literal-quote + start-string.

The 7 Core Expression Functions

Function	Where It Works	What It Returns
@dataset()	Inside datasets only	The dataset own parameter values
@item()	Inside ForEach only	The current loop element
@activity()	Inside pipelines	Another activity output
@pipeline()	Inside pipelines	Pipeline-level properties
@trigger()	Inside pipelines	Trigger information
@variables()	Inside pipelines	Pipeline variable values
@concat(), etc.	Anywhere	Utility functions

@dataset() Explained

What It Does

@dataset() accesses the parameters defined on the dataset itself. Used ONLY inside the dataset Connection tab. Never in a pipeline activity.

Real Example

Dataset DS_SqlDB_SourceTable has parameters: SchemaName and TableName.

In the dataset Connection tab:

Schema field: @dataset().SchemaName
Table field:  @dataset().TableName

This tells the dataset: “When someone uses me, look at my own parameters to know which table to connect to.”

The Analogy

Think of a dataset as a function definition:

def read_table(SchemaName, TableName):
    # @dataset().SchemaName = using the function own parameter
    connection = f"SELECT * FROM {SchemaName}.{TableName}"
    return execute(connection)

The function does not know what values it will get. It just says “use whatever SchemaName and TableName I am given.”

What @dataset() Returns

It returns the parameter value that was passed to the dataset by the pipeline activity:

• Copy activity passes SchemaName = "SalesLT" –> @dataset().SchemaName returns "SalesLT"
• Copy activity passes TableName = "Customer" –> @dataset().TableName returns "Customer"

Common Error

{"errorCode": "InvalidTemplate", "message": "The template function 'dataset' is not defined or not valid"}

This means you used @dataset() in a pipeline activity. Only datasets understand dataset().

WRONG (in pipeline): @dataset().SchemaName
RIGHT (in pipeline): @item().SchemaName

@item() Explained

What It Does

@item() accesses the current element in a ForEach loop. Each iteration, @item() points to a different element from the input array.

Real Example

Lookup returns this array:

[
    {"TableName": "Customer",  "SchemaName": "SalesLT", "FolderName": "sqldb/Customer"},
    {"TableName": "Address",   "SchemaName": "SalesLT", "FolderName": "sqldb/Address"},
    {"TableName": "Product",   "SchemaName": "SalesLT", "FolderName": "sqldb/Product"}
]

Iteration 1:

@item().TableName  = "Customer"
@item().SchemaName = "SalesLT"
@item().FolderName = "sqldb/Customer"

Iteration 2:

@item().TableName  = "Address"
@item().SchemaName = "SalesLT"
@item().FolderName = "sqldb/Address"

The Analogy

metadata = [
    {"TableName": "Customer", "SchemaName": "SalesLT"},
    {"TableName": "Address",  "SchemaName": "SalesLT"},
]

for item in metadata:           # ForEach activity
    schema = item.SchemaName    # @item().SchemaName
    table = item.TableName      # @item().TableName
    copy_data(schema, table)    # Copy activity

Case-Sensitive

@item().TableName     -- CORRECT (matches SQL column)
@item().tablename     -- WRONG (returns null or error)
@item().TABLENAME     -- WRONG (case mismatch)

Only Works Inside ForEach

ForEach Activity
  |-- Copy Activity     --> @item().TableName  WORKS
  |-- Stored Procedure  --> @item().SchemaName  WORKS

Outside ForEach
  |-- Set Variable      --> @item().TableName  ERROR

@activity() Explained

What It Does

@activity() accesses the output of a previously executed activity.

Lookup Output (First row only UNCHECKED — returns array)

@activity('Lookup_Metadata').output.value              -- array of all rows
@activity('Lookup_Metadata').output.count               -- number of rows (e.g., 5)
@activity('Lookup_Metadata').output.value[0].TableName  -- first row TableName

Lookup Output (First row only CHECKED — returns single object)

@activity('Lookup_MaxValue').output.firstRow              -- the single row
@activity('Lookup_MaxValue').output.firstRow.MaxValue     -- a column value

Copy Activity Output

@activity('Copy_TableData').output.rowsRead        -- rows read (e.g., 847)
@activity('Copy_TableData').output.rowsCopied       -- rows written (e.g., 847)
@activity('Copy_TableData').output.copyDuration      -- seconds (e.g., 3)
@activity('Copy_TableData').output.throughput        -- KB/s
@activity('Copy_TableData').output.dataRead          -- bytes read
@activity('Copy_TableData').output.dataWritten       -- bytes written
@activity('Copy_TableData').output.errors[0].Message -- error message (on failure)

Name Must Match Exactly

Activity name: Lookup_Metadata
CORRECT: @activity('Lookup_Metadata').output.value
WRONG:   @activity('LookupMetadata').output.value       -- missing underscore
WRONG:   @activity('lookup_metadata').output.value      -- wrong case
WRONG:   @activity('Lookup_Metadata ').output.value     -- trailing space

Tip: Copy-paste activity names into expressions. Never type them manually.

@pipeline() Explained

Access pipeline-level properties:

@pipeline().RunId                        -- unique run GUID
@pipeline().Pipeline                     -- pipeline name
@pipeline().DataFactory                  -- workspace name
@pipeline().parameters.ParamName         -- pipeline parameter value

Use case: Audit logging with @pipeline().RunId links every log entry to a specific pipeline execution.

@trigger() and @variables() Explained

@trigger()

@trigger().name              -- trigger name
@trigger().startTime         -- when it fired
@trigger().scheduledTime     -- when it was scheduled
@trigger().outputs.windowStartTime  -- tumbling window start
@trigger().outputs.windowEndTime    -- tumbling window end

@variables()

Note the different syntax — parentheses with string name:

@variables('CurrentTable')             -- uses ('name') syntax
@pipeline().parameters.ParamName       -- uses .property syntax
@item().TableName                      -- uses .property syntax

Utility Functions

String Functions

@concat('a', 'b', 'c')                              -- "abc"
@concat(item().SchemaName, '.', item().TableName)    -- "SalesLT.Customer"
@substring('Hello World', 0, 5)                      -- "Hello"
@replace('Hello World', 'World', 'ADF')              -- "Hello ADF"
@toUpper('hello')                                     -- "HELLO"
@toLower('HELLO')                                     -- "hello"
@trim('  hello  ')                                    -- "hello"
@length('Hello')                                      -- 5
@coalesce(null, 'default')                           -- "default"
@guid()                                               -- new GUID

Date/Time Functions

@utcnow()                                        -- "2026-04-05T14:30:00Z"
@formatDateTime(utcnow(), 'yyyy-MM-dd')          -- "2026-04-05"
@formatDateTime(utcnow(), 'yyyy')                -- "2026"
@formatDateTime(utcnow(), 'MM')                  -- "04"
@formatDateTime(utcnow(), 'dd')                  -- "05"
@formatDateTime(utcnow(), 'HH')                  -- "14" (24-hour)
@formatDateTime(utcnow(), 'mm')                  -- "30" (minutes)
@addDays(utcnow(), -1)                           -- yesterday
@addHours(utcnow(), -6)                          -- 6 hours ago

Building Date-Partitioned Folders

@concat(item().FolderName, '/',
        formatDateTime(utcnow(), 'yyyy'), '/',
        formatDateTime(utcnow(), 'MM'), '/',
        formatDateTime(utcnow(), 'dd'))

Produces: sqldb/Customer/2026/04/05

Conditional Functions

@if(equals(item().LoadType, 'Full'), 'full-load', 'incremental')
@if(greater(activity('Copy').output.rowsCopied, 0), 'Copied', 'Empty')

How Values Flow Through the Pipeline

Step 1: Lookup reads metadata
        @activity('Lookup').output.value = [{TableName:"Customer",...}]

Step 2: ForEach iterates
        @item() = current row from the array

Step 3: Copy Source (PIPELINE scope) passes values to dataset:
        SchemaName param <-- @item().SchemaName = "SalesLT"
        TableName param  <-- @item().TableName  = "Customer"

Step 4: Dataset Connection (DATASET scope) uses its own parameters:
        Schema: @dataset().SchemaName --> receives "SalesLT" from Step 3
        Table:  @dataset().TableName  --> receives "Customer" from Step 3
        Result: SELECT * FROM SalesLT.Customer

Step 5: Copy runs, output available:
        @activity('Copy').output.rowsRead = 847

Step 6: Stored Procedure uses both:
        @item().TableName = "Customer" (still in ForEach)
        @activity('Copy').output.rowsCopied = 847

Key insight: @item() PASSES values from the pipeline. @dataset() RECEIVES them inside the dataset.

The Scope Rules

Function	Works In	Does NOT Work In
@dataset()	Dataset Connection tab	Pipeline activities
@item()	ForEach child activities	Outside ForEach, Datasets
@activity()	Pipeline (after referenced activity)	Datasets
@pipeline()	Any pipeline activity	Datasets
@trigger()	Any pipeline activity	Datasets
@variables()	Any pipeline activity	Datasets
@concat(), @utcnow()	Anywhere	—

Simple rule: @dataset() lives in datasets. Everything else lives in pipelines. Utility functions work everywhere.

Common Mistakes and How to Fix Them

Mistake	Error	Fix
@dataset() in pipeline	“dataset not defined”	Use @item() in pipeline
@item() in dataset	“item not defined”	Use @dataset() in dataset
@item() outside ForEach	“item not defined”	Only use inside ForEach
@concat(@item()...)	Parse error at nested @	Remove inner @: @concat(item()...)
@item().tablename	Null value	Match exact case: @item().TableName
Double quotes in string	Parse error	Use single quotes
Wrong activity name	Activity not found	Copy-paste exact name
Missing quotes in SQL	SQL syntax error	Use doubled single quotes for literals

Quick Reference Cheat Sheet

DATASET (Connection tab only):
  @dataset().SchemaName
  @dataset().TableName
  @dataset().ContainerName
  @dataset().FolderName

PIPELINE (ForEach activities):
  @item().TableName
  @item().SchemaName
  @activity('Name').output.value        -- Lookup array
  @activity('Name').output.firstRow     -- Lookup single row
  @activity('Name').output.rowsRead     -- Copy metric
  @activity('Name').output.rowsCopied   -- Copy metric
  @activity('Name').output.copyDuration -- Copy metric
  @pipeline().RunId                     -- run GUID
  @pipeline().parameters.ParamName      -- parameter
  @variables('VarName')                 -- variable
  @trigger().name                       -- trigger name

ANYWHERE (utility):
  @concat('a', 'b')
  @utcnow()
  @formatDateTime(utcnow(), 'yyyy-MM-dd')
  @if(condition, trueVal, falseVal)
  @guid()
  @coalesce(val, 'default')

Wrapping Up

ADF expressions follow a simple pattern:

• @dataset() = what parameters does THIS dataset have? (inside datasets)
• @item() = what is the current row in the loop? (inside ForEach)
• @activity() = what did THAT activity produce? (after an activity)
• @pipeline() = what is the pipeline own info? (anywhere in pipeline)

Once you understand these four scopes, every expression becomes readable. The key is knowing which scope you are in — dataset or pipeline — and using the right function.

Related posts: – Parameterized Datasets Deep Dive – Metadata-Driven Pipeline in ADF – Incremental Data Loading – Common ADF/Synapse Errors

Bookmark this page. Questions? Drop a comment below.

---

Naveen Vuppula is a Senior Data Engineering Consultant and app developer based in Ontario, Canada. He writes about Python, SQL, AWS, Azure, and everything data engineering at DriveDataScience.com.