Bulk Data - Pagination

Overview

Use pagination when you need to retrieve large datasets in smaller batches. Dakota supports this through max_num (page size / limit) and offset (starting position) inside the filters object.

Parameter	Description
`max_num`	Number of records to return per request (limit / page size). Recommended range: 20–50
`offset`	Number of records to skip before returning results (defaults to `0` if omitted)
`order_by`	Sort order for stable paging (recommended for bulk pulls)

How offset and max_num work

Request	`offset`	`max_num`	Returns
Page 1	`0`	`50`	Records 1–50
Page 2	`50`	`50`	Records 51–100
Page 3	`100`	`50`	Records 101–150

Formula: offset = (page_number - 1) * max_num

Response shape

List responses return an envelope:

{
  "records": [ /* page of records */ ],
  "next_offset": 50
}

Field	Meaning
`records`	Array of records for the current page
`next_offset`	Offset to use for the next request. `-1` means there are no more pages

count_only responses return:

{
  "record_count": 160167
}

How to Use

Optionally call with count_only: true to get record_count.
Request page 1 with offset: 0 and your chosen max_num.
For the next page, set offset to the previous response’s next_offset (or add max_num to the current offset).
Stop when next_offset is -1 (last page, empty result, or past end). You can also stop when records.length < max_num.

Empty and last-page behavior

Scenario	`records`	`next_offset`
Full page	`max_num` items	next page offset (for example `10`, `20`, `50`)
Last page (partial)	fewer than `max_num` items	`-1`
No matches (empty filter)	`[]`	`-1`
Offset at/beyond total count	`[]`	`-1`

Account Examples

First Page of Accounts

{
  "module": "account",
  "filters": {
    "fields": ["sfid", "name", "type", "aum__c", "lastmodifieddate"],
    "order_by": "lastmodifieddate:DESC",
    "max_num": 50,
    "offset": 0
  }
}

Next Page of Accounts

{
  "module": "account",
  "filters": {
    "fields": ["sfid", "name", "type", "aum__c", "lastmodifieddate"],
    "order_by": "lastmodifieddate:DESC",
    "max_num": 50,
    "offset": 50
  }
}

Paginate Accounts with a Filter

{
  "module": "account",
  "filters": {
    "fields": ["sfid", "name", "type", "aum__c", "lastmodifieddate"],
    "order_by": "name:DESC",
    "max_num": 50,
    "offset": 0,
    "filter": [
      {
        "aum__c": {
          "$between": [100, 500000]
        }
      }
    ]
  }
}

Count Accounts Before Paging

{
  "module": "account",
  "count_only": true
}

Contact Examples

First Page of Contacts

{
  "module": "contact",
  "filters": {
    "fields": ["sfid", "firstname", "lastname", "email", "lastmodifieddate"],
    "order_by": "lastmodifieddate:DESC",
    "max_num": 50,
    "offset": 0
  }
}

Next Page of Contacts

{
  "module": "contact",
  "filters": {
    "fields": ["sfid", "firstname", "lastname", "email", "lastmodifieddate"],
    "order_by": "lastmodifieddate:DESC",
    "max_num": 50,
    "offset": 50
  }
}

Paginate Contacts with a Date Filter

{
  "module": "contact",
  "filters": {
    "fields": ["sfid", "firstname", "lastmodifieddate", "contact_type__c"],
    "order_by": "firstname:DESC",
    "max_num": 50,
    "offset": 0,
    "filter": [
      {
        "lastmodifieddate": {
          "$between": ["2025-01-01", "2025-01-20"]
        }
      }
    ]
  }
}

Count Contacts Before Paging

{
  "module": "contact",
  "count_only": true
}

Best Practices

Prefer next_offset === -1 as the primary stop condition for bulk jobs.
Always include order_by so page results stay consistent across requests.
Use a max_num between 20 and 50 for reliable bulk pulls.
Use count_only first when you need the total record count before paging.
Apply the same filter and order_by on every page of a bulk job.
max_num works as either a number (50) or a string ("50").
Request fields use payload names (for example sfid, lastname); responses may return mapped names (for example account_id / contact_id, last_name).

Token expiry

Access tokens expire after 18,000 seconds (5 hours). When pulling large datasets (for example hundreds of thousands of Account or Contact records), a long-running pagination script can exceed this window. If the token expires mid-run, later page requests will fail with an unauthorized / invalid token response. Recommendations:

Track elapsed time while paging.
Refresh or re-authenticate and obtain a new token before the 5-hour limit if the job may run longer.
Resume from the last successful offset / next_offset after getting a new token so you do not restart from page 1.

Get started

Fields

Bulk Data - Pagination

Overview

How offset and max_num work

Response shape

How to Use

Empty and last-page behavior

Account Examples

First Page of Accounts

Next Page of Accounts

Paginate Accounts with a Filter

Count Accounts Before Paging

Contact Examples

First Page of Contacts

Next Page of Contacts

Paginate Contacts with a Date Filter

Count Contacts Before Paging

Best Practices

Token expiry

​Overview

​How offset and max_num work

​Response shape

​How to Use

​Empty and last-page behavior

​Account Examples

​First Page of Accounts

​Next Page of Accounts

​Paginate Accounts with a Filter

​Count Accounts Before Paging

​Contact Examples

​First Page of Contacts

​Next Page of Contacts

​Paginate Contacts with a Date Filter

​Count Contacts Before Paging

​Best Practices

​Token expiry

Overview

How offset and max_num work

Response shape

How to Use

Empty and last-page behavior

Account Examples

First Page of Accounts

Next Page of Accounts

Paginate Accounts with a Filter

Count Accounts Before Paging

Contact Examples

First Page of Contacts

Next Page of Contacts

Paginate Contacts with a Date Filter

Count Contacts Before Paging

Best Practices

Token expiry