Incremental Extract via API Call

Knack Community
1

Hi Community!

I’m a newbie here and looking for any amount of guidance possible! Currently, we are API calling on the data daily for ingestion into our BI tool. At the moment we do a full extract followed by the same thing the next day. This can be taxing on our system not to mention the calls can fail fairly regularly due to timing out.

Does anyone know if Knack can support passing parameters through the JSON API call to limit data to specific criteria? i.e. Newest increment records, newest dates, etc.

Thank you!

2

Hi David,

You can append filters to the API url.

For example:

var api_url = 'https://api.knack.com/v1/objects/object_1/records';

// field_3 is a date/time field
var filters = [
  // Filter for records with a value for this field in the last three months
  {
    "field":"field_3",
    "operator":"is during the previous"
    "range":3,
    "type":"months"
  }
];

// Add filters to route
api_url += '?filters=' + encodeURIComponent(JSON.stringify(filters));

Here’s a related article: Constructing Filters.

For example, you can add a Created At date field to your records, then filter for records where Created At is during the previous 24 hours. Or you could mark the record as Extracted (yes/no) after you’ve extracted it, then filter for records where Extracted is no.

Cheers