My application has a Javascript function that makes a copy of an existing set of records that is shown on the screen in a table. I accomplish this by looping through the data model and making object-based API calls to POST data back to Knack. The problem is that the 10 calls/1 second limit is quickly encountered and results in partial copies of data when records are missed.
Knack documentation has pseudo-code that shows (in theory) how a sleep/wait function could delay things but I cannot for the life of me figure out how to throttle my requests. Should I be using some type of retry logic instead each time it fails? If so, how would I accomplish this? Here's a sample of my data:
Knack.showSpinner();
// Get table data
var data_view_287 = Knack.models['view_287'].data.toJSON();
// ************ Loop through Dispatch Details table
for(x = 0; x < data_view_287.length; x++){
item = data_view_287[x];
var objJSON = {
field_250: record.id,
field_245: item.field_245_raw[0].id,
field_261: item.field_261,
field_280: item.field_280,
field_242: item.field_242,
field_243: item.field_243,
field_246: item.field_246_raw[0].id,
field_247: item.field_247,
field_248: item.field_248,
field_286: item.field_286
}
// POST to Records
$.ajax({
url: 'https://api.knack.com/v1/objects/object_99/records',
type: 'POST',
headers: {
'Authorization': Knack.getUserToken(),
'X-Knack-Application-Id': Knack.application_id,
'X-Knack-REST-API-Key': 'not-shown',
'Content-Type': 'application/json'
},
data: JSON.stringify(objJSON),
success: function(records) {
}
}); //End POST
} //End For-loop
Knack.hideSpinner();
I want to add something here, also because I have observed some behaviour with the knack API which I don't fully understand.
The scenario: I have two tables(A and B) from two different objects. Both those tables has a checkbox. When I press the button for each checked box of table B I create a new record connected to the checked object B with a field named with the object checked with table A. This means that if I select 10 objects from A and 10 Objects from B, I am creating 100 connected records ad once.
Now, first of all, for the benefit of testing, I've created this scene and I realized that I was not getting any 429 error. The calls were supposedly launched almost all together and slowly slowly the results were getting back and the records were created.
I thought that knack API had a sort of cache / queue so after the 10 calls per second limit was putting the other on hold. regardless, the more the time was passing the less the results were coming (for example, I got the first 40 results in about 45 seconds, and the other 60 in minutes). Even if I was not getting any error, I was totally generating a sort of delay (I'm talking about VIEW based requests since everything is in the Javaascript and I'm not happy to expose my API key).
So, without any error to intercept, I thought about throttling the API requests in ADVANCE. How? With a deferred execution. I'm basically counting the number of iterations, every 10 I delay the execution by 1 second. Meaning: the first 10 are executed at second "0", the next at second "1" and so on. So I am sure that the request from 90 to 100 are executed about 10 seconds AFTER I hit the button.
Do you think this approach makes sense ? Following the code I am using
$("#bulk").click(function(){
// calculate total number of requests
var count = ($('#view_xxx tbody input[type=checkbox]:checked').length)*($('#view_yyy tbody input[type=checkbox]:checked').length);
var ctr = 0;
var ctr2 = 0;
$('#view_XXX tbody input[type=checkbox]:checked').each(function(i) {
//obmitted code that works with data
$('#view_380 tbody input[type=checkbox]:checked').each(function() {
//obmitted code that preapers all api Headers and Data
ctr++;
var timeout = Math.floor(ctr2 / 10) *1000; // this gives the milliseconds that I have to delay
setTimeout(function(){
$.ajax({
url: api_url,
headers: headers,
type: 'POST',
data: JSON.stringify(data),
complete: function(){ ctr2 ++; } // this is another counter that tells me when all the requests are done
}).done(function(responseData){ console.log(responseData); }); // This is where to check the response
},timeout);
});
});
});
That's perfect!!! Thanks so much for sharing that. I've been trying workarounds for the past few weeks that were overly complicated and not accomplishing what I was looking for. You saved the day.
Hi Christopher,
We've had success catching this error and re-trying within ajax calls using the error handler. A good example is here https://stackoverflow.com/questions/10024469/whats-the-best-way-to-retry-an-ajax-request-on-failure-using-jquery.
Below is the code adopted from that, and using a retryLimit of 3 seems about right for our apps.
$.ajax({
url : 'someurl',
type : 'POST',
data : ....,
tryCount : 0,
retryLimit : 3,
success : function(json) {
//do something
},
error : function(request, status, error ) {
if (request.status == 429) {
this.tryCount++;
if (this.tryCount <= this.retryLimit) {
console.log('try again...');
$.ajax(this);
return;
} else {
console.log('update failed!');
}
} else {
//handle other errors
}
}
});
A 2023 update - now that the API is far more responsive (a good thing) we’re finding the retryLimit of around 10 to be optimal now.