Whenever someone upvotes something on a votable or forumable board, we count that upvote but we also recompile the page. Imagine a scenario where you have tons of people on one page where there are tons of things to be upvoted and the upvote command is coming in so frequently, we can hardly catch up constantly recompiling the page.
I would like to implement a solution where we put page recompilations behind a permissive limiter such that the first request to compile goes in right away, but if another request comes in within a certain short period of time, we delay the recompilation from happening. This does not impede other logic like recording the number of upvotes.
If another request comes in during that delay, we make the delay longer but we don't queue up two recompilations. Another example would be if we get 1000 requests to recompile within a minute, we don't do 1000 recompilations, we do only one, and that's after a delay that is tied to that velocity of requests.
Now the problem with this approach is that we have all these executions happening in the context of lambda. How could we adjust our architecture such that we could allow for this kind of limiting? I want to see an implementation plan.
Implementation idea
You want to wait until you have a certain number of records (or a certain amount of time passes) before spinning up a Lambda, rather than firing one Lambda per event.
The Solution: Batch Window (SQS, Kinesis, or DynamoDB Streams).
How to do it:
In the Lambda Trigger configuration (Event Source Mapping), set the Batch Window (MaximumBatchingWindowInSeconds) and Batch Size.
Example: Set Batch Size to 100 and Batch Window to 30 seconds.
Result: Lambda will wait up to 30 seconds to gather 100 records. If only 50 arrive in that time, it runs with 50. This significantly reduces costs and "cold starts" by processing items in bulk.
So if I did it this way, I could make a separate lambda function specifically for handling emails for upvoting, but that would also mean the upstream logic would need to call this function if certain conditions of the incoming email exist. Maybe I need a dispatching lambda for incoming emails and from there it can call lambdas for commenting versus anything else.