logo
February 10, 2023

[TIL] How to trigger a lambda function only if multiple s3 events are met?

How to trigger a lambda function only if multiple s3 events are met

Is there any way to define an S3 trigger that says "only run after these 3 files have been uploaded"?

In this blog, we will delve into the intricacies of defining S3 triggers that meet multiple event requirements.

As far as I know, there is no direct way to trigger Lambda on the basis of such conditions. We need to write your logic in a separate lambda which will trigger your function.

I think we have some approaches here

Approach 1. Batch processing with Zip file uploads

Once the zip is uploaded to S3, the lambda gets notified, processes that zip, extracts all three files, does some processing, and triggers the state machine with processed data.

Approach 2. Evaluating the existence of files before triggering the Lambda function

The S3 uploads would trigger an AWS Lambda function. This function should check for the existence of the other files. If they are present, it should continue processing. If they are not present, it should exit.

Approach 3. Implementing manually triggered SNS Events

What if it simply notified an SNS topic that it had completed a batch of S3 uploads? You could subscribe your Lambda function to that SNS topic.

Reference

  • https://stackoverflow.com/questions/58922234/how-to-trigger-aws-lambda-function-when-multiple-files-in-s3-are-ready
  • https://stackoverflow.com/questions/34376697/aws-want-to-upload-multiple-files-to-s3-and-only-when-all-are-uploaded-trigger