Using S3 connector in ETL pipeline

S3 upload/download capabilities

Depending on the type of connection established, the capabilities associated with uploading and/or downloading files with S3 connector are as follows:

• OAuth 2.0: Upload/Download both (Read and Write). However, user can restrict permissions

• Access Key: Upload/Download both (Read and Write). However, user can restrict permissions

• Public: Download only (Read only).

Prerequisites

• The following permissions are required for file access in S3:

  • s3:PutObject

  • s3:GetObject

  • s3:DeleteObject

  • s3:ListBucket

  • GetObjectTagging

  • PutObjectTagging

• There can be one root user and multiple IAM users.

• Multiple IAM accounts can read and write between Langstack and AWS.

• There can be multiple buckets and the bucket detail can be edited in the connector.