Comment on page
Sync Files from Amazon S3
Connecting an Amazon S3 Data Source
Amazon Simple Storage Service (S3) is a popular solution for storing data in the cloud. Amazon S3 stores resources in "buckets". Plainsight supports syncing data sets from both public and private buckets. You will need access to an Amazon Web Services (AWS) account to take advantage of Amazon S3.
On-Demand users: Assets synced from S3 will have thumbnails counted towards storage usage. Any clips or frames generated from synced video files will also count towards usage.
- 1.In the "Sources" tab of your dataset, click "Add Data"
- 2.Select the "Amazon S3" option
- 3.Enter a name for your data source in the Data Source Name field. This is how it will display in your list of data sources. Example, Chess Images.
- 4.Enter your Bucket Name. This is the globally-unique name for your bucket, so be sure this is spelled correctly with no trailing spaces. Example, my-images
- 5.Enter an optional folder path in the Folder Path field. Only files from this directory will be synced. Ex: images/cows
- 6.If your bucket is private you will need to authenticate it by entering an Access Key ID and a Secret Access Key in their respective fields.
- 7.Optionally, enter any Filters for your files.(optional) File Extension: Specify one or more file extensions to sync within the bucket folder path. Hit the
returnkey between each one. Supported types are: JPG, PNG, GIF, BMP, TIFF, WEBP, MP4, MOV, AVI, FLV, MKV, and WEBM.(optional) File Name Filter: Specify one or more glob patterns to filter files by name. Hit thereturnkey between each one. Check out the File Name Filter Guide for more information on how to use this filter. - 8.When you are finished, click "Add Source". If there are any errors, they will be displayed on the fields in red.
The bucket will begin syncing. You will be taken to the GCS source view as the syncing begins. The Processing Log shows the status of each asset. Synced images will be listed in the Images tab, and videos are listed in the Videos tab.
Public vs. Private buckets
AWS S3 buckets have varying levels of permissions. In general, public buckets are available to everyone, whereas private buckets require security tokens to access.
If your bucket is private, you will need to add your security credentials using the Access Key ID and Secret Access Key fields.
Buckets are private by default, but you can easily make them public during creation. For more detailed information on how to give buckets public read access, see this page.
Note: if you are using a public bucket, you will need to grant Public access to Everyone along with List objects access.
Last modified 1yr ago