Sync Files from Amazon S3
Connecting an Amazon S3 Data Source

What is Amazon S3?

Amazon Simple Storage Service (S3) is a popular solution for storing data in the cloud. Amazon S3 stores resources in "buckets". Plainsight supports syncing data sets from both public and private buckets. You will need access to an Amazon Web Services (AWS) account to take advantage of Amazon S3.
Read more about Creating a bucket in AWS.

Syncing an Amazon S3 Data Source

See Supported File Formats for a list of compatible files types.
    1.
    In the "Sources" tab of your dataset, click the "Amazon S3" option
    2.
    Enter a name for your data source in the Data Source Name field. This is how it will display in your list of data sources. Example, Chess Images.
    3.
    Enter your Bucket Name. This is the globally-unique name for your bucket, so be sure this is spelled correctly with no trailing spaces. Example, my-images
    4.
    Enter an optional folder path in the Folder Path field. Only files from this directory will be synced. Ex: images/cows
    5.
    If your bucket is private you will need to authenticate it by entering an Access Key ID and a Secret Access Key in their respective fields.
    6.
    Under "Filters", enter an optional File Name Filter. You can use this field to filter files within a data source by name. Check out the File Name Filter Guide for more information on how to use this filter. Example, *-june-*
    7.
    When you are finished, click "Add Source". If there are any errors, they will be displayed on the fields in red.
The bucket will begin syncing. You will be taken to the GCS source view as the syncing begins. The Processing Log shows the status of each asset. Synced images will be listed in the Images tab, and videos are listed in the Videos tab.

Creating a bucket

For instructions on setting up an Amazon S3 bucket, see here.
Public vs. Private buckets
AWS S3 buckets have varying levels of permissions. In general, public buckets are available to everyone, whereas private buckets require security tokens to access. If your bucket is private, be sure the "Public Bucket" is unchecked when adding the data source. You will need to add your security credentials using the Access Key ID and Secret Access Key fields. Buckets are private by default, but you can easily make them public during creation. For more detailed information on how to give buckets public read access, see this page. Note: if you are using a public bucket, you have to grant Public Access to Everyone along with List objects access. For more information, see step 4c here.

Last modified 2mo ago