Sync Files from Google Cloud Storage
Connecting a Google Cloud Storage Data Source

What is Google Cloud Storage?

Google Cloud Storage (GCS) is a popular way to store data in the cloud. Like Amazon S3, data is stored in "buckets". Plainsight supports syncing data sets from both public and private buckets. You will need access to the Google Cloud Platform to take advantage of GCS.

Syncing a Google Cloud Storage Source

For a list of supported file types, see Supported File Formats.
    1.
    In the "Sources" tab of your dataset, click the "Google Cloud Storage" option
    2.
    Enter a name for your data source in the Source Name field. This is how it will display in your list of data sources. Example, Chess Images
    3.
    Enter your GCS Bucket Name. This is the globally-unique name for your bucket, so be sure this is spelled correctly with no trailing spaces. Example, my-images
    4.
    Enter an optional folder path in the Folder Name field. Only files from this directory will be synced. Ex: images/cows
    5.
    If this is a private bucket, enable the "Private Bucket" toggle. This will allow you to upload a Service Account Key JSON File with credentials to access a private bucket. More info
    6.
    Enter an optional Filter. You can use this field to filter files within a data source by name. Check out the File Name Filter Guide for more information on how to use this filter.
    7.
    When you are finished, click "Add Source". If there are any errors with your input, they will be displayed on the fields in red.
Adding a GCS bucket
The bucket will begin syncing. You will be taken to the GCS source view as the syncing begins. The Processing Log shows the status of each asset. Synced images will be listed in the Images tab, and videos are listed in the Videos tab.

Creating a GCS bucket

For instructions on setting up a Google Cloud Storage bucket, see here.
Bucket Permissions
GCS buckets have varying levels of permissions. In general, public buckets are available to everyone without credentials to access them, whereas private buckets require Service Account Keys to use. Buckets are private by default, but you can make them public by following these steps. A private bucket will need to grant authenticated users read access in order for the platform to use it. A service account will need to be added as a member of the bucket with a "Storage Object Viewer" role. Private buckets will require credentials tied to this service account. You can see more on creating a Service Account Key JSON file.
Last modified 2mo ago