Comment on page
Sync Files from Google Cloud Storage
Connecting a Google Cloud Storage Data Source
Google Cloud Storage (GCS) is a popular way to store data in the cloud. Like Amazon S3, data is stored in "buckets". Plainsight supports syncing datasets from both public and private buckets.
Walk-through syncing a GCS datasource
- 1.In the "Sources" tab of your dataset, click "Add Data"
- 2.Select the "Google Cloud Storage" option
- 3.Enter a name for your data source in the Source Name field. This is how it will display in your list of data sources. Example, Chess Images
- 4.Enter your GCS Bucket Name. This is the globally-unique name for your bucket, so be sure this is spelled correctly with no trailing spaces. Example, my-images
- 5.Enter an optional folder path in the Folder Name field. Only files from this directory will be synced. Ex: images/cows
- 6.If this is a private bucket, enable the "Private Bucket" toggle. This will allow you to upload a Service Account Key JSON File with credentials to access a private bucket. More info
- 7.Optionally, enter any Filters for your files.(optional) File Extension: Specify one or more file extensions to sync within the bucket folder path. Hit the
returnkey between each one. Supported types are: JPG, PNG, GIF, BMP, TIFF, WEBP, MP4, MOV, AVI, FLV, MKV, and WEBM.(optional) File Name Filter: Specify one or more glob patterns to filter files by name. Hit thereturnkey between each one. Check out the File Name Filter Guide for more information on how to use this filter. - 8.When you are finished, scroll down and click "Add Source" at the bottom right. If there are any errors with your input, they will be displayed on the fields in red.
The bucket will begin syncing. You will be taken to the GCS source view as the syncing begins. The Processing Log shows the status of each asset. Synced images will be listed in the Images tab, and videos are listed in the Videos tab.
Bucket Permissions
GCS buckets have varying levels of permissions. In general, public buckets are available to everyone without credentials to access them, whereas private buckets require Service Account Keys to use.
Buckets are private by default, but you can make them public by following these steps.
A private bucket will need to grant authenticated users read access in order for the platform to use it. A service account will need to be added as a member of the bucket with a "Storage Object Viewer" role. Private buckets will require credentials tied to this service account. You can see more on creating a Service Account Key JSON file.
Last modified 5mo ago