Links
Comment on page

Configure Label Detection

Label detection is a computer vision task to detect any entities in an image such as objects, products, animal species, scene, location, action, activities, and more.
It differs from object detection in that it does not return a bounding box or polygon around a particular instance. Instead, it will returns the label description and confidence scores.
Image credit: Alex Knight on Unsplash.
For example, the image above may return the following labels:
Description
Score
Tire
0.9670996069908142
Bicycle
0.9648268818855286
Wheel
0.9450284242630005
Automotive lighting
0.9129207730293274
Building
0.8878806233406067
Label detection can be useful if you do not have a custom model trained with your specific data or if you are looking to sort through the type of data you have before you label.
This block uses the Label Detection Google Cloud Vision API. Responses are returned in Plainsight format. Read the docs for more info:
Private cloud users must activate the Google Cloud Vision API in order to use this block.

Adding a Label Detection Block:

To add a label detection block:
  1. 1.
    Navigate to the "Pipeline Blocks" tab.
  2. 2.
    Under the Prediction section, click on the Label Detection box to configure this processing block. You may need to click "Show All" or use the search box to find the block.
  3. 3.
    Enter the desired Result Limit between 1-50. This will limit the number of results returned per image for this block.
  4. 4.
    Click "Add to Pipeline" to add the block to your pipeline. This will bring you back to the "Pipeline Blocks" tab. From there you can add additional blocks and drag and drop to rearrange around other blocks as needed.
Image Sizing
Recommended image size for label detection: 640 x 480
Vision API requires images to be a sufficient size so that important features within the request can be easily distinguished. Sizes smaller or larger than these recommended sizes may work. However, smaller sizes may result in lower accuracy, while larger sizes may increase processing time and bandwidth usage without providing comparable benefits in accuracy.
Image file size should not exceed 20 MB.

Example Output

{
"type": "multiSelect",
"children": {},
"data": {
"selected": [
"Tire",
"Bicycle",
"Wheel",
"Automotive lighting",
"Building",
"Bicycle wheel",
"Lighting",
"Electricity",
"Vehicle",
"Neighbourhood"
]
},
"scores": [
0.9670996069908142,
0.9648268818855286,
0.9450284242630005,
0.9129207730293274,
0.8878806233406067,
0.8852881193161011,
0.8629685044288635,
0.8295934796333313,
0.8165712356567383,
0.8109857439994812
]
}