Comment on page


Use Polygon labels for Semantic & Instance Segmentation


Polygons enable semantic and instance segmentation.
Architectures such as FCN and Mask R-CNN support polygons (or pixel masks) as input. Common use cases include visually segmenting the road, pedestrians, and cars for automated vehicles.
Pixels categorized for person, bike, and road.

Define a Polygon Label

  1. 1.
    Navigate to the "Label Definitions" tab of your dataset
  2. 2.
    Type label name in the "Label Name" field
  3. 3.
    Choose Polygon from the "Label Type" drop-down
  4. 4.
    Select label color from the color chooser menu
  5. 5.
    Optionally configure AutoLabel if desired
  6. 6.
    Click "Save" button
  7. 7.
    Repeat as needed for multiple labels
After defining a Polygon label, you can use it to label an image:
Polygon label types also support Sublabels:

Data Format

A Polygon schema element creates an array of objects with data consisting of an array of vertices. Each vertex is a XY coordinate relative (0.0 to 1.0) to width and height respectively. The last point is repeated indicating the Polygon is closed.
"type": "polygon",
"data": {
"points": [
[0.747344340429344, 0.15742085125935987],
[0.9739569760244967, 0.45670599815143104],
[0.927144772766141, 0.9173846907546139],
[0.46022637620936213, 0.9198246950741422],
[0.07559490210070685, 0.8696470576290873],
[0.07105101385880182, 0.47930310191599174],
[0.3470671249450289, 0.19757298531948478],
[0.5400276738305427, 0.18165081542981515],
[0.5234445622475404, 0.34212221989113467],
[0.5948034201128737, 0.34212221989113467],
[0.6007213926803049, 0.18082102823538931],
[0.7763268499780116, 0.18344523182787914],
[0.747344340429344, 0.15742085125935987]
"children": {}

Labeling Polygons

Once you've defined your polygons, you can begin annotating your data.

Training Instance or Semantic Segmentation Models

SmartML supports training polygon labels for either an instance or semantic segmentation model. You can select which type of model output you want to use for training when configuring your SmartML model.

Semantic Segmentation

Semantic segmentation assigns a semantic label to each pixel in an image. Individual object instances are not delineated. For example, if there are multiple "zebra" objects in the image, they all received the same global "zebra" label.
Identifying zebras with semantic segmentation
Select Semantic Segmentation under "Model Output Options" when configuring your new model version:

Instance Segmentation

Instance segmentation goes a step further and distinguishes between individual object instances. For example, if there are multiple "zebra" objects in an image, each object will be assigned an individual "zebra" label.
Identifying zebras with instance segmentation
Select Instance Segmentation under "Model Output Options" when configuring your new model version:
For more information on training models with SmartML, check out our training guide: