Skip to main content

Use cases and quickstart

Every method reads a color from a crop; the filter turns that into three things.

Two compose files. docker compose up (the default, docker-compose.yaml) is the whole-frame color-cycle demo — no detector, no GPU (scenario 2). The per-detection and per-part scenarios below need the detector, so they use docker-compose.detector.yaml.

1. Color per detection (the main use case)Direct link to 1. Color per detection (the main use case)

An operator watches a street feed and wants every car and person tagged with its color, live, so the footage can later be searched for "blue car" or "red shirt".

VideoIn ──▶ object detector ──▶ filter-color-attribute ──▶ Webvis

The detector emits boxes; this filter adds color + color_margin to each detection and draws a viz overlay for Webvis.

docker compose -f docker-compose.detector.yaml up # needs an NVIDIA GPU
# http://localhost:8000 — boxes labelled with the color of each car/person

2. Whole-frame color (no detector) — the default demoDirect link to 2. Whole-frame color (no detector) — the default demo

Point the filter straight at a video with no detector upstream. With FILTER_WHOLE_FRAME_FALLBACK=true (the default) it reads the dominant color of the whole frame and writes it to meta.frame_color. Cheap (one read per frame), useful for a quick "what color is this feed" probe or for pre-cropped single-object streams.

services:
video_in: { image: openfilter-video-in, environment: { FILTER_OUTPUTS: "tcp://*:5550" } }
color:
image: filter-color-attribute
environment:
FILTER_SOURCES: "tcp://video_in:5550"
FILTER_METHOD: hsv # the default demo runs the CPU engine (no GPU)
webvis: { image: openfilter-webvis, environment: { FILTER_SOURCES: "tcp://color:5552;viz" } }

3. No-GPU classical modeDirect link to 3. No-GPU classical mode

Swap the engine with one env var, no rebuild. hsv/rgb need no GPU and no model download, at a real accuracy cost (see the table in the README).

FILTER_METHOD=hsv docker compose up

4. Measure what per-object scoring savesDirect link to 4. Measure what per-object scoring saves

The filter scores each object once instead of once per frame (FILTER_TRACK_COLORS, on by default). To put a number on it, run the ingestion-shaped pipeline twice and diff the dumps:

VideoIn ──▶ filter-sam3-detector ──▶ filter-color-attribute ──▶ recorder
docker build -t filter-color-attribute:bench .
export VIDEO=$PWD/parking.mp4 # a clip where objects persist across frames
mkdir -p output

RUN=off TRACK_COLORS=false docker compose -f docker-compose.bench.yaml up --abort-on-container-exit
RUN=on TRACK_COLORS=true docker compose -f docker-compose.bench.yaml up --abort-on-container-exit

python scripts/bench_track_colors.py output/off.json output/on.json

The report gives crops scored on each side and the reduction factor, and next to it the color agreement between the two runs — read them together. A large reduction with poor agreement means colors are freezing on the wrong read, which is a regression however fast it is. Raise FILTER_CONFIRM_READS if agreement is low; lower it if you want the saving sooner and the scene is easy.

TuningDirect link to Tuning

  • Custom palette — restrict or extend the color vocabulary: FILTER_COLORS="['red','blue','white','black','gray']".
  • Abstain instead of guess — with siglip2, set FILTER_MARGIN_THRESHOLD=0.03 so low-confidence reads (typical at night) come back as "unknown" rather than a wrong color.
  • Turn off the overlay — FILTER_DRAW_VISUALIZATION=false if you only consume the JSON metadata downstream and don't need the viz topic.

Per-part clothing color (people)Direct link to Per-part clothing color (people)

Each garment's color is read from its actual pixels instead of the whole box, so the face, hair and background don't pollute it (needs the detector so there are person detections to parse):

docker compose -f docker-compose.detector.yaml up

Each person detection then carries a parts map with one entry per garment found, out of upper, dress, pants, skirt, shoes, hat, bag and scarf — e.g. "blue shirt / gray pants / white shoes / red hat". Garments that aren't visible are simply absent. The viz overlay shows them.

This runs a second GPU model (mattmdjaga/segformer_b2_clothes) per person, so it is on by default only for siglip2; hsv/rgb cannot use it. Set FILTER_PERSON_PARSING=false to force the plain whole-box read. Detections shorter than FILTER_MIN_PARSE_HEIGHT (64px) skip segmentation and take the whole-box read — they are still answered, never dropped.

Searching by garment word. The parser has a single Upper-clothes class, so the filter cannot tell a t-shirt from a hoodie. It emits the coarse part and exports GARMENT_ALIASES so search can expand queries: hoodie/t-shirt/jacket → upper, cap/beanie → hat, jeans/shorts → pants. This buys search recall, not sub-type precision.

It segments and reads clothing regions only. It does not classify skin, race, or any protected attribute, by design.

Other attributesDirect link to Other attributes

The engine is prompt-driven, so the same filter extends beyond color: point FILTER_COLORS (the vocabulary) at any attribute the SigLIP text encoder can name — e.g. vehicle type or clothing type — without retraining. Color is just the first vocabulary.