Use cases and quickstart
Every method reads a color from a crop; the filter turns that into three things.
Two compose files.
docker compose up(the default,docker-compose.yaml) is the whole-frame color-cycle demo — no detector, no GPU (scenario 2). The per-detection and per-part scenarios below need the detector, so they usedocker-compose.detector.yaml.
1. Color per detection (the main use case)Direct link to 1. Color per detection (the main use case)
An operator watches a street feed and wants every car and person tagged with its color, live, so the footage can later be searched for "blue car" or "red shirt".
VideoIn ──▶ object detector ──▶ filter-color-attribute ──▶ Webvis
The detector emits boxes; this filter adds color + color_margin to each detection and
draws a viz overlay for Webvis.
docker compose -f docker-compose.detector.yaml up # needs an NVIDIA GPU
# http://localhost:8000 — boxes labelled with the color of each car/person
2. Whole-frame color (no detector) — the default demoDirect link to 2. Whole-frame color (no detector) — the default demo
Point the filter straight at a video with no detector upstream. With
FILTER_WHOLE_FRAME_FALLBACK=true (the default) it reads the dominant color of the whole
frame and writes it to meta.frame_color. Cheap (one read per frame), useful for a quick
"what color is this feed" probe or for pre-cropped single-object streams.
services:
video_in: { image: openfilter-video-in, environment: { FILTER_OUTPUTS: "tcp://*:5550" } }
color:
image: filter-color-attribute
environment:
FILTER_SOURCES: "tcp://video_in:5550"
FILTER_METHOD: hsv # the default demo runs the CPU engine (no GPU)
webvis: { image: openfilter-webvis, environment: { FILTER_SOURCES: "tcp://color:5552;viz" } }
3. No-GPU classical modeDirect link to 3. No-GPU classical mode
Swap the engine with one env var, no rebuild. hsv/rgb need no GPU and no model download,
at a real accuracy cost (see the table in the README).
FILTER_METHOD=hsv docker compose up
4. Measure what per-object scoring savesDirect link to 4. Measure what per-object scoring saves
The filter scores each object once instead of once per frame (FILTER_TRACK_COLORS, on by
default). To put a number on it, run the ingestion-shaped pipeline twice and diff the dumps:
VideoIn ──▶ filter-sam3-detector ──▶ filter-color-attribute ──▶ recorder
docker build -t filter-color-attribute:bench .
export VIDEO=$PWD/parking.mp4 # a clip where objects persist across frames
mkdir -p output
RUN=off TRACK_COLORS=false docker compose -f docker-compose.bench.yaml up --abort-on-container-exit
RUN=on TRACK_COLORS=true docker compose -f docker-compose.bench.yaml up --abort-on-container-exit
python scripts/bench_track_colors.py output/off.json output/on.json
The report gives crops scored on each side and the reduction factor, and next to it the
color agreement between the two runs — read them together. A large reduction with poor
agreement means colors are freezing on the wrong read, which is a regression however fast
it is. Raise FILTER_CONFIRM_READS if agreement is low; lower it if you want the saving
sooner and the scene is easy.
TuningDirect link to Tuning
- Custom palette — restrict or extend the color vocabulary:
FILTER_COLORS="['red','blue','white','black','gray']". - Abstain instead of guess — with
siglip2, setFILTER_MARGIN_THRESHOLD=0.03so low-confidence reads (typical at night) come back as"unknown"rather than a wrong color. - Turn off the overlay —
FILTER_DRAW_VISUALIZATION=falseif you only consume the JSON metadata downstream and don't need theviztopic.
Per-part clothing color (people)Direct link to Per-part clothing color (people)
Each garment's color is read from its actual pixels instead of the whole box, so the face, hair and background don't pollute it (needs the detector so there are person detections to parse):
docker compose -f docker-compose.detector.yaml up
Each person detection then carries a parts map with one entry per garment found, out of
upper, dress, pants, skirt, shoes, hat, bag and scarf — e.g. "blue shirt /
gray pants / white shoes / red hat". Garments that aren't visible are simply absent. The
viz overlay shows them.
This runs a second GPU model (mattmdjaga/segformer_b2_clothes) per person, so it is on by
default only for siglip2; hsv/rgb cannot use it. Set FILTER_PERSON_PARSING=false to
force the plain whole-box read. Detections shorter than FILTER_MIN_PARSE_HEIGHT (64px)
skip segmentation and take the whole-box read — they are still answered, never dropped.
Searching by garment word. The parser has a single Upper-clothes class, so the filter
cannot tell a t-shirt from a hoodie. It emits the coarse part and exports GARMENT_ALIASES
so search can expand queries: hoodie/t-shirt/jacket → upper, cap/beanie → hat,
jeans/shorts → pants. This buys search recall, not sub-type precision.
It segments and reads clothing regions only. It does not classify skin, race, or any protected attribute, by design.
Other attributesDirect link to Other attributes
The engine is prompt-driven, so the same filter extends beyond color: point FILTER_COLORS
(the vocabulary) at any attribute the SigLIP text encoder can name — e.g. vehicle type or
clothing type — without retraining. Color is just the first vocabulary.