roboflow · greeshmab21 · Jul 2, 2026 · Jul 2, 2026 · Jul 6, 2026 · Jul 6, 2026
@@ -0,0 +1,114 @@
+---
+name: supervision
+description: use when working with roboflow/supervision
+---
+
+# supervision
+
+`supervision` is a model-agnostic computer vision library (roboflow/supervision) for working with detection/segmentation results: building `sv.Detections`, drawing with annotators, tracking objects across frames, and processing video/streams.
+
+This skill covers the patterns an agent gets wrong most often. For anything not covered here, read the source under `src/supervision/` rather than guessing at an API — many method/parameter names look plausible but don't exist (see "common mistakes" in each reference file).
+
+## Most common pattern: detect + annotate
+
+The standard loop is: run a model, wrap its output in `sv.Detections`, draw boxes and labels, output/save the frame.
+
+```python
+import cv2
+import supervision as sv
+from ultralytics import YOLO
+
+model = YOLO("yolov8n.pt")
+box_annotator = sv.BoxAnnotator()
+label_annotator = sv.LabelAnnotator()
+
+image = cv2.imread("image.jpg")
+result = model(image)[0]
+detections = sv.Detections.from_ultralytics(result)
+
+labels = [
+    f"{class_name} {confidence:.2f}"
+    for class_name, confidence in zip(detections["class_name"], detections.confidence)
+]
+
+annotated = box_annotator.annotate(scene=image.copy(), detections=detections)
+annotated = label_annotator.annotate(
+    scene=annotated, detections=detections, labels=labels
+)
+
+cv2.imwrite("annotated.jpg", annotated)
+```
+
+See `references/detection.md` for building `Detections` from other sources (inference, SAM3) and filtering, and `references/annotators.md` for the full annotator list and the compose pattern.
+
+The repo also publishes `docs/llms.txt` for general model-level facts and API surface; this skill focuses specifically on the mistakes agents repeatedly make in practice (wrong method names, deprecated APIs, silently-ignored kwargs) with runnable patterns, rather than restating the API reference.
+
+## Critical decision: InferencePipeline vs sv.process_video
+
+These solve the same problem — "run a model over every frame of a video/stream" — but they are not interchangeable. Picking the wrong one is the single most common architectural mistake in supervision code.
+
+**Use `sv.process_video` when:**
+
+- The input is a finite video *file* on disk and you want an output video file.
+- You bring your own model call inside a `callback(frame, frame_index) -> np.ndarray`.
+- You want the simplest possible script — no threading, no queues.
+
+```python
+import supervision as sv
+
+
+def callback(frame: sv.numpy.ndarray, frame_index: int) -> sv.numpy.ndarray:
+    result = model(frame)[0]
+    detections = sv.Detections.from_ultralytics(result)
+    return box_annotator.annotate(scene=frame.copy(), detections=detections)
+
+
+sv.process_video(
+    source_path="input.mp4",
+    target_path="output.mp4",
+    callback=callback,
+    show_progress=True,  # NOT `progress=True` — see references/video.md
+)
+```
+
+**Use `roboflow/inference`'s `InferencePipeline` when:**
+
+- The source is a *live* stream — webcam, RTSP, or an infinite feed — not a video file you're transcoding.
+- You need the model inference itself decoupled/threaded from frame reading for real-time throughput (InferencePipeline runs inference in a background thread and calls your `on_prediction` callback as results become available).
+- You don't need a saved output video, or you'll build one yourself (e.g. with `sv.VideoSink`) inside the callback.
+- You want Roboflow-hosted or local Roboflow models run for you, rather than calling `model(frame)` yourself each iteration.
+
+```python
+from inference import InferencePipeline
+import supervision as sv
+
+box_annotator = sv.BoxAnnotator()
+
+
+def on_prediction(result, video_frame):
+    detections = sv.Detections.from_inference(result)
+    annotated = box_annotator.annotate(
+        scene=video_frame.image.copy(), detections=detections
+    )
+    cv2.imshow("frame", annotated)
+    cv2.waitKey(1)
+
+
+pipeline = InferencePipeline.init(
+    model_id="your-model/1",
+    video_reference=0,  # webcam, or an RTSP URL
+    on_prediction=on_prediction,
+)
+pipeline.start()
+pipeline.join()
+```
+
+**Rule of thumb:** file in, file out → `sv.process_video`. Live/streaming source, or you need async/threaded inference → `InferencePipeline`. Don't reach for `InferencePipeline` just to process a static mp4 — it adds threading complexity `process_video` doesn't need, and don't use `process_video` on an infinite/live source — it assumes a finite frame count from `VideoInfo`.
+
+## Reference files
+
+- `references/detection.md` — building `sv.Detections`, key attributes, filtering, common mistakes.
+- `references/annotators.md` — annotator classes, correct parameter names, the compose pattern.
+- `references/tracking.md` — tracking with the `trackers` package (`ByteTrackTracker`), why `sv.ByteTrack` is deprecated, correct parameter/method names, filtering by `tracker_id`.
+- `references/video.md` — `sv.process_video`, `VideoInfo`, `VideoSink`.
+- `references/utils.md` — `PolygonZone`, `LineZone`, `sv.Color` / `sv.ColorPalette`.
@@ -0,0 +1,60 @@
+# Annotators
+
+All annotators implement `.annotate(scene, detections, ...) -> np.ndarray`. They never mutate `scene` in place as documented usage — pass `scene.copy()` (or reuse the returned array as the next annotator's input) rather than the original frame if you need the original preserved.
+
+## Common annotator classes
+
+| class                       | draws                                                    | notable params                                                       |
+| --------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------- |
+| `sv.BoxAnnotator`           | rectangle bounding boxes                                 | `color`, `thickness`                                                 |
+| `sv.RoundBoxAnnotator`      | rounded-corner boxes                                     | `color`, `thickness`, `roundness`                                    |
+| `sv.BoxCornerAnnotator`     | corner-only brackets                                     | `color`, `thickness`, `corner_length`                                |
+| `sv.LabelAnnotator`         | text labels (needs `labels=[...]`)                       | `color`, `text_color`, `text_scale`, `text_padding`, `text_position` |
+| `sv.RichLabelAnnotator`     | text labels with custom font/unicode support             | `font_path`, `text_color`, `text_scale`                              |
+| `sv.MaskAnnotator`          | filled segmentation masks                                | `color`, `opacity`                                                   |
+| `sv.PolygonAnnotator`       | mask/box outline as polygon                              | `color`, `thickness`                                                 |
+| `sv.EllipseAnnotator`       | ellipse under each box (good for people-tracking)        | `color`, `thickness`, `start_angle`, `end_angle`                     |
+| `sv.CircleAnnotator`        | circle around each box center                            | `color`, `thickness`                                                 |
+| `sv.DotAnnotator`           | filled dot at box center/anchor                          | `color`, `radius`, `position`                                        |
+| `sv.TriangleAnnotator`      | triangle marker above box                                | `color`, `base`, `height`                                            |
+| `sv.HaloAnnotator`          | glow/halo around mask                                    | `color`, `opacity`, `kernel_size`                                    |
+| `sv.HeatMapAnnotator`       | cumulative heatmap across frames                         | `position`, `opacity`, `radius`                                      |
+| `sv.BlurAnnotator`          | blur out detected regions                                | `kernel_size`                                                        |
+| `sv.PixelateAnnotator`      | pixelate detected regions                                | `pixel_size`                                                         |
+| `sv.TraceAnnotator`         | draws tracked path history, needs `tracker_id`           | `color`, `position`, `trace_length`                                  |
+| `sv.CropAnnotator`          | pastes a zoomed crop of each detection back on the scene | `position`, `scale`                                                  |
+| `sv.IconAnnotator`          | places a custom image/icon at each detection             | `icon_path` or `icon_resolver`, `icon_scale`                         |
+| `sv.PercentageBarAnnotator` | draws a confidence bar under each box                    | `color`, `height`, `width`                                           |
+
+Most annotators accept `color=sv.Color(...)` / `sv.ColorPalette(...)` and a `color_lookup` argument (`sv.ColorLookup.CLASS`, `.INDEX`, `.TRACK`) controlling whether color is assigned per class, per detection index, or per tracker id.
+
+## Compose pattern (chain annotators)
+
+Annotators are meant to be chained — call each one with the previous output as the new `scene`:
+
+```python
+box_annotator = sv.BoxAnnotator()
+label_annotator = sv.LabelAnnotator()
+trace_annotator = sv.TraceAnnotator()
+
+annotated = scene.copy()
+annotated = trace_annotator.annotate(scene=annotated, detections=detections)
+annotated = box_annotator.annotate(scene=annotated, detections=detections)
+annotated = label_annotator.annotate(
+    scene=annotated, detections=detections, labels=labels
+)
+```
+
+Order matters visually — draw fills/masks/traces first, then outlines, then labels on top so text isn't obscured.
+
+## Common mistake
+
+```python
+# WRONG — this class does not exist in supervision
+annotator = sv.BoundingBoxAnnotator()
+
+# RIGHT
+annotator = sv.BoxAnnotator()
+```
+
+Other name mix-ups worth double-checking against `src/supervision/annotators/core.py` before using: `sv.LabelAnnotator` (not `TextAnnotator`), `sv.MaskAnnotator` (not `SegmentationAnnotator`), and `sv.TraceAnnotator` (not `PathAnnotator` / `TrajectoryAnnotator`).
@@ -0,0 +1,94 @@
+# Detections
+
+`sv.Detections` is a single dataclass-like container used across the whole library. Every model integration converts its native output into one of these — always prefer the `from_*` constructor over building `Detections(...)` by hand.
+
+## Creating Detections from common sources
+
+```python
+import supervision as sv
+
+# Ultralytics (YOLOv8/v9/v10/11, SAM, etc.)
+result = model(image)[0]
+detections = sv.Detections.from_ultralytics(result)
+
+# Roboflow inference (hosted or `inference` package)
+result = model.infer(image)[0]
+detections = sv.Detections.from_inference(result)
+
+# Segment Anything (SAM / SAM2 / SAM3)
+sam_result = mask_generator.generate(image)
+detections = sv.Detections.from_sam(sam_result)
+
+# Transformers (e.g. DETR, Grounding DINO via HF pipeline)
+detections = sv.Detections.from_transformers(transformers_results, id2label=id2label)
+```
+
+Each `from_*` method normalizes the model's native output into the same set of attributes below — this is the whole point of using them instead of parsing raw model output yourself.
+
+## Key attributes
+
+| attribute    | shape / type                     | notes                                                                                                 |
+| ------------ | -------------------------------- | ----------------------------------------------------------------------------------------------------- |
+| `xyxy`       | `np.ndarray (N, 4)`              | float, `[x1, y1, x2, y2]` per box, always present                                                     |
+| `confidence` | `np.ndarray (N,)` or `None`      | float scores                                                                                          |
+| `class_id`   | `np.ndarray (N,)` or `None`      | integer class ids                                                                                     |
+| `tracker_id` | `np.ndarray (N,)` or `None`      | set after running a tracker, not by detection alone                                                   |
+| `mask`       | `np.ndarray (N, H, W)` or `None` | boolean segmentation masks                                                                            |
+| `data`       | `dict`                           | extra per-detection arrays, e.g. `data["class_name"]`; also accessible via `detections["class_name"]` |
+
+`len(detections)` gives the number of boxes (`N`). `Detections` is empty-safe: with zero detections, arrays have shape `(0, 4)` / `(0,)` rather than being `None`.
+
+## Filtering patterns
+
+`Detections` supports NumPy-style boolean-mask indexing directly — this is the correct and idiomatic way to filter. It does **not** have a `.filter()` method.
+
+```python
+# keep only class_id == 0 (e.g. "person")
+detections = detections[detections.class_id == 0]
+
+# confidence threshold
+detections = detections[detections.confidence > 0.5]
+
+# combine conditions
+detections = detections[(detections.class_id == 0) & (detections.confidence > 0.5)]
+
+# keep detections inside a set of classes
+detections = detections[np.isin(detections.class_id, [0, 2, 3])]
+
+# by area
+detections = detections[detections.area > 1000]
+
+# slicing / indexing a single detection
+first = detections[0]
+```
+
+`Detections` also supports `+` to merge two instances and `sv.Detections.merge([d1, d2])` for combining more than two.
+
+## Common mistakes
+
+```python
+# WRONG — Detections has no .filter() method
+detections = detections.filter(lambda d: d.class_id == 0)
+
+# RIGHT — boolean-mask indexing
+detections = detections[detections.class_id == 0]
+```
+
+```python
+# WRONG — comparing class_id to a class name string
+detections = detections[detections.class_id == "person"]
+
+# RIGHT — class_id is an integer id; compare to the class name via class_name data,
+# or map the name to its integer id first
+detections = detections[detections["class_name"] == "person"]
+```
+
+```python
+# WRONG — assuming confidence/class_id are always populated
+avg_conf = detections.confidence.mean()  # crashes if confidence is None
+
+# RIGHT — guard when the source may not populate a field (e.g. some SAM masks
+# have no class_id/confidence)
+if detections.confidence is not None:
+    avg_conf = detections.confidence.mean()
+```
@@ -0,0 +1,95 @@
+# Tracking
+
+## sv.ByteTrack is deprecated — use the `trackers` package
+
+`supervision`'s built-in `sv.ByteTrack` is deprecated. The current, maintained way to track objects is the standalone `trackers` package (`pip install trackers`), which provides `ByteTrackTracker` (plus `SORTTracker`, `OCSORTTracker`, `BoTSORTTracker`). It still consumes/returns `sv.Detections`, so everything else in this skill (filtering, annotating) works unchanged — only the tracker object and its update method differ.
+
+```python
+import cv2
+import supervision as sv
+from trackers import ByteTrackTracker
+
+tracker = ByteTrackTracker(
+    track_activation_threshold=0.25,
+    lost_track_buffer=30,
+    minimum_consecutive_frames=3,
+    minimum_iou_threshold=0.3,
+)
+
+trace_annotator = sv.TraceAnnotator()
+box_annotator = sv.BoxAnnotator()
+label_annotator = sv.LabelAnnotator()
+
+for frame in frames:
+    result = model(frame)[0]
+    detections = sv.Detections.from_ultralytics(result)
+
+    detections = tracker.update(detections)  # NOT `.update_with_detections()`
+
+    labels = [f"#{tracker_id}" for tracker_id in detections.tracker_id]
+
+    annotated = trace_annotator.annotate(scene=frame.copy(), detections=detections)
+    annotated = box_annotator.annotate(scene=annotated, detections=detections)
+    annotated = label_annotator.annotate(
+        scene=annotated, detections=detections, labels=labels
+    )
+
+tracker.reset()  # call between videos/streams to clear track state
+```
+
+`ByteTrackTracker` is stateful, like the old `sv.ByteTrack` — create **one** instance per video/stream and reuse it every frame; creating a new instance per frame resets tracking.
+
+### Correct constructor parameters (`ByteTrackTracker`)
+
+- `track_activation_threshold` (default `0.25`) — minimum detection confidence to start a new track. Not `confidence_threshold`.
+- `lost_track_buffer` (default `30`) — frames to keep a track alive with no matching detection before dropping it.
+- `minimum_consecutive_frames` (default `3`) — consecutive detections required before a track is confirmed; suppresses spurious one-frame detections.
+- `minimum_iou_threshold` (default `0.3`) — IOU threshold for matching detections to existing tracks. This replaced the old `minimum_matching_threshold` name.
+
+### Correct method name
+
+- `tracker.update(detections) -> Detections` — NOT `update_with_detections()` (that was the `sv.ByteTrack` method name) and NOT plain `update()` semantics from other libraries — pass the `Detections` object, get a new `Detections` back with `tracker_id` populated.
+- `tracker.update(detections, timestamp=...)` — pass a monotonic `timestamp` in seconds if your pipeline has irregular/dropped frames, so Kalman prediction and lost-track pruning match the real time gap instead of assuming a fixed frame rate.
+- `tracker.reset()` — clears all track state; call this between videos, not just once at startup.
+
+```python
+# WRONG — this is the deprecated sv.ByteTrack method name
+detections = tracker.update_with_detections(detections)
+
+# RIGHT — ByteTrackTracker from the `trackers` package
+detections = tracker.update(detections)
+```
+
+## Legacy: sv.ByteTrack (deprecated, still present in supervision)
+
+You may still encounter `sv.ByteTrack` in older code. It behaves the same way conceptually but with different names — recognize it, don't write new code against it:
+
+```python
+tracker = sv.ByteTrack(
+    track_activation_threshold=0.25,
+    lost_track_buffer=30,
+    minimum_matching_threshold=0.8,  # note: different default/name than the new package
+    frame_rate=30,
+)
+detections = tracker.update_with_detections(detections)  # old method name
+```
+
+If you see this pattern in an existing codebase, prefer migrating it to `ByteTrackTracker` from `trackers` rather than extending it further.
+
+## Filtering by tracker_id
+
+After tracking, `tracker_id` is just another attribute you can boolean-index on, same as `class_id`:
+
+```python
+# only detections that have been assigned a tracker id (drop unmatched, if any)
+detections = detections[detections.tracker_id != None]  # noqa: E711 (elementwise, not `is not None`)
+
+# keep only a specific tracked object
+detections = detections[detections.tracker_id == 7]
+
+# exclude ids you've already counted/processed
+seen_ids = {1, 2, 3}
+detections = detections[~np.isin(detections.tracker_id, list(seen_ids))]
+```
+
+Note `tracker_id` is `None` on a `Detections` object until it has been passed through `tracker.update(...)` at least once — accessing it before that raises/returns `None`, it is not auto-populated by `from_ultralytics` / `from_inference` alone. Also note `detections.tracker_id != None` (elementwise numpy comparison) is intentional here, not a mistake — `is not None` would do a Python identity check on the whole array instead of an elementwise mask.