Imagine building your own football analytics engine, one that watches a match and rebuilds it as a live top-down minimap, draws a ring around every player and tracks their movement, marks whoever has the ball, and keeps a running count of possession, passes and turnovers, all from an ordinary broadcast clip and nothing else. The rich data behind this kind of analysis usually comes from expensive dedicated tracking systems, the multi-camera rigs installed in stadiums and the proprietary software sold to broadcasters and clubs, so we set out to see how much of it we could reproduce ourselves with open-source tools, running everything on a single laptop.
This post covers what we built, how each stage of the pipeline works, and how you could assemble the same thing yourself, none of which required training a model from scratch or renting a GPU.
The inspiration
Unsurprisingly, this idea has already been tackled and PlayVision already offers exactly this kind of system for basketball, analysing game footage to follow every player, the ball, and the hoop, mapping the players onto court coordinates, and producing what the company says is more than a million data points per game, all so that coaching staff can stop tagging film by hand. What makes it relevant to us is how it was built, because PlayVision runs on Roboflow's platform, using its annotation tools, hosted training, and an active learning loop to keep the models accurate as footage arrives from arenas whose camera angles and lighting the system has not seen before.
The most useful lesson we took from their account is that the data feeding the model matters more than the architecture, since the case study is almost entirely about annotation and correction, keeping the training data clean as conditions changed from one venue to the next. As co-founder Marc Zoghby put it: "Roboflow saves you a lot of heavy lifting as a developer."
Roboflow also publishes the core building blocks as open source, so we took those, pointed them at football, and gave ourselves a weekend project challenge.
What we built
We built an overlay that reads a football clip and annotates every frame as it plays. Each player is drawn with a coloured ring and a number that stays attached to them as they move, the player currently on the ball gets a second ring in a contrasting colour, and a minimap in the top corner rebuilds the whole pitch from above so you can see the shape of both teams' formations at once. Underneath the minimap, a panel keeps a running tally of possession share, completed passes and turnovers for each side, the player in possession at that instant, and the leaders by passes made and time spent on the ball, while a short caption reports each pass the moment it lands or goes astray.
The full overlay, with player rings coloured by team, the red ring marking the player on the ball, and the panel below tracking possession, passes and top performers as the clip plays.
We ran it on two public Bundesliga broadcast clips from the sample footage the Roboflow examples ship with, with the whole pipeline running locally on an Apple Silicon laptop and no cloud inference involved at any stage.
Every player's position is projected from the broadcast frame onto a top-down pitch, which is what makes possession and distances measurable in the first place.
It is a proof of concept, so it holds up well on the two clips we rendered but starts to slip the moment you ask more of it: the team a player is assigned to can flip for a single frame when someone is half-hidden behind another player, and a shirt number occasionally hops onto the wrong player when two of them cross and the tracker briefly mixes them up. These are the exact errors PlayVision's active learning loop is built to grind down over many games, and we left them in the clips on purpose so you can see where the system still struggles.
How it works
The stages are chained together, each one feeding the next, and the sections below follow a single frame the whole way through, from the raw broadcast image to the finished overlay.
Detection
Detection is handled by a YOLO model fine-tuned on football footage, which returns four classes per frame: players, goalkeepers, referees, and the ball. We did not train this model ourselves; it comes pretrained on a public football dataset hosted on Roboflow Universe, and it was accurate enough out of the box that we spent almost no time on this stage.
Tracking
On its own, detection produces boxes with no memory, so the same player arrives as a fresh and unrelated box in every frame, and ByteTrack is what stitches those boxes into continuous tracks with stable identifiers, so that player 7 stays player 7 from one frame to the next. Almost everything downstream, the numbering, the possession counts, the pass detection, depends on that identity holding steady across the clip.
Telling the teams apart without reading shirts
Assigning each player to a team is the one genuinely clever step, and it comes from Roboflow's sports package. Shirt numbers are useless here because on a broadcast wide shot they are only a few pixels tall and half the players are facing away from the camera, so the package works from overall appearance instead, cropping each detected player, passing the crop through SigLIP to produce an embedding that captures kit colour and pattern, reducing those embeddings to three dimensions with UMAP, and running KMeans to separate them into two clusters that correspond to the two teams. All of this is fitted during a short pass over the opening frames, so by the time the overlay starts drawing, the system already knows what each kit looks like for this particular match.
Finding the ball
The ball is small, moves quickly, and spends much of the match hidden behind players, which is enough to make a detector running on the full frame lose it constantly, so the fix is sliced inference: the frame is cut into tiles and the detector runs on each tile separately, where the ball is large relative to the slice and far easier to spot, and the detections are stitched back together afterwards. A short tracking buffer carries the ball marker through the frames where it disappears completely, so it does not flicker on and off as players cross in front of it.
From broadcast angle to bird's-eye
A second model finds known points on the pitch, the penalty box corners, the centre circle, the line intersections, and matching those to their real positions on a standard pitch lets you solve a homography, the transform that warps the slanted broadcast view into a flat top-down map. We smooth that transform across frames so the minimap does not jitter every time the camera pans, and once it is in place every player box can be converted into an (x, y) coordinate on the pitch.
The homography carries one built-in blind spot, and it is easiest to see in the minimap whenever the ball is played long. The transform assumes every point sits flat on the pitch, so a ball in the air is projected as though it were lying on the grass and slides toward or past the top edge of the map, and because the system only has a flat detection with no sense of height, it cannot tell whether that ball is still in the air, sitting in the goal, or already out of play. A camera mounted directly overhead would remove the error, since a ball's position on a true top-down view barely depends on how high it is, but a broadcast feed never gives you that angle.
Turning positions into stats
With everyone placed on a common pitch map, the remaining statistics are mostly geometry: the player closest to the ball is taken as the likely owner, and accumulating that judgement over time gives you each team's possession share. When the owner changes and the change holds for a few frames, the system records a pass if the ball stayed on the same team and a turnover if it did not, and counting those per team produces completion rates, the top passer, and time on the ball.
The stats panel updating as the clip plays: possession share and pass completion for each team, who has the ball right now, and the leaders by passes and time on the ball.
Build your own
The whole stack is open source. supervision provides the computer vision plumbing, the detection containers, the tracker, the slicer, the annotators, and the video reading and writing. roboflow/sports adds the football-specific parts, the team classifier and the pitch geometry. Ultralytics YOLO runs the detection models themselves, and the sports examples ship with pretrained football weights you can use directly or retrain from the Roboflow Universe datasets.
Most of the work is wiring about five supervision primitives together, and detection and tracking come down to a short loop:
import supervision as sv
from ultralytics import YOLO
model = YOLO("football-player-detection.pt")
tracker = sv.ByteTrack()
for frame in sv.get_video_frames_generator("match.mp4"):
result = model(frame, imgsz=1280)[0]
detections = sv.Detections.from_ultralytics(result)
detections = tracker.update_with_detections(detections)
The team classifier takes two calls: you collect player crops from a quick pass over the clip, fit the classifier once, then predict per frame:
from sports.common.team import TeamClassifier
classifier = TeamClassifier(device="mps")
classifier.fit(crops)
team_ids = classifier.predict(crops)
Ball detection wraps the model in sv.InferenceSlicer, the homography comes from cv2.findHomography on the pitch keypoints, and the minimap is drawn with draw_pitch from the sports package. Roboflow's own walkthrough, track football players with computer vision, is the right place to start if you want the detector and tracker running end to end.
A few specifics are worth knowing before you start: the homography will jitter badly unless you smooth it across frames, and team assignment only stays stable if the opening pass sees enough of both kits, so a clip that begins mid-throw-in can throw the clustering off. The ball depends entirely on the sliced inference, since a detector running on the full frame will lose it for long stretches, and tracker identities survive almost everything except two players crossing at close range, which is the source of the occasional number swap. All of this is ordinary engineering tuning, and it accounts for most of the two days the project took.
What we learned
Our experience lined up with PlayVision's: pretrained detection got us most of the way on the first day, and after that the work was all glue, keeping identities stable through occlusions, smoothing the pitch transform, and turning a stream of coordinates into events that mean something to someone watching.
The same shift puts this within reach of almost anyone, because what used to require a specialist team and a broadcast production budget now takes a few open-source libraries and a laptop, and a single engineer can stand up a credible analytics overlay in a matter of days. Every match at the 2026 World Cup is already being filmed from the angles this pipeline needs, so the data is there for anyone willing to extract it.
