The point is to ask once per card, not once per frame
Card recognition already works. Point a good model at a trading card and the name, the set and the number come back in about half a second. Running that against video is where it falls apart — not technically, but in sheer volume.
Video is mostly repetition. A card held up for four seconds at fifteen frames a second is sixty pictures of one card, and asking an identification endpoint sixty times gets you the same answer sixty times over. Every naive streaming pipeline does exactly that, which is why streaming analysis has a reputation for being something only a funded platform can run.
cardstream exists to delete that repetition. It is a small state machine between your video and the recognition call, built almost entirely from signals it works out for itself — has the scene settled, is this the same card as a moment ago, is there even a card in frame. The call fires when a genuinely new card has settled, and not otherwise. Everything else runs on hardware you already own.
One call per card, not per frame
One hour of show at fifteen frames a second — the top of the rate the project is verified against — and roughly a hundred distinct cards put in front of the lens. Two ways to analyse exactly the same video:
identification calls — one per frame, most of them re-asking a question that was answered a moment ago.
identification calls — one per distinct card, whatever your frame rate, however long you hold each card up.
Around 540× fewer calls, and the gap is not a trick of the numbers: it is the whole design. Hold a card still and the count does not move. Run the camera faster and the count does not move. The call count tracks the cards you show, not the hours you stream.
Who keeps it running
Built by people who ship recognition
cardstream comes from the team behind Ximilar’s collectibles recognition. The identification endpoints it calls are the ones we run in production, so the client and the service are maintained by the same people.
The work is in the open
The full package is on GitHub — the state machine, both deployment shapes and the browser UI. Nothing about the decision logic is hidden behind a service you cannot inspect.
Tested without a network
The whole suite runs offline, with fakes standing in for the model files and the HTTP calls. Fork the repo on a laptop with no API key and you can still tell whether you broke something.
What the code is not allowed to forget
-
One call per card, not per frame
Every design decision starts here. If a signal can be computed locally, it runs before the one call that leaves the machine.
-
One copy of the decision logic
When to call lives in exactly one module. Not spread across a driver, a transport and a UI where three people can change it three ways — one file, readable in a sitting.
-
Drivers stay thin
Scheduling, logging and I/O belong to the driver; what counts as a new card belongs to the engine. Swapping how frames arrive never quietly changes when a call goes out.
-
Nothing blocks the frame loop
Decode, detection, HTTP and disk all run off the loop. A slow identification drops a frame; it never turns your stream into a growing lag.
-
No provider lock-in, including ours
The identification call is one swappable step behind a small interface. Every gate, throttle and cache keeps working if you point it somewhere else entirely.
-
Self-hosted by default
Your hardware, your key, your video. In client-side mode a single crop per distinct card is the only thing that leaves the machine.
Things the cards taught us
Some of this is counter-intuitive enough that we write it down where the next person will find it — in the repo, next to the code it constrains.
A helpful hint can cost you accuracy
Telling the endpoint which game a card is from switches off its own writing-system classifier, which then falls back to latin — so a Japanese card quietly matches its English print. Measured, not guessed, which is why setting a game without a writing system is treated as a mistake.
A slightly loose crop matches better
A tight cut around the card looks right and performs worse. The crop that goes out carries a deliberate margin, because a little context beats a clean edge.
Recovering beats pretending
When a call hangs past its timeout the pipeline unsticks itself and moves on rather than waiting on a result that may never come. The limitation is written down in the repo instead of papered over.
Read it, run it, send patches
The repository is public: one installable Python package with independent extras and a browser UI. The locator and embedding weights are ours, trained and published Apache-2.0 alongside the releases. Issues and pull requests are welcome — especially reports from real shows, which surface things no test rig does.
Recognition itself is the Ximilar endpoint and always will be; that is the part with the card database behind it. Everything around it is yours to change.
Open source. Self-hosted. Your stream, your stack.
No platform in the middle, no seat licence: connect the Ximilar API or your own identification system, and cardstream calls it once per distinct card instead of once per frame. Whether you're breaking on Whatnot, running a Fanatics Live-style show, or streaming your own live commerce setup, run cardstream tonight on the hardware you already have.