On 24 July 2026, Microsoft announced on its Linux and Open Source blog that it was open-sourcing MLVC, the ML Video Codec. The code is at github.com/microsoft/mlvc under the MIT licence, with trained weights, training scripts and the conversion tooling to target different NPUs.
The paper behind it, MLVC: Multi-platform Learned Video Codec for Real-World Deployment, went up on arXiv a month earlier.
This is not a research demo that ran once on a lab GPU. MLVC is the product iteration of the DCVC line that Microsoft Research has been publishing since 2021, and Microsoft says it is already rolling out in Microsoft Teams for peer-to-peer calls, with live telemetry, A/B testing and a fallback to conventional codecs when the hardware or the network says no.
The headline claim is big enough to be worth checking properly, so that is what this post does: the numbers, what they actually measure, what is really in the repository, and whether any of it touches a streaming stack this year.
If you want the short version first, we keep a plain definition of what MLVC is in our streaming glossary.
The Headline Numbers
Microsoft's own comparison, at equivalent subjective quality:
| Resolution | Bitrate saved vs H.264 | Bitrate saved vs H.265 |
|---|---|---|
| 360p | 87.8% | 75.5% |
| 540p | 82.7% | 65.4% |
The concrete version is easier to hold on to. A 360p 30 fps call that needs 1 Mbps of H.264 needs about 122 kbps of MLVC. Roughly one eighth of the bits, in real time, on a laptop NPU.
Now the fine print, because it matters a lot.
These are human scores, not PSNR
Those percentages are MOS-based, from an ITU-T P.910 subjective test where human viewers scored the clips. The source material is the Video Conferencing Dataset, which Microsoft also built and open-sourced. So this is people on webcams, at 360p and 540p, judged by eye.
Measure the same codec with PSNR instead, across the paper's wider test set, and the BD-rate gain lands closer to 52%. Still a very large number. Not 87.8%.
The anchor is the weakest H.265 you can buy
The comparison point is hardware H.265 on Intel Quick Sync, not a well-tuned x265 running slow presets. Hardware encoders constrained to conferencing latency are the floor of what H.265 can do, not the ceiling.
There is also no comparison against AV1 or VVC anywhere in the paper. The authors say why: they omit VTM and ECM because those have no commodity hardware implementations, so there is nothing to benchmark real-time against. Fair as a research decision, but it means nobody has yet shown you MLVC against a modern AV1 encoder.
The Real Achievement Is Determinism, Not Compression
Here is the part that got buried under the bitrate numbers, and it is the more interesting engineering story.
Neural codecs have beaten conventional codecs on coding efficiency for a while. They could not be deployed, because they could not be decoded reliably on a different machine than the one that encoded.
Entropy coding needs the encoder and the decoder to compute identical probabilities. Run the same network on an Apple Neural Engine and a Qualcomm Hexagon and you do not get identical floating point results, because the compilers pick different kernels, fuse operators differently, and round differently. The bitstream then decodes into garbage.
The paper puts a number on this. DCVC-RT, unconstrained, scores about 69.6% BD-rate improvement when encoder and decoder sit on the same platform. Across platforms its BD-rate is infinite, which is the polite way of saying the video does not decode at all.
How they fixed it
MLVC stops asking the two sides to agree by luck. It transmits the entropy model's scale parameters explicitly through the hyperprior, so both ends read the same values out of the bitstream instead of each computing their own.
Around that, the team removed everything else that diverges. Exotic activation functions are gone in favour of ReLU and LeakyReLU with ReGLU gating. INT8 quantisation is rejected outright, because INT8 convolution is not reproducible across vendors and older Apple silicon simulates it in FP16 anyway. The whole thing runs in FP16.
That discipline costs real compression. A perceptually trained DCVC-RT scores 81.9% where MLVC scores 75.5%. Microsoft gave up about six points of BD-rate to buy a codec that decodes on hardware it does not own.
For anyone who has shipped video, that is obviously the right trade, and it is the first time a learned codec has made it.
What Is Actually in the Repository
Worth looking at directly, because the repository tells a slightly different story than the announcement.
The repo was created on 28 January 2026 and sat there with nothing but a licence, a README stub and Microsoft's standard governance files for six months. The actual code landed on 22 July in a single commit titled "Hello MLVC", pushed by Tanel Pärnamaa, the paper's first author. At the time of writing it has 8 commits, 189 stars, 10 forks and no tagged releases at all.
So this is a code drop, not a project developed in the open. That is not a criticism, just the correct expectation to have. All four open issues are Dependabot bumps for transformers, jupyterlab and cryptography. The only outside contribution merged so far is a one-line fix adding #include <cstdint> to EntropyCoder.h so the C++ entropy coder builds on GCC 13 and newer Clang. Small, but a good sign: someone outside Microsoft tried to compile it within days, and Microsoft merged the patch.
What you get
- Four checkpoints. The full MLVC at 18.3 million parameters and the lighter MLVC-S at 5.4 million, each in a PSNR-optimised and a perceptually optimised variant. The perceptual ones are trained with LPIPS and a face-segmentation mask, which tells you exactly what they were built for.
- The full training pipeline, for both the image and video models, driven by YAML configs. Microsoft also documented how it collected training data, which is more than most releases do.
- Export tooling that converts a model to CoreML for Apple, ONNX for Intel and OpenVINO, and QNN for Qualcomm, at fixed input dimensions. Supported runtimes are ONNX Runtime on CPU and GPU, ONNX Runtime QNN, OpenVINO and Windows ML.
- A C++ rANS entropy coder in
packages/msrtc_ranswith Python bindings, plus benchmark tooling that computes BD-rate, MS-SSIM, LPIPS, VIF and DeQA.
What you do not get is a C++ codec library you can link into an application. Microsoft says that is coming in a follow-up release. Today the usable surface is Python, and you need Python 3.12 or 3.13 and uv to install it.
Where the Frame Rates Land
Speed is the reason this is interesting, so the resolution ladder matters. Encoding throughput from the paper:
| Resolution | MLVC, Apple M3 Pro | MLVC, 3-vendor average | MLVC-S, Apple M3 Pro |
|---|---|---|---|
| 360p | 129.5 fps | 103 fps | not published |
| 540p | 65.7 fps | 49 fps | not published |
| 720p | 33.8 fps | not published | 83.8 fps |
| 1080p | 15.6 fps | not published | 39.5 fps |
Decoding runs a few frames per second slower than encoding across the board. On Intel Lunar Lake the 1080p figure is 10.5 fps, and the three vendors tested are Apple M3 and M4, Intel Lunar Lake and Qualcomm Snapdragon X Elite.
The honest summary: 540p at 30 fps is the shipping claim, on all three vendors, at under half the NPU. 1080p real time exists only on the cut-down model, on Apple, in one direction. The paper also notes that encoding and decoding simultaneously on one device at 1080p and above is still a problem, which is precisely what a two-way call needs.
Microsoft's roadmap follows that: stabilise 540p and improve loss resilience first, then 1080p and broader streaming scenarios.
Why You Cannot Put This in a Streaming Ladder
Compression efficiency has never been what decides a codec's fate in streaming. Decoders do.
We wrote a whole post about how HEVC never became the future it was promised to be, and the reason was licensing and device support, not coding tools. AV1 only started mattering to real deployments when phones shipped hardware decoders for it.
The format is the weights
MLVC has a harder version of that problem. There is no MLVC bitstream standard, because the trained weights are the format. A stream is only decodable by a device holding the exact model that encoded it, running it closely enough to reproduce the arithmetic.
No conformance streams. No MIME type. No packaging story, no DRM story, and nothing in a smart TV, a set-top box or a browser that can play it. Your CDN can carry it, and then nothing at the other end can open it.
The content is not your content
The impressive numbers come from static talking heads at low resolution, with a model trained on faces using a face ROI mask. Sport, film grain, fast pans and high motion are exactly where conventional codecs earn their heuristics, and the paper does not claim those.
It even quantifies one weakness: the long-term reference mechanism costs about 2 percentage points of BD-rate on abrupt scene changes, mitigated with scene-cut detection at deployment time. A movie is mostly scene changes.
And nobody has measured the power draw
The paper's own closing limitation. Real time is solved, they say, so power consumption is the next frontier. Running a neural network on the NPU for every frame of a two-hour film on a battery device is a very different proposition from a fixed-function decoder that has been optimised for a decade.
Where It Does Make Sense Today
Everywhere both ends are computers you control, and the network is the constraint rather than the silicon.
That is conferencing first, which is why Teams is the launch vehicle. It is also contribution feeds over bad uplinks, remote production, drones, robotics and surveillance links, and cloud gaming or remote rendering. Microsoft explicitly asks for help on all of those in the announcement, alongside streaming and VOD, plus platform ports to widen NPU coverage.
If you are building anything real-time and peer-to-peer on modern hardware, this is worth an afternoon of your time right now. If you are running an OTT service, it is not.
The Part That Should Actually Worry Codec People
Not the 75%. The slope.
Microsoft claims MLVC's coding efficiency improves with model capacity and training compute, in the way every other machine learning system has for the last few years.
Conventional codecs improve by roughly 30 to 50% per generation, and a generation takes the better part of a decade of standardisation, followed by years of waiting for hardware, followed by a patent pool argument. If a learned codec keeps getting better every time someone points more compute at it, and it runs on NPUs that ship in everything anyway, the shape of that competition changes.
It is MIT licensed, which quietly removes the other thing that killed HEVC. And it degrades into a normal engineering problem: you ship a model version, you negotiate it like you negotiate a codec, you fall back when the far end cannot run it.
None of that is happening this year. The gap between "runs at 540p on my M3" and "plays on a five-year-old TV in someone's living room" is the same gap that has taken every codec fifteen years to cross. But the last time compression got interesting enough to bother reading a paper about was a while ago, and this one is worth the hour.
What To Do About It
Nothing changes in your encoding ladder. Keep shipping H.264 for reach, HEVC and AV1 where the devices support them, and revisit this when three things happen:
- 1080p real time on the full model, not just MLVC-S on Apple.
- A C++ library you can actually link into a product.
- Any decoder outside Microsoft's own products.
Clone the repo if you are curious, because the training pipeline and the cross-platform analysis are instructive whether or not you ever ship a neural codec. The appendix on why INT8 quantisation diverges across vendors is a compact explanation of a problem that usually takes far longer to describe.
Frequently Asked Questions
Is MLVC free to use?
Yes. Microsoft released it under the MIT licence, which covers the model source, the trained weights and the training scripts. There is no patent pool attached, which is a meaningful difference from HEVC.
Can I stream MLVC to a smart TV or a browser?
No. Decoding requires a device that can run the neural network, and the exact model that encoded the stream. No TV, set-top box, phone or browser has an MLVC decoder today, and there is no bitstream standard for one to implement.
How much bandwidth does MLVC actually save?
Microsoft reports 87.8% less bitrate than H.264 and 75.5% less than hardware H.265 at 360p, judged by human viewers under ITU-T P.910. Measured with PSNR across a wider test set, the saving is closer to 52%. Both figures are against hardware encoders on video conferencing content.
What hardware does MLVC need?
An NPU. Microsoft tested Apple Neural Engine on M3 and M4, Intel Lunar Lake and Qualcomm Snapdragon X Elite, and reports real-time 540p at 30 fps on all three at under half the NPU's capacity.
Is MLVC better than AV1?
Nobody has published that comparison. The paper benchmarks only against H.264 and H.265, and omits AV1, VVC and ECM because they have no commodity hardware implementations to compare against in real time.
What is the difference between MLVC and DCVC?
DCVC is the Microsoft Research line of neural codecs published since 2021. MLVC is the productised version, trading roughly six points of compression efficiency for the ability to decode reliably across NPUs from different vendors.
Need Help Deciding What This Means for Your Stack?
Codec decisions are expensive when they go wrong, and most of the cost lands years later in device support and licensing rather than in bitrate. Our streaming and broadcast talents have shipped codecs, encoding ladders and delivery stacks at scale, and you can brief one directly about the problem you are actually facing.