Finance and defence got forward deployed engineers. Media did not. Here is what the role looks like when your data runs at 25 frames a second and your failures happen on air.
Over the last eighteen months, the fastest growing job title in software has been one almost nobody outside enterprise AI had heard of. Postings for forward deployed engineer grew more than 800% between January and September 2025, and more than 1,000% year over year into 2026. OpenAI built an entire business around the role, backed by more than four billion dollars. Anthropic signed a 1.5 billion dollar joint venture with Blackstone and Goldman Sachs whose whole purpose is to place these engineers inside banks.
Banking has them. Defence has them. Healthcare has them. Broadcast and streaming, an industry that spent all of 2026 buying agentic AI at NAB and IBC, mostly does not.
The chart above is one segment of a real HLS stream, one bar per frame. The tall orange spikes are keyframes, and they land on the dashed lines marking a scene change. That alignment is the whole argument of this article in one picture: the encoder already worked out where the content changes, and it wrote the answer into the file. Most AI pipelines pointed at video ignore that and pay a model to work it out again.
This article is an attempt to define the missing job, because the work already exists and the people doing it have no name to put on it.
What a forward deployed engineer actually is
The model comes from Palantir, which has run it for roughly two decades. A forward deployed engineer embeds with a customer, works inside their environment, and owns a system end to end: scoping it, writing the production code, and keeping it running once it is live.
One line separates the role from consulting:
Consultants deliver reports. Forward deployed engineers deliver a running system.
The reason the role exploded is not fashion. MIT's GenAI Divide study looked at 300 public AI deployments alongside interviews and surveys, and found that 95% of enterprise AI pilots produced no measurable impact on profit. The headline number has drawn fair methodological criticism, and it is not the interesting part. The interesting part is where the failures clustered. It was not model quality. It was integration: the unglamorous work of connecting a capable model to a real workflow, real data, and real people who have to use it on a Tuesday.
The same study found that specialised outside teams succeeded roughly 67% of the time, against roughly 33% for internal builds. Not because the outsiders had better models. Because they had done it before, and knew where it breaks.
Why media needs its own version of the role
Every industry says its data is special. Most are wrong. Media is one of the exceptions, and the reasons are structural rather than cultural.
A generic AI engineer has spent their career on text, tables, and single images. All three are small, addressable, and forgiving. Video is none of those things. It arrives at 25, 30, 50 or 60 frames a second, it carries its own clock, it is legally encumbered, it is often encrypted, and when it fails it fails in front of an audience at a known time.
Below are nine things that a generic AI engineer has never had to learn, and that decide whether a media AI project ships or joins the 95%.
1. Video is not a context window. The engineering is the reduction
A 90 minute match at 25 frames a second is 135,000 frames. At 50 fps it is 270,000. You cannot send that to a model, and the naive fix, sampling every Nth frame, either misses the moment that mattered or costs more than the content earned.
The real work is deciding which frames the model ever sees, and the cheapest signal for that is one you already paid for. Your encoder made a decision about every single frame when it compressed the file. I-frame placement marks scene changes. Motion vectors describe what moved. Frame types tell you where complexity sits. Recent research such as CodecSight formalises this: use codec signals to steer what the vision model looks at, instead of re-deriving it with expensive inference.
This is exactly what our free Video Quality Analyzer surfaces. It reads I, P and B frame types, frame reordering, B-frame runs and macroblock decisions straight out of the bitstream. Everything in that chart was computed once, by the encoder, and most AI pipelines throw it away.
There is a stronger version of the same idea. Adrian Roe, CEO of id3as and the driving force behind the Norsk live media server, has described three ways to build automatic 16:9 to 9:16 reframing: run object detection locally, send frames to a cloud model, or do both by detecting objects locally and sending the model only the resulting coordinates rather than the pictures. The third option cuts token usage by around two orders of magnitude, and it often produces better results, because the model is no longer reasoning about a mass of pixels irrelevant to the question. Sending less is not a compromise here. It is the improvement.
2. The answer has to land on frame 4102, not near it
Timecode is the primary key of this industry, and it is a minefield. Drop frame versus non drop frame. 29.97 versus 30. Presentation timestamps versus decode timestamps. Wall clock versus media clock. EXT-X-PROGRAM-DATE-TIME and whether anyone upstream set it correctly.
An AI system that reports "the goal happens around 42:17" is useless to the editor who has to cut on a frame. Generic AI ships approximately. Broadcast needs exact, and knowing why those two differ is most of the job. If you have never had to explain B-frame reordering to a data scientist, in the sentence where you also explain that frame 4102 is not the 4102nd frame displayed, you have not done this work yet.
3. Where the model runs is a contract question, not a preference
Pushing a pre-transmission asset to a third party API can breach a distribution agreement, break an embargo, or violate the content security terms a studio imposed on you. TPN, MovieLabs and CDSA requirements are not suggestions, and "we used a cloud API" is not a defence.
There is a harder version of the same constraint. You cannot decode an encrypted stream. Every quality metric that needs actual pixels is off the table the moment DRM is in play, which kills a good number of architectures before a line of code is written. Our Stream Analyzer detects DRM and gates every decode-based metric on it, because reporting a confident number you could not possibly have measured is worse than reporting nothing.
Knowing what you are not allowed to look at is half the design.
4. Quality here has a number, and someone signs off on it
VMAF. PSNR. SSIM. EBU R 128 and ATSC A/85 for loudness. Apple's HLS authoring specification. This industry decided a long time ago that "does it look right" was not good enough, and built measurements instead.
An engineer who proposes a smart AI-driven encoding ladder without a VMAF curve attached has proposed something no broadcaster can approve, no matter how good the idea is. The number is not bureaucracy. It is how a person puts their name on a change that will go out to millions of devices.
Here is what that looks like in practice. This segment measures 4.03 Mbps against a declared peak of 3.30 Mbps. A player that trusted the manifest has under-provisioned for it, and on a constrained connection that is precisely how a rebuffer begins. Nobody noticed because the average looked fine. Averages are where this kind of problem hides, which is why the industry measures instead of looking.
5. The unit economics are the pilot
In most industries the unit of AI cost is a request. In media it is an hour of content, and an archive runs to tens of thousands of hours.
This is where media AI projects actually die: not on accuracy, on arithmetic. A per-asset cost that looks trivial in a demo becomes a six figure invoice the moment you point it at the library. Real time work such as live captioning needs sub-five-second latency on always-on GPUs. Batch analysis of an archive can run on cheap off-peak compute overnight. Choosing which parts of a workflow belong in which bucket is the single highest value decision in the project, and it happens before anyone writes code.
Latency is part of the same budget. A round trip to a cloud model typically costs somewhere between two and ten seconds, which is fine for tagging an archive overnight and disqualifying for anything that has to keep up with a live event. The rule that follows is one that JP Saibene, CEO of the media engineering firm Qualabs, puts well: architect so that the expensive path is the exception rather than the default. Broadcast spent decades in a world where an encoder set to 1080p produced 1080p and the invoice was known in advance. Consumption-priced inference removes both of those certainties at once, and a demo that never had a quota attached is how a single clip ends up costing a hundred dollars.
Streaming has done a version of this arithmetic before, on delivery. The number that ever mattered there was not the cost of one stream in a demo, it was egress at scale, and the teams who modelled it upfront were the ones who kept their margins. Inference is a different calculation, but it is the same habit, and it is the habit that decides whether a media AI project survives contact with the archive. Our CDN Cost Optimizer does it for the delivery side.
6. Deciding what matters inside the frame
Reframing 16:9 to 9:16 for vertical distribution sounds like a crop. It is a saliency problem: which subject carries the shot, when does attention move, and what happens when two people are talking at opposite edges of frame. It is judged frame by frame, by humans, and it is unforgiving.
This is real AI applied to pixels rather than to transcripts, and it is a good interview topic precisely because everyone underestimates it. Our Vertical Reframe tool is still marked beta, and that label is not modesty. It is a status report on the problem.
The reason is worth spelling out, because it says something about the state of the field. The tool lets you choose which vision model does the looking: Claude, Gemini, GPT, Grok, Llama, Qwen, Pixtral, a local model through Ollama, or your own if you have one. That menu is not a feature we were proud of adding. It exists because there is no settled answer about which model is good at this. Different models win on different content, and the failure modes vary more than any benchmark suggests, so the choice is exposed rather than made on your behalf.
Notice what that implies, because it generalises well beyond reframing. When a capability is mature, nobody ships a model picker. You ship the answer. A picker is an admission that the field has not converged yet, and anyone planning a project on top of automatic reframing should price that in rather than trust a demo reel where somebody chose the clip. It also tells you where the durable engineering actually sits. Not in the model, which will be swapped several times before this is solved, but in the harness around it: how you sample, how you evaluate the output, and what happens when the answer is obviously wrong.
The failure mode is worth picturing. Norsk demonstrated automatic reframing on a fencing bout, a sport that is inherently horizontal and therefore a deliberately hostile test. The model locked onto a large poster of two fencers hanging in the background and framed the shot on that, ignoring the actual competitors. It was not confused about what fencing looks like. It found something that looked more like fencing than the fencing did. Every subject-tracking system has a version of this failure waiting in it, and finding yours before an audience does is the work.
7. An AI system in a broadcast chain is a component with an SLA, not a demo
A hallucinated answer in a chatbot is an annoyance. A hallucinated caption on live television is an accessibility failure with a regulator attached. Current ASR reaches word error rates under 5% on clean broadcast dialogue, which is genuinely comparable to a human transcriber, and still not the same thing as being safe to broadcast unattended.
The patterns that work are confidence gates, human in the loop where the stakes justify it, and a deterministic fallback for when the model is unavailable or unsure. Knowing which parts of a workflow can tolerate probabilistic output at all is a design skill, not a policy question.
It also has to be auditable after the fact, which is harder than it sounds when the component is probabilistic. The practice Norsk described for automated ad insertion is the right shape: record every frame sent to the model, along with the model's state immediately before and immediately after that frame. When something goes out at the wrong moment, you can reconstruct what the system believed and what changed its mind. Broadcast has always been able to answer "why did that happen" about any other component in the chain. An AI component does not get an exemption from that question just because the answer is harder to produce.
Our AI Ad Generator is the honest illustration of what this costs. The model is the smallest part of it. The rest is a render farm, a job queue, cancellation, a sweeper for stuck jobs, web push so the user is told when the tab is closed, payments, and an archive. That ratio is what shipping AI to real users looks like.
8. Media systems outrun the engagement
A 24/7 channel is not delivered, it is handed over. Someone has to run it at three in the morning, and that person was not in the workshop where the architecture was agreed.
Here is a concrete example of the kind of knowledge that only comes from operating one. Across our platform, H.264 encoding levels are locked and identical in three separate code paths. Change one and playback breaks at discontinuity boundaries, because some decoders reset when the level changes mid-stream. Nobody learns that from a paper. You learn it from a channel that stuttered at the exact moment one asset handed over to the next, and from the week it took to work out why.
That is the work the words "forward deployed" are doing. The deliverable is not a system that worked on the day you demonstrated it. It is one somebody else can run on a night you are not there.
9. The agent surface is the next thing to get right
Agents can now operate media systems directly. We wrote about running a 24/7 TV channel through an AI agent over MCP, and the interesting question turned out not to be whether an agent can schedule a channel. It can. The question is which nineteen tools you expose, what an agent is permitted to touch in a live playout system, and what happens when it is confidently wrong at 02:00.
Somebody has to design that surface. That person needs to understand both halves.
What this sounds like when practitioners say it out loud
None of the above is theoretical, and I am not the only person arguing it. A webinar called When AI Runs The Stream puts two people who build this for a living on the same hour, and they land on most of the nine points above from a completely different direction.
Adrian Roe is co-founder and CEO of id3as, the company behind Norsk, a live media server and SDK used to build streaming workflows in code. Juan Pablo Saibene, who goes by JP, is co-founder and CEO of Qualabs, a Montevideo engineering firm whose teams embed inside the engineering organisations of video technology and media companies in the United States and Europe. The session was moderated by Eric Schumacher-Rasmussen.
Read that second description again, because it is the argument of this whole article stated as a company profile. Embedding engineers inside a media customer's own organisation to build and run the thing is forward deployed engineering. It is already happening in our industry. It just does not have the name yet.
What makes the hour worth your time is that neither of them is selling the easy version. The question they keep returning to is not whether AI can do something to a live stream. It is whether you can run it a thousand times and trust the output every time, in front of a real audience, on a real budget, without breaking what already works.
Three patterns from it are worth stealing outright, because each one is a design decision rather than a model choice.
Design so that mistakes are cheap
Adrian Roe's test for whether a job suits AI at all comes down to three questions. Can you frame the problem narrowly enough to play to the model's strengths? Is there wiggle room in the timing? And when it gets things wrong, is the mistake cheap?
The examples are instructive because they are so unglamorous. If you are auto-inserting an ad at a change of ends in tennis, use an L-bar or a squeezeback rather than a full-screen break, because a mistimed L-bar is mildly annoying while a mistimed full-screen break destroys the moment. If you are switching between two redundant feeds on a quality score, the switch is hitless when both feeds are healthy, so an unnecessary switch costs the viewer nothing. Get the design right and the model is allowed to be wrong occasionally, which is the only condition under which you can ship it at all.
Turn the model's answers into deterministic ones
Saibene, whose teams sit inside client engineering organisations, described a quality-of-experience diagnosis system built as a funnel. The first layer is a plain threshold over a sliding window, no AI at all, answering only whether something is wrong. The second matches the anomaly against causes already seen, again deterministic. Only the third layer, reached when the first two find nothing familiar, hands the open-ended question to a model.
The important part is what happens next. When the model proposes an explanation for a genuinely new fault, it is not simply acted on. It is offered to an experienced engineer, and if they approve it, that fault is added to the deterministic layer and diagnosed without a model ever again. The human is there, in Saibene's phrasing, "not as a safety blanket but as a mechanism". The system gets more deterministic, and cheaper to run, the longer it operates. That is the opposite of how most AI deployments age.
The bottleneck moves, it does not disappear
Saibene's second example is the one most teams have not seen coming. Generative tools can produce thousands of clips, and every one of them still has to be checked by somebody before it goes out. The constraint did not go away when generation got cheap. It moved from making to validating, and validation is the part nobody has automated.
So the interesting product is not another highlight generator. It is the validator: give it a clip and a spec, get back a publishable or not-publishable verdict. Some of those checks are pure arithmetic, such as confirming a crop really is 9:16. Some genuinely need judgment, such as whether a sponsor is actually visible or whether the commentary matches the action. Knowing which is which is the entire design.
Two smaller details from the same hour are worth carrying around. On a live monitoring dashboard showing something like eighty numbers, Roe pointed out that only two of them come from AI, and that this ratio is correct rather than a shortfall. And on cost, Roe's compression of the whole problem is hard to improve on: context is king, context is cost, and too much context is poor quality.
Forward deployed engineer, consultant, or systems integrator?
The industry already has good consultants and good systems integrators. The distinction is worth stating plainly, because the words get used interchangeably and they should not be.
| Role | What you receive | Who owns it after |
|---|---|---|
| Consultant | An assessment, a recommendation, an architecture | You do, from scratch |
| Systems integrator | A vendor's product, installed and configured | You and the vendor |
| Vendor forward deployed engineer | That vendor's product, made to fit your stack | The vendor, while you remain a customer |
| Independent media FDE | Code that runs in your environment | You, with the patterns transferred to your team |
The role is real inside vendors already. Both Bitmovin and CAMB.AI currently post roles titled forward deployed broadcast or streaming engineer, sitting between their engineering teams and their marquee media customers. That is a strong signal: the companies closest to the problem worked out they needed this person before the rest of the industry named the job.
The difference with an independent one is simple and it is not a criticism of vendors. A vendor's forward deployed engineer is paid to make one product fit. An independent one is paid to make your stack work, including the parts of it you would rather replace.
There is a strategic version of this worth thinking about before you sign anything. As agents start orchestrating media workflows, the valuable layer is not the model, which will be replaced several times this decade. It is the orchestration and the context underneath it: your assets, your rules, your operational knowledge. JP Saibene of Qualabs, crediting Andy Beach's work on this, puts the risk plainly: let a vendor own that layer while you do not own the context beneath it, and you end up renting your own workflow back from your suppliers. Whoever builds that layer for you should be building it so that you keep it.
Five questions that tell you whether someone is real
If you are hiring for this, the CV will not help you. Everyone's CV says AI now. These five questions separate people who have shipped video from people who have only ever sent text to an API.
1. "We want to auto-tag a 40,000 hour archive. Walk me through the cost model."
You want them to start with the unit, not the model. Hours of content, cost per hour, batch versus real time, and what runs on off-peak compute. If they open with which model they would pick, they have never had this project killed by a finance director.
2. "Our masters are DRM protected. How does that change your approach?"
The right answer is that it eliminates anything requiring decode, and pushes you to container level metadata, sidecar assets, or an analysis point upstream of encryption. A wrong answer is enthusiasm.
3. "The model says the highlight is at 42:17. What do I do with that?"
You are listening for frame accuracy, timecode format, and whether the output can drive an EDL, an SCTE-35 marker, or a MAM record. "It gives you a timestamp" is not an answer, it is a restatement.
4. "How do you sample frames from a two hour asset?"
Every Nth frame is the weak answer. Shot boundary detection is a decent answer. Using I-frame placement and motion vectors the encoder already produced is the answer from someone who has actually had to pay for inference at scale.
5. "What happens when the model is unavailable during a live show?"
There must be a deterministic fallback and a defined degraded mode. If the answer involves retries and nothing else, that system has never been on air.
What it costs, and what it should return
A word of caution about the numbers circulating. Total compensation of 300,000 to 1.2 million dollars gets quoted for forward deployed engineers, and it is real, but it describes in-house packages at frontier AI labs in the United States, heavily weighted toward equity. It is not what media pays, and it is not what an independent specialist in Europe charges. Read those figures as evidence that the market is repricing the role, not as a rate card.
What is worth measuring is the other side. The cost of the 95% is a pilot that consumed a year of a team's attention and shipped nothing. Against that, the relevant question is not the day rate. It is how quickly someone can tell you that the project as scoped will cost four times its budget at archive scale, or that DRM makes the proposed approach impossible, and whether they can then build the version that works.
That conversation should happen in the first week, not the third quarter.
Frequently asked questions
What is a Media Forward Deployed Engineer?
An engineer who embeds with a broadcaster, streaming service or media company and owns an AI system end to end inside their environment: scoping it, writing the production code, and keeping it running. The difference from a general forward deployed engineer is domain depth in video and audio, meaning codecs, timecode, streaming protocols, content security and the economics of processing content at archive scale.
How is a forward deployed engineer different from a consultant?
A consultant delivers a report or an architecture and leaves the building of it to you. A forward deployed engineer writes and ships the production code inside your environment, then transfers the patterns to your team. The deliverable is a running system, not a document.
Why do so many media AI projects fail?
MIT's research found that 95% of enterprise AI pilots delivered no measurable financial impact, and that the failures clustered on integration rather than on model quality. In media specifically there is a second common cause: unit economics. Costs that look negligible per asset become prohibitive across an archive of tens of thousands of hours, and that arithmetic is often done after the pilot rather than before it.
Can a generic AI engineer do this work?
Partly, and that is the trap. They can build a convincing prototype quickly. What tends to be missing is frame-accurate timecode handling, awareness that encrypted content cannot be decoded, the ability to produce output a broadcast pipeline can actually ingest such as SCTE-35 markers or TTML captions, and a realistic cost model at scale. Those gaps usually appear after the pilot has been approved.
Do I need to hire one full time?
Usually not at first. The role is most valuable at the point of highest uncertainty: scoping, choosing what to build, and getting the first system into production. Many media companies engage a specialist for that phase and then run the result with their own team, which is the intended outcome, since transferring the patterns is part of the job.
What should I ask a candidate?
The five questions above cover cost modelling, content security, frame accuracy, frame sampling strategy and failure behaviour on air. They are difficult to answer convincingly without having shipped a media AI system, and easy to answer for someone who has.
Where to find one
The tools linked throughout this article are free to try. They are not the point of it, but they are the evidence for it: every one of them started as a problem that came up in production and had to be solved properly rather than described in a slide. One of them is still in beta, for reasons the article has already explained.
If you are trying to get AI into a broadcast or streaming workflow and you need someone who already knows where it breaks, describe the role and we will scope it with you, including an honest day rate before anyone gets on a call.
Describe the role you need to fill
And if you read the nine points above and recognised your own job, the label is being attached to this work right now. When that happens, generalists arrive and claim it. The people who have actually been doing it should be findable under that name first: join the talent network.