LinkedIn trains its AI job search 8x faster
An 8B relevance oracle and a 1.7B engagement teacher train a 0.6B ranker that now serves natural-language job search for US users.
In short
LinkedIn says it cut the training time for its AI job search ranker by roughly 8x by having two differently specialized teacher models jointly train one compact 0.6B student.
At a glance
- Student ranker has 0.6B parameters; teachers are an 8B relevance oracle and a 1.7B engagement model.
- Training is about 8x faster: local teacher replicas give 3x, multi-node training up to 3.5x more.
- FSDP2 adds roughly 20 percent, multi-node H200 clusters up to 30 percent; LiGer doubles batch size.
- NDCG@10 rose 24.48 percent, from 0.7583 to 0.9432, reported as an offline measurement.
- Ranking throughput went from about 290 to over 2,000 items per second per GPU; live for US users.
LinkedIn has described the training setup behind its natural-language job search, and the headline number is a roughly 8x reduction in training time. The trick is splitting the teaching job in two: one large model judges how well a posting fits the query, a smaller one carries behavioral signals, and both hand their knowledge to a compact 0.6B student. That student is the model ranking results for US members today.
Why two teachers instead of one
Relevance and engagement are different questions, and a single teacher tends to blur them. LinkedIn assigns an 8 billion-parameter relevance oracle to the first and a 1.7 billion-parameter engagement teacher to the second. The 0.6 billion-parameter student learns from both, which keeps serving cost low without asking one small model to rediscover either signal on its own.
Distillation runs in two modes. Online, the teachers score alongside training, which keeps iteration quick. Once their judgments settle, cached teacher outputs drive an offline pass, cutting the overhead of keeping large teachers served. The serving layer is built on SGLang, an open-source LLM serving engine.
Where the 8x comes from
No single change carries the number. Running teacher replicas locally accounts for a 3x acceleration, and spreading training across nodes adds up to 3.5x on top. FSDP2 contributes about 20 percent, and multi-node H200 clusters up to another 30 percent. LiGer works on the memory side, allowing batches twice as large.
One documented dead end is worth noting: FP8 mixed precision produced no benefit for models below 8B parameters. That is a useful negative result for teams assuming lower precision always pays.
What changed for search results
On quality, LinkedIn reports NDCG@10 climbing 24.48 percent, from 0.7583 to 0.9432. The metric scores how well the top ten hits are ordered, with 1.0 as a perfect ranking. Separately, structured pruning and context compression lifted serving throughput from roughly 290 to more than 2,000 items per second per GPU.
What the write-up does not say
The account gives no explicit baseline for the 8x figure, so it is unclear whether the comparison is against a single-teacher pipeline or an earlier stack. The NDCG numbers are presented as offline evaluation, with no live A/B results attached. There are no cost-savings figures either. Every number here comes from LinkedIn by way of InfoQ; we have no independent verification of the measurements.
FAQ
What is multi-teacher distillation?
Several larger models pass their judgments to one small model, which learns to reproduce them. At LinkedIn one teacher covers relevance and a second covers engagement signals.
How large is the model serving LinkedIn job search?
The production ranker has 0.6 billion parameters. Its teachers are an 8B relevance oracle and a 1.7B engagement model, which do not run at serving time.
How much better are the search results?
LinkedIn reports NDCG@10 rising 24.48 percent, from 0.7583 to 0.9432. That figure is described as an offline measurement rather than a live A/B test result.
