News and publishing

Recommendation systems for news and publishing

In news, every item is cold, most items are worthless within a day, and the editor has to be able to outrank the model. What that means for the recommender, and how NeuronSearchLab handles it.

What is a recommendation system for news?

A recommendation system for news ranks articles, videos, newsletters and podcasts for each reader across a publisher's surfaces: the homepage, the article page's 'read next', section fronts, app feeds and email digests. It differs from other recommenders in that almost every item is new, item value decays within hours, and editorial judgement about what matters has standing that the model must not silently override.

Cold start is not an edge case in news, it is the steady state. A recommender that needs a hundred interactions before it can place an item is useless on a story published twenty minutes ago, which is exactly when placing it is worth the most.

Publishers also carry obligations other industries do not: source and viewpoint diversity, transparency about why content was shown, and a duty not to build a feed that narrows a reader's world. These are increasingly regulatory as well as editorial, which makes explainability part of the specification rather than a nice-to-have.

Where NeuronSearchLab fits for a publisher

NeuronSearchLab is an API-first recommendation platform for building, evaluating and operating personalised recommendations. For a publisher, the part that matters most is usually the control and explanation layer rather than the ranking itself.

Editorial rules - pin this story to the top of the homepage for three hours, cap coverage of one topic at two slots, never place this article next to that one, guarantee the investigation a position for the day it lands - apply at request time, scoped to a named surface, with no retrain and no deploy, and previewable before they go live.

Every result decomposes into the interests, interactions, content similarities and rules that produced it, so 'why did this reader see this story' has an answer you can give to an editor, to a reader, or to a regulator. Diversity, novelty and catalogue coverage are reported alongside relevance and are available as release guardrails, so a model cannot be promoted for raising clicks while collapsing the reader's world onto one topic.

A single recommendation model gives you one fixed way of ranking. NeuronSearchLab trains and compares several architectures on your own data, keeps the winner behind one unchanging API, and gives you evaluation, experimentation, explainability and editorial control over whatever is serving. The general case is set out on recommendation engines.

What makes this hard

The constraints that separate this industry from every other one using the same underlying models.

Permanent cold start

Every article is new. The items with the most commercial value - the story that broke an hour ago - have the least behavioural signal, and by the time they have enough, the value has gone.

Items decay in hours

A recommender with a nightly retrain is recommending yesterday's news. Freshness has to be a first-class ranking term and the pipeline has to react to publication within minutes.

Editorial authority is real

There are stories that must be seen whether or not a reader is predicted to click them. A system that cannot express that, reversibly and with an audit trail, will not be allowed near the homepage.

Clicks are a treacherous objective

Optimising click-through on a news homepage produces a predictable outcome, and every newsroom can name it. Time spent reading, subscription conversion and return visits are the objectives that survive contact with the newsroom.

Diversity is an obligation, not a metric

Source diversity, viewpoint diversity and topic breadth are increasingly things a publisher must be able to demonstrate, not just measure. That means reporting exposure across sections, sources and desks, not only engagement.

The subscription funnel changes the target

An anonymous reader, a registered reader and a subscriber are three different ranking problems with three different objectives. One model with one objective serves at most one of them well.

Where recommendations appear

The placements teams in this industry actually build, and what each one is for.

Personalised homepage

The hardest surface: editorially curated slots and personalised slots on the same page, with a clear rule about which wins where.

Section and topic fronts

Personalisation constrained to a section, where the editorial hierarchy is stronger and the model's job is ordering within it.

App feed

Sequential ranking over a fast-moving catalogue, with freshness, breaking-news overrides and read-state handling.

Newsletters and digests

Batch personalisation with a long horizon and an unsubscribe as the cost of getting it wrong.

Paywall and offer placement

Which article to meter, and which to show a subscription prompt against, ranked with the same signals as the content itself.

Signals worth modelling

What to send, and why. Most disappointing recommenders in this industry are disappointing because of what never reached them.

Reading depth and dwell

Scroll depth and time on page separate a headline that worked from an article that was read. In news the gap between those two is unusually wide.

Recency and publication time

Not a tiebreak. For a large part of a news catalogue, recency is the dominant term, and the decay curve differs by section.

Content embeddings

Text embeddings over the article itself are what make a story rankable at minute zero, and what let a topic cap be enforced on meaning rather than on tags.

Return visits

The habit metric. A reader who comes back tomorrow is worth more than one who read four more articles today, and only one of those is easy to measure.

Subscription state

Anonymous, registered and subscribed readers want different things and should be ranked with different objectives, expressed as segments rather than as one averaged model.

How to know it is working

Offline ranking metrics narrow the field; an online test against these decides. The benchmark methodology sets out how the offline side is measured.

Time spent reading

The engagement metric that does not reward headline inflation, measured per variant online.

Return visit rate

The habit metric, and the closest proxy for the thing a subscription business actually sells.

Subscription conversion

Slow, sparse and the one that pays. Needs a longer experiment window than the newsroom will initially want to give it.

Topic and source diversity

Exposure distribution across desks, sections and sources - reported as a first-class output, and usable as a release guardrail.

Time to first impression for new articles

How long a published story waits before it reaches anyone. The direct measure of whether cold start is solved.

Frequently asked questions

What is the best recommendation system for a news website?

One where content-based signals carry the cold-start load and editorial rules can override the model at request time. Pure collaborative filtering performs badly on news because the items with the most value have the least interaction history. In practice publishers run a hybrid: content embeddings and recency for placement, behavioural signals for personalisation, and hard editorial controls on top.

How do news recommenders handle articles published minutes ago?

By ranking on the text of the article itself. A content embedding places a new story next to similar existing stories immediately, so it can be shown to the readers most likely to read it before it has any interaction history. Recency is then a ranking term in its own right, with a decay curve that differs by section.

Can editors override a recommendation system?

In NeuronSearchLab, yes: pin, boost, bury, cap, exclude or guarantee an article on a named surface, applied at request time, with no retrain or deploy, previewable before publishing and reversible afterwards. You also set how hard an editorial decision pushes against personalisation rather than choosing between a fully automated feed and a fully manual one.

How do you avoid creating a filter bubble?

Measure exposure, not just engagement. Report the distribution of impressions across topics, sections and sources per reader and in aggregate, set diversity and coverage as release guardrails so a model cannot be promoted for narrowing the feed, and use topic caps to bound how much of one subject a single slate can carry.

Does personalisation hurt editorial authority?

Only if the system is built so the two compete. The workable arrangement is editorially fixed slots plus personalised slots on the same surface, with the model ordering within editorial constraints rather than against them - and with every result explainable, so a disagreement can be settled by looking at what actually happened.

See it on your own catalogue

Send a catalogue and an event stream and get ranked results back the same day. The free tier includes 1,000 recommendation requests a month and needs no card.

Start free Talk to us