Airbnb cuts recommendation staleness from two days to seconds

A guest searches for a stay, taps a listing, searches again, and the recommender does not notice until tomorrow. InfoQ reported on 28 October that Airbnb has closed that gap. The company extended its open-source Chronon feature platform with Push Mode and a Near Real Time Model Transform. Embedding staleness went from nearly two days to under a minute. Typical updates now arrive in ten to thirty seconds. Airbnb says it saw a 1.67 percent improvement in offline NDCG. In online A/B testing, uncancelled bookings rose by about 0.33 percent.

The system behind these numbers is called JourneyFormer. It is a Transformer encoder that reads a guest's booking, review and browsing history. The embedding it produces goes into a low-latency store. The ranking system reads the latest embedding at search time and combines it with the live query. The Transformer never sits in the search path.

The staleness problem, in Airbnb's words

Before the change, JourneyFormer ran as a daily batch. The embedding store held values from last night's run. Anything a guest did after the batch was invisible to the next ranking until tomorrow's run. Processing delays pushed effective staleness close to two days. A guest who clicked twenty listings that afternoon still ranked as the guest from yesterday morning.

Push Mode and the NRT Model Transform

Airbnb added two capabilities to Chronon. Push Mode has the streaming job publish a notification when a feature value is committed. Downstream work reacts to that notification. Nothing polls. Nothing waits for a scheduled job.

The NRT Model Transform runs an already deployed model inside the streaming pipeline. Airbnb uses the same JourneyFormer encoder for the near real-time path. One inference implementation covers both the batch job and the stream. The pipeline captures events like searches and listing views. It joins them with the guest's longer-term sequence data. Inference then runs when the feature state is ready. The new embedding then goes into the same low-latency store the batch system already wrote to.

The online ranking path does not change. The embedding it reads is just more current.

Pengyu Hou, senior software engineer in infrastructure at Airbnb, is quoted in the InfoQ post. He describes the change as closing the gap between event-driven data and model inference. He says the NRT Model Transform lets Chronon call an already deployed model directly from the streaming flow.

What Airbnb shipped back

Push Mode and the NRT Model Transform are in the public Chronon repository. A team already running Chronon can read the design and decide where it fits. Feast, another feature platform InfoQ names, has cached prediction architectures where feature updates trigger inference. Chronon's design pulls streaming and model transformation into the feature platform itself. The same code runs on the offline and online sides.

A team that runs its best model only overnight is paying for staleness in every ranking. Airbnb's result says that cost is 1.67 points of NDCG, plus 0.33 points on a booking rate. Both figures are small per request. Across millions of searches, Airbnb has shown they add up to be worth the stream.

Source

• Airbnb Cuts Recommendation Staleness from 2 Days to Seconds with Chronon, by Leela Kumili, InfoQ
• The guest journey, updated in real time: Extending Airbnb's sequence recommender with Chronon, Airbnb Engineering
• github.com/airbnb/chronon