Racing Post
Real-time data for betting shops nationwide.
Multiple XML feeds and data sockets from different providers, aggregated, normalised, and pushed to touchscreens in betting shops across the UK. The kind of system where being a second late means showing the wrong odds.
What the system does
Racing Post had data coming from everywhere: XML feeds, data sockets, several providers, each with its own format, timing and quirks. All of it had to become one reliable stream feeding live displays in betting shops across the country.
I designed and built the data aggregation and distribution system. Ingestion from every source, normalisation into a common format, validation to catch bad data before it reaches a screen, and real-time push to the touchscreen displays. When a race is about to start and odds are moving, you can't afford lag. When a feed goes down, the system has to handle it gracefully without the punters in the shop noticing anything.
There are no maintenance windows when the shops are open all day, so it was built to run continuously.
Under the Hood
Ingestion takes XML feeds and socket connections from each provider and normalises them into one shape. Validation sits between that and the screens, so bad data from a provider never reaches a shop. Distribution is a real-time push to the touchscreens, and the design made adding a new source a small job when one came along.
Monitoring and alerting watched data quality all day. When a provider went down, the displays carried on with what they had rather than going blank, and the punters in the shop never knew.