Top 3 things we learned at the Innovate East data science hackathon.

The Problem

Did you know that 3.1 billion litres of water is leaked every single day in the UK? That’s the equivalent of 1,268 Olympic swimming pools of water wasted on a daily basis. Ouch! Naturally Ofwat has started holding water companies to account, giving them challenging targets to reduce mega litres lost through leaks. No business would ever want to lose their valuable product – and that goes for water companies too. So why was it happening, and what could we do about it?

The Hackathon

We set about exploring these issues in a leakage hackathon, held at Innovate East. Organised by some of the biggest names in the water industry (including Anglian Water, Essex & Suffolk Water, and Northumbrian Water), Innovate East was a fun three-day event in a field in Ipswich with the ethos of “working together to solve common issues.” Around 1,800 people attended the festival, which included pop-up food trucks, teepees, evening entertainment and even an appearance by Carol Vorderman. There was a buzz of excitement in the air as a diverse range of people came together to help solve this, and other issues – women, men, people from all around the world, IT experts, data scientists, mathematicians, statisticians and researchers from a range of companies. Each team was asked to focus on four challenge areas:

1. How we can identify leaks in their early stages using standard flow, pressure data, and advanced data from noise and transient loggers?

2. To what extent do network operations cause new leaks?

3. How can we use open data and data about customer behaviours to develop a better assumption model for unmeasured customers?

4. How can we separate customer water usage from leakage, looking at seasonal variations in demand and consumption?

The first day involved learning about the leakage process, data, measurements, and asking the experts any questions. Then participants were asked to choose their area of focus based on the four key questions, form teams and start sketching out ideas. Day two centered around creating a prototype and testing it out with SMEs using anonymised data (as well as the help of techies and facilitators). By day three the teams had finished their prototype and had started preparing their pitch in time for the presentations that afternoon. Each team had seven minutes to pitch their prototypes to a panel of judges including Martin Jackson from Northumbrian Water, Luke Stockdale from Essex & Suffolk Water, David Jacobs from Anglian Water, Matt Edwards from Anglian Water and Tom O’Farrell from Aiimi.

The Results

The judges were absolutely blown away by the standard of the solutions! The overall winner of the hackathon was team Aquaflyers for their DMA DNA solution, whose idea is being explored further by all three water companies - Anglian, Essex & Suffolk and Northumbrian. The challenge they solved was “How can we determine what is customer usage and what is leakage looking at seasonal variations in demand and consumption?” From this they came up with the concept of DMA DNA, that each District Metered Area (DMA) has a unique makeup based on the types of users within it (industrial, commercial, residential, etc) which forms its DNA. This knowledge would allow water companies to identify when usage is abnormal in a given day/time period. From there, they analysed five different data sets including: rainfall data, temperature data, soil moisture, domestic consumption data and non-household logged data. They plotted these on a graph and produced a Power BI interactive prototype of each DMA’s DNA.

In John Barwood’s words, who did the data analysis and presentation for Aquaflyers:

“It was a very good team to be a part of, people naturally moved into roles which suited their skills and everyone tried to contribute positively towards the challenge. The diversity in backgrounds of the team definitely helped to bring a wide variety of skills and ideas into the mix.”

Another team, Leaky Blinders, looked at whether network operations did indeed cause new leaks – and the answer was yes, especially routine jobs such as fire hydrant repairs. They displayed their findings in two visualisations, one in a Power BI and the other with Kibana (using elastic search) that enabled users to see a near real-time spatial distribution of network jobs. This meant they could identify hot spots on the network which could trigger an early investigation of leaks. It would also allow analysts and managers to drill down into the details of jobs – so they could see where they were performed, what job types they were, if there had been any previous leaks in the area, etc.

In data scientist Sneha Eashwer Singhraj’s words from Team Leaky Blinders:

“The thing that really made Innovate East special for us was the collaboration within our diverse team. We had a huge amount of subject matter experts from data scientists to a leakage expert to a lead data architect. Bringing these diverse brains together and collaborating on the same goal meant we could pull together a really useful visualisation quickly, which had the perfect mesh of data and subject-matter expertise.”

Top Three Things We Learned

• Diversify your teams - The more diverse your teams are, the greater variety of ideas you’ll have and the better your overall product. The teams that succeeded most at Innovate East had a range of women, men, people from around the world and experts from different backgrounds as well as seniority who all solved problems in varying and innovative ways which fostered creative thinking.
• Be realistic – It's great to have big ideas but be realistic about what you can achieve in the timeframe you’re given. You could have the coolest app with the most beautiful interface, but if it’s not going to help solve a leakage problem it won’t win anything.
• Have fun! - This is your chance to shine so enjoy yourselves, meet new people and get involved in making something amazing.

We will definitely be using these tips for our upcoming hackathons at our new headquarters in Milton Keynes in 2020! At Aiimi we have plans to hire 100 data experts over the next three years, and turn our new office into a state-of-the-art Hackathon Zone and Data Hub. We look forward to working together on many more innovative ideas.

Thank you so much to #InnovateEast for hosting our hackathon – we had a blast!