Building IoT at scale – lessons from the past 3 years

Part one – takeaways:

1. Real IoT at scale is hard to do, which is one reason why there are so few examples today
2. The chasm between running a pilot and moving to production at scale is wide and deep
3. But some people will win very big from IoT in the coming years
4. Poor air quality kills more than 50,000 people a year in the UK
5. Our project is designed to do something to help directly address the issue
6. Air quality measurement is also really hard to do properly
7. But many people seem to think they can do it, with no specialist knowledge
8. After three years, we are finally rolling out the project we thought would take 16 months
9. We’ve learned a lot in the process, and I’m going to cover much of it in the coming weeks – see at the end of the post for some of the subjects I’ll be covering

Real-world IoT – more a very long team-based assault course than a sprint

This is a blog series about making the Internet of Things a reality.

Unusually, it won’t just be a series of thought-pieces from a consultant or futurist (both of which I am as it happens), but a series of posts about the development and roll-out of complex sensor technologies and a big data platform – a large and live end-to-end IoT project called AirSensa.  Not a pilot, or a seed-funded experiment, or a guy in a shed with a Raspberry Pi, but a large network of complex sensors delivering environmental data from hundreds (thousands eventually) of locations.

This is IoT in practice, and in retrospect – all the technology is done, all the hurdles cleared, and we’re now growing the network at a healthy clip.

We set out our goals for the project in 2012 – improved public awareness, schools engagement, free data for policy-makers and planners, and to be a platform for innovation.  But most of all, to generate lots of decent data from lowish-cost sensors that wouldn’t, in the end, be rubbish.  How hard could it be?  Well… very, as it turns out.

To borrow an analogy from our CTO, doing real IoT is not like climbing a mountain; it’s like climbing an entire mountain range.  There isn’t a single problem to solve to achieve IoT in the real world, there are dozens of them, and only some are technical.  This blog is for people who want to take IoT seriously – whether you’re a big business thinking about how the IoT can improve processes, reduce your cost base, or increase revenues, or in central or local government looking at how to provide cheaper, more effective, services.  We’ve made, and learned from, most of the mistakes.  Hopefully I can help you to avoid as many of them as possible.

Winning, and winning big, but not yet

Where to start?  Well, first let’s be clear about what IoT is and isn’t, because there’s a lot of hot air around at the moment.  Wikipedia defines IoT as:

“…the network of physical objects or “things” embedded with electronics, software, sensors, and network connectivity, which enables these objects to collect and exchange data. The Internet of Things allows objects to be sensed and controlled remotely across existing network infrastructure, creating opportunities for more direct integration between the physical world and computer-based systems, and resulting in improved efficiency, accuracy and economic benefit.”

That’s not a bad definition (actually it goes on a bit, but that’s enough for now).  Three key points from this definition:

• Things can exchange data      i.e. it’s two- or multi-way communication – one of the keys of IoT is the ability to accommodate autonomous operation, and for processing and data storage (for example) to be executed where and when it makes sense – some call it ‘fog computing’ – this is not all about a big central control system and billions of dumb things.  The IoT will have intelligence where it is needed to execute most efficiently.
• Which allows integration between the physical and digital world     i.e. it includes interactions with/between processes and people as well as things.  People will be both source and user of data; business and technical processes will be too, interacting with things to achieve their goals.
• To create improved efficiencies, and economic benefit     i.e. real and measurable outcomes.  The point of all technology (though this is often overlooked) is to help create real and useful outcomes.  Without such outcomes, there is no economic sustainability.  Many of the grant-funded IoT projects to date don’t seem to have taken that into account.

There are a number of quite different things going on right now in IoT that are being conflated – power station control systems, industrial automation, smart cities, and wifi-enabled kettles are all labelled ‘IoT’ in some way.  Of course there are some common elements, and a later post in this series will lay out a model for the world of IoT, but for now, I’ll just say Smart Home does not equal Smart City does not equal Industrial IoT.  Different aspects of connected things have very different technical, procedural and security requirements, and very different desired outcomes.  It’s like putting a Caterpillar earth-mover and a Renault Twingo in the same bracket, because they’re both motorised vehicles.

But those caveats aside, there are big wins available for those that work out how to make the Internet of Things really work (whatever that means for them – new products, better customer engagement, lower costs, smarter cities, international competitiveness, etc.).  Because although much of what is called IoT has been around for ages – a repackaging of a basket of existing technologies and ideas – we are at a point at which enough ingredients exist, and there’s enough need, to really make it work.

But as I said up front, real IoT at scale is hard to do, and financially sustainable models are in their infancy.  Huge claims are being made – estimates about the number of connected things by 2020, or the ‘tsunami’ of data that will be available to everyone to do amazing things with big data, analytics, etc.  Well, not so fast.  I think it will take 5 years before the hockey stick really kicks in.

In the longer term though it can also be argued that much of our thinking today is underplaying the long-term opportunities.

Overestimating the short-term and under-estimating the long-term

It seems to me that the Internet of Things is describing a similar path to the many truly disruptive technologies that have preceded it (television, mobile telephony, the Internet itself, etc.) – we overestimate the short-term impact and under-estimate the long-term, fundamental changes that they will bring.  We don’t really know what opportunities will emerge, because those changes are generally unforeseeable and unknowable (sorry to all the futurists and consultants out there).

Why do we get the whole short-term / long-term thing wrong?  There are many reasons, including:

• Technology people think that everyone gets as excited about stuff as they do (they don’t)
• We misjudge the speed at which cultural and lifestyle changes happen (very slow until (and IF) a tipping point is reached)
• Most of us even fail to spot ideas that actually will indisputably make life easier – like Uber – which then spread like wildfire (otherwise VCs would be successful all the time)
• There’s little money to be made in saying ‘don’t worry, it won’t happen for ages’
• We simply aren’t very good at contemplating anything outside our cultural frame of reference – true visionaries are few and far between.  There’s a reason that futurists in the 1950s saw a millennial future of flying cars with really BIG fins at the back.

In all probability, for the next five years most of the value to be generated around IoT will accrue to infrastructure suppliers and systems integrators.  But at some point, winners will start to emerge – both in the technology industry and across many business sectors.

I’ll have a go at where those winners might emerge as this series develops, but first, in the next post, I’ll cover why we started this project and why we set our project objectives the way we did.  Stay tuned.

Upcoming posts

In future posts, I’m going to look at many of the lessons we’ve learned about IoT, including (but not limited to, and not necessarily in this order):

1. How to set project objectives: what question are you trying to answer?
2. A model for the IoT implementation stack, which will help to show (among other things) why it’s so hard to get a solution implemented at scale
3. Why smart cities are such a challenge
4. Technology design ethos – open vs closed
5. Sensor selection / sensitivity
6. Industrial design
7. Electronics design
8. Sensor estate management
9. Automated service and preventative maintenance
10. Data and platform algorithms
11. Problems with comms and the data networks
12. Firmware and the difficulty of recognising all boundary conditions
13. Testing / calibration processes and challenges
14. Data reliability & provenance
15. Hardware problems – RF/comms & noise
16. Security (or why some people have too much time on their hands)
17. Data visualisation
18. Site selection, challenges
19. Political considerations
20. Does citizen science work?

In the meantime, if you’d like to get in touch:

Email: js@jonathansteel.com

Twitter: @jonathansteel