IoT and BigData — The coming $2 Trillion Tsunami

Your car is no longer just a transportation device. It is an internet endpoint. As is your toaster, your microwave and your refrigator. And don’t even get me started on your television; it was already gathering and transmitting information before the term ‘IoT’ was coined. IoT, when combined with a streaming data processing pipeline, will change the way we look at everyday objects.

Where does BigData Fit in ?

There’s a perception out there that BigData is only for BIG companies, the Googles and Amazons of the world. Companies that need to store terabytes, petabytes and exabytes of data. Typical examples of BigData analysis include:

  1. Google, analyzing, in real time, everything that everyone is searching for across the world.
  2. Facebook, trying to recognize a face among a billion other faces.

Asking the Right Question — Where can I fit in IoT into my business process model?

Enter IoT, and a 2 Trillion Dollar Opportunity

Anything that MOVES is an IoT opportunity !

Vehicles can produce upwards of 560 GB data per vehicle, per day. The same can be said of almost any device with electronic components. What about things without electronic components?

Vehicle and Appliance Breakdowns

Say, for example, you are Toyota or Honda and are sell a few hundred million cars every year, worldwide. Now, say you want to get crash data from all the Toyotas that get into crashes, on a daily interval; i.e. Toyota would like immediate notification along with a ‘crash dump’ of all on-board data. This is a problem that BigData (combined with IoT) is already solving.

Google’s Pixel Buds, Douglas Adams’ BabelFish is a Reality!

While I was working on this article, Google announced the release of an IoT device that has AI built into it. It has a built-in natural language processor, which allows it to translate upto 40 languages on the fly! Think about that! It IS actually straight out of science fiction (all of us Hitchhiker’s Guide to the Galaxy fans know what I am talking about). Douglas Adams, with his wild imagination, had conjured up a small fish (appropriately named the BabelFish), which, when inserted into one’s ear (painfully), translated any alien language. If only Adams was alive today; Google’s PixelBuds would have brought tears of joy to his eyes.

Sound Waves Move As Well !

Of course, the spoken language is nothing but a vibration of air; in other words, movement of air molecules. And movement, any movement, is an IoT opportunity. Google has shown the world what can be done when you combine existing NLP technologies with IoT; all of a sudden, the world has shrunk. One can travel to foreign lands without fear of a ‘language barrier’.

Which BigData Engine is winning — Hadoop or Spark?

Large scale data processing engines come in two flavors — Hadoop and Spark. Hadoop is the industry standard. Created by Apache, it was the defacto implementation of the famous MapReduce search algorithm, developed at Google.

Who is leading the BigData (Platforms and Products) War?

Of course, it is hard to predict, however, these companies, in my opinion have distinguished themselves through innovative, development time saving product offerings.

  • Cloudera — Cloudera QuickStart VM and Cloudera’s Hadoop Cluster are both huge time savers for companies looking to get up and running quickly.
  • AWS / Azure — both have mature BigData ecosystems that leverage their own, individual strengths.

Who will win the IoT war?

There are more IoT startups than you can throw a brick at. Some have distinguished themselves, by endearing themselves to the developer community.

  • Notables — Autonomo (with a Solar Panel and Long Battery Life), Wio Link

Summary and Next Steps

Companies that think BigData isn’t for them, aren’t asking the right questions. There isn’t a single industry I can think of that will not be revolutionized by this coming Tsunami. As I was writing this post, Michael Dell announced a $1 Billion investment into IoT R and D. That’s quite a powerful endorsement from a tech leader!

  1. Follow Anuj on Twitter.
  2. Thoughts? Comments? BigData + IoT Success Stories?For a free 15 minute consultation, talk to me, I’m listening…..

Ask me about Cloud Security, Google Cloud, AWS, Cloud Infrastructure, Migrations, Docker and Containerization, Quantum Physics, Relativity Theory and Blockchain

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store