Posted in

Building The Big Data Platform for Your Smart City

As governments worldwide aim to become smarter like Dubai, Barcelona or Singapore, their ultimate goal is to become percipient, astute, shrewd, and quick. That means that smart city governments need to use emerging technologies like Big Data and Internet of Things to be able to understand their citizens affinities in real-time and to use this ability to assess, make decisions, and react either in milliseconds, seconds, minutes, or days and months.

The latency of decisions usually depends on the scale of the problem such as an anomaly, hazardous event, or a city wide disaster. To illustrate, if there is a pipeline leakage, modern smart city platforms can detect the threat and automatically shut down the gas supply in this area. These types of analytics applications are latency-sensitive and need to be taken in milliseconds.  Also, the ability to take fast decisions across local cities problems is now possible through emerging technologies like fog computing.

With all the advanced technologies that can help build Big Data and Internet of Things platforms for a smart city, we still face challenges to make these platforms realize their full potential. From my experience with clients working on smart city plans, privacy and security are not the only government concerns, interoperability is a main issue. Thats why our ability to unlock data in a way that governmental agencies are able to move it from one system to another in a safe, secure, and confidential manner is key.

When we discuss data in a smart city, we are talking about semi-structured data from sensors, unstructured data from social media or images/videos collected from video surveillance, or other shared open data like bus time schedules or city maps. In order to reach full interoperability, we need a joint strategy for both, technical and organizational interoperability to be connected and collaborative respectively. Organizational interoperability contains aspects like organizational strategies and policies, laws, business processes, cost, and collaborative work while technical interoperability deals with data lineage, semantics, and infrastructure.

Thats why we are witnessing governments appointing chief data scientists like DJ Patil in the White House. National chief data scientists can act as instigators to push this full interoperability across the government and work on both an inward and outward collaboration across governmental agencies and external data partners and citizens.

Another interesting opportunity lies in adopting technologies like data virtualization which can help integrate data without moving it from its physical storage. Data virtualization is suitable in such public sector use cases since it tackles regulatory constraints, departmental politics and traditional data governance methodologies that might limit the ability to access data within the source databases. Other than setting the right joint strategy, making cities connected for collecting data is just the rst step towards smart cities.

The essential step is being able to build up a scalable real-time big data analytics platform for all the layers between the data sources and applications. Most of the solutions in the market focus on  functional requirements and architecture designs at the abstract level.  Such platforms will eventually help data scientists have a unique source to access, visualize and tell stories based on the data flowing within the big data platform of the smart city.

Data is a tool like any tool that acts as a means to an end. Successful companies and governments in the 21st century are using such tools to build data-driven products. Thats why we need to see how our own culture could tailor this tool to be used collaboratively in building a smarter governance ecosystem.

What do you think about other challenges facing governments planning their smart city initiatives? Feel free to share your thoughts and comments below.

Photo by Ali Rebaie: Top of Burj Khalifa, Dubai.

Ali Rebaie is a principal analyst at Rebaie Analytics Group. He is also a prominent blogger and keynote speaker. A data science anthropologist, and phenomenologist, Rebaie has been studying the impact of data patterns that govern changes in business, human affairs, and culture. His research, keynotes, and consulting help data natives and business executives to draw power from these universal patterns to better understand its impact on people and the role each one of us play in the data era.

As a part of his work at Rebaie Analytics Group, Rebaie has led and developed several data-driven strategies for different industries such as media, marketing, retail, oil & gas, public sector, and transportation to help them become competitive in the new economy. Ali has contributed and quoted in leading technology and media outlets including WIRED, CIO, Yahoo, Computer World etc...

Ali also appeared in several lists of "Who's Who in Big Data" and as one of the top 100 big data influencers worldwide. Ali is a frequent speaker at international conferences and regularly trains on data science and visualization. As a School of Data fellow, he aims at growing a community of data enthusiasts, spreading data skills and educating on the use of big data for public good.

As a part of his work with big data and analytics, Rebaie has led and developed several technology projects across business intelligence for different industries such as media, marketing, oil & gas and transportation. Ali is a member of the internationally renowned Boulder BI Brain Trust (BBBT). If you would like to read my upcoming posts about "Big Data", you can connect with himvia Twitter, like his page on Facebook, or sign-up to my his blog.

Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.