A Big Data Use Case: Building a Big Data Repository

This is a great use case that describes the benefits of building a Big Data Repository and then applying the right set of predictive analytics and filtering models against 80 years of historical data.

The Los Angeles Police Department Predicts and Fights Crime with Big Data
When the movie Minority Report was released in 2002, the predicting and consequently preventing of crime was still science fiction. It was far away and criminals-to-be had not to fear about predictive analytics being used by the police. Fast-forward to 2014 and things have changed. Although we are still very far away from the Minority Report scenario, predicting were crime is likely to occur is currently being used by the Los Angeles and Santa Cruz Police Departments. And with success; as there have been a 33% reduction in burglaries, 21% reduction in violent crimes and 12% reduction in property crime in the areas where predictive software is being used.

It all started with predicting earthquakes. The LAPD used a mathematical model that was used to predict aftershocks during earthquakes and started feeding it with crime data. Although it is still extremely difficult to predict earthquakes, predicting aftershocks is relatively a lot easier. Whenever an earthquake has happened, there is a high probability that aftershocks appear close by in space and time. This mathematical model, developed by Assistant Professor George Moher, is capable of defining patterns that can be used to predict new aftershocks.

As it appears, crime data shows similar patterns and therefore they were capable of feeding the model with 13 million crimes of over the past 80 years. This treasure trove of data helps them to understand the nature of crime. It appeared that whenever a crime had appeared somewhere, more crime is likely to occur and the patterns of those criminal activities are similar to aftershock patterns. When they plugged the old crime data into the equation, it generated predictions that matched what had happened in the past.

The LAPD already used the vast data set to showcase which areas in Los Angeles are hotspots of crime. This information however is only about where crime had happened. With Big Data they are now trying to predict crime. The LAPD started with a pilot project in applying the mathematical model of Moher to predict the areas where crime is likely to occur. Together with among others the University of California and the company PredPol (with the mission to place officers at the right time and location to give them the best chance of preventing crime) they managed to improve the software and the algorithm. Nowadays they identify crime hotspots where crime is likely to happen on a given day and it is used by Police Officers in their daily job. Getting the police officers to start trusting and using the software, however, was not an easy task however.

At first they were reluctant to use the software as described in this interesting BBC documentary. Police offers are assigned to a 500 square feet geographical area, where the algorithm predicts crime is likely to occur in a 12-hour shift. During their shift, the police officers are instructed to patrol these areas as often as they can and to look for criminal activities or evidence that criminal activities are about to occur. The Proof of Concept was monitored in real-time at the crime centre in downtown LA. At the start of the proof, the officers were not happy about being told by an algorithm where to patrol, but when they saw the evidence that crime did indeed became less they were soon convinced. Today, the model is continuously being updated with new crime data as crime occurs, to make the predictions even more accurate.

But Big Data analytics is not only used to fight violence or theft but also to combat for example insurance fraud. The Durham police for example used analytics to find a large part of dubious claims in a large pile of insurance data. By combining data and analyzing it they managed to stop a cash for crash scam that involved an organized crime group faking accidents and claiming several times for that accident.

The crime data does not only help to prevent crime from happening, but it also helps to understand the bigger picture about why crime occurs. For example as Former policeman Shaun Hipgrave, now a security consultant at IBM, explains in a post at BBC, When you use Big Data you can see the relationships between one family and another troubled family and you see the absences from school. This more holistic view is very efficient in understanding and fighting the source of criminal activities. As a result, the PredPol software is currently being trialed in over 150 cities across America and it is likely to be rolled out in more cities.