Visual AI: The Shiny Technological Object That Glitters Like Gold

The 1999 cinematically genius, pop science fiction film “The Matrix” gave license to some compelling computer-simulated reality theories. Fact following fiction, the classic sci-fi film, and others before and after it have profoundly affected critical thinking and the future orientation of science and technology, especially artificial intelligence (AI).

Computational power has given rise and substance to AI, including machine learning (ML), natural language processing (NLP), and optical character recognition (OCR). Over the past few decades, computer vision (visual AI) has been illuminated on the technological scene driven by the mind-boggling amounts of information that flow through the physical world and the global economy. By the end of 2023, the world will have 103 zettabytes (i.e., 103T GB) of data according to IDC.

Visual AI, or Computer Vision, is a technological innovator shaping business environments across nearly every industry, including energy, manufacturing, retail, construction, and mining. It can “observe and comprehend” input in real-time and allows computers and systems to capture and interpret meaningful information from image and video data, much like humans. Because computers can simultaneously interpret several mediums of perception, they can go beyond human visual power and recognize and identify patterns, images, movements, etc., that humans cannot. Using real-world visual data, machines can be taught to accurately identify and classify objects, interpret things that play out over multiple scenes, make decisions, and act based on insights.

Modern computer vision approaches utilize AI and deep learning models that automatically learn features from which advanced hierarchical tasks are composed-like detecting and classifying objects or interpreting situations-attaining performance levels and accuracy that exceed human abilities. Deep learning is a form of machine learning particularly well-suited for inherently hierarchical data sources, like video.

Visual AI delivers real, actionable insights by analyzing new and diverse data sources using existing cameras, CCTV feeds, drone footage, handheld video devices, and more. Visual AI technology can be deployed in hours or days to address critical problems, including:

• Health and safety (HSE) initiatives
• Security
• Visual inspection
• Productivity
• Situational awareness (e.g. facial identification)

Case / Capability Examples

Energy: Asset Tracking and Defect Detection

Cameras located on drill sites record 24/7, allowing the operator to go back and review the footage when something happens. In contrast, the visual AI platform analyzes the information it receives in real time and can do everything from identifying the location of assets to recognizing potentially problematic or dangerous situations. When identified, it automatically raises alarms to prevent injuries and determine whether safety procedures are being followed.

Adding cameras to drones allows visual AI to monitor pipeline leaks (oil, methane, etc.) and other potential security issues. Under human supervision, these drones can fly autonomously, providing extended and continuous vigilance to any developing situation. In fact, they can be scheduled to launch automatically, fly prescribed routs, and then return to their charging base.

Deep learning models can be trained on historical data to predict high risk situations by interpreting activities leading to the events, analyzing the issues, and initiate measures to prevent repeat incidents. In leveraging IIoT and AI, massive amounts of data can be quickly sorted and evaluated for quick, appropriate decision-making regarding safety and efficiency.

Facility Protection: Industrial and Manufacturing Fires

Fires in industrial and manufacturing facilities cause an estimated $1.2 billion in damages, 279 injuries, and 18 civilian deaths every year. The National Fire Protection Association (NFPA) reported in 2018 that more than 37,000 industrial and manufacturing fires occur yearly. The NFPA attributes those fires to five leading causes:

• Electrical hazards
• Machinery and equipment
• Hot Work
• Combustible dust
• Flammable liquids and gases

Smoke and flame detectors are legacy technologies with limitations that have always been unacceptable, particularly during a fire’s ignition and growth phases. They cannot detect an exact position of a fire, how the fire is spreading, or its size – all of which are necessary data points for emergency responders.

In one instance, an oleochemical company lost roughly $1M in critical assets in a massive fire that spread out of control despite traditional heat and smoke detection methods that were in place. Visual AI can detect and pinpoint the location of fires as soon as they start. Coupled with machine learning, visual AI can recognize fire signatures and immediately raise real-time alarms.

After the fire, the oleochemical company facility was equipped with over 200 CCTV cameras with visual AI technology, capable of detecting fire signatures in seconds and immediately deploy real-time alarm and safety systems.

Construction: Death on the Job

An AFL-CIO report released in April 2022 titled “Death on the Job (The Toll of Neglect, 2022)” outlined some startling worker safety statistics. A few of them include the following:

• Every day, 340 workers die from occupational injuries from hazardous workplaces.
• More than 4,700 workers were killed on location while on the job.
• The fatality rate in the workplace was 3.4 per 100,000 workers.

The cost of job injuries and illnesses is enormous-estimated at $176 billion to $352 billion annually – with widespread underreporting of incidences. The average accident cost an employer over $120,000, with two thirds coming in the form of indirect costs (e.g. downtime, investigation, liability, etc.). With only 1,719 OSHA inspectors (755 federal and 964 state) available to assess 10.4 million workplaces under its jurisdiction, companies are turning to technology to help prevent workplace accidents.

With construction being so high on the most dangerous jobs list, visual AI has become the de facto safety technology for the construction industry. Visual AI uses an organization’s existing camera infrastructure, whether fixed, drone-mounted, or mobile phones, to contextualize what it “sees.” It can identify potentially unsafe practices and conditions before they become accidents – such as a missing piece of PPE, a dangerous path in which someone is walking, or an action a worker is taking or about to take.

It provides a wide range of capabilities without needing on-staff data science expertise or resource-intensive manual analysis of video feeds. Users gain the described benefits through simple drag-and-drop interfaces that allow flexible operations while ensuring continuous awareness of safety events or conditions that might lead to problems later.

Warehouse Operations: A Macro View

A distribution hub for a major American food and beverage company wanted to improve productivity within their warehouse operations and monitor essential key performance indicators associated with volume utilization and cost reductions. Before initiating a visual AI platform, their process of monitoring key performance indicators like CBM, turnaround time (TAT), asset utilization, person utilization, and safety compliance was largely manual, time-consuming, and inaccurate.

A visual AI platform was plugged into their existing CCTV cameras to detect and monitor specific scenarios tailored to the customer’s requirements and provide real-time analytics, including dock utilization, truck TAT, forklift utilization, safety compliance, near-misses, and more.

The customer today has a holistic picture of their warehouse, empowered to analyze operational efficiency accurately in real time. Their visual AI dashboard provides a macro view, which can be drilled down to individual asset, truck, and dock utilization metrics, resulting in substantial improvements in operational efficiency, a comprehensive view of their workforce utilization, and the ability to make well-informed business decisions in real time.

Snack Food Production: The Chips May Fall

Creating the perfect potato chip is far more than dunking a spud in a vat of oil. Quality control is essential. Potatoes used for chip production can have multiple types of defects, including undesirable color (greening or browning), external discoloration, or internal discoloration. A batch of potatoes inflicted with any of these defects that goes through the entire chip production process can mean the difference between selling thousands of potato chips at a profit or discarding the entire run at a considerable loss.

A leading potato chip producer used manual inspections, which resulted in the company discarding up to 7% of its finished product because of inferior quality end products due to defective raw materials. The flawed chips were rejected at the packaging step-after the entire lot had gone through all the time and resource-intensive slicing, segregating, blanching, and frying processes.

The company’s manual inspection process was slow, required additional labor, and increased the chances of error. The manual process was subjective, and fluctuated based on the observation skills of human scanning teams working in multiple shifts across multiple sites. Subtle discoloration cues were perceived differently from one another. Regardless of the company’s manual process, it was unreliable and costly.

The chip company implemented an advanced computer vision-enabled scanning solution to streamline its visual scanning process and reduce potato chip wastage. Sample chips were segregated before accepting a delivery lot, with roughly 300 chip samples reserved for quality control inspection. After the representative samples were washed, sliced, fried, and dried, the sample chips were scanned through a conveyor machine integrated with CCTV cameras running a visual AI platform.

As the potato chips passed through the camera scanner, the visual AI technology accurately detected the overall total potato defects (TPOD) percentage of the batch based on specific parameters set by the customer:

• Undesired color (browning): Edge Touched/Untouched – 17.5MM
• and above
• Internal discoloration: Inside the periphery – 4.75MM to 17.5MM
• External discoloration: Touching the edge – 4.75 MM to 17.5 MM
• Greening: Any amount

The visual AI platform dashboard provided insight into the percentage of the correct chips vs. TPOD per batch or lot. Operators could then drill down to evaluate specific chip defects or get the big picture with analytical tools tracking defect trends over a selected time range. Any discrepancies between what the visual AI platform flagged as inferior and what the human operators observed could be commented per image result inside the product’s dashboard-an easy and efficient way to log notes about each defect or false positive, track all historical data, and share evidence of issues with the potato vendors.

Based on the automated scanning results of batch sample runs through the visual AI system, the chip manufacturer can now accurately and confidently assess the TPOD of a given lot before making the highly consequential decision of accepting or rejecting it. Visual AI proved better than 90% effective at identifying undesirable potatoes before they entered the manufacturer’s potato chip processing line. They have since experienced a 15% improvement in the number of defects caught by visual inspection, with consistent result sets that do not fluctuate based on the variability of testers and shifts. Quality control vastly improved across their sites. They have achieved standardized and objective defect identification across dozens of processing lines with fewer workers per shift while building a valuable database of results to apply toward quality and profitability enhancements.

School Safety: Technology Investment for Safer Schools

School leaders across the United States have been struggling to thwart mass shootings since 1999, when two teens killed 12 students and a teacher at Columbine High School in Littleton, Colorado. Since that historic day, more than 330,000 students have experienced gun violence at school.

School security procedures are more widespread now than ever but are still inadequate or incomplete in emergencies. Interior locks on classroom doors, active shooter preparedness plans, tighter building access control, personal identification badge systems, metal detectors, gunshot detectors, bullet-proof backpacks, additional security staff, and human-monitored camera surveillance fail to detect and prevent active shooter incidents adequately. Most are reactive solutions limited in function.

Even traditional high-tech closed-circuit television (CCTV) systems have limitations when it comes to safety and security. CCTV systems operate by capturing and storing files for a designated period to be reviewed post-incident. Someone may be responsible for monitoring real-time footage to detect evolving situations, but it is an almost impossible task for a human, especially when presented with dozens, if not hundreds, of camera feeds. In contrast, CCTV systems equipped with visual AI can analyze data feeds in real time to identify issues as they occur and take immediate mitigating action.

Visual AI can utilize a school’s existing camera infrastructure to proactively detect everything and anything unusual – a weapon, an open door, unauthorized access, a license plate, vehicle identification, suspicious behavior, etc. Upon detecting a threat, security breach, or other developing situation, it can transform data into real-time actionable insights that instantaneously send alerts, alarms, and designated actions like initiating lock-down procedures.

For example, when a rifle or handgun is detected, visual AI immediately provides role-based alerts to notify designated personnel of the threat with vital details (e.g., location, time, and visual ID). Alerts are sent via web and mobile apps, email, and text and can tie into a school’s onsite security system. In this way, information sharing happens in real-time, automatically, providing school staff, first responders, and children precious extra time to act.

Retail Gas Stations: Competitive Intelligence in Dynamic Environments

Gas station owners know that creating customer-centric strategies is the key to increased customer loyalty and revenues. Historically, consumer insights have been acquired through manual surveys and market research, which provideinformation on only a tiny fraction of customers. With the advent of CCTVs, GPS data, digitized purchasing information, and shopper movements, they could ascertain additional intelligence. Still, the time and effort it took to manually analyze the data and watch hours of CCTV footage were impractical.

A major petroleum company with over 10,000 retail outlets sought a real-time, automated data collection solution to gather customer insights related to waiting time, fueling time, demographics, repeat customers, and vehicle information, as well as to track Standard Operating Procedure (SOP) compliance and identify and mitigate safety risks.

It deployed a visual AI technology to monitor retail customer time and behavior at gas pumps and then matched the information with license plates and vehicle types in the company’s CRM system. It provided the means to track customer business volume and loyalty and optimize employee performance, safety, and cleanliness, which resulted in significant customer loyalty and revenue increases.

The customer analytics were unparalleled. The visual AI technology focused on situational awareness, including detailed customer, vehicle, and store analytics that unlocked exciting opportunities and trackable ROI.

Congregations of People: Facility and Patron Safety

Increasing security cameras in retail stores, shopping malls, concert venues, faith-based institutions, and game parks pose a significant workflow challenge for security staff. While it’s undoubtedly beneficial to have extra “eyes” watching out for personal safety threats, active shooters, dangerous behavior, fraud, theft, loitering, or soliciting, the burden shifts to persistently and consistently monitoring video feeds to identify and respond to threats in real time.

Deep learning models trained in behavior detection automate this task, freeing security staff to patrol and respond to events faster. Computer vision-enabled solutions can automatically identify and alert security staff to suspicious behavior, aggressive interactions, slips, and falls, and crowd

behavior, acting as virtual “spotters” for their human counterparts.

Conclusion

Visual AI is not a new shiny technological object. Organizations embracing visual AI are unlocking the value of automated real-time visual analytics and alerts using existing camera infrastructures to transform safety, security, productivity, visual inspection, and situational awareness. They are uniquely positioned at the forefront of disruption.

The viability and value of visual AI are well established, and future possibilities are limitless. Visual AI will become ubiquitous as advancements in custom vision, new methodologies, and algorithms continue to progress.