In this blog, we’ll explore edge AI concepts, trends, and use cases, and provide guidance for building AI solutions at the edge.
Edge computing is one of the hottest trends in IT today. By the end of 2023, there will be 43B connected devices in market, and IDC research predicts more than half of new enterprise IT infrastructure will be at the edge. Additionally, Gartner research predicts that by 2025, more than 75% of enterprise data will be created and processed at the edge. It should come as no surprise, then, that this hot topic coincides with another trend – the explosion of artificial intelligence (AI) and machine learning (ML.) To take advantage of these trends and the actionable insights hidden within the troves of data collected, teams need a way to centrally train and manage locally run AI and ML models at the edge to accelerate results and outcomes.
In this blog, we’ll explore edge AI concepts, trends, and use cases, as well as provide practical guidance for organizations looking to take advantage of the data and opportunities that exist at the edge.
First, edge computing is nothing new. It refers to distributing data storage and workloads close to where the data is being generated and where actions are being taken with the goal of improving scalability, performance, and security. As seen in the image below, edge computing can take many forms; however, to keep things simple we will limit the scope to compute capabilities, device size, and location.
Now that you have a high-level understanding of what edge computing is, let’s jump into the primary factors driving AI and ML to the edge using an example. Consider the shop floor of an industrial factory— it likely has a lot of machinery and possibly other wired or wireless endpoints and actuators such as, cameras, sensors, and more. Without edge AI/ML solutions, all the data collected from those machines and endpoints must get sent to a centralized data center or cloud computing infrastructure for AI/ML processing and analysis. In most circumstances, the results are sent back to on-site operational technology (OT) systems to perform post-processing for optimization, on-site alerting, and other applications.
By moving and processing the AI/ML workloads locally on or near the factory floor, you can immediately imagine reduced strains on network bandwidth from not having to transfer significant volumes of data back and forth. Multiply this by scenario by the number of other factory locations in your supply chain and you can begin to see significant cost savings from reduced inbound and outbound traffic transfer fees. Other obvious benefits of moving AI/ML workloads to the edge include increased privacy and security because your organization’s proprietary data is not exposed to network vulnerabilities and attack or may not even have to leave the factory. Similarly, many OT systems on the factory floor are executing critical functions in near real-time, necessitating low latency, which only real- time edge AI/ML results can provide.
Ultimately, sending AI analysis to the point of data collection can ensure that all the machines and factory floor assets remain fault tolerant, all while enabling faster, more secure AI results, and safer and more efficient operations.
Edge AI presents endless possibilities. Getting started doesn’t have to be hard. The following steps outline a high-level implementation approach that organizations can follow to start reaping the benefits of moving AI/ML workloads to the edge.
There are likely many processes ripe for AI/ML across your organization. Getting this step right is critical. Get it wrong and you may end up with a few disenfranchised stakeholders that will make any future attempts an uphill battle. That said, it is important to get the right mix of stakeholders in the room at the onset of this kind of initiative. The more hats the merrier; but, at a minimum be sure to have representation from business/operational, IT and security, data, and engineering groups. Equally important is not to assume everyone has the same background and understanding of AI/ML concepts and techniques.
An assessment rubric for prioritizing and picking the right edge AI/ML use case can be a useful tool. At the very least, you should consider the following criteria when picking an edge AI use case.
Outcomes & Results
Resources & Readiness
Example Use Cases by Industry
As discussed, there are a lot of different things to consider when designing your edge AI architecture–compute capabilities, device size, and location. A recently published article on edge AI architectures goes into more detail on four different architecture design patterns.
Ultimately, the resulting design should be driven by your use case and the outcomes you are looking to achieve. That said, it is important to determine upfront where the AI workloads need to be performed/processed so that your resulting use case is fast, scalable, and secure. If fast response is important for your use case, your edge AI solution should support low latency inferences. This can facilitate real-time results if you need them.
Also, consider a central management hub for your ML/AI models. Managing your models in a central location allows you to deploy new versions of models easily and automatically to your edge devices, enables collaboration amongst teams, and the ability to reuse or update models as needed. The concept of a central management hub also enables transparency by presenting model training framework and data, version history, and expected performance. A central hub also ensures results reproducibility by providing an audit trail of past predictions, and the ability to set governance controls.
Next, your solution architecture should be device agnostic, and be able to support a wide range of chips, and data transfer protocols. While many IoT devices come built with ARM chips, you’ll want to also make sure they work for AMD chips. Same goes for data transfer protocols. For example, while MQTT might be the standard in manufacturing, you’ll also want to make sure your architecture woks for gRPC, REST, etc. This can save a lot of time as you build future solutions and applications that will interact with the AI/ML inference results. Planning for spotty or disparate network connectivity can also ensure that your models run as expected, always, even if or when your devices go offline. Your architecture should support the ability to operate in a disconnected environment.
Once you have the use case identified, the solution architecture designed, it is time to execute an implementation plan that will deliver the projected outcomes and results. The implementation plan is a necessary asset that can be iterated on in parallel to ensure coordination, collaboration, and accountability across involved stakeholders. Look to structure the plan with milestones at the 30-60- 90-day points. For example, strive to have an AI/ML model prototype developed by day 30, your edge devices, central management hub, and networking infrastructure in place by day 60, and the edge AI/ML results integrated into business or operational workflows and applications by day 90. At this point, you are ready to begin educating the end users and collecting feedback and lessons learned that will help you scale adoption and use across the organization successfully.
Building AI solutions for the edge might be the future, but by using some of the recommendations in this white paper and adopting a location-centric approach will help put the foundation in place to build these solutions today. By abstracting some of the complexities of an infrastructure-centric approach to AI deployment, you can use the ideas from this white paper to support the efficient development of scaled edge AI solutions that run in any location - in the cloud, on- prem, and of course, at the edge.