Edge AI vs Distributed AI
Distributed AI is a paradigm of computing which allows you to scale your applications across distributed cloud environments. Distributed cloud environments is something that allows you to have a interface for application lifecycle management across the cloud, on-premise and edge environments.
Let’s take a look at the evolution of Distributed AI from the cloud based AI. In the Cloud based AI, lets consider that you operate a plant at a particular location. The business operations from within the plant results in the generation of large volume of data which is passed on to the cloud. The cloud environment consists of an orchestration platform (Eg. Kubernetes), an AI based middleware and an application layer used for training and inferencing the model. The decisions generated by the model is then transmitted back to the plant. However, this could run into various challenges related to connectivity at your core location, which has led to the emergence of the Edge AI.
The next stage of evolution i.e. Edge AI consists of the orchestration platform, AI middleware and the application deployed right within the plant. The core still performs the same functionality as it did in the Cloud based AI model, the model is trained in the core and deployed within the plant where inferences are made by the application. The application generates decisions which drives the business process and feeds data back into the computational stack present within the plant. The decision making process is localized and we no longer need sending the data regularly to the core location and wait for the decisions from the core. However data still needs to be sent to the core to re-train the AI pipelines and deploy it back in the plant.
However when it is required to implement the Edge AI model across a large number of locations and a variety of applications, we run into certain challenges. In order address these challenges a new distributed AI model was introduced. Distributed AI is very similar to Edge AI except that it follows the hub and spoke model.
There might be a case where you have vast amount of data in a public cloud but might want to consume AI capabilities from another cloud. In this case, the first cloud is called as a ‘spoke’ where the data is present, while on the other hand the cloud which contains AI capabilities is called as the ‘hub’ which acts as a control pane. The control pane manages the deployment of applications from the hub to spoke while taking control of the data lifecycle from the hub.
Challenges around Distributed AI and their solutions
When large amounts of data is being sent by multiple applications to the hub, it poses a challenge called ‘data gravity’. It involves huge costs in training and analyzing the data and also the network bandwidth issue that comes along the way. This can be managed by a process called ‘intelligent data collection’, meaning only the data that is required is collected. Also, ‘Heterogeneity’ is another issue that is related to the variety of applications in the spoke. For example each of your plants might be manufacturing a slightly different product mix. A single trained pipeline within the hub might not be applicable for all the different spokes. To tackle this issue, it is vital to adapt to and monitor each of the different spoke.
The data generated from the spoke might have a variety of data for example sensor data, images, sounds etc. This causes problems in deploying at scale. The solution to this lies in automating the data and AI lifecycle meaning when the data is to be purged, replicated and when the data is to be collected among others.