AWS Certified AI Practitioner Practice Exam

For the lowest latency inference on edge devices, which solution should a company implement?

1. Deploy optimized small language models on edge devices

2. Deploy optimized large language models on edge devices

3. Use a centralized small language model API

4. Use a centralized large language model API

The correct answer is: Deploy optimized small language models on edge devices

Opting for optimized small language models on edge devices is ideal for achieving the lowest latency in inference. This approach allows the processing to happen locally on the device itself, minimizing the need for data transmission to and from a centralized server. The reduced distance for data communication significantly lowers latency, as the model can quickly respond to input without waiting for the network round-trip time associated with cloud-based solutions. Small language models are inherently less computationally intensive than larger counterparts, making them more suited for environments with limited processing power and memory. This also contributes to faster inference times since smaller models require fewer resources to execute. In scenarios involving centralized APIs, whether for small or large language models, there is always an inherent delay due to the need to send data over the network, wait for the model to process the data, and receive the results. This latency can be significant compared to local inference, especially in applications requiring real-time responses. By deploying optimized small language models directly on edge devices, a company can ensure rapid responses to user inputs, making this solution particularly effective for applications where speed is critical.