FastAPI is built on asynchronous principles, allowing it to handle many connections concurrently and efficiently. This chapter focuses on applying these principles to your machine learning API. You will learn how to define asynchronous route handlers using async and await and understand where they offer the most advantage, typically in I/O-bound preprocessing or postprocessing steps.

We will address the common challenge of integrating CPU-bound tasks, like model inference, into an asynchronous application, presenting strategies to prevent blocking the server's event loop using techniques like run_in_threadpool. Additionally, you'll learn to implement background tasks for actions that can run after a response has been sent, such as logging detailed prediction results or sending notifications. The chapter concludes by examining key performance considerations specific to serving ML models via APIs, ensuring your application remains responsive under load.