This chapter focuses on how to reduce costs when deploying large language models in production.
Once your responses are accurate, the next challenge is cost control.
Using APIs like OpenAI’s can cost several cents per query at scale.
Hosting your model is also expensive due to computational requirements.
Engineers need to be aware of tradeoffs and optimization opportunities.
One common method is caching, but it doesn’t work the same way for LLMs.
You also can't move the LLM to the client side due to size and computational limitations.
In short, there are some optimizations that are necessary before deploying an LLM app to production. This chapter discusses the most common ones.