FAQ | Can I use the cache to reduce the cost?#

The LLM Mesh provides caching capabilities that an administrator can configure at the LLM connection level. By caching responses to common queries, the LLM Mesh avoids unnecessary re-generation of answers, reducing avoidable costs.

To enable the use of cache:

  1. In the Administration panel > Connections, select your LLM connection or create a new one (New connection > LLM Mesh).

  2. Under Usage Control, check the Caching option.

Screenshot of the cache settings in LLM connections.