Inspired by my experience consulting for companies and the rise of inference time compute for LLMs (as well as the acceptance of users waiting while the LLM *thinks*), I suggest many teams working on RAG should rethink if you can improve your accuracy by relaxing latency.
