1 hour ago · 7 min read1372 words · Tech · hide

When building applications with AI, we often ask models to generate text and then try to make sense of that text in our code. For instance, imagine you’re building a feedback management application. You want to categorise incoming feedback as a bug report, a feature request, or general feedback. You might ask an LLM to classify the message and return the result as JSON. Your application then parses that JSON, checks whether the expected fields exist, and uses the result to decide what to do next. This works, but there’s an interesting question here: if all we need is a decision, why ask the model to generate a whole lot of text in the first place? That’s simply the waste of precious tokens and time. Instead, we could ask the model to evaluate a set of predefined questions and return the answers directly. Google seems to have been thinking along similar lines. It recently introduced the Gemini Decisions API, a new API powered by the gemini-decision-1 model that’s designed specifically…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town