Optimizing prompts for different requests

[Thirteenth]

We have an AI assistant on our electronics store that handles everything from simple product questions to comparisons between several laptops and longer pre-purchase conversations. Every request currently uses the same system prompt, but every time we improve it for one kind of question something else seems to get worse. Adding more detail helps with complicated comparisons, then customers asking basic questions start getting ridiculously long answers. Cutting it back fixes those, but the assistant becomes less useful when someone actually needs help choosing between products. We have feedback from completed chats and can tell whether a conversation led to a product view, cart addition, or the customer giving up. How can we use those results to optimize which prompt setup works best for different kinds of requests instead of maintaining one increasingly bloated prompt for everything?

 

[Egglex]

The problem isn't really that your prompt is too long or too short, it's that you're asking one version to behave well across completely different jobs. A customer asking whether a laptop has Thunderbolt doesn't need the same instructions as someone comparing four models and talking through tradeoffs for twenty minutes. I'd split traffic into a few intent buckets first, give each one its own prompt version, then score those versions against both answer quality and the outcomes you already track. That gives you a clean way to see which setup works for quick factual questions, comparisons, and longer buying conversations without letting one giant prompt keep growing.

 

[Lovol]

You've already got useful feedback coming out of the chats, so there's no reason to judge prompt changes only by reading a handful of transcripts. The stronger approach is to let different request types try different configurations and feed the result back into the selection process, so the system can learn which setup performs best for each kind of conversation. You can do prompt optimization here https://eignex.com/ . It can choose among different prompt styles for each request and learn from the score you send back after the run. That lets short product questions stay concise while comparison and pre-purchase chats can use a more detailed setup when the results show that it works better.