Loading…
15
Showerthought: I tried both a big open model and a small fine-tuned one for a customer service bot, and the small one won
Last month, my team built a test bot using GPT-4 for a client's online help page. It was smart but slow and gave weirdly long answers. Then we fine-tuned a much smaller Llama model on just their past support tickets for about $200 in compute costs. The small model answered in under 2 seconds every time and used the client's exact wording, which customers liked way more. The big model felt like showing off, but the small one just solved the problem. Has anyone else found that smaller, focused AI works better for specific business jobs?
3 comments
Log in to join the discussion
Log In3 Comments
annawebb6mo ago
Your test only shows what works for one simple task. Big models can handle surprise questions a small bot would fail at completely. That two second speed won't matter when a confused customer asks something not in your old tickets. You're basically building a fancy search engine, not real intelligence. Relying on that small box could really backfire when you need to understand new problems.
5
luna8246mo ago
Ugh, my friend's small bot totally froze when a client asked about their weird new error code.
4
rileyp496mo ago
Wait, you hit the exact same wall we did last year. Our small bot crashed on a weird shipping question too. So we set up a simple rule: if the bot's confidence score drops below 80%, it instantly hands off to a human agent with the full chat history. You keep the fast answers for common stuff but never leave a customer stuck. Isn't that better than one slow, expensive model trying to do everything?
1