Back
EP 007
19 Min
A third-party AI tool got Shahaf Tsaig's team to 70% accuracy and stopped there. They built their own and reached 99%, while holding a 95% CSAT across 24/7 support for more than 1,300 B2B customers.
Most support leaders buy an AI tool, accept that it is not quite right, and live with the compromises. Shahaf Tsaig did the opposite. As Director of Support and Program Management at logz.io, he spent a decade writing code before he ran support, and that background led him to build his own Gen AI workflows in-house. On this episode of the Fini Podcast, he walked through what he built, what failed first, and why the knowledge base matters more than the model.
Meet Shahaf Tsaig
Shahaf has led a 24/7 global support organization at logz.io for five years, serving over 1,300 B2B customers at a 95% CSAT score. He started as a software developer and spent ten years writing code before moving into support operations. He is clear that the edge is not reading code line by line. It is not being afraid to build, so his team could implement the exact use cases they needed instead of settling for an off-the-shelf fit.
Why he stopped buying and started building
The first attempt used a third-party tool. It promised quick setup, but categorization accuracy landed around 70%. At that level you have to re-check every ticket, so it saves nothing. After several months giving the vendor a fair chance, the team moved the work in-house. Within two months they hit 95% categorization accuracy, and after a year of running it, close to 99%. Same problem, very different result.
Three workflows: categorize, summarize, resolve
He built in order. First, ticket categorization, tagging each chat by type (question, configuration, or bug) and by product domain, which answers where the workload sits and where resolution is slow. Second, Summary AI, which condenses long chats so an engineer picking up a shift does not lose ten to fifteen minutes reading history. Third, Resolution AI, which uses the knowledge base to identify the issue and hand the engineer the fix. It is not fully end to end, because many fixes need an API call or an R&D change, but it gives every shift a reliable baseline.
Automation without killing CSAT
Holding 95% CSAT while automating heavily comes down to one rule: the AI helps the engineer, it does not replace them. The insights go to the support engineer, who decides what reaches the customer. Shahaf ran a chatbot test and watched customers ask for a human after a few messages, even when the answers were identical. He keeps people in the loop, and he is firm that the knowledge base, which everything depends on, should be owned and verified by humans. Fini takes the same view of where automation should hand off to a person.
What support leaders should take from this
Do not accept 70%. At that accuracy you re-check everything and save nothing. Push to the high 90s or keep iterating.
Invest in the knowledge base first. It has to be structured, complete, and updated with every feature and bug fix, or the AI gives wrong answers. Keep humans in charge of it.
Keep a human in the loop. Send AI insights to the engineer, not straight to the customer. Many customers still prefer a person.
Define your use cases before you build. Align them with how you actually support your product, then automate the highest-volume ones first.
You do not need a big budget. Shahaf built all three workflows with his existing team, no extra spend, and won buy-in by agreeing to be judged on results.
Automation covers gaps, it does not close them. Support exists to handle what the product cannot, so some tickets will always need a person.
Listen to the full episode
Shahaf goes deeper on the build, the failure with the vendor, and running 24/7 support across time zones in the full episode of the Fini Podcast. You can follow his work on LinkedIn.
Self-learning, knowledge-grounded resolution is what Fini is built for. Book a demo to see it on your own tickets.
Is it better to build AI support workflows in-house or buy off-the-shelf?
It depends on your team, but Shahaf Tsaig's experience shows building in-house can be far more accurate. A third-party tool plateaued at about 70% categorization accuracy, while his in-house build reached 95% in two months and close to 99% within a year.
Why was the off-the-shelf AI tool only 70% accurate?
At 70% you have to manually verify every ticket, so it saves no time. The gap came from a generic tool that was not tuned to logz.io's specific products and use cases, which is why the team rebuilt it around their own knowledge base.
Can AI resolve support tickets end to end?
Not always. Shahaf's Resolution AI identifies the issue and gives the engineer the fix, but many resolutions require an API call or an R&D change, so a human completes them. Support exists to cover product gaps that automation cannot close.
Why is the knowledge base so important for AI support?
The AI is only as good as the knowledge it draws on. It must be structured, cover every use case, and stay updated with new features and bug fixes, or it returns wrong answers. Shahaf keeps humans in charge of maintaining it.
