Back
EP 007
19 Min
A third-party AI tool got Shahaf Tsaig's team to 70% accuracy and stopped there. They built their own and reached 99%, while holding a 95% CSAT across 24/7 support for more than 1,300 B2B customers.
Most support leaders buy an AI tool, accept that it is not quite right, and live with the compromises. Shahaf Tsaig did the opposite. As Director of Support and Program Management at logz.io, he spent a decade writing code before he ran support, and that background led him to build his own Gen AI workflows in-house. On this episode of the Fini Podcast, he walked through what he built, what failed first, and why the knowledge base matters more than the model.
Meet Shahaf Tsaig
Shahaf has led a 24/7 global support organization at logz.io for five years, serving over 1,300 B2B customers at a 95% CSAT score. He started as a software developer and spent ten years writing code before moving into support operations. He is clear that the edge is not reading code line by line. It is not being afraid to build, so his team could implement the exact use cases they needed instead of settling for an off-the-shelf fit.
Why he stopped buying and started building
The first attempt used a third-party tool. It promised quick setup, but categorization accuracy landed around 70%. At that level you have to re-check every ticket, so it saves nothing. After several months giving the vendor a fair chance, the team moved the work in-house. Within two months they hit 95% categorization accuracy, and after a year of running it, close to 99%. Same problem, very different result.
Three workflows: categorize, summarize, resolve
He built in order. First, ticket categorization, tagging each chat by type (question, configuration, or bug) and by product domain, which answers where the workload sits and where resolution is slow. Second, Summary AI, which condenses long chats so an engineer picking up a shift does not lose ten to fifteen minutes reading history. Third, Resolution AI, which uses the knowledge base to identify the issue and hand the engineer the fix. It is not fully end to end, because many fixes need an API call or an R&D change, but it gives every shift a reliable baseline.
Automation without killing CSAT
Holding 95% CSAT while automating heavily comes down to one rule: the AI helps the engineer, it does not replace them. The insights go to the support engineer, who decides what reaches the customer. Shahaf ran a chatbot test and watched customers ask for a human after a few messages, even when the answers were identical. He keeps people in the loop, and he is firm that the knowledge base, which everything depends on, should be owned and verified by humans. Fini takes the same view of where automation should hand off to a person.
What support leaders should take from this
Do not accept 70%. At that accuracy you re-check everything and save nothing. Push to the high 90s or keep iterating.
Invest in the knowledge base first. It has to be structured, complete, and updated with every feature and bug fix, or the AI gives wrong answers. Keep humans in charge of it.
Keep a human in the loop. Send AI insights to the engineer, not straight to the customer. Many customers still prefer a person.
Define your use cases before you build. Align them with how you actually support your product, then automate the highest-volume ones first.
You do not need a big budget. Shahaf built all three workflows with his existing team, no extra spend, and won buy-in by agreeing to be judged on results.
Automation covers gaps, it does not close them. Support exists to handle what the product cannot, so some tickets will always need a person.
Listen to the full episode
Shahaf goes deeper on the build, the failure with the vendor, and running 24/7 support across time zones in the full episode of the Fini Podcast. You can follow his work on LinkedIn.
Self-learning, knowledge-grounded resolution is what Fini is built for. Book a demo to see it on your own tickets.
Leo: Welcome back to the Fini podcast. I'm your host Leo. Most CX podcasts interview strategists and thought leaders who talk about the future of AI. Today we're doing something different. My guest is Shahaf Tsaig, Director of Support and Program Management at logz.io, where he's been for the past five years, leading a 24/7 global support organization, serving over 1,300 B2B customers with a 95% CSAT score. But here's what makes Shahaf different. He started as a software developer. He spent a decade writing code before moving into support operations. He understands both the technical and operational sides of customer support. He's not here to talk about AI strategy. He's here to talk about what actually happens when you implement AI in a production support environment. What works, what fails and what it takes to get from pilot to scale. Shahaf, welcome to the show.
Leo: You spent 10 years as a software developer before moving into support leadership, and that's pretty unusual. Most support leaders come up through customer service. Can you give me one example of a decision you made at logz.io that you could only make because you understand the code behind it?
Shahaf: It's not exactly that I understand the code behind it, it's more about not being afraid of doing things to code or finding solutions by ourselves. It gave me the ability to make an easy solution to implement different Gen AI solutions as part of our support organization.
Leo: When you say it helps you eliminate that fear, would you say that other support leaders have that fear when they're facing automation because they just don't know what's behind it?
Shahaf: I think so. Most support leaders will choose an off-the-shelf product and start using it and live with different compromises because it's not 100% adjusted to what they need. From my perspective, it's very easy to implement the use cases we need with existing Gen AI tools available in the market, and there's no real need to live with these compromises.
Leo: Most companies pilot around simple FAQ deflection and stop there. In your case you customized it a lot. What was the first workflow you automated, and what made you confident enough to push it into production?
Shahaf: The first one was very basic. It was the categorization of chats and tickets. We categorize with two layers. The first is question, configuration or bug. Question is when everything works and the customer doesn't understand what to do, or the documentation isn't clear. Configuration means the system operates well but someone from the company has to configure something. Bug means something doesn't work as expected. The second layer is the specific product domain. Categorization is the base for understanding where the workload is, where we're good, where we're bad, and whether resolution time differs by domain. It's an easy use case to implement with Gen AI. We did it manually in the beginning, then tested the solution side by side until we reached sufficient results.
Leo: Which workflow do you think has saved you and your team the most time?
Shahaf: All of them saved us time. The first was categorization. The second was a summary of the different chats, which we call Summary AI. Summary is essential because chats can be very long, and if you switch the support engineer between shifts, the new engineer has to read it all, which can take up to 10 or 15 minutes. The summary gave us good insight about what's going on and what the next step is. The third was Resolution AI, which actually gave us the solution. Based on the categorization and summary, and on a solid knowledge base, we found a resolution for the majority of use cases.
Leo: Many leaders think there's a tension between automating more and customer satisfaction, but you maintain a 95% CSAT while pushing heavy automation. How are you automating without killing your CSAT, and what guardrails do you have?
Shahaf: Gen AI doesn't come to replace the people, it helps us. The insights the AI gives are not communicated directly to the customer. They're given to the support engineer, who decides what to do with them. The same people who at home use Gen AI constantly, when it comes to a service they should get, prefer to talk to a human. We implemented a chatbot and saw that after a few messages people asked to speak with a human, even when given the same answers.
Leo: When you first deployed Gen AI, what was the biggest failure, and what did you learn?
Shahaf: The biggest failure was that at first we didn't implement the solutions in-house. We used a third-party supplier with a ready solution. It was supposed to be quick and easy, but the results weren't good. The categorization accuracy was about 70%, which is not enough, because at 70% you have to verify each ticket again and again, so it doesn't save you anything. We spent several months giving it a fair chance, then decided it wasn't good enough and switched to implementing it in-house. With in-house tools we reached 95% accuracy after about two months, and almost 99% after a year.
Leo: What are the main tickets your AI can handle end to end?
Shahaf: It's not really end to end. First, you have to invest a lot in the knowledge base. It must be robust, cover all the use cases, and be written in a very structured way so the tools can find the right solution. It must be updated with all new features, changes and bug fixes, otherwise it gives a wrong answer. Second, when handling a configuration or a problem, in most cases you have to do something, and that requires the right API or way to make the change. We were limited by how many use cases we could fully solve. Most required manual processing, so Resolution AI gave the support engineer the instructions about how to solve it rather than solving it completely.
Leo: Do you see the industry moving toward AI taking more of these actions, so you can automate more of these tickets?
Shahaf: It depends. Support exists to cover gaps in the product, whether mistakes or planned gaps that R&D decided not to implement. Closing those gaps is usually at the bottom of the queue. It depends on the company. In most cases the gaps still exist, and support is still required to close them.
Leo: Some leaders think the knowledge base can be automatically updated now. Do you still see a need for humans in the loop there?
Shahaf: At least for now I don't see it happening, but it might in the near future. In any case, I would keep a human in the loop, at least to verify what's written so we can take decisions based on it.
Leo: You probably needed executive buy-in. How did you sell this internally, and what was the business case?
Shahaf: It wasn't a hard sell, because I created all of these tools myself with the support team, without additional resources, without additional budget, and without affecting the day-to-day work. I found time slots where the team could progress. The agreement with management was that they would test the results, and if the results were good, we're all good. The results were good, so everyone was happy.
Leo: What's been the hardest part of running 24/7 support that most people underestimate?
Shahaf: The level of service. With 24/7 support, different people do different shifts. You may have stronger team members during work hours and less strong members during off hours. Training the team to a sufficient level is the biggest challenge. Resolution AI helps a lot, because it gives a baseline for the majority of use cases with the exact way to solve them, so we can keep a high level of service during all operating hours.
Leo: There are support leaders being told to implement AI who don't know where to start. Where should they begin?
Shahaf: The actual implementation is not that hard. Start by defining the exact use cases you want to solve, aligned with your support methodology. The second thing is to invest in the knowledge base. The knowledge base is critical, and in my opinion that's something humans should do.
Leo: Your title is Director of Support and Program Management. Why did it make sense to combine those?
Shahaf: It wasn't planned, it developed within the company. The role helped me understand what features are planned and the roadmap, so I could prepare the support group for these changes in a better way. It's not essential, because the gap can be closed in other ways, but in my case it helped.
Leo: Do you have a closer relationship with product because of this role?
Shahaf: Theoretically yes, but product decisions and prioritization are taken based on real product needs. I don't think it's correct to push product to solve customer requests if that's not the real priority. There should be a good balance between the roadmap and supporting customer needs.
Leo: How can support leaders find that balance?
Shahaf: I always push to see customer issues solved, but the balance should be kept. If it's a high-priority problem or bug, it should be somewhere in the roadmap in the near future, depending on the customer, their real priority, and the impact on other customers. I believe the allocation for customer needs should be somewhere between 10 to 15 percent of capacity, including features, not only bugs.
Leo: What made you switch from software development to support leadership?
Shahaf: I love to engage with customers. In R&D you don't communicate much with customers. When you talk with customers you feel the real requirements, whether the product is efficient and helps them. For me it's more interesting.
Leo: Rapid fire. First response time or resolution time, which KPI matters more?
Shahaf: As a single KPI, it's always a combination. Both are super important, and both should come together.
Leo: True or false, in three years most Tier 1 support will be fully automated.
Shahaf: I don't believe the product gaps that would allow complete automation will be closed. A big part of the work will be automated, but there will still be a part that won't be.
Leo: Finish the sentence: the company that wins in 2026 isn't the one with the best AI, it's the one with the best...
Shahaf: Service.
Leo: Shahaf, this was really great. Where's the best place for people to follow your work?
Shahaf: On LinkedIn.
Leo: Perfect. Thanks for joining us. And for everyone listening, if you want more conversations about what it actually takes to implement AI in customer support, make sure you're subscribed to the Fini podcast and we'll see you next time.
Shahaf: Thanks a lot.
Is it better to build AI support workflows in-house or buy off-the-shelf?
It depends on your team, but Shahaf Tsaig's experience shows building in-house can be far more accurate. A third-party tool plateaued at about 70% categorization accuracy, while his in-house build reached 95% in two months and close to 99% within a year.
Why was the off-the-shelf AI tool only 70% accurate?
At 70% you have to manually verify every ticket, so it saves no time. The gap came from a generic tool that was not tuned to logz.io's specific products and use cases, which is why the team rebuilt it around their own knowledge base.
Can AI resolve support tickets end to end?
Not always. Shahaf's Resolution AI identifies the issue and gives the engineer the fix, but many resolutions require an API call or an R&D change, so a human completes them. Support exists to cover product gaps that automation cannot close.
Why is the knowledge base so important for AI support?
The AI is only as good as the knowledge it draws on. It must be structured, cover every use case, and stay updated with new features and bug fixes, or it returns wrong answers. Shahaf keeps humans in charge of maintaining it.
