Back
EP 006
25 Min
Rohit Tripathi joined a two-person team supporting six brand-new AI products at Zapier, inheriting a backlog of 700 tickets. When the product itself is unpredictable, telling a real bug from a hallucination is the whole job. Rohit Tripathi is one of the CX leaders featured in our Hall of Fame.
Supporting AI products is harder than supporting normal software, because the thing you are debugging can be wrong in ways no one designed. Rohit Tripathi has spent more than ten years in technical support, including a pilot team at Zapier that supported its entire AI line while it was still being built. On this episode of the Fini Podcast, he explained how he triages AI issues, how he gets engineering to act, and what he would automate first.
Meet Rohit Tripathi
Rohit is an application support engineer at Last Yard, and over the past decade he has worked in technical support at HP, Microsoft, Automate.io, Notion, and Zapier. At Zapier he was one of two people supporting six new AI products at once (chatbots, tables, interfaces, and more), debugging live systems, logging bugs in Jira, and working directly with product managers to prioritize fixes by real customer impact.
Clearing a 700-ticket backlog from both ends
The pilot inherited about 700 pending tickets. With two people, Rohit ran a simple play: one person worked from the newest ticket backward, the other from the oldest forward, and they met in the middle. The logic was to protect the experience of new customers without abandoning the people who had been waiting for months. It worked, taking the backlog down to roughly 100 to 150 in three to four months.
Bug, hallucination, or user error?
When you are not sure whether an issue is a product bug, a hallucination, or a user mistake, Rohit relies on two habits: watch for patterns and reproduce the issue. If several users report the same experience, it is probably a real bug, so he recreates it to see how far it goes before deciding to escalate. He is blunt that hallucination is a genuine problem, because AI will confidently make up answers, so the person receiving an AI suggestion has to double-check it before passing it to a customer.
Getting engineering to actually prioritize
Logging a bug and hoping is not a strategy. Rohit escalates with impact: which customers are affected, whether they are paying, and how badly the bug blocks them from using the product. A white-space error is not a refund-blocking failure, and he sets the noise level accordingly. His larger point is that the feedback loop matters more than anything. When communication between support and engineering is flawless and detailed, engineers who never talk to customers can still serve them well.
What to automate first, and what to automate last
First, capture knowledge before it disappears. In fast teams, a tier-one engineer hits a problem, posts in Slack, and three teams solve it, then the answer is lost in the thread and someone re-solves it a week later. Rohit wants a tool that watches those discussions, files the answer, and surfaces it on the next matching ticket. That is the knowledge architecture problem most teams ignore. Last to automate: anything touching payment or personal data, and any multi-turn conversation that drags on. If a customer comes back more than about five times unsatisfied, a human should step in, which is exactly where good guardrails earn their keep.
What support leaders should take from this
Reproduce before you escalate. Recreate the issue on your end to confirm it is a bug, not a feature request or user error.
Triage by pattern. Multiple customers reporting the same thing is your strongest signal of a real bug.
Escalate with impact, not volume. Name who is affected, whether they pay, and how badly they are blocked. Set the urgency to match.
Capture tribal knowledge. Get answers out of Slack threads and into a repo the next agent can find, or you will solve the same issue forever.
Cap the back-and-forth. If the AI cannot resolve a conversation in a few turns, hand it to a human before a hallucination does damage.
First fix the customer, then fix the issue. Acknowledge the frustration before you debug. That is still what separates a person from a robotic tone.
Listen to the full episode
Rohit shares more on supporting AI in production, the support-to-engineering loop, and advice for tier-one agents who want to level up, in the full episode of the Fini Podcast. You can connect with him on LinkedIn.
Resolution that stays grounded instead of guessing is what Fini is built for. Book a demo to see it on your own tickets.
Leo: Welcome back to the Fini Podcast. I'm your host Leo. My guest today is Rohit Tripathi, currently an application support engineer at Last Yard. Over the past 10 years, he's worked in technical support at Zapier, Notion, Automate.io, Microsoft, and HP. At Zapier, he was part of a two-person pilot supporting the entire AI product line (chatbots, interfaces, AI actions) while those products were still being built. That meant debugging complex systems, logging bugs in Jira, and working directly with product managers to prioritize fixes based on real customer impact. Today we're going to dig into what it actually takes to support AI products in production, how to bridge the gap between support and engineering, and what's changed in technical support over the last decade. Rohit, welcome to the show.
Rohit Tripathi: Thank you for having me, Leo.
Leo: Let's get right into that pilot. At Zapier you were part of a two-person team supporting the entire AI product line, and that's a lot of surface area for two people. What was the hardest part about supporting AI products that are still being built? And how do you triage a ticket when you're not even sure if it's a user error, a bug, or just the AI being unpredictable?
Rohit Tripathi: When Zapier launched its new AI line it included about six products, each with its own product manager and development team. There was huge hype around ChatGPT coming out. Zapier even introduced a period where everybody in the company was encouraged to experiment with AI and brainstorm how it could be useful. I was given a chance to join the pilot support team covering all of these products at once. I was nervous but excited, because it was a two-person job and six products. When I joined, there were about 700 tickets pending, a backlog from when the products were first released. So I came up with a simple strategy: since there were two of us, one would start from the newest ticket toward the oldest, and the other from the oldest toward the newest, and we'd meet midway. The logic was that we shouldn't hurt the experience for new customers while still making sure we answered the older ones. That strategy worked, and we brought the volume from 700 down to about 100 or 150 in three to four months. On the second part: when you're unsure whether it's a bug or a user error, it's about observing trends and patterns. If multiple users complain about a similar experience, there's a good chance it's a product bug. I always believe in reproducing the issue to see how far it goes, then deciding whether it's a bug, a missing feature, or purely a user error, and whether to report it to engineering.
Leo: A lot of support teams just log bugs and hope engineering fixes them, but engineering usually has thousands of things ahead of them. What did you do differently that actually got engineering to listen and prioritize?
Rohit Tripathi: At the end of the day it's about the type of customers using the product. There are customers on a free plan testing it out, versus paying customers who are patrons of the product. Naturally, paying customers get more attention when something is a valid bug rather than a user error. We never ignored free customers, everyone got timely responses, but for prioritizing bugs we weighed who is paying and the impact of the bug, whether it's a roadblock or a minor white-space error. Based on that I'd choose the noise level at which I needed someone to look at it. Because it was a two-person team, the product managers and engineers were attentive and gave us the confidence to be vocal when something needed urgent attention.
Leo: You started in tier-two technical support at Microsoft and now support complex systems after years at automation and AI companies. What's the biggest difference between tier-two support at Microsoft and product specialist support at a company like Zapier or Notion?
Rohit Tripathi: The goal of the job is still the same: we face customer problems and do our best to unblock them. Support is almost an unfortunate job, because you're talking to people who'd rather be doing 50 other things but have to reach out because something that should work isn't. The technologies have changed. I started as a tier-one technician at Microsoft on the download, install and activation process for Windows and Office, and on my first day they released Office 365 and Windows 10 at the same time, so there was a lot of chaos. I've always enjoyed fast-moving environments. But the bottom line is the same: big systems designed to work, with some cog that isn't working. Our job is to find what's wrong, oil it up, and get it running again. And if the feedback loop between support and engineering is seamless, with the detail passed back, it helps engineers serve customers better.
Leo: You worked at both Zapier and Automate.io, so you know how automation works under the hood. What's the one mistake you see support teams make when they try to automate?
Rohit Tripathi: People try to solve a problem with whatever resources they have, fixated on the problem itself. What I do differently is take a step back and look at the bigger picture: why is this even a problem, is there a simpler way? Eight times out of ten it's not the literal problem that needs a solution, it's the overall workflow that needs simplifying. At Zapier with 700 tickets, the bigger picture was that the customer who wrote five months ago would keep waiting if we only worked today's tickets, so we started from both ends. Most of my automations are really simple, but they reduce the overall noise in day-to-day work.
Leo: If you were building a support team from scratch, what's the first thing you'd automate with AI, and what's the last?
Rohit Tripathi: In companies with a small headcount and no dedicated technical-writing team, I'd first harness the information shared in Slack. When a tier-one contributor hits an issue and posts it, and tier two, tier three and engineering jump in to solve it, what stops the next tier-one person from hitting the same issue? If there's no way to capture that into documentation, the information is lost in the void of Slack, and people reinvent the wheel. A tool that watches what's discussed, files it in a repo, and surfaces it on the next matching ticket would be a real difference-maker for level one and level two support. The last thing I'd automate is anything with personal or payment information, until we have systems that can be trusted with it. I'd also set a threshold: if an AI solution goes back and forth more than about five times and the customer is still unsatisfied, that's when a human needs to jump in, because hallucination is a real problem and AI confidently makes up answers, so responses need to be monitored.
Leo: You were supporting AI products while also using AI tools yourself. What's it like to use AI to support AI products? Does it help, or create more confusion?
Rohit Tripathi: It helps a lot, the pros are far ahead of the cons. The con is that AI only understands patterns it has seen before, and new companies ship products the models haven't seen, so hallucination will be a thing until AI is far more advanced. The onus is on the recipient to double or triple-check before giving a response to a user. If you're not 100% sure, check with someone on the team. One thing I've noticed in good engineering firms is that people are eager to help, all you have to do is ask.
Leo: You've been in technical support for over 10 years. What's the one thing that hasn't changed, no matter the product?
Rohit Tripathi: One of my managers used to say: first fix the customer, then fix the issue. The first thing you do is understand the customer's frustration. That's what distinguishes a human agent from an AI system, the ability to actually listen and respond accordingly, rather than a robotic automated tone that people dislike. That's what makes technical support a humane job, and that part hasn't changed.
Leo: Have you seen AI become ready to handle agitated customers and mimic that human tone, or is that still where humans need to be escalated to?
Rohit Tripathi: With companies trying to automate support it's increasingly possible, and AI can beat humans on response time because everything is indexed. But on overall customer experience, knowing you're talking to someone who understands your situation and gives a response that works for both sides, that's where the focus needs to be. If that gap can be bridged, it would be a real difference-maker for the support industry.
Leo: Imagine someone in tier-one or tier-two support who wants to jump to a product specialist or technical support engineer role. What's your advice, and what should they be learning?
Rohit Tripathi: The technology space has evolved so much it's barely recognizable. With tools like Cursor and Claude Code, the playground has leveled, a support engineer and a programmer are closer than ever. Make the most of the resources you have, keep a positive learning mindset, and stay curious about new tools, even via a quick daily newsletter on what's changing. Pair that with people skills and a small side project, and you have a real shot at these companies.
Leo: Rapid fire. Ticket queue or Slack escalations, which do you prefer to manage?
Rohit Tripathi: Ticket queue.
Leo: True or false, in three years most technical support will be handled by AI with minimal human oversight.
Rohit Tripathi: False.
Leo: Finish the sentence: the best technical support engineers aren't the ones who know the most, they're the ones with the best...
Rohit Tripathi: People skills.
Leo: Rohit, this was great. Where's the best place for people to connect with you?
Rohit Tripathi: On LinkedIn, at linkedin.com/in/simplyrohit. Or come have a chat with me in Perth in one of the cafes, we've got great cafes here.
Leo: Perfect, Rohit, thanks for joining us.
Rohit Tripathi: Thank you for your time, Leo.
Leo: And for anyone listening, if you want more honest conversations about what it actually takes to implement support AI in production, make sure you subscribe to the Fini Podcast and we'll see you next time.
How do you tell an AI bug from a hallucination or a user error?
Watch for patterns and reproduce the issue. If several customers report the same behavior, it is likely a real bug, so recreate it before escalating. Because AI can confidently produce wrong answers, treat unverified AI output as a possible hallucination and double-check it against a reliable source.
How do you get engineering to prioritize support escalations?
Escalate with impact rather than raw volume. Specify which customers are affected, whether they are paying, and how badly the issue blocks them from using the product, then set the urgency to match. A tight, detailed feedback loop helps engineers who never talk to customers fix the right things.
What should support teams automate first?
Rohit would first capture knowledge that gets lost in Slack threads, filing it where the next agent can find it so the same issue is not solved repeatedly. Most of his automations are simple and aimed at reducing day-to-day noise rather than building complex workflows.
When should a human take over from AI?
For anything involving payment or personal data, and whenever a conversation runs long. If a customer comes back more than about five times still unsatisfied, a human should review the conversation, since hallucinations make unmonitored long exchanges risky.
