'Done is better than perfect' will wreck your customer-facing AI
Yuri Vonchitzki"Done is better than perfect." A good mantra for writing an internal memo or testing out an ad hook. But when it comes to building automated systems with AI, going live will torpedo your reputation in a matter of days.
After building and deploying over 2,500 live automations and AI systems for more than 300 companies, the single biggest mistake I see is when people treat fragile AI systems accessing live business data like lean software experiments.
A team integrates an LLM into their n8n (or Make) scenario, provides direct access to their CRM and email infrastructure, tests out 3 prompts in the afternoon, and then pushes the thing live to their customer accounts. Fast forward to Thursday morning and it has hallucinated and provided custom discounts to 60 inbound leads, has misclassified some enterprise clients as spam, and sent out 120 blank/broken messages. So now you have to spend the weekend cleaning up their mess.
And if you ask why they went live without guardrails, they tell you it's part of their iterative shipping, and they want to learn from the live customer feedback. It's not lean development, it's system negligence.
The autonomy your AI system did not earn
We nearly made this mistake at Ninjabot. When we first built our automated AI voice and chat agents, they sounded amazing and delivered on what we wanted: humanlike tone, booked appointments like a pro. Then we dove into the conversation logs and realized the AI was making up services that the client did not offer, agreeing to project scopes the agency could not complete, and booking appointments in slots that were already full.
We were giving the automated AI system more freedom than it was ready for. Immediately we took it out of production and placed it back in a sandbox. From there, we built automated regression suites and tested it against edge cases using our historical data.
Whenever an LLM provider updates their API or changes model weights, we run those test suites before taking that AI to the customer. We make sure that even if prompt behavior has shifted by a fraction, we will not send that to customers.
You do not want to give an automated AI system full autonomy on day one. Instead, you should give it an assisted mode in which a human reviews the output of the model. Or you can give it a delegated mode, where it handles certain narrow tasks and low risk tasks under supervision. But you should only give an automated AI system full autonomy after you have enough real operational data to show that it does not deviate from your rules.
The two buckets
To stop shipping broken AI workflows, split your backlog into two distinct buckets.
Bucket 1: Internal AI systems
This is where an 80% accurate build is completely fine.
When I'm building something for my team that will transcribe calls and give me action items in a Google Sheet, or make updates for me in Slack for projects, I deploy it straight away. If the AI model makes something up or misses an action item, my team will let me know in 2 seconds. I'll just adjust the system prompt or the parsing logic in the scenario and off I go.
Your team is your safety net. This is one place where moving quickly can save you hours each week. Don't spend a month trying to make a perfect AI pipeline for your team, you're just burning payroll.
Bucket 2: Customer-facing AI systems
Here, an 80% success rate means one out of every five customer interactions is a failure.
If your AI system misinterprets your customer's angry email, or it hallucinates a refund policy that you don't offer, or it charges the wrong card because an LLM used the wrong invoice ID, the fact that you're running a lean business won't be a good excuse. They'll think you're just incompetent.
If an AI model is directly touching a customer, or if it's writing to your live database without human in the loop, 80% is a fail grade. The path that it handles automatically must be 100% certain.
Automate less of the flow, but automate it completely
Many founders want to automate the entire customer lifecycle with AI. They want an AI that will get a lead, qualify them, write an email, overcome objections, negotiate pricing, and generate a contract.
In practice, they wind up building a Frankenstein's monster of AI that will do this sometimes, but requires the founder to spend hours of every day babysitting their system, correcting its mistakes. You haven't automated your sales pipeline. You've built a full-time babysitting job for yourself.
Instead, automate the things you can automate completely, and save the judgment for humans. In our own sales pipeline we use automated AI systems for very specific things: parsing a messy form into structured data, augmenting data from a form with data from other places, checking if the lead meets our qualification criteria, and routing the lead.
When a lead does meet our qualification criteria, we send them a link directly to our calendars so they can book a time with us right away. This way, customers get an answer to their inquiry in 30 seconds instead of 2 days, and our system works 100% of the time.
Don't try to make your AI system actually have a sales call with the customer, negotiate prices, or draft service agreements. If someone says they've built an AI agent that autonomously does all of that for you, and is replacing your sales team, then they're selling snake oil. Selling is a human process. You can use automated AI systems to make the process easier for your team, but let your team do the actual sales job.
The 10-minute boundary test
When someone tells you that their automated AI system is "90% ready", what they usually mean is that they have not defined any boundaries. They are still adding more and more instructions to the prompt, and adding conditional branches to handle all the edge cases that happen twice a year.
Take a sheet of paper. Take 10 minutes and write down the conditions needed for the AI system to actually create business value. What are the 2 or 3 clear conditions that cover the majority of the use cases? Everything else, have a hard fallback. An alert in Slack, an email to an ops manager, a review ticket, anything.
I audited a project for a client where their automated AI system was trying to classify and route customer support tickets to the right category. There were 18 different (and complex) categories. Their automated AI system was launched 5 weeks later, because it kept getting confused between 2 rare types of international billing inquiries.
When we looked at their data, we found that 2 categories alone accounted for 82% of their support tickets. The other 16 categories were just edge cases that rarely occurred.
We re-wrote the automated AI system to only handle those 2 categories. If a support ticket did not clearly match one of those 2 categories, we set the AI system to route it to a human operator for handling.
That AI system with only 2 categories went into production within 48 hours. The company was able to automate over 4/5ths of their support volume overnight, and was able to manually handle the edge cases in 20 minutes a day. This is how you deploy automated AI that actually makes money.
Strip the complexity
A software company approached us with 150 automated scenarios that made their business work. They had AI prompts in dozens of the scenarios and no one understood how everything was tied together. When a change was made to one scenario, it caused two other customer facing scenarios to fail. They had a maintenance bug that took them two weeks to figure out.
We audited their business requirements and rebuilt the architecture. We restructured their scenarios into 18 scenarios, each isolated and with guardrails around all of the model calls. We were able to reduce their system by 88% while having their business work reliably. Fewer errors occurred and they stopped spending their Fridays fixing problems in their automated scenarios.
A complex set of automated AI scenarios isn't a sophisticated system. It's simply technical debt.
The question to ask before implementing an AI model in a workflow is: Who is responsible to catch the error if the system fails at 2:00 AM?
If it's you or your team that is responsible, release the system at 80% and work on it as you learn. If it's your customer that is responsible, reduce the scope, route all exceptions to a person, and prove out the automated AI system on small, verified tasks. Only increase the scope of the system once it has earned it through data.
Build a calmer business,Yuri