“A man who carries a cat by the tail learns something he can learn in no other way,” said Mark Twain. Discovering hard truths through hands-on experience is how everyone is learning AI. That includes this CTO of a Fortune 500 solutions integrator. So take it from me: You’re likely repeating the sins of our past.
How do I know? Confession time. We didn’t used to have a global contract with the frontier model providers. We do now, but back then, we had a bunch of well-meaning teammates swiping their corporate credit cards. For $20 a month, they got to do whatever they wanted. But once we looked closely at it, the realization came. We were spending a lot of money, and $20 a month for some turned into $100 to $200 for others. One agreement with all the model providers turned countless expense reports into a single line item that someone owns.Â
We fell into the next trap, not understanding the true cost of tokens. We had developers who built tools and workflows, not thinking beyond the point of innovation to the cost. Three words no CTO wants to see in print sum it up: Mistakes were made.
And how could it have been otherwise? Everyone in the C-suite has the same thought: Let’s go build AI teams everywhere! It’s a healthy instinct, and we’ve acted on it as much as anybody else. We have AI teams in partner management; we have AI teams in finance; you name the functional part of our business, we have AI teams. Each team is picking its own favorite frontier model, tools—their own everything. Same tune, different song.
THE LAST TIME THIS HAPPENED
In 2009, the software development world went through a major shift that evolved into what we know as DevOps. Instead of having enormous teams working on projects, you created smaller teams. There’s an old axiom from Jeff Bezos that a team shouldn’t be bigger than what two pizzas would feed.
Engineering leaders said, “Let’s try this DevOps thing.” They assigned five, six, seven of their best people to a DevOps project and it usually worked well. They deployed more and more DevOps teams and were shipping features and capabilities much faster.
But Murphy’s law remained undefeated, and before long, a few ugly warts appeared. Each DevOps team started finding its own tools. One project would use a programming language, a framework, and a persistence layer. Another would use a completely different set of tools. And like a pile of clothes hangers, it all culminated in a big, tangled mess. The industry came to two realizations: We need to put some controls on all of this, and we’re building and compounding technical debt faster than we’re adding value to the business.
What ultimately happened? Platform engineering teams built internal developer platforms. They said, “Here’s the golden path. If you stay on the golden path, we’ll support you. If you stray from it, we’re still here, but you’ll have to find your own way.” This approach worked so well that in 2023, Gartner predicted 80% of software engineering organizations would have platform teams by 2026.
Meanwhile, the cloud came to prominence, in part because of this DevOps movement. They fed on each other. It seemed cheap because you could spin up a server so quickly or consume an API. Then come to find out that if you’re not careful, it can cost you an arm and a leg, and likely your head. So another discipline came into being called FinOps—financial operations.
Part of it was making sure that you understood your bill from cloud providers. Part of it was expressed most often as a concern: Maybe we shouldn’t be using the most expensive technology for everything we do. Developers always take the shortest path. If a button will solve their problem, they’re pushing that button. And typically, the fastest, most expensive button is going to do it faster and better. So they just pushed that button over and over again. FinOps put a meter on the button and made someone answerable for the number.
That problem is still being solved. Which is exactly when AI showed up on 337 S&P 500 earnings calls in a single quarter, the most in a decade.
SAME TUNE, DIFFERENT SONG
Now we’re coming to similar realizations, especially with tokenomics, which is a FinOps conversation. We’re hearing CIOs and CFOs say, “Perhaps we should start consolidating the number of tools we’re using.” Starting to sound familiar, yes?
The fix should sound familiar too. Customers come to us with a bunch of plans. In the past, they worked on all of them independently. That’s the sprawl. My recommendation: Find the most important plan and build that one first. Use it as a foundation for the next plan, and the next one. We built platform engineering to do this for developers. We need to do it again for the AI side of the house.
Specifically, the first project defines a small set of capabilities that solve for that problem. Then the second project builds on what you learned from the first one and adds more. And then as you add the third, fourth, and fifth, you start creating a flywheel with this capability already built in.
You’re creating an accelerating factor instead of what we had: five teams, five frontier models, five sets of tools, five angry emails from the CFO. That’s a very expensive pile of clothes hangers to untangle. Five separate stacks become technical debt that compounds faster than you can add capability. Somebody has to be the expert in all five. The goal is minimizing the blast radius of what you implement.
Twain was right. Some lessons can only be learned through painful experiences. But with the scar tissue we accumulated early, the most valuable lesson is this: Let someone else carry the cat.
Juan Orlandini is chief technology officer of North America for Insight Enterprises.
Â
