6. Treat AI systems more like workers than software
Some enterprises still govern AI like traditional applications. But according to Kale, AI systems behave more like workers and less like deterministic software.
“You cannot just deploy once and be done,” he says. “Like workers, they need ongoing oversight.”
That ongoing oversight is becoming a core accountability function. Employees are not hired, trained, and then left unsupervised indefinitely. Managers monitor performance, provide feedback, evaluate changing responsibilities, and intervene when behavior drifts from expectations. Kale argues that AI systems increasingly require similar treatment.
Traditional software can often be reviewed and approved at release time because its behavior remains relatively stable between versions. AI systems are different. Models evolve, prompts change, retrieval systems are updated, and the information available to agents changes continuously.
That challenge extends beyond internally developed systems. Enterprises must also monitor the third-party AI services they rely on. Not only do vendor models evolve on their own, but vendors also update software and capabilities behind the scenes.
“The vendor we approved last quarter is functionally a different vendor this quarter,” Kale says.
As a result, accountability cannot end when a system is deployed. Someone must remain responsible for monitoring performance, reviewing changes, assessing risk, and determining whether systems continue to operate within acceptable boundaries. Kale points to CoSAI’s AI Shared Responsibility Framework as one emerging effort to clarify those responsibilities across enterprises, software vendors, model providers, and infrastructure operators.
The organizations making the most progress are discovering that accountability cannot be assigned on paper and forgotten. As AI systems become more autonomous, accountability is becoming an operational capability built into data governance, observability, escalation processes, and ongoing oversight. For IT leaders, the challenge is no longer defining responsibility. It is making responsibility enforceable.
Related reading: