Companies have spent the past two years teaching employees how to use AI. Most of that training has focused on the basics, such as prompting skills. It has helped people become comfortable with AI and begin to see its value. But that level of interaction is only the beginning, and it will not hold up as AI systems become more autonomous.
The newest generation of AI systems is not simply waiting for instructions. Agents now pursue goals, call tools, make recommendations, trigger actions, and hand work from one system to another. The human role changes with them: the employee is no longer simply a user but the person responsible for directing and overseeing the agent. As Gianpaolo Barozzi, Cisco’s 3P CTO, explains, “Agentic AI changes the relationship between people and technology. It requires new ways of setting boundaries, calibrating trust, and maintaining human accountability.”
Most companies are not training for that. They are teaching people how to get more out of AI. They are not teaching people how to lead it. This is becoming one of the most important capability gaps in the next phase of enterprise AI. Without that management discipline, employees may grant agents more trust and autonomy than the work or the technology warrants.
The risk is not that agents will be obviously bad. The greater risk is that they will be useful, fast, fluent, and confident enough that people begin to relax their judgment at exactly the moment they need to sharpen it. In a 2025 MIT Sloan Management Review and BCG study, 76% of executives said they viewed agentic AI more as a coworker than as a tool. That language is directionally right. Agents will increasingly function like teammates. They will participate in work, shape decisions, coordinate tasks, and take on pieces of execution that used to belong only to people.
But “teammate” is not a single relationship. Once an agent begins participating in the work, people need to understand what role it is playing and manage it accordingly. In our research, people frequently approached agents through one of five mental models: tool, intern, service provider, teammate, or expert. Each model creates different expectations about competence, autonomy, trust, supervision, and accountability.
When an agent is treated like a tool, the human expects it to perform a bounded task on command. The person operates the system, evaluates the output, and remains responsible for the result. This works for narrow, repeatable tasks but becomes insufficient as the agent gains autonomy. An example of AI being used as a tool would include an employee using an agent to summarize a meeting transcript or reformat data into a standard report.
When an agent is treated like an intern, the human provides context, inspects the work closely, corrects mistakes, and gradually expands its responsibilities. The agent may be capable, but it is still learning the organization, the work, and the standards required to perform well. For example, a manager might ask an agent to draft a client briefing, then review the work closely and coach it on what the organization considers important.
Wharton professor Ethan Mollick has used the “AI intern” analogy to describe this relationship. Like a new employee, the AI needs a defined role, sufficient context, clear assignments, and ongoing evaluation of where it is, and is not, reliable. It can produce first drafts, conduct initial research, analyze data, or prepare a briefing, but the human must inspect the work and provide the judgment the agent lacks.
When an agent is treated like a service provider, the human defines the desired outcome, scope of work, deliverables, performance standards, decision rights, constraints, and escalation requirements. The agent is given discretion over how to execute the work within those agreed parameters, while the organization maintains appropriate visibility, review points, and control. For example, a procurement agent might manage an end-to-end sourcing process against defined cost, quality, risk, and compliance targets, while escalating exceptions or consequential decisions for human approval.
As Hala Jalwan, co-founder and CEO of Rivio.ai, a start-up that builds such procurement agents designed to operate as service providers by taking on work traditionally handled by outsourced and offshore teams, explains, “When an agent takes responsibility for a body of work, the human role becomes more managerial, not less important. Someone still has to set the objective, define the boundaries, determine when the agent should escalate, and remain accountable for the outcome.”
When an agent operates like a teammate or peer, the relationship is collaborative and ongoing. The agent helps develop ideas, coordinates work, responds to feedback, and participates in shared problem-solving. Its value comes not only from what it knows, but from how it works alongside others. Even so, the human remains accountable for the outcome. For example, an agent might participate throughout a product launch by developing options, tracking decisions, coordinating follow-ups, and adapting as the team’s direction changes.
BNY offers an enterprise example of what it takes to support this kind of human-to-agent relationship. As BNY CIO and Global Head of Engineering Leigh-Ann Russell explains, “We launched our first digital employee in 2025 with the same approach as our enterprise-wide systems—governance was built in from the start. We understood that, over time, digital agents and people would work side by side. Digital employees are onboarded, governed, and continuously monitored with the same rigor we expect across our technology estate,, while accountability always remains with our people.” The example illustrates that treating an agent as a colleague does not mean treating it as an equal bearer of responsibility. It requires clear oversight, performance expectations, and human ownership of the outcome.
When an agent is treated like an expert, the relationship is more consultative. The human turns to it for specialized knowledge, analysis, or recommendations that may exceed their own capabilities. The primary risk is deference: because the agent appears authoritative, people may be less likely to question its conclusions. Its expertise should inform the decision, not own it. For example, a leader might ask an agent to analyze a complex dataset and recommend where the business faces the greatest operational risk.
Research at Procter & Gamble illustrates how AI can play this expert role within a broader collaborative relationship. In a field experiment involving professionals working on product innovation challenges, AI helped employees generate solutions that extended beyond their own areas of specialization. Commercial professionals incorporated more technical thinking, while research and development professionals incorporated more commercial thinking. Less experienced participants were also able to perform more like teammates with deeper product-development expertise. In this setting, AI acted as a source of specialized knowledge that broadened the human’s perspective but the employee still had to assess whether its recommendations fit the business context.
Each model can be useful, and each creates different risks when applied carelessly. A person who sees an agent as an expert may fail to question it. Someone who views it as a teammate may assume it shares organizational context or responsibility. Someone who treats it as a service provider may delegate an outcome without maintaining sufficient visibility. And someone who treats it like an intern may waste time micromanaging work it can already perform reliably.
Nor should an agent be placed permanently into one category. The same system may function like an expert when analyzing a large dataset, a service provider when executing a defined procurement workflow, a teammate when helping a group solve a problem, and an intern when navigating a new or ambiguous situation. The appropriate model depends on the task, context, risk, and the agent’s demonstrated reliability. Calling an agent a teammate may recognize that it is participating in the work rather than merely producing outputs, but it does not tell us how much authority to grant it, how closely to supervise it, or how much confidence to place in its judgment.
This is why companies need to build human-agent fluency: the ability to recognize which relationship the work requires and manage the agent accordingly. That is very different from prompt engineering. Prompt engineering asks, “How do I get a better answer from the machine?” Human-agent fluency asks, “What role should this agent play, and what does that role require of me?” As Tom Lamberty, Senior Consultant in Cisco’s 3P Tech Office, puts it, “How we define an agent’s role shapes how we work with it: how much authority we give it, how closely we question it, and where we retain judgment. We shape our agents, and then our agents shape us.”
That is also the better lesson from recent research on agent framing. Research by Boston University’s Emma Wiles found that people caught 18% fewer errors when work was described as coming from an agentic “AI employee” rather than a chatbot. The finding should not lead companies to abandon teammate language altogether. It should lead them to take that language more seriously. When agents function like teammates, they require clear standards, challenge, escalation, and ownership. A teammate relationship without appropriate oversight is not teaming. It is overtrust.
Organizations should therefore make the expectations associated with each model explicit: what the agent may decide, when it may act independently, how its work will be reviewed, and what would justify changing the relationship. Employees must also learn to recognize when they are granting an agent too much, or too little, trust.
That is the difference between AI usage and agentic readiness. A workforce is not ready simply because employees use AI frequently. It is ready when people can identify the relationship the work requires, manage it appropriately, and recalibrate it as the task, risk, and agent capability evolve. Across every model, human accountability for the outcome remains constant.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
This story was originally featured on Fortune.com