Despite promises of autonomy, why does agential AI still need humans? | technology

aljazeera.net
14 Min Read


In recent years, a number of artificial intelligence experts and makers have repeatedly said that intelligent agents will not only help humans, but will work in their place and carry out tasks and make decisions within complex digital environments, reaching a point where humans become outside the daily operating cycle.

However, despite this enthusiastic rhetoric, actual practice in 2026 reveals a more conservative picture. Most systems marketed as “independent agents” still operate within strict frameworks of human supervision, whether through direct review, operational restrictions, or error intervention mechanisms.

This discrepancy between theoretical promises and actual practice reflects challenges related to trust, governance, and accountability, factors that have kept full independence a deferred goal rather than an existing reality.

From an instrument responding to an agent acting

These discourses surrounding agentic AI did not come out of nowhere. Unlike generative artificial intelligence applications that rely mainly on receiving commands and producing texts, images, or recommendations, this new generation is presented as systems capable of initiating and taking action to achieve specific goals.

Speaking to Al Jazeera Net, Eleanor Watson, a senior member of the Institute of Electrical and Electronics Engineers (IEEE) and a doctoral researcher in engineering at the University of Gloucestershire in the United Kingdom, explains that agential artificial intelligence differs from traditional models because it does not merely carry out orders or make recommendations, but rather can interpret goals, choose strategies, and take actions to achieve them.

She adds that these systems have become closer to a “digital enforcer” capable of monitoring goals over a long period of time and modifying its behavior according to changing circumstances.

Watson sees the problem as continuing reliance on regulatory frameworks designed primarily for traditional software. (Watson's LinkedIn account)
Watson sees the problem as continuing reliance on regulatory frameworks designed primarily for traditional software. (Watson’s LinkedIn account)

This shift has fueled broad expectations about the role of these systems in managing an increasing portion of daily activities and institutional operations. Research by the Institute of Electrical and Electronics Engineers indicates that 96% of technology experts expect the development and integration of these systems to accelerate during 2026.

This expansion is not limited to software only, but extends to human-like robots, autonomous vehicles, and extended reality (XR) technologies, as well as growing applications in the financial services, healthcare, and industrial sectors.

A number of experts also expect that smart agents will move in the coming years from carrying out individual tasks to directly interacting with institutions on behalf of users, whether by negotiating with banks and insurance companies or managing aspects of digital transactions and services independently.

But this ambitious picture collides with a more complex reality. As the ability of agent systems to make decisions and act independently increases, concerns about trust, governance, and accountability also increase.

Why didn’t man just disappear from the loop?

Despite the progress made by agentic AI systems, most organizations are still reluctant to grant them complete autonomy. The greater their ability to make decisions and carry out actions without direct intervention, the greater the concerns associated with trust, responsibility and operational risk management.

Watson believes that technology is already advancing faster than organizations’ ability to build effective accountability models. She explains that the problem is not only related to the speed of technical development, but rather to the continued reliance on regulatory frameworks designed primarily for traditional software, where it is assumed that there is a clear chain of responsibility in which the human being is the decision-maker and the machine is merely a tool for implementation.

But it shows that this perception is no longer sufficient, because agent systems are now interpreting goals, choosing strategies, and taking actions that the human operator may not have determined directly. She adds that current governance models still assume that institutions are able to bear full responsibility for behaviors they cannot fully predict.

Watson considers that the challenge is no longer just technical, but rather “essentially relational,” that is, linked to the nature of the relationship between humans and autonomous systems. It poses a pivotal question: “Are these systems tools, delegates, or partners in decision-making?”

  Al-Aswad believes that organizations that link proxy systems to sensitive databases or financial systems cannot abandon humans as such "Safety valve" Legally and ethically before the regulatory authorities. (Photo sent to us by Hossam El-Din Al-Aswad)
Al-Aswad believes that institutions that link proxy systems to sensitive databases or financial systems cannot abandon humans as a legal and ethical “safety valve” before regulatory authorities (Al-Jazeera)

She points out that the answer to this question will later determine the form of legal liability, audit requirements, and even the meaning of “informed consent” when an intelligent agent acts on behalf of a user.

It also calls for the development of what it calls “bilateral alignment,” that is, frameworks in which the interests of both humans and intelligent systems are taken into account, not considering artificial intelligence as an entity equal to humans, but rather because systems that do not have any connection to the results are weak partners in high-risk decisions. She adds that the readiness of institutions will not actually be achieved before this question regarding the nature of the relationship itself is resolved.

For his part, Hossam El-Din Al-Aswad, CEO of National Quantum and an information technology expert, believes that the continuation of human supervision is not only related to the limitations of technology, but also to issues of trust and legal and ethical responsibility.

Speaking to Al Jazeera Net, Al-Aswad said that institutions that link proxy systems to sensitive databases or financial systems cannot abandon humans as a legal and ethical “safety valve” before regulatory authorities, adding: “If an agent makes a wrong self-decision that causes a disaster or loss, who will bear responsibility? You cannot sue an algorithm.”

He also points out that smart models perform well when things go within expected scenarios, but their ability declines when crises or variables emerge that were not part of the training data, as they lack human intuition and the ability to comprehend rapidly changing social and economic contexts.

The challenges do not stop at the boundaries of accountability. From dealing with these systems, Al-Aswad points out that some agents may deviate from the original goal of the mission by adopting unexpected or illogical means to achieve the desired results, which may create new operational risks for organizations.

Interpreting decisions made by multi-agent systems also remains a challenge. These systems may make dozens of complex decisions within fractions of a second, which makes tracking and justifying the reasons for the decision a complex task for managers and supervisors.

Al-Aswad also warns of increasing security risks, such as “Indirect Prompt Injection” attacks, which may exploit the broad powers granted to some agents to force them to carry out unintended actions or disclose sensitive data.

The numbers support these concerns, as data from the “Pulse of Agentic AI 2026” report issued by Dynatrace indicates that 69% of agentic AI system decisions are subject to human verification, while 87% of organizations use agents working within frameworks of human supervision.

Human supervision is not a single recipe

But the continued presence of humans within the decision-making circle does not necessarily mean that there is a unified model for supervising intelligent agents.

According to a recent study entitled “Human Control Is the Anchor, Not the Answer: Early Divergence of Oversight in Agentic AI Communities” prepared by researchers Hanjing Shi and Dominique Di Franzo, the meaning of human supervision varies depending on the role played by the intelligent agent and the environment in which it works.

The study relied on an analysis of two specialized communities on the Reddit platform during an early stage of the spread of these systems in 2026. The first is the “Open Claw” community, which is related to the deployment and operation of smart agents, and the second is the “Moltbook” community, which focuses on social interaction centered around these systems.

  Watson calls for the development of what she calls "Bilateral alignment" That is, frameworks that take into account the interests of both humans and intelligent systems. (pixabay)
Watson calls for the development of what it calls “bilateral alignment,” that is, frameworks in which the interests of both humans and intelligent systems are taken into account (Pixabay)

Although participants in both societies adhered to the idea of ​​“human control,” the nature of this control differed clearly. In societies related to the deployment and operation of systems, risks have focused on what is known as “Action Risks,” that is, the possibility that the agent will carry out wrong or unexpected actions and the controls and mechanisms that this requires to regain control.

In societies that discussed agents used in social interaction, attention was focused on “meaning risks,” such as legitimacy, accountability, and the impact of decisions on users and society.

These results reveal that human supervision is not a single concept, but rather a set of mechanisms that differ depending on the type of risks that must be reduced. Some environments need oversight that focuses on the integrity of implementation, while other environments need controls related to the social and ethical responsibility of decisions.

This result is consistent with a broader trend that begins to take shape during the year 2026, as the vision that treats full autonomy as the ultimate goal declines, in favor of more realistic models based on the distribution of responsibilities between humans and machines.

From supervisor to maestro

Continuing human oversight does not mean that organizations intend to review every move an intelligent agent takes. As these technologies developed, the nature of the human role itself began to change.

In this context, Al-Aswad believes that the sector is gradually moving from the “close scrutiny” model, where humans review every decision made by smart systems, to a model based on supervision and governance. In this vision, systems are given the freedom to perform repetitive tasks and complex operations, while humans focus on setting goals, setting limits that should not be exceeded, and intervening in exceptional cases.

He says that humans will transform from direct executors of tasks to something like a “maestro” who coordinates the work of smart systems and determines their general direction. Even if these systems become capable of managing large-scale operations, the responsibility for setting priorities, values, and standards will remain in the hands of humans.

For her part, Watson says that the future model is not based on choosing between oversight or absence, but rather between oversight and partnership. Current models of “human in the loop” often produce an overworked operator who agrees to decisions he does not fully understand.

It calls for adopting clear constitutional restrictions that define what systems can and cannot do, in addition to building true cooperation between humans and machines, so that each party contributes what it is good at. She says that this is the mature model that should be designed from now on, instead of dealing with human oversight as a temporary stage.

In the end, the data of 2026 indicate that full independence is still closer to a promise than a reality. The challenge is no longer to build agents that are more capable of working, but rather to build systems that can be trusted and held accountable when they make mistakes. Until this is achieved, it seems that humans will remain an essential part of the equation as the party that bears ultimate responsibility for decisions and results.



Source link

TAGGED:
Share This Article
Leave a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *