AI and Incompetence a Dangerous Combination

In recent times, new technologies such as digitization, process automation, artificial intelligence, robotics, virtual and augmented reality, and others. have acquired particular relevance and will surely continue to make greater leaps in the coming years (or months), at a greater speed, as a result of the speed with which we are breaking down the physical barriers of our computing capabilities.

This dizzying technological evolution raises questions about how prepared we are as a society to adapt to these sudden changes with such a profound impact on the way we are used to living and interacting in our world.

Much has been said about the ever-widening gap between each technological leap and our ability to respond from a regulatory perspective in a way that reduces the risks that misuse or uncontrolled use of technology can cause (many of us remember the freedom and unbridled freedom that was the creation of the internet or, later, the streaming and file-sharing platforms in their early days, to cite an example).

However, and speaking specifically about artificial intelligence (which is possibly the pinnacle and transition between the 4th and 5th Industrial Revolutions) there is another factor that we may not have paid much attention to, but which I would like to point out: the risk posed by the use of artificial intelligence by those who do not have the necessary skills.

Artificial intelligence is a powerful tool that is quickly changing our way of life. It has been integrated into everything, from our computers, cellphones, wearables, and communications devices to cars, humanoid robots, and the analysis of large volumes of information.

But in many cases, we have embraced their adoption wholeheartedly without taking any precautions, which poses risks. On the one hand, these are still emerging technologies; on the other hand, we still lack the appropriate regulatory and control frameworks to prevent misuse or abuse of these technologies. Also, we have not stopped to consider the appropriate characteristics and profiles for the personnel in charge of the operation, maintenance, and control of these artificial intelligence systems.

They say that even more terrifying than artificial intelligence is natural stupidity, and this makes sense beyond the irony of the statement. Operating systems that use AI definitely require specialized skills that allow the human element to address, contextualize, refine, and improve the AI's responses and thought processes.

Aspects such as the validation of information sources, logical processes, necessary iterations, semantics and prompts, the use of heuristics, among others, require that humans still be able to oversee and refine the response level of artificial intelligence systems. Likewise, the human factor can be better equipped to identify false responses, misinterpretations, or flaws in reasoning within these systems.

It is well known that, in comparative terms, computational heuristics will not necessarily work on the same basis as human heuristics; after all, the trend of artificial intelligence is not toward becoming an imitator of humans, but rather a probability calculator, and it is in this sense that the intervention of human beings with instinct, context, and experience becomes more relevant to moderate the results of AI.

We are increasingly seeing ERP, Document Management, Laboratory Information Management, Logistics, MES, SCADA, and other systems incorporate some degree of artificial intelligence into their updates, and this trend continues to grow. Having said that, I believe that, as AI becomes increasingly integrated into computerized systems to enhance their capabilities and improve their benefits, it will be necessary to establish new job skills for the personnel involved in its use, maintenance, and control.

The lack of a clearly defined profile and the necessary skills for personnel involved in managing computerized systems that use artificial intelligence could add new risks to their use. Some of these risks may include:

• Unquestioning acceptance of AI process results

• Deficiencies in the quality of information used to feed systems

• System configuration and design without considering all aspects required by business and regulatory processes, resulting in rework and hidden costs

• Poor design of logical processes without considering additional contextual information

• Introduction of biases into processes and heuristics

• Inefficient management of system updates and changes

• Migration from manual to digital and AI-powered processes with poorly defined planning and responsibilities

• Cost overruns and rework due to the lack of a comprehensive plan

• Regulatory non-compliance regarding the integrity of critical data

• Resistance to change

Contrary to popular belief, the use of artificial intelligence not only does not encourage a lack of thought or intellectual apathy in humans, but rather requires the opposite to operate correctly. Among the skills that might be necessary for the profiles of users or personnel in contact with computerized systems powered by artificial intelligence are:

• Critical Thinking

• Analytical Thinking

• Knowledge Management

• Communication and Writing Skills

• Digital Mindset

• Proficiency in the use of computer systems

• Hardware and Software Knowledge

• Mastery of the business processes served by computerized systems

• Advanced Knowledge of Data Integrity through the design and validation of computerized systems

• User Requirements Development

• Development of Service Level Agreements with suppliers

• Systemic Thinking

Perhaps it is time to look at these aspects and reflect on them as part of our preparation for the adoption of these new technologies, since they are ultimately designed to serve humans and fortunately (for the time being) the human presence “in the loop” will still be necessary to ensure that we obtain not only the results we want, but also those we need from the use of these new technologies.

What do you think?