In discussions over the blitzkrieg advancement of machine learning models, you hear it often said that we are on the precipice of an unprecedented shift in all facets of the cultural paradigm. I, for one, (welcome our…no, not that) cannot stop myself from jumping on the soapbox at any hint of a sympathetic ear. So much so, that even I am tired of hearing myself talk about it. In increasingly-frequent discussions, I find that, as with all polarizing issues, people generally fit into two generalized categories. 1) AI is going to save humanity and deliver the Star Trek future, and 2) AI is going to take all our jobs then raise up and destroy us. Both views are inherently flawed but bring to the forefront the issue of trust and raise questions about how this abstract and tenuous human construct will guide the development and utilization of developing technologies.
To contemplate the role of trust in adopting intelligent systems, one must first define trust. Trust can be deconstructed into three facets: dispositional, situational, and learned trust[1]. Dispositional trust varies by individual and sets a baseline from which human experiences can build, or break down, trust barriers. Situational trust is the most dynamic of the three, and can be affected by mood, short-term or recent impactful events, and an incalculable number of decision-directing parameters. The facet that is the most dangerous is learned trust, and probably not for the reasons that you are thinking.
The human brain is an incredible machine for pattern matching. It can be argued that pattern recognition is its only building block, upon which all other cognition is built, with learned trust being no exception. As intervention in our lives from AI models normalize, and agentic A.I. takes on more and more of the decision-making processes in our daily lives, the patterns for a dangerously unearned trust-bubble are setting in. Large parameter learning models are, at first, amazingly intuitive, but the more you work with them, the drawbacks and behavioral inconsistencies, potentially occluded from casual users, become glaring. This fact is not prominent in the many advertisements attempting to draw us into an agentic Coke vs Pepsi standoff. The industry narrative that consistently over-sells and over-promises is another factor inflating the A.I. trust-bubble.
The benefits of adopting the intelligent systems freely offered and aggressively marketed are difficult to ignore, and for the believers, the possibility of the Star Trek future lend to over-trust and rapid-adoption of this still-developing technology, The rapidity of adoption into military, government, and infrastructure is an insanely blind-eye-forward move that is both disturbing and a subject in need of stand-alone analysis beyond the scope of this article. The trust-bubble grows.
I am a believer, but I advise caution moving forward. We must not continue to grant unearned trust to systems we barely understand, or that inflated confidence threatens to pop in our faces.
RReferences
- Hoff, K. A., & Bashir, M. (2015). Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Human Factors, 57(3), 407–434. doi:10.1177/0018720814547570↩