Does Zuckerberg’s AI Vision Have a Blind Spot?

Mark Zuckerberg’s latest essay on Meta’s AI strategy deserves more attention than a typical technology announcement. His argument is not simply that AI should become cheaper, more capable and more widely available. It is that powerful AI should be distributed to billions of individuals as a deliberate counterweight to the power of governments, corporations and other institutions. That is a much bigger idea.

For centuries, institutions have held an advantage because they could concentrate expertise, information and resources in ways individuals could not. Highly capable personal AI begins to erode that asymmetry. Imagine a citizen approaching a tax authority with an AI adviser, an employee challenging a corporate decision with an AI employment specialist or a small business accessing capabilities previously affordable only to major corporations. The significance is not simply productivity. It is a redistribution of capability.

In my reading of his thesis however, there is an obvious paradox. Meta proposes to decentralise institutional power through infrastructure it provides, if personal AI becomes deeply integrated into our finances, health, relationships, work and decisions, questions of privacy, control, portability and dependency become as important as the intelligence itself.

There is also a broader consequence. If highly capable intelligence becomes effectively free, intelligence itself stops being scarce. The premium shifts elsewhere, knowledge becomes less valuable than knowing what questions to ask. Analysis becomes less valuable than deciding whether to trust it. Recommendations become less valuable than judgment and judgment remains inseparable from accountability for consequences as I wrote about earlier – The Expanding Reach, the Shrinking Judgement.

That has profound implications for employers, professional services firms and universities. Much of today’s education and professional development model is built around acquiring knowledge, producing analysis and learning through junior work. AI increasingly performs all three.

However, Zuckerberg’s vision contains, in my view, a significant blind spot. Giving people access to extraordinary intelligence does not automatically give them the judgment to use it.

AI can produce an extraordinarily sophisticated answer to a badly framed problem. Users may omit critical context, embed false assumptions in a question or simply lack enough subject knowledge to recognise that an apparently authoritative answer is wrong. The danger is therefore not simply AI hallucination. It is human inability to recognise flawed AI reasoning.

As AI becomes more capable, persuasive and personalised, that risk will increase. People naturally defer to apparent expertise. A system that communicates confidently, remembers our history and understands our preferences could acquire enormous influence over decisions involving health, money, employment, politics and relationships.

Then we have the matter of Open Models, they are a double edged sword. Openness can promote innovation and scrutiny but models can also be modified, safeguards removed and behaviours manipulated. Most users will have little ability to determine whether the intelligence advising them is unbiased, authentic or subtly designed to influence them.

That leads to the uncomfortable question of accountability. When an AI informed decision causes harm, who is responsible? ‘The AI recommended it’ cannot become an acceptable defence for a doctor, director, engineer, public official or citizen.

We have arguably passed an AI event horizon where society is now being reshaped at machine speed and in compressed time. Basic AI literacy is no longer enough. We need AI judgment literacy, the ability to frame questions properly, challenge assumptions, evaluate evidence, recognise uncertainty, detect bias and ultimately accept responsibility for decisions. Otherwise we risk replacing one dependency with another as citizens become less dependent on institutions but more dependent on algorithms they are poorly equipped to question or trust with legitimacy as I shared thoughts on at the weekend – Are Universities Educating Students for a World That Is Disappearing?

Zuckerberg may be right that AI could remove the historic intelligence advantage institutions have held over individuals but the greater challenge of democratising AI for me may not be giving everyone access to intelligence, it may be ensuring that billions of people are educated enough to avoid digital enslavement/dependency, to disagree with it.

The Greeks warned us repeatedly about giving humans godlike powers before they had acquired godlike wisdom. AI may simply be the first time we have attempted the experiment at global scale. Recall Prometheus who stole fire from the gods and gave it to humanity. Zuckerberg’s proposition has resonance as it is to essentially hand everyone a flamethrower for free. Democratising power is admirable; checking whether people know which end to point away from themselves and each other might also be prudent.