Beyond the Obvious

“Moving through unnecessary complications with clear, direct and flexible thinking. Seeing with new eyes, looking beyond the obvious, evolving established ways of thinking, reappraising old assumptions, finding new answers, harnessing potential."


AI Isn’t Misaligned … It May Be Perfectly Aligned with Our Imperfections.

AI Reflects Humanity’s Shattered Mirror

As one does I flicked on the radio a few nights ago heading home from th egym and BBC Radio 4 had a professor from the University of California waxing lyrically on the topic of ‘Alignment’ in the context of AI. This is a well exercised subject that tries to answer the question of ‘how do we make an AI pursue goals that are aligned with human goals and values?’. The presumption being such ‘alignment’ would mitigate rouge AI behaviour. Oh the blissful innocence in such a belief.

The question may sound like a plausible approach to governing AI behaviour and mitigating the doom scenarios that abound, but the kicker is this assumes human goals and values constitute a sufficiently coherent and observable target to which a system can actually be aligned, a questionable assumption in itself.

As I stated in an earlier piece ‘AI Won’t Take Over the World, It May Just Destabilise It’, the model remains a compressed representation of reality rather than reality itself. Applying this to the alignment question, an AI model does not encounter humanity or the physical world directly, it encounters representations of them, notably training data, reward signals, simulations, tool outputs, human feedback, policies and subsequently whatever observations are available to an agent at runtime. So there are several successive compressions as you flow through these inputs from real world through human observation, recorded information, the selected training corpus, model representation, alignment intervention, model behaviour to finally reach real-world consequences. At every boundary/step information is lost, selected or transformed. That means alignment is being attempted against a cropped representation of humanity, rather than humanity itself.

When I use the term compression I do not confine this merely to the suppress phenomena, it can disproportionately amplify them. If a behaviour, ideology, linguistic convention or social pattern is overrepresented in the corpus or reinforced through training and evaluation, for example, it can become disproportionately influential within the resulting model.

As I see it so far, there are at least three different sources of values in such a system. There is the humanity as represented by its recorded behaviour; the choices of those constructing and curating the model and the subsequent alignment choices of those deciding what behaviour should be rewarded, discouraged or prohibited. That makes the concept of human alignment an unusually slippery concept because humanity itself cannot be said to be aligned.

Remember the training corpus contains both our values and our violations of them. This produces the paradox that human generated information contains extraordinary quantities of evidence about cooperation, compassion, fairness and morality. Whilst the same corpus contains war, fraud, coercion, racism, manipulation, revenge, deception, exploitation and almost every other behaviour humans regard as undesirable.

This may all sound aligned until reality supplies competing objectives and the model enters a world of value collisions. Humans resolve these inconsistently through law, ethics, professional and moral judgement, culture, precedent, context and when all else fails improvisation. I am unaware of a latent universal answer sitting inside human knowledge waiting for a sufficiently intelligent model to discover it.

Consequently, asking AI to learn what humans believe is right may produce something quite different from asking it to behave in ways humans have decided are permissible. The second is arguably much more tractable.

This in my interpretation leaves us with the current state of AI that possess enormous knowledge of human social interaction while lacking the embodied experience from which much human intuition originates. It may understand millions of descriptions of grief without grieving, analyse fear without fearing, or model physical danger without ever having experienced pain.

Alignment then becomes not merely the challenge of teaching machines our values, but recognising that we are asking a bounded mathematical representation of an inconsistent species to infer stable rules from evidence demonstrating that humans routinely violate the very rules we say we value.

That dear reader may mean the ultimate safety mechanism being pursued or sold as alignment is not perfect alignment at all. It is accepting that alignment will always be imperfect and engineering limits on authority, reversibility, human intervention and accountability around that imperfection.

My conclusion so far from my musing on this is that AI is perhaps not fundamentally misaligned with humanity, It may be extraordinarily well aligned with the imperfect representation of humanity we have given it.


Leave a Reply