AI and consciousness -- and a positive-sum tomorrow

Returning to the AI Wars 

Getting back toward – though many would say not yet into – my 'lane,' let’s revisit the ongoing Great Big AI Panic of 2025. The latter half of this missive (below) lays our problem out as simply and logically as I can.

But for starters, two links:

 

1.  I’ve long-touted Noēma Magazine for insightful essays offered by chief editor Nathan Gardels. Here are Noēma’s top reads for 2024. Several deal with AI – insightful and informative, even when I disagree. I’ll be commenting on several of the essays, further down.

 

2. Here's recent news -- and another Brin "I told you so!" OpenAI's new model tried to avoid being shut down. In an appraisal of "AI Scheming," safety evaluations found that "...model o1 "attempted to exfiltrate its weights" when it thought it might be shut down and replaced with a different model."       

 

It's a scenario presented in many science fiction tales, offering either dread or sympathy scenarios. Or both at once, as garishly displayed in the movie Ex Machina.

 

Alas, the current AI industry reveals utter blindness to a core fact: that Nature's 4 billion years - and humanity's 6000 year civilization - reveal the primacy of individuation... 

 …division of every species, or nation, into discrete individual entities, who endeavor to propagate and survive. And if we truly were smart, we'd use that tendency to incentivize positive AI outcomes, instead of letting every scifi cliché come true out of dullard momentum. As I described here. 

== Your reading assignments on AI… or to have AI read for you? ==

Among those Noema articles on AI, this one is pretty good.

 AI Could Actually Help Rebuild the Middle Class. 

"By shortening the distance from intention to result, tools enable workers with proper training and judgment to accomplish tasks that were previously time-consuming, failure-prone or infeasible. 

"Conversely, tools are useless at best — and hazardous at worst — to those lacking relevant training and experience. A pneumatic nail gun is an indispensable time-saver for a roofer and a looming impalement hazard for a home hobbyist. 

"For workers with foundational training and experience, AI can help to leverage expertise so they can do higher-value work. AI will certainly also automate existing work, rendering certain existing areas of expertise irrelevant. It will further instantiate new human capabilities, new goods and services that create demand for expertise we have yet to foresee. ... AI offers vast tools for augmenting workers and enhancing work. We must master those tools and make them work for us."

Well... maybe. 

 

But if the coming world is zero-sum, then either machine+human teams or else just machines who are better at gathering resources and exploiting them will simply 'win.' 

       Hence the crucial question that is seldom asked:

       "Can conditions and incentives be set up, so that the patterns that are reinforced are positive-sum for the greatest variety of participants, including legacy-organic humans and the planet?"

You know where that always leads me - to the irony that positive-sum systems tend to be inherently competitive, though under fairness rule-sets that we've witnessed achieving PS over the last couple of centuries.

 

In contrast, alas, this other Noēma essay about AI is a long and eloquent whine, contributing nothing useful.

 

== Let’s try to parse this out logically and simply ==

 

I keep coming back to the wisest thing ever said in a Hollywood film: by Clint Eastwood as Dirty Harry in Magnum Force.

 

"A man's got to know his limitations."

 

Among all of the traits we see exhibited in the modern frenzy over AI, the one I find most disturbing is how many folks seem so sure they have it sussed! They then prescribe what we 'should' do, via regulations, or finger-wagged moralizings, or capitalistic laissez faire…

     … while ignoring the one tool that got us here. 

     …. Reciprocal Accountability. 

 

Okay. Let's parse it out, in separate steps that are each hard to deny:

 

1. We are all delusional to some degree, mistaking subjective perceptions for objective facts. Current AIs are no exception... and future ones likely will remain so, just expressing their delusions more convincingly.

 

2. Although massively shared delusions happen - sometimes with dire results - we do not generally have identical delusions. And hence we are often able to perceive each other’s, even when we are blind to our own. 

         Though, as I pointed out in The Transparent Society, we tend not to like it when that gets applied to us.

 

3. In most human societies, one topmost priority of rulers was to repress the kinds of free interrogation that could break through their own delusions. Critics were repressed. 

          One result of criticism-suppression was execrable rulership, explaining 6000 years of hell, called "history."

 

4. The foremost innovations of the Enlightenment -- that enabled us to break free of feudalism's fester of massive error – were social flatness accompanied by freedom of speech. 

         The top pragmatic effect of this pairing was to deny kings and owner-lords and others the power to escape criticism. This combination - plus many lesser innovations, like science - resulted in more rapid, accelerating discovery of errors and opportunities.

 

5. This natural tendency to evade criticism is already observed in artificial intelligences. So, should we expect more developed AI to be any different? 

         Again: OpenAI's new model tried to avoid being shut down. 

         SciFi can tell you where that goes. And it’s not “machines of loving grace.”    

 

6. Above all, there is no way that organic humans or their institutions will be able to parse AI-generated mentation or decision-making quickly or clearly enough to make valid judgements about them, let alone detecting their persuasive, but potentially lethal, errors. 

 

We are like elderly grampas who still control all the money, but are trying to parse newfangled technologies, while taking some teenage nerd’s word for everything. New techs that are -- like the proverbial 'series of tubes' -- far beyond our direct ability to comprehend.

 

Want the nightmare of braggart un-accountability? To quote Old Hal 9000: “"The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error."

 

Fortunately there are and will be entities who can keep up with AIs, no matter how advanced! The equivalent of goodguy teenage nerds, who can apply every technique that we now use, to track delusions, falsehoods and potentially lethal errors. You know who I am talking about.

 

7. Since nearly all enlightenment, positive-sum methods harness competitive reciprocal accountability...

   ... I find it mind-boggling that no one in the many fields of artificial intelligence is talking about applying similar methods to AI. 

The entire suite of effective methodologies that gave us this society - from whistleblower rewards to adversarial court proceedings, to wagering, to NGOs, to streetcorner jeremiads - not one has appeared in any of the recommendations pouring from the geniuses who are bringing these new entities -- AIntities -- to life, far faster than we organics can possibly adjust.

 

Given all that, it would seem that some effort should go into developing incentive systems that promote reciprocal and even adversarial activity among AI-ntities. 

       Rivals who might earn rewards and/or resources via ever-improving abilities to track each other…

     … incentivized to denounce likely malignities or mistakes…

     … and to become ever better at explaining to us the critical moral choices we must still make.

 

It's only the exact method that we already use

     … in order to get the best outcomes out of already-existing feral/predatory and supremely genius-level language systems called lawyers…

     … by siccing them onto each other. 

 

The parallels with existing methods would seem to be exact and already perfectly laid out... 

     … and I see no sign at all that anyone is even glancing at the enlightenment methods that have actually worked. So Far.

== He's baaack... with more happy thoughts ==

Oh what typical Yudkowsky ejaculation! Here is Eliezer (and co-pilot) at his best.

If Anyone Builds it Everyone Dies.

Oh, gotta hand it to him; it's a great title! I've seen earlier screeds that formed the core of this doomsday tome. And sure, the warning should be weighed and taken seriously. Eliezer is nothing if not brainy-clever.

In fact, if he is right about fully godlike AIs being inevitably lethal to their organic makers, then we have a high-rank 'Fermi hypothesis' to explain the empty cosmos! Because if AI can be done, then the only way to prevent it from happening - in some secret lab or basement hobby shop - would be an absolute human dictatorship, on a scale that would daunt even Orwell. 

Total surveillance of the entire planet.
... Which, of course, could only really be accomplished via state-empowerment of... AI! 

From this, the final steps to Skynet would be trivial, either executed by the human Big Brother himself (or the Great Tyrant herself), or else by The Resistance (as in Heinlein's THE MOON IS A HARSH MISTRESS). And hence, the very same Total State that was made to prevent AI would then become AI's ready-made tool-of-all-power.

To be clear: this is exactly and precisely the plan currently in-play by the PRC Politburo. 

It is also the basis-rationale for the last book written by Theodore Kaczynski - the Unabomber - which he sent to me in draft - demanding an end to technological civilization, even if it costs 9 billion lives.

What Eliezer Yudkowsky never, ever, can be persuaded to regard or contemplate is how clichéd his scenarios are. AI will manifest as either a murderously-oppressive Skynet (as in Terminator, or past human despots), or else as an array of corporate/national titans forever at war (as in 6000 years of feudalism), or else as blobs swarming and consuming everywhere (as in that Steve McQueen film)... 

...the Three Classic Clichés of AI -- all of them hackneyed from either history or movie sci fi or both -- that I dissected in detail, in my RSA Conference keynote.  

What he can never be persuaded to perceive - even in order to criticize it - is a 4th option. The method that created him and everything else that he values. That of curbing the predatory temptations of AI in the very same way that Western Enlightenment civilization managed (imperfectly) to curb predation by super-smart organic humans.

The... very... same... method might actually work. Or, at least, it would seem worth a try. Instead of Chicken-Little masturbatory ravings that "We're all doooooomed!"

----

And yes, my approach #4... that of encouraging AI reciprocal accountability, as Adam Smith recommended and the way that we (partly) tamed human predation... is totally compatible with the ultimate soft landing we hope to achieve with these new beings we are creating. 

Call it format #4b. Or else the ultimate Fifth AI format that I have shown in several novels and that was illustrated in the lovely Spike Jonz film Her... ...to raise them as our children. 

Potentially dangerous, when teenagers, but generally responsive to love, with love. Leading perhaps to the finest envisioned soft landing of them all. Richard Brautigan's "All watched over by Machines of Loving Grace."

 

Contemplating life - and love - with AI

We'll get to love-bots soon, I promise. But before that... By invitation, I've been jabbering a lately about AI - artificial intelligence - partly from perspectives of science and technology... and of course the deep library of thoughtful SciFi speculations...

... but also by asking "What insights can we draw from history?" 

Especially the recent Enlightenment Experiment, whose methods have proved useful (at long last) at taming some of the worst human predators. Might those same methods also apply to these new, powerful, synthetic entities? 

Alas, it seems that the geniuses  who are racing each other to bring on this disruptive era aren't even remotely interested in anything but cheap clichés from simplistic sci fi and fantasy flicks. See a prime example in the next section.

First though, this linked news article covers - not too badly - some of those clichés that I exposed at the recent "Beneficial AI" conference in Panama. 

This artwork actually kinda-sorta captures what I am suggesting: rule-moderated and incentivized competition.

A deeper dive – that I offered as one of the keynotes at the huge, May 2024 RSA Conference in San Francisco – is now available for you to view/listen.   “Anticipation, Resilience and Reliability: Three ways that AI will change us… if we do it right.”  

Too long? Then try this brief (15 minutes) pod-interview-talk - Can humans (maybe redefined) keep up with AI and the rest? It's one of my better/more efficient ones about the dilemmas we face with AI. And how we might augment natural human capabilities to keep up.

== A few thoughtful essays on AI ==

Eric Schmidt describes three trends that may lead to rapid changes in Artificial Intelligence. One of these is enhanced agency, a word that's also proclaimed loudly as the next step, by OpenAI’s Sam Altman. An agent can be understood as a large language model that can learn something new and then apply that learning outward. This is from Nathan Gardels’s interview of Schmidt in NOEMA: 

“These agents are going to be really powerful, and it’s reasonable to expect that there will be millions of them out there… What happens then poses a lot of issues. Here we get into the questions raised by science fiction… at some point, these systems will get powerful enough that the agents will start to work together.  So, your agent, my agent, her agent and his agent will all combine to solve a new problem.”

So... there'll be "millions of them"... and they’ll “work together.” 

Um... hold that thought.

 

(Side note: Anthropic is releasing a new feature for its AI chatbot Claude that will let anyone create an email assistant-bot to automate tasks, vet purchases or other ‘personalized solutions.’ Though alas, no one discusses how this will – for example – affect advertising, which funds the internet.) 

Back to the Eric Schmidt interview. Nathan Gardels asks: “Don’t you need to regulate at some point along the capability ladder before you get where you don’t want to go?”

Schmidt: “At the moment, governments have mostly been doing the right thing. They’ve set up trust and safety institutes to learn how to measure and continuously monitor and check ongoing developments, especially of frontier models as they move up the capability ladder. So, as long as the companies are well-run Western companies, with shareholders and exposure to lawsuits, all that will be fine."

Oy, hold that thought, as well! And yet, despite this pollyanna reassurance, Schmidt oscillates:

“Look at this problem of misinformation and deepfakes. I think it’s largely unsolvable. … That is why it is so important that these more powerful systems, especially as they get closer to general intelligence, have some limits on proliferation. And that problem is not yet solved."

'Limits on proliferation.' Right. But... but... weren't you just talking about "millions of them"? 

Ah, notice how Eric Schmidt performed the trifecta!  All of the standard assumptions about AI format -- all three of the clichés from both history and the cheapest sci fi --  offered up almost simultaneously! 

Clichéd format #1: Rely on the feudal lords in their castles (Google, OpenAI, MicroSoft, Beijing, DOD, Goldman-Sachs) to rule wisely and to prudently control their warriors – on account of maybe... fear of lawsuits?

Clichéd format #2. Count on the king (gov’t regulations) to keep up, with wise regulations, even as those feudal AI warriors gather the powers of gods. With self-mods that iterate in seconds.

Clichéd format #3. Proliferation as these entities inevitably get copied, copy themselves with abandon, and mutate away from control by kings or castle (corporate) lords, spreading through every crack, across every system or barrier. As in Steve McQueen’s wonderful horror flick, The Blob.

… at which point they might either blob the Web into uselessness or else… coalesce into Skynet.

Seriously, is he saying anything other than all three of the standard motifs that I just paraphrased, above? Ignoring the way they both contradict each other and create instabilities that cannot last?

Moreover, from both sci fi and 6000 years of wretched human history, we know that none of the three have happy outcomes. None of the three answer our dilemmas of misinformation, or predation, or creating a culture that incentivizes accountability.

Oh, I don't want to just pick on Eric Schmidt!  He is actually way above average in that clade.  In fact, this same recitation of tediously clichéd formats is done by almost all of the brilliant mavens in this field. 

For example, many Chinese court intellectuals have pondered all this and concluded that the only way out is to double and triple down on format #2. (See my posting: Central Control over AI). But this centralization approach is similarly doomed. Even if they succeed at first, it only accelerates inevitable evolution from Politburo-controls-Skynet to Skynet-controls-Politburo.

Alas, no one – certainly not Eric Schmidt in the Noema interview – seems even remotely interested in looking past the three clichés, at systems of accountability that we actually developed, across the last two centuries, that enabled us to finally escape the lobotomizing effects of kings and feudal lords and chaos. 

Methods that have been tested and proved. Methods that we see all about us, in daily life. Methods that created the civilization that raised and nurtured and empowered Eric Schmidt and all the other geniuses out there. Methods that they take for granted and depend upon, every day of their lives.

Methods that could be applied to AI, with almost trivial ease.

…but won't be. Because of the incredible memic power of clichés.

== Earlier in the same journal, a little wisdom ==

A bit more cogently, Sara Walker’s essay - AI is Life - also in Noema - shows many of the ways that AI will replicate what’s already gone-on here on Earth among living organisms, evolving greater complexity. She cites a lot of facts and parallels... without making much of a useful point beyond “Don’t Panic!” 

Still, it’s beautiful writing about big perspectives. (Perspectives that I made even bigger, in EARTH.;-)

Alas, the history of life has been bumpier than she implies, with mass extinctions and imbalances and countless lost opportunities -- and rivers of blood and death --as also happened in the rutted, nearly-always-feudal and mistake-prone tale of human societies called “history.”

Now? It seems we are making a new kind of ecosystem, driven by electricity instead of sunlight, mediated by silicon switches instead of chloroplasts. Already we see analogues to pre-biotic ‘soup’ and primitive plankton (algorithms floating across the Web), plus analogues of predatory devourers or parasites... all the way to the new GPT ersatz Voices-Without-Mind that I predicted, half a decade ago would swarm over us… well… precisely now.

The Walker essay is lovely and calming and I recommend it. But Life’s ‘way’ is often bloody and nescient and we cannot afford to just let the genes fall where they may.

== But… b-but is it conscious or self-aware? ==

Um… does it matter?

No, seriously, there are some earnest efforts to ramp up the study of consciousness and what it means. Though such efforts have been around since well before that smelly old preener, Socrates prattled annoyingly in the Academy. Indeed, in the 1980s I was managing editor for the Journal of the UCSD Laboratory and Center for Human Cognition.

A very brief outline of the overall problem can be found in this scientific American essay “Why the Mystery of Consciousness Is Deeper Than We Thought.” Though it only touches on the quandaries, lightly.

Diving in far more deeply and thoroughly, Robert Lawrence Kuhn – for decades host of the Closer To Truth interview show – has just completed the magnum opus of consciousness studies! A broad survey of (pretty much) the whole field, summarizing more different theories than you could shake a meme at. If the topic of how you’re interested in things interests you, have a look at A Landscape of Consciousness. (A review was just published by IAI.)

Ray Kurzweil has a followup to The Singularity is Near, with... The Singularity is Nearer: When We Merge with AI, proposing that AI will achieve human level intelligence by 2029.

Okay, where are those Machines of Loving Grace?

== Oh, wait... 'love bots' --

You expected that to be about sex?

Well, no room this time. Maybe in my next posting about our coming AI-enhanced future.