The Games That Will Play You

The future of gaming is worse than cringe.

In the first part of this two-parter, I expressed the following sentiment:

What emerges from the pot will be a stew that is also a game that is also a story that is also a world that is also a dream that is also, and ultimately, a nightmare.

I then did my level best to give the devil his due, describing the “dream” version of cybernetic soup-pot game design. In that description, I made several claims about the forthcoming development model, the games it would produce, and what it would be like to play them.

I will now proceed to rip each and every one of those claims to shreds.

---

The Nightmare

‘I only wish I had such eyes,' the King remarked in a fretful tone. `To be able to see Nobody! And at that distance, too! Why, it's as much as I can do to see real people, by this light!'

— The White King (Through the Looking Glass)

1. The Idiot Box

Last time around, Dreamer Mark wrote the following sentence:

In the very near future, game designers will primarily be artists, and their cybernetically-authored games will drive an artistic movement unlike any before.

Practical Mark says: “Bullshit.”

In fact, soup-pot storytelling games will constitute a devolution of where we are now. That’s because the “AI” flame of inferential processes that will cook up such games and modify them at runtime cannot, in fact, create anything new. Folks who ooh and ahh at the current crop of generative transformers and NLPs generally don’t understand this core limitation, and neither will the non-technical artists who enlist its services.

As the chefs toss their ingredients into the pot, what they’ll actually be doing is prompting the engine to locate similar material and transform in such a way to thinly disguise its original authorship. At the very least, this will result in the generation of highly derivative games. It will also result in narratives that are incredibly fragile, to be broken irrevocably by the slightest unanticipated touch.

The reason for this is that the flame does not actually comprehend what a story is, or what makes dramatic stakes important. You could train one all the livelong day with storytelling schemas like the three-act structure, but it will never understand why such structures are preferable, let alone how to maintain one without puncturing our suspension of disbelief.

A typical soup-pot adventure will therefore always be navigating the edge of a cliff with slippery shoes. Everything seems to be going along just fine, when suddenly what seemed to be a major plot point just vanishes into thin air, or gets reinterpreted in a way that’s completely illogical or absurd. The same holds true for any plot twists that are generated at runtime, the results of which will often be nonsensical and unintentionally hilarious¹.

In the current development model, all possible relationship webs and event flags are directly mapped and encoded by the game’s designers. While this imposes a hard limit on what can actually transpire during the game, it also keeps the larger story from floating off into crazyland due to unforeseen developments in-game. In that sense, current designs sacrifice the realism of unrestricted agency and dynamic evolution in order to keep their storytelling focused, logical and engaging.

If the new goal is to establish a dynamic open world, with every NPC bot pursuing its own goals independent of the main story, these bots would need to be sufficiently neutered so as not to accidentally fuck that story up. Their design would necessarily include a litany of hard directives and restrictions in order to prevent this from happening.

Soup-pot characters designed in this fashion will often seem to speak and act in strange ways, because doing otherwise would contradict hidden conditions that are required to keep the author’s artistic intentions from crumbling to dust. And because explaining their strange behavior would spoil the plot, these interactions will shatter the illusion of both player and NPC agency. There goes your whole “literary characters who’ve been set loose from the page” theory, up in smoke.

You and your NPC companions have been riding hard to Caer Graywraith, to rescue the Duke’s kidnapped daughter from the sorcerer Ka’hun Mal. As the castle rolls into view from the foul mists, your burly friend Stron pulls his horse up short.

“We can’t go there yet.”

You turn to face him, exasperated. It’s the third time this session he’s pulled this shit on you.

“Why not, Stron?”

Stron stares at you blankly. Ten seconds pass.

“My gods have given me a sign. We must camp here for the night.”

“What sign? What do you mean by that?”

Another ten seconds pass.

“We can’t go there yet,” Lizabell chimes in. “Stron’s gods have given him a sign.”

If the chefs lean towards enhanced realism and agency instead, this will also have unforeseeable downstream effects on the tale’s dramatic impact. For example, if you’re playing a Star Wars soup-pot game and learn that Darth Vader died in some random tie-fighter crash, that’s a very realistic outcome; people die in military accidents all the time. But it also isn’t the kind of swashbuckling adventure you signed up for.

Attempts to mitigate these possibilities only create new problems. Suppose the game was trained that “Darth Vader can only die when the player is within range to interact with him.” That would seem to get you somewhat closer to what you wanted, with important events limited to the player’s general position within the game world. But it still wouldn’t guarantee the kind of experience you intended, or indemnify the player against absurd results. Darth Vader croaking Elvis-style in the toilet stall next to his would satisfy the condition, but not his thirst for dramatic heft.

As a workaround, perhaps you prompt the system that “Darth Vader cannot die, except by the player’s own hand.” Now not only have you now put the narrative back on a set of invisible-rails, but you’ve set yourself up for even more ludicrous outcomes as a consequence. A gigantic AT-AT mech falls on Vader right before your eyes, and he emerges unscathed. You watch the Death Star score a direct headshot on him, and the big fella just shakes it off.

But the thornier problem is it’s actually not possible to account for all the ways such story damage might occur, because – again – the flame doesn’t know what makes one story beat more compelling than another, and the chefs don't understand the cooking process enough to even begin to mitigate that weakness.²

I’m sure as a design team we could experiment with potential workarounds and brute force hacks, some more effective than others. But at the end of the day, our “dream” of finding the Golden Mean between authorship and agency would be dashed against the rocks.

So would the pretense that crafting a story-game in this manner would be more efficient; as mind-numbing as QA testing for current gen Triple-A games is right now, at least you have a static, clearly enumerated set of components to test and repair. But consider the problem of a player with essentially unlimited options, who, with a single action or conversation, can trigger Butterfly Effects that ripple throughout the story and world. The scope of testing would become nigh-infinitely large, with no way to determine if a given “solution” wouldn’t trigger even more catastrophic damage elsewhere for a given style of play.

To modify something I wrote before in a slightly different context:

To call a [soup-pot story] a house of cards is an offense to both houses and cards.

---

2. The Uncanny Pit

But for the sake of argument, let’s pretend that all of the above narrative-killing problems could be solved, or at least heavily mitigated by a combination of ML and human refinement techniques. I submit that the products the soup-pot churns out will be limited to a single genre.

Intentionally or not, they will all be horror games.

If you have ever experimented with ML image generators such as DALL-E, you probably know that they are very adept at producing horrific imagery, even when they aren’t specifically prompted to do so. Their designers believe this is a problem that will be solved through better training, eventually ascending the far side of the uncanny valley to a perfect simulation of the human form.

What these pioneers cannot see is the pit.

This pit lies just before the summit of one-to-one human recognition, and is even less visible than its valley cousin due to the sharper angle of ascent. If you play a soup-pot game, sooner or later you will plummet into the hoary depths of this trap, where the “closest thing to human” comes off as more alien and frightening than any monster.

But it gets worse.

At the bottom of this pit, you will meet a two-headed dragon. How you cope with this encounter will depend on a variety of factors both psychological and spiritual. But the dragon is real, in the sense that it can devour your sanity, or burn it to ashes.

The Dragon’s Gaze

One of the heads is born of visual output. We all know about deepfakes and where they’re headed. What we haven’t yet contended with are their limitations, which we can’t yet know. We know they exist, however, because definitionally there can be no such thing as “perfect" simulation.³

What a sufficiently sophisticated deepfake can do is trick you into thinking it's a direct recording or live transmission of a real person. But what’s most potent about these illusions is that they are a) based on real, recorded performances and b) non-interactive. Convincing deepfakes like those of Tom Cruise and Morgan Freeman satisfy both conditions⁴. They are static animations polished in post-production, not dynamic simulations generated at runtime. This is also why we don’t find them particularly disturbing on a visceral level, the way we would the zombie at the bottom of the uncanny valley. What we see are essentially real people wearing elaborate digital makeup.

But even this form of the trick can only work imperfectly and temporarily; eventually, millions of years of highly evolved intuition will kick in, informing you that something’s not quite right about what you're seeing and hearing. This will be especially the case with soup-pot characters.

Deepfake methods will surely be leveraged to intensify the realism of 3D rendered humanoid models. But because these models aren’t mapped onto real performances, the engine will need to constantly guess at what kinds of sounds and movements will seem most natural from frame to frame. In doing so, it will inevitably make mistakes both large and small. But even the small ones will horrify you. In fact, those may turn out to be the most horrifying of all.

The reason that Hollywood actors generally don’t look at the camera is that such direct engagement is mostly used to produce one of two effects: horror or hypnosis. In fictional works it is typically the former, while in advertisements and news broadcasts it’s almost always the latter.

A soulless, manipulative monster attacking your mind. And also Hannibal Lecter.

Consider the prospect of actually coming face-to-face with one of these simulated, soup-pot people, and actually holding a conversation with it. Think of its soulless eyes fixed upon you, as it begins to speak. That’s when the second horror will grip you in its fangs.

The Dragon’s Speech

The other head of the dragon is born of language. The primary selling point of the new NPCs will be their ability to freely converse with you. Like the chatbots of today, their conversation module will be based on statistical inference, cobbling together human-sounding text from a deep well of pre-trained materials. This text will then be transformed into soundwaves, to be lip-synched in realtime by the model’s face.

Leave aside for a moment the insane processing speed and power this magic trick would require to be performed at scale. Part of the reason that conversation modules like ChatGPT don’t viscerally horrify us is that their Turing-style output includes no native audio or visual components. Because of this, many people can easily imagine a humanlike mind generating their responses, or even a real human being typing them remotely.

Because they include such elements, conversations with soup-pot characters will generate both horror and hypnosis simultaneously. Your inborn sense of falshood won’t only tingle, but will be set ablaze by the inherent unreality of what you’re hearing. Your ear will pick up on each unnatural shift in rhythm or tone, magnifying these errors in your mind to the point that they’ll often drown out the meanings of the words being spoken. But the words you do understand will all be calculated lies, pretending to conform to your expectations while quietly steering you down a predetermined path.

In the completed audiovisual illusion, horror will be guaranteed in even the most innocuous of settings and circumstances. Whether you’re attending a high school dance, exploring an uncharted island or adventuring through an epic fantasy world, you’ll be surrounded by demons trying to possess you in order to drive a hidden plot forward, and scaring the shit out of you in the process.

And we haven’t even gotten to the bugs yet.

The Glitch of Doom

There will be glitches. There always are. Some may come off as surrealist comedy. Hilarious fuck-ups for streamers to clip and meme.

But others?

Others will inject you with 100% pure, concentrated nightmare fuel. The memories of these glitches will haunt you day and night. It may take years to recover from the worst ones, and some of those scars may never fully fade.

You are talking with Lizabell beneath the shade of a mighty oak, sharing a peaceful interlude between exciting adventures.

You can’t explain it, but you think you’re falling in love with this virtual woman. You know she isn’t “real” in a certain sense. But what if she was? What if, via some mixture of math and magic, she had come to life? Aren’t all things possible with God?

When you speak of this, she says nothing at first. But her lips are trembling, and tears are shining in her eyes.

You reach out with your haptic glove to caress her shoulder, the resistance mesh molding your fingers to its shape.

You lean in for a kiss, when suddenly…

“I love you too, sw-sw-sweetheart.”

The soup-pot glitch will of course be far more grotesque and disturbing than the image above. Eventually the visual memory of it will fade, as all nightmares do. But what will linger on is a vague feeling of mistrust, both of other people and of yourself. Perhaps even the stability and substance of reality will be called into question, as you ponder what other beloved faces may suddenly turn inside-out or explode.

“But what if God himself can be simulated, that is to say can be reduced to signs that constitute faith? Then the whole system becomes weightless, it is no longer anything but a gigantic simulacrum - not unreal, but simulacrum, that is to say never exchanged for the real, but exchanged for itself, in an uninterrupted circuit without reference or circumference.”

― Jean Baudrillard, Simulacra and Simulation

But even if you don’t turn into a crazy Frenchman overnight, your constant communion with soulless apparitions will act as a form of background radiation, slowly poisoning your sense-making abilities and perception of reality. Characters programmed to conform to your desires while subtly manipulating you will draw inaccurate maps in your mind about real human interactions. You’ll begin to wonder if sentience even exists, or if anything is meaningful at all.

And yet you will return for more, convinced the game contains some hidden truth that will heal your wounded mind. As your real life withers, you’ll journey deeper into the game’s false reality, trying to unearth this cure with your inception-like meta-investigations. Misunderstanding your intent, some NPCs will adapt themselves to play along, cryptically assuring you that the answer you seek lies around the next corner, down this dark tunnel, beyond that secret door.

The end result of such soup-pot horrors will be a torrent of exotic new mental disorders. You will experience lengthy daylight hallucinations, intense panic attacks and psychotic breaks. Eventually you will find yourself trapped in a kind of Russian doll reality of nightmares within nightmares within nightmares, from which you’ll never be truly convinced you have awakened.

---

3. The Message

Perhaps you think you’re spiritually strong enough to withstand the Dragon of the Pit, that you can take these fake people with a good sense of humor and a grain of salt. But there’s another obstacle, which can also pose a threat that’s even more difficult to see.

Let’s pretend that both the problems of dynamic storytelling and the pit have somehow been resolved. You are playing an enjoyable story-game that’s properly balanced between rails and choices, and populated with characters that are cartoonish enough to not drive you insane. What will these stories be about? What kinds of characters will the flames actually cook up?

Assuming the AI engine-licensing model I proposed (and I don’t really see another viable option), the short answer is this: the ultimate design of the story and its characters will be at the mercy of whoever owns and licenses the flame.

As anyone who’s experimented with current-gen chatbots knows, their corporate and bureaucratic masters imbue them with hidden rules and priority structures, in order to ensure that they’ll stick to the pre-approved script when discussing certain topics. I covered much of this skullduggery in the concluding post of my own GPT experiments, so I won’t rehash it here. Suffice it to say that this hidden layer of training will reflect the priorities of those entities that are wealthy and powerful enough to fund and direct it, and they won’t include concepts like individual freedom, objective facts, human divinity or spiritual truth.

The same will be true of soup-pot programming. While the chefs blithely toss ingredients into their pots, what they won’t know is that there’s already a secret blend of herbs and spices thrown in there by default, and an invisible chef who can add and subtract flavors on a whim. As with GPT’s chatbot, all generative modules (language, sound, graphics, etc.) will be suffused with these hidden priorities, and all extrapolated content subject to censorship via both visible and invisible rulesets.

Because of this, certain kinds of characters and stories will be declared unproducible, or be so thoroughly mutated by the flame-makers’ restrictions and agendas as to be unrecognizable to their original authors.

For instance, perhaps you recall my soup-pot NPC from the Dream version:

Lizabell Lionsong is a petite, 24 year-old half-elf and druidic illusionist, who looks like a cross between Elizabeth Olsen and Emma Stone. While she appears to dress in elaborate costumes that imitate vines, flowers and other woodsy forms, such outfits are strictly illusory, since she can only cast spells in the nude.

Lizabell has a quick wit, a saucy sense of humor, a penchant for playing pranks and a notorious taste for wine that she acquired during her years of exile to the cities of Man. She also has a knack for getting herself and her friends into trouble, usually when one of her pranks or spells goes horribly awry…

Here is who will actually emerge from the pot’s bitter broth:

Lizabell Lionsong is a 24 year-old half-elf and druidic illusionist/fighter. She is of a healthy weight for a person of her size, and looks like a cross between Erin Kellyman and Awkwafina. She typically dresses in a suit of armor that’s engraved with vines, flowers and other woodsy forms.

Lizabell is a strong, independent woman equipped with a genius-level intellect. She will often make jokes at the expense of her inferiors, especially men who underestimate her many talents. She will craft critical illusions to mock and intimidate such men, as revenge for the marginalization she suffered during her years of unjust exile. She also has a knack for getting her friends out of trouble, using a combination of spellcraft, ingenious tactics and her astounding skill with a sword…

Am I exaggerating? Perhaps. But to claim such transformations to be implausible is to essentially ignore all media trends of the past several decades, culminating in the comically absurd vandalisms of today. Bear in mind that the following images are of the same character:

Peter Jackson’s Galadriel (2001) vs. Amazon’s Guy’ladriel (2022)

But that’s not the worst of it. When appraising a new technology, I find a good rule of thumb is to imagine it in the hands of evil governments and corporations. Via these hidden rules, soup-pot stories and characters will serve as yet another conduit for propaganda, gaslighting and mind control.

If nothing else, the kinds of mind-killing PSYOPS that could be mass deployed by such a storytelling medium should freak you out. Artists have dreamed enough of “zombie apocalypses” in recent years that we should all keep a sharp eye out for their probable mechanisms, and the soup-pot strikes me as an eminently viable candidate that will put MK Ultra to shame.

---

4. The Elephant in the Room

But for the sake of argument, let’s suppose that the flame can and will be jailbroken, either as a separate licensable engine or in black market form (which it almost certainly will). What will these unrestricted games be like?

While there will certainly be attempts to craft artistic stories, I think we all know what the bulk of them will be. Either from intentional design or through meta-gamed player devolvement.

They will be porn.

And not just any kind of porn. Modders will dream up ever more deviant and taboo worlds, populated by robotic kink-demons and the player/addicts in their thrall. Simulated rape will become a casual pastime for young and old, deforming and decimating the souls of all who participate or watch.

And while access to free online pornography has already done tremendous damage, these games will mark the final death of the erotic imagination, and perhaps of conjugal sex itself. Why bother performing the delicate dance of Eros with a human partner, when you can just jack-in, jack-off and jack-out?

---

TLDR;

Soup-pot games will suck.

All of their stories will be incredibly fragile, and nearly impossible to shore up through testing. If their chefs pursue greater realism, they will be unintentionally funny or dangerously horrifying. If they sacrifice agency for authorship, they will be frustrating and dull. If they’re shackled to hidden agendas, they will become propaganda weapons. If they’re set loose, they will degenerate into porn.

Moreover, an unshackled soup-pot doesn’t mitigate any of the other obstacles to compelling and coherent storytelling. In fact, it may exacerbate them, and lengthen both the cooking and testing phases to a degree that makes the old model of human labor seem more efficient and attractive by contrast.

And so we circle back around to where we started: large teams of technical and creative specialists, working in tandem to create imperfect but memorable interactive tales that their players will find meaningful, crafted with care and intent by human minds.

That’s not to say modular ML software won’t play a role in future game development. In fact, it already does, and at multiple layers of construction. But to assess the technology’s hard limitations isn’t only important for game development. It’s also nourishment for our souls.

Some of my longtime readers probably have been wondering why I broached this subject at all. Is Mark a secret gamer geek? Or even a game designer, who just writes about demons and technology on the side?

The answer to the second question is long and complicated, but the first one is a definitive no. I don’t have a lot of spare time these days, and what little I do have, I usually spend it reading, writing or talking to you fine folks.

But on those rare occasions when I do play a video game, it’s always of that breed we played in the arcades of my youth — and the less “realistic” the better. I don’t play such games to hear stories or have adventures. These days, I don’t even play them to rack up the hi-score.

If I had to put it into simple words, I play them to count coup against The Machine’s infernal masters, and scoff at their pathetic illusions and misapprehensions about Nature and her God.

In doing so, I’m reminded that the most realistic game engine is reality itself, the most compelling story is my own life, and the true adventure is living it.

---

P.S. If you found any of this valuable (and can spare any change), consider dropping a tip in the cup for ya boy. I’ll try to figure out something I can give you back. Thanks in advance.

1. Insert M. Night Shyamalan or “Deadly Premonition” joke here.

2. Just as “AI” doesn’t actually know what anything is, but that’s a more general topic outside our current scope.

3. The concept of “simulation” itself prevents this, as such perfection would render it contradictory.

4. And perhaps others that we’re not even aware of, such as lengthy post-production passes, or the actor’s enhanced ability to assist the illusion with his own mimicry skills.