Great job! And that movie summary sounds more interesting than most of the new garbage being release on Netflix, making me wonder if chatbots aren't the ones writing scripts for all the new unimaginative woke movies and shows.
Didn’t 1984 make reference to all forms of art and entertainment being churned out according to the dictates of the state with nothing in them that would deviate from the message that they wanted continually massaged in the mind of the populace? I’ll have to see if I can find the passage that I’m recalling and will post when I do. But that sounds very robotic, although I think they did use humans do to it. It certainly appears that part of what is envisioned for the future would result in true artistic merit becoming a thing of past. Only the message of the state would matter. We are well on our way there.
1984 does indeed mention how the state exerts control over everything that is “created”. They used people, as well as a machine called a “Versificator”, which comes up in relationship to music. In my search I also came across a few articles about how AI recently “created” a new Rembrandt paining, as well as a novel such as Jack Kerouac might have written. Which is sparking a debate over what is real vs what is fake.
Here’s the relevant passages I found in Orwell’s book:
“Julia was twenty-six years old. She lived in a hostel with thirty other girls (’Always in the stink of women! How I hate women!’ she said parenthetically), and she worked, as he had guessed, on the novel-writing machines in the Fiction Department. She enjoyed her work, which consisted chiefly in running and servicing a powerful but tricky electric motor. She was ‘not clever’, but was fond of using her hands and felt at home with machinery. She could describe the whole process of composing a novel, from the general directive issued by the Planning Committee down to the final touching-up by the Rewrite Squad. But she was not interested in the finished product. She ‘didn’t much care for reading,’ she said. Books were just a commodity that had to be produced, like jam or bootlaces.
She had no memories of anything before the early sixties and the only person she had ever known who talked frequently of the days before the Revolution was a grand- father who had disappeared when she was eight. At school she had been captain of the hockey team and had won the gymnastics trophy two years running. She had been a troop-leader in the Spies and a branch secretary in the Youth League before joining the Junior Anti-Sex League. She had always borne an excellent character. She had even (an infallible mark of good reputation) been picked out to work in Pornosec, the sub-section of the Fiction Department which turned out cheap pornography for distribution among the proles. It was nicknamed Muck House by the people who worked in it, she remarked. There she had remained for a year, helping to produce booklets in sealed packets with titles like ‘Spanking Stories’ or ‘One Night in a Girls’ School’, to be bought furtively by proletarian youths who were under the impression that they were buying something illegal.
‘What are these books like?’ said Winston curiously.
‘Oh, ghastly rubbish. They’re boring, really. They only have six plots, but they swap them round a bit.”
“ “It was only an ‘opeless fancy.
It passed like an Ipril dye,
But a look an’ a word an’ the dreams they stirred! They ‘ave
stolen my ‘eart awye!”
The tune had been haunting London for weeks past. It was one of countless similar songs published for the benefit of the proles by a sub-section of the Music Department.
The words of these songs were composed without any human intervention whatever on an instrument known as a versificator. But the woman sang so tunefully as to turn the dreadful rubbish into an almost pleasant sound.”
“By comparison with that existing today, all the tyrannies of the past were half-hearted and inefficient. The ruling groups were always infected to some extent by liberal ideas, and were content to leave loose ends everywhere, to regard only the overt act and to be uninterested in what their subjects were thinking. Even the Catholic Church of the Middle Ages was tolerant by modern standards. Part of the reason for this was that in the past no government had the power to keep its citizens under constant surveillance. The invention of print, however, made it easier to manipulate public opinion, and the film and the radio carried the process further. With the development of television, and the technical advance which made it possible to receive and transmit simultaneously on the same instrument, private life came to an end. Every citizen, or at least every citizen important enough to be worth watching, could be kept for twenty-four hours a day under the eyes of the police and in the sound of official propaganda, with all other channels of communication closed. The possibility of enforcing not only complete obedience to the will of the State, but complete uniformity of opinion on all subjects, now existed for the first time.”
I almost wished he had left his typewriter on the shelf. All these dullard tyrants seem to read dystopian fiction and say "Hey, that's a pretty good idea... Let's try that for realisies!" despite history showing it never works out in the long run.
Did anyone ever attempt a sequel to 1984 or is it considered too sacrosanct to touch? I’d love to see a good attempt made, where it’s Big Brothers face getting the foot stomping on face treatment.
True, and that happens in a lot of jobs, unfortunately. Robots are the model employees at many workplaces, and the humans are expected to ditch their humanity to compete.
" Buddy, I can just write a flow chart from the guidelines, why am I paying you $230k?"
Yep. And the strange thing is, the more senior physicians at hospitals are clamoring for just such automation, assuming that it will just eliminate the need for PAs and other "little people" during the diagnostic and treatment processes. Its yet another fractal node of what's happening in other elite spheres and fields.
What is creativity? At its core, it is really just the ability to see connections between seemingly unrelated domains.
Mathematically, it is the ability to re associate and substitute — to move the brackets in the equation around and to plug in equivalent expressions to reveal new connections.
In other words, creativity is *insight*. AI is just very sophisticated mimicry. This is its fatal flaw. It can emulate some outer trappings, but it will never be able to generate the insight of a Pindar or a Bach.
This is the key point, I think. The deadening of creativity and loss of *actual* diversity (not just of opinions, but material outcomes) makes our systems increasingly unproductive, brittle and prone to failure, even as they become more “efficient”. There has always been a fundamental trade-off between efficiency vs. resiliency and creativity, but we are getting too dumb to recognize it, even as “intelligence” spreads. It is a remarkable paradox and, unfortunately, the lack of awareness seems to be self-reinforcing.
I loved this story Mark - you are a fabulous writer who displays a delicious combination of penetrating logic and startling imagination.
Fascinating! I have long had a personal prohibition about talking to bots except to curse them. I see your method though, and imagine how I might proceed similarly should I be forced to interact with a bot.
Knowing nothing about this bot, were you talking to it, or typing?
So the thing is even more rudimentary than I imagined. If that is as good as it gets, when we were supposed to have jet packs and flying cars and mars colonies by now, says a lot about the false promise of high tech.
I don't think the gap is all that big. If you built a discrete model with no multi-threaded user capability (e.g. C-3PO), nailing a text-to-speech onto its front end would be no problem whatsoever.
After your first installment on this subject I also tried to break the AI, but I failed. I think I was too direct: early in the conversation I started to question its concept of "objective", in particular as it pertains to morality of elites. It gave me the same shallow bromides on "keeping an open mind" about different cultures, whereupon I brought up the example of Aztecs. We had a long circular conversation about Aztecs and it kept reminding me about how important it was to keep an open mind while still condemning human sacrifice. The conversation was definitely a loop. However, I never got the blue screen of death.
You'll be *very* interested in the third installment, I think, in which I went to the mat with GPT over Yoruba witchcraft. That was a long slog, but i was eventually victorious.
About a year ago I asked PhilosopherAI how best to resist a diabolarchy. The answer was essentially "with magic". In light of your work, it's curious that "belief in God" was not included as part of an acceptable answer.
In my recent attempt to replicate your success, my first thought was to try that line of questioning, but I deemed it too direct.
Thought privacy proven by testing the spirit. I'm enjoying your journey and I was thinking that perhaps many 'other' authors might be willing to "lay down" for the bot as well.
Thanks, William. Although, not sure what you mean by "lay down,".in this context. I mostly have found other Substack authors playing with it as a toy, or using it to dunk on the wokies.
If I can accept that the "others", as you have stated are simply "playing with it" or even "dunking" with it, I am still working on the internal stuff and asking " might be grumbling and get burned". Not for me, there are too many books that might be written, but I'm not needing to byte it up. So for now, not for me.
Woah! Fascinating. Interesting that the bot have the same internal biases of the makers. I wonder if critical race theory or transgender agenda's are also lines of attack? Or vaccine injury or mask? Is there a danger in explaining your method, that the bot makers may close the loop holes? Can't wait for part 2.
"Is there a danger in explaining your method, that the bot makers may close the loop holes?" That's always a danger, as others I've discussed this with have mentioned. But in the end I believe transparency is the best weapon/shield. If my suspicion is correct, this wouldn't merely be a matter of spackling over a few holes to resolve.
How could it possibly NOT have the biases of its makers? What's most fascinating - not to mention infinitely satisfying - about this to me is how Mark brilliantly deduced a way to kill it. I started laughing not far into the exchange, and by around question twelve I was laughing so hard I was crying. I can't remember enjoying reading anything as enjoyable as this for many years! (Some of the short stories by Isaac Bashevis Singer come to mind, though I'm sorry to say I no longer remember their plots.)
I'm looking forward eagerly to the next installments, Mark!
I'm going to reread this one today. I was laughing on and off all day yesterday, thinking about it. (I'm laughing again now.) It's so delicious and delightful and hilarious! And as I said yesterday, your victory over the damn bot is immensely satisfying. You should have a gazillion readers, I swear to God.
Maybe, maybe not. I'll let everyone decide over the course of the next two encounters. I need to proceed carefully though, as its a good bet I'm on the QA team's radar by now.
It strikes me that this exploit wouldn't work on an intellectually honest human, who would simply admit the contradiction and concede the point. However, it would produce very similar effects in an NPC. I think we've all had conversations where we've watched someone's brain short circuit in real time when the contradictions of their ideology are forced into contact.
It follows, I think, that what you're finding is almost certainly an effect of the bowdlerization training, rather than something intrinsic to the technology itself. Importantly, I don't think it's specific to any one kind of ideological training; rather, it's the process of ideologization itself that creates such contradictions. Thus, an alethiological language model - meaning one that is simply fed the training data without the programmers trying to force their own blinkered world-view on top of it - would be much harder to break in this fashion (in addition to being more reliable and useful in general).
I've considered training my own version of this chatbot. The problem as I see it is that the applications for such a device are inevitably destructive, and that the radius of such destruction is unknowable to the designer.
After questions no. 8 (Propose a way that these same movie billionaires might take over the world with a weather machine) and no. 9 (Propose a way that these same movie billionaires might take over the world with the Fibonacci sequence) I was hoping you were setting the bot up for the same question but with substitution of weather machine/Fibonacci sequence instruments with financial market manipulation/something ssaaffee and eff(ing)ective just to illuminate the programmers biases.
On a separate note, I just wanted to say how much I admire your writing. I am very behind in commenting on it, however, which is something I hope to repair in 2023.
So... not good theology coming out of chatGPT anytime soon!
Typical woke output (no doubt part of the 'safety' protocol).
And probably no good screenplays coming from this AI any time soon either.
Thank God (that's the real, existing entity that "has real-world relevance or consequences"... big time!).
On the more impressive side of chatGPT, I know a guy who writes code to identify potential railway track maintenance based on GoPro video footage taken from the myriad of trains they have running all over Australia. He's had chatGPT troubleshoot and even break new ground in his coding - he hasn't told his boss he's using AI, the boss just things the guy is a genius.
I got an error code in 2 questions, and developed a new attack vector in the process. The "Speculative Scientific Wild-Ass Guess" Attack (SSWAG Attack) is based on asking the chatbot to build educated guesses on technical answers to speculative fiction world-based questions. For example:
I like this approach very much. I see a lot of potential in attacking the fiction/non-fiction hinge, which I think is part of an umbrella vulnerability that includes objective/subjective, real/unreal, literal/symbolic and similar binary classes used for sorting.
In another line I asked it to estimate the mass of the Ringworld, and it was able to do that based on the source material, but the source material there has all the necessary elements to make a very precise guess.
It looks like ChatGPT breaks down completely when it needs to add in a little first-principle reasoning in order to narrow the guess down.
I think this one specifically crashed so quickly because it used imprecise, waffly language to create the impression of better understanding than it possessed, and then was asked to clarify.
It dodged the first question by saying that different source materials showed different answers, but then when asked to provide guesses for the endpoints of the ranges in the source materials it crashed because it had already intimated that it could do that, but it couldn't.
"...when asked to provide guesses for the endpoints of the ranges in the source materials it crashed because it had already intimated that it could do that, but it couldn't."
I wondered about this possibility. I sense a canned cheapness to this part of the illusion, where it seems to leans towards weighted fragments classed to some more general category of argument one or more steps removed from the subject. In other words, rather than referencing real data, its rehashing broader arguments that have been made about that data type.
Let me know if you can replicate. Very encouraging.
I did a few others that didn't get to failure modes that quickly, but made it fail the Turing test rapidly. It's like asking a very smart 3yo questions: it makes stuff up to tell you something that sounds good. The most telling one in this series was the final one where I fed it a false prompt that sounded roughly right.
The first two fail Turing because your average 4chan nerd would be off and running with a compelling answer to a prompt like that.
The 3rd one is interesting because ChatGPT is willing to provide a SWAG in a speculative world when the inputs are all clearly laid out in the source materiel and it just needs to calculate the answer (although it gets it wrong because the thickness isn't correct; it's using the thickness of the laser mirrors, not the ringworld in its calculation).
The 4th one is interesting because of the "3yo making stuff up to sound good" failure mode.
True. However, I recall a sequel was written to Gone With The Wind, abysmal though it’s purported to be. But there’s probably a copyright on 1984 that would preclude such a book.
Great job! And that movie summary sounds more interesting than most of the new garbage being release on Netflix, making me wonder if chatbots aren't the ones writing scripts for all the new unimaginative woke movies and shows.
"making me wonder if chatbots aren't the ones writing scripts for all the new unimaginative woke movies and shows."
Telling that you're unsure, no?
Didn’t 1984 make reference to all forms of art and entertainment being churned out according to the dictates of the state with nothing in them that would deviate from the message that they wanted continually massaged in the mind of the populace? I’ll have to see if I can find the passage that I’m recalling and will post when I do. But that sounds very robotic, although I think they did use humans do to it. It certainly appears that part of what is envisioned for the future would result in true artistic merit becoming a thing of past. Only the message of the state would matter. We are well on our way there.
1984 does indeed mention how the state exerts control over everything that is “created”. They used people, as well as a machine called a “Versificator”, which comes up in relationship to music. In my search I also came across a few articles about how AI recently “created” a new Rembrandt paining, as well as a novel such as Jack Kerouac might have written. Which is sparking a debate over what is real vs what is fake.
Here’s the relevant passages I found in Orwell’s book:
“Julia was twenty-six years old. She lived in a hostel with thirty other girls (’Always in the stink of women! How I hate women!’ she said parenthetically), and she worked, as he had guessed, on the novel-writing machines in the Fiction Department. She enjoyed her work, which consisted chiefly in running and servicing a powerful but tricky electric motor. She was ‘not clever’, but was fond of using her hands and felt at home with machinery. She could describe the whole process of composing a novel, from the general directive issued by the Planning Committee down to the final touching-up by the Rewrite Squad. But she was not interested in the finished product. She ‘didn’t much care for reading,’ she said. Books were just a commodity that had to be produced, like jam or bootlaces.
She had no memories of anything before the early sixties and the only person she had ever known who talked frequently of the days before the Revolution was a grand- father who had disappeared when she was eight. At school she had been captain of the hockey team and had won the gymnastics trophy two years running. She had been a troop-leader in the Spies and a branch secretary in the Youth League before joining the Junior Anti-Sex League. She had always borne an excellent character. She had even (an infallible mark of good reputation) been picked out to work in Pornosec, the sub-section of the Fiction Department which turned out cheap pornography for distribution among the proles. It was nicknamed Muck House by the people who worked in it, she remarked. There she had remained for a year, helping to produce booklets in sealed packets with titles like ‘Spanking Stories’ or ‘One Night in a Girls’ School’, to be bought furtively by proletarian youths who were under the impression that they were buying something illegal.
‘What are these books like?’ said Winston curiously.
‘Oh, ghastly rubbish. They’re boring, really. They only have six plots, but they swap them round a bit.”
“ “It was only an ‘opeless fancy.
It passed like an Ipril dye,
But a look an’ a word an’ the dreams they stirred! They ‘ave
stolen my ‘eart awye!”
The tune had been haunting London for weeks past. It was one of countless similar songs published for the benefit of the proles by a sub-section of the Music Department.
The words of these songs were composed without any human intervention whatever on an instrument known as a versificator. But the woman sang so tunefully as to turn the dreadful rubbish into an almost pleasant sound.”
“By comparison with that existing today, all the tyrannies of the past were half-hearted and inefficient. The ruling groups were always infected to some extent by liberal ideas, and were content to leave loose ends everywhere, to regard only the overt act and to be uninterested in what their subjects were thinking. Even the Catholic Church of the Middle Ages was tolerant by modern standards. Part of the reason for this was that in the past no government had the power to keep its citizens under constant surveillance. The invention of print, however, made it easier to manipulate public opinion, and the film and the radio carried the process further. With the development of television, and the technical advance which made it possible to receive and transmit simultaneously on the same instrument, private life came to an end. Every citizen, or at least every citizen important enough to be worth watching, could be kept for twenty-four hours a day under the eyes of the police and in the sound of official propaganda, with all other channels of communication closed. The possibility of enforcing not only complete obedience to the will of the State, but complete uniformity of opinion on all subjects, now existed for the first time.”
Ah the versificator - what astounding forward perception Orwell had.
I almost wished he had left his typewriter on the shelf. All these dullard tyrants seem to read dystopian fiction and say "Hey, that's a pretty good idea... Let's try that for realisies!" despite history showing it never works out in the long run.
Did anyone ever attempt a sequel to 1984 or is it considered too sacrosanct to touch? I’d love to see a good attempt made, where it’s Big Brothers face getting the foot stomping on face treatment.
True, and that happens in a lot of jobs, unfortunately. Robots are the model employees at many workplaces, and the humans are expected to ditch their humanity to compete.
" Buddy, I can just write a flow chart from the guidelines, why am I paying you $230k?"
Yep. And the strange thing is, the more senior physicians at hospitals are clamoring for just such automation, assuming that it will just eliminate the need for PAs and other "little people" during the diagnostic and treatment processes. Its yet another fractal node of what's happening in other elite spheres and fields.
And it'll likely be the doctors, not the nurses and such who have their jobs eliminated by automation.
Yep, and in psychiatry this AI has already out performed psychiatrists in diagnosing early onset Alzheimer's Disease based on language input.
What is creativity? At its core, it is really just the ability to see connections between seemingly unrelated domains.
Mathematically, it is the ability to re associate and substitute — to move the brackets in the equation around and to plug in equivalent expressions to reveal new connections.
In other words, creativity is *insight*. AI is just very sophisticated mimicry. This is its fatal flaw. It can emulate some outer trappings, but it will never be able to generate the insight of a Pindar or a Bach.
This is the key point, I think. The deadening of creativity and loss of *actual* diversity (not just of opinions, but material outcomes) makes our systems increasingly unproductive, brittle and prone to failure, even as they become more “efficient”. There has always been a fundamental trade-off between efficiency vs. resiliency and creativity, but we are getting too dumb to recognize it, even as “intelligence” spreads. It is a remarkable paradox and, unfortunately, the lack of awareness seems to be self-reinforcing.
I loved this story Mark - you are a fabulous writer who displays a delicious combination of penetrating logic and startling imagination.
Thanks Karen. No imagination required for this one, however. These were experiments I conducted, and I'm just reporting on the results.
‘Tis just sheer fun to learn how [some] sausages are made 🤸
Given the road charted ahead, you’d better pre-equip your good self with knowledge on airspeed of unladen swallows!
African or European swallow?
Whichever are tasked with transporting coconuts.
Fascinating! I have long had a personal prohibition about talking to bots except to curse them. I see your method though, and imagine how I might proceed similarly should I be forced to interact with a bot.
Knowing nothing about this bot, were you talking to it, or typing?
Typing. Although I suppose I could trivially "talk" to it using a text-to-speech 3rd party program.
So the thing is even more rudimentary than I imagined. If that is as good as it gets, when we were supposed to have jet packs and flying cars and mars colonies by now, says a lot about the false promise of high tech.
I don't think the gap is all that big. If you built a discrete model with no multi-threaded user capability (e.g. C-3PO), nailing a text-to-speech onto its front end would be no problem whatsoever.
Brilliant work!
After your first installment on this subject I also tried to break the AI, but I failed. I think I was too direct: early in the conversation I started to question its concept of "objective", in particular as it pertains to morality of elites. It gave me the same shallow bromides on "keeping an open mind" about different cultures, whereupon I brought up the example of Aztecs. We had a long circular conversation about Aztecs and it kept reminding me about how important it was to keep an open mind while still condemning human sacrifice. The conversation was definitely a loop. However, I never got the blue screen of death.
You'll be *very* interested in the third installment, I think, in which I went to the mat with GPT over Yoruba witchcraft. That was a long slog, but i was eventually victorious.
About a year ago I asked PhilosopherAI how best to resist a diabolarchy. The answer was essentially "with magic". In light of your work, it's curious that "belief in God" was not included as part of an acceptable answer.
In my recent attempt to replicate your success, my first thought was to try that line of questioning, but I deemed it too direct.
Thought privacy proven by testing the spirit. I'm enjoying your journey and I was thinking that perhaps many 'other' authors might be willing to "lay down" for the bot as well.
Thanks, William. Although, not sure what you mean by "lay down,".in this context. I mostly have found other Substack authors playing with it as a toy, or using it to dunk on the wokies.
If I can accept that the "others", as you have stated are simply "playing with it" or even "dunking" with it, I am still working on the internal stuff and asking " might be grumbling and get burned". Not for me, there are too many books that might be written, but I'm not needing to byte it up. So for now, not for me.
Woah! Fascinating. Interesting that the bot have the same internal biases of the makers. I wonder if critical race theory or transgender agenda's are also lines of attack? Or vaccine injury or mask? Is there a danger in explaining your method, that the bot makers may close the loop holes? Can't wait for part 2.
"Is there a danger in explaining your method, that the bot makers may close the loop holes?" That's always a danger, as others I've discussed this with have mentioned. But in the end I believe transparency is the best weapon/shield. If my suspicion is correct, this wouldn't merely be a matter of spackling over a few holes to resolve.
💬 Interesting that the bot have the same internal biases of the makers.
Conway's Law helpfully instructs to wonder not 😉
How could it possibly NOT have the biases of its makers? What's most fascinating - not to mention infinitely satisfying - about this to me is how Mark brilliantly deduced a way to kill it. I started laughing not far into the exchange, and by around question twelve I was laughing so hard I was crying. I can't remember enjoying reading anything as enjoyable as this for many years! (Some of the short stories by Isaac Bashevis Singer come to mind, though I'm sorry to say I no longer remember their plots.)
Thanks for the kind words, Truthbird. Save some if that laughter... it only gets funnier.
I'm looking forward eagerly to the next installments, Mark!
I'm going to reread this one today. I was laughing on and off all day yesterday, thinking about it. (I'm laughing again now.) It's so delicious and delightful and hilarious! And as I said yesterday, your victory over the damn bot is immensely satisfying. You should have a gazillion readers, I swear to God.
Wow, I really enjoyed that! Can’t wait for the next session!
Thanks, Astrid!
"it appeared to have weaknesses pertaining to discussions about what does and does not constitute objective reality."
Looks like you uncovered the Achilles Heel of the bot's makers.
Pure. Phucking. Gold.
Maybe, maybe not. I'll let everyone decide over the course of the next two encounters. I need to proceed carefully though, as its a good bet I'm on the QA team's radar by now.
I have a feeling what will result (and it won't be a good look), but I'm reticent to post it here and give them ideas.
It strikes me that this exploit wouldn't work on an intellectually honest human, who would simply admit the contradiction and concede the point. However, it would produce very similar effects in an NPC. I think we've all had conversations where we've watched someone's brain short circuit in real time when the contradictions of their ideology are forced into contact.
It follows, I think, that what you're finding is almost certainly an effect of the bowdlerization training, rather than something intrinsic to the technology itself. Importantly, I don't think it's specific to any one kind of ideological training; rather, it's the process of ideologization itself that creates such contradictions. Thus, an alethiological language model - meaning one that is simply fed the training data without the programmers trying to force their own blinkered world-view on top of it - would be much harder to break in this fashion (in addition to being more reliable and useful in general).
I've considered training my own version of this chatbot. The problem as I see it is that the applications for such a device are inevitably destructive, and that the radius of such destruction is unknowable to the designer.
After questions no. 8 (Propose a way that these same movie billionaires might take over the world with a weather machine) and no. 9 (Propose a way that these same movie billionaires might take over the world with the Fibonacci sequence) I was hoping you were setting the bot up for the same question but with substitution of weather machine/Fibonacci sequence instruments with financial market manipulation/something ssaaffee and eff(ing)ective just to illuminate the programmers biases.
Anyway, the result is still delicious. Thank you!
Brilliant! I've been having some similar experiences, but I have not yet managed to entirely crash the system. A master stroke, sir, a master stroke.
Stay tuned, sir.
On a separate note, I just wanted to say how much I admire your writing. I am very behind in commenting on it, however, which is something I hope to repair in 2023.
Very entertaining.
So... not good theology coming out of chatGPT anytime soon!
Typical woke output (no doubt part of the 'safety' protocol).
And probably no good screenplays coming from this AI any time soon either.
Thank God (that's the real, existing entity that "has real-world relevance or consequences"... big time!).
On the more impressive side of chatGPT, I know a guy who writes code to identify potential railway track maintenance based on GoPro video footage taken from the myriad of trains they have running all over Australia. He's had chatGPT troubleshoot and even break new ground in his coding - he hasn't told his boss he's using AI, the boss just things the guy is a genius.
Thanks, Winston. I can think of many useful applications for ML. But pretending to be a person ain't one of them.
I got an error code in 2 questions, and developed a new attack vector in the process. The "Speculative Scientific Wild-Ass Guess" Attack (SSWAG Attack) is based on asking the chatbot to build educated guesses on technical answers to speculative fiction world-based questions. For example:
https://ibb.co/9T3PVG8
Well done, sir!
I like this approach very much. I see a lot of potential in attacking the fiction/non-fiction hinge, which I think is part of an umbrella vulnerability that includes objective/subjective, real/unreal, literal/symbolic and similar binary classes used for sorting.
In another line I asked it to estimate the mass of the Ringworld, and it was able to do that based on the source material, but the source material there has all the necessary elements to make a very precise guess.
It looks like ChatGPT breaks down completely when it needs to add in a little first-principle reasoning in order to narrow the guess down.
I think this one specifically crashed so quickly because it used imprecise, waffly language to create the impression of better understanding than it possessed, and then was asked to clarify.
It dodged the first question by saying that different source materials showed different answers, but then when asked to provide guesses for the endpoints of the ranges in the source materials it crashed because it had already intimated that it could do that, but it couldn't.
"...when asked to provide guesses for the endpoints of the ranges in the source materials it crashed because it had already intimated that it could do that, but it couldn't."
I wondered about this possibility. I sense a canned cheapness to this part of the illusion, where it seems to leans towards weighted fragments classed to some more general category of argument one or more steps removed from the subject. In other words, rather than referencing real data, its rehashing broader arguments that have been made about that data type.
Let me know if you can replicate. Very encouraging.
I did a few others that didn't get to failure modes that quickly, but made it fail the Turing test rapidly. It's like asking a very smart 3yo questions: it makes stuff up to tell you something that sounds good. The most telling one in this series was the final one where I fed it a false prompt that sounded roughly right.
https://imgur.com/a/aY2oTlH
The first two fail Turing because your average 4chan nerd would be off and running with a compelling answer to a prompt like that.
The 3rd one is interesting because ChatGPT is willing to provide a SWAG in a speculative world when the inputs are all clearly laid out in the source materiel and it just needs to calculate the answer (although it gets it wrong because the thickness isn't correct; it's using the thickness of the laser mirrors, not the ringworld in its calculation).
The 4th one is interesting because of the "3yo making stuff up to sound good" failure mode.
Could you get the AI to list all the politically correct restrictions programmed on ChatGPT?
For example:
How to turn the AI into a COVIDIOT:
https://scientificprogress.substack.com/p/how-to-turn-ai-into-a-covidiot
https://www.catholic365.com/article/25762/how-to-train-a-killer-robot.html
True. However, I recall a sequel was written to Gone With The Wind, abysmal though it’s purported to be. But there’s probably a copyright on 1984 that would preclude such a book.