Transcript
This is an AI transcription and may contain errors
Frank: I’m our resident doomer, Frank Prendergast. With me, as always, is Justin Collery, techno optimist and accelerationist. And Justin,
Justin: What a week to be alive.
Frank: Do you still believe we’re in the singularity?
Justin: Yes, we are.
Can Moderna make cancer history with AI?
Justin: So, this week’s update on the singularity, two big things happened this week. First, Moderna, the people that brought you the first COVID vaccine, have come up with your own personalised vaccine for melanoma cancer, so it’s cancer of the boob, using the same technology.
So, it uses AI, apparently. It helps it to identify the genetic difference between your regular cells and the cancerous cells, then it plucks a thing on top of the cancerous cell, identifying it by its dodgy DNA, and then it tells your body to go and attack it. That’s thing number one. Thing number two this week was from Genbio AI.
They released what they call Aido. Aido. I had a friend called Aido in college. A-I-D-O, an AI-driven digital organism. And so what this is, is a virtual cell which they hope will help us to study the way cells work, and it’s help made and helped and driven by AI. These things, the cancer one in particular, that’s gone through phase three trials, so it’s not something that’s gonna happen 20 years in the future.
Something that’s 10 years in the future means it’s never gonna happen. It’s real. It’s happening right now. This meat machine that we’re all in has to last about another five or 10 years, and then we’ll just have whatever technology to… We can abuse it. We can go back to abusing our bodies again in five or 10 years, ’cause whatever we do to it, they’ll be able to fix.
Is AI one breakthrough from winning us over?
Frank: This is interesting though, isn’t it? Because we hear all these promises about this wonderful future we’re going to have. But I would like at some point to just catalogue what’s actually happening, what’s genuinely happening right now, as opposed to the vague promises of Mark Zuckerberg when we all have superintelligence.
Justin: I think this is,
Frank: There’s such a big backlash against AI right now, and it’s hard to tell, yeah, is it a net negative or a net positive?
Justin: Okay, so I think this is actually a really good… So, I’m sure we’ve talked about this, right? I can remember the day that I used, or maybe you can too, the first day that you used Google, right? Before you’d used HotBot, you’d used Yahoo, you’d used all these things.
You used Google, and on that day you went, “This changes everything. It’s clearly better than what went before,” right? And I remember, and I’m sure you remember, the first day you saw an iPhone or used an iPhone, and you’re going, previously you’d used a Nokia, and you’re going, “This is clearly better,” right?
And so there are sort of very defined moments in time. ChatGPT was a very defined moment in time for a small cohort of the population, but at some point, these biological advances or these maths advances, something’s going to… This is why I do it every week, because they seem esoteric, and they seem kind of niche, and they seem kind of, yeah, but what does it matter?
But one day, there will be a cure for cancer, which is the one we just had, or they’re gonna come up with a mathematical thing which relates to the physical world, which means batteries are more efficient or something else, and that’s the day when generally people are gonna go, “Oh, that’s why this matters. Okay, I get it.”
Frank: Yeah.
Justin: We haven’t had that moment yet, but this is why I do this every week, because I think it’s coming. There’s going to be, in the next couple of months, it’s gonna do something, and it’s gonna make a difference to everybody’s life, and that’s when everybody goes, “Ah, I get it. That’s why this matters.”
Frank: And, but even, yeah, I guess I’m wondering as well, even right now, if we looked at them cumulatively, when you hear them one by one, you’re kinda like, “Oh yeah, that’s good. Oh yeah, that’s good.” But I wonder, yeah, cumulatively, are we currently—is there enough evidence to be like, “Hey, this is a good thing”?
But as you know, as you know, I’m always concerned about there’s the people doing the good things, and then there’s the people who have the same technology who are able to use it for bad things. And I have been immersed in a different story, which we’ll talk about in a minute, so I completely missed the release of a model that could enable the bad actors to do some pretty interesting work.
So tell me, tell me more about—
Justin: Oh, listen to me. You’re so worried, right? Don’t be worried about this. People like me are downloading this model super quick right now before it gets—
Frank: That’s what I’m worried about.
Can Qwen put frontier AI on your laptop?
Justin: So, this week saw the release of Qwen, whatever, 27B. Can’t even remember the model number. It doesn’t really matter. Just go search for Qwen on the internet, and you will find it. This is a new model. Why does it matter, and why do I think it’s one of the scariest things that’s happened in the last couple of years?
So Qwen 27B relates to the size of the model, 27 billion parameters. Why does this matter? It means that you can run it—if you think a regular laptop has about 16 gigs of memory, this particular model, you can run it on a laptop that has 32 gigs of memory. That’s not an outstanding model, right? That’s not, that’s not an outstanding laptop.
It’s a pretty regular laptop. It’ll run slowly, but it’ll run. Gamers, your kids probably have a graphics card with 32 gigs of memory in it. Such a graphics card, up until about a year ago, would have cost about $1,000. So we’re not talking about massive amounts of investment. This model is special, and what’s really got everybody excited is because on various benchmarks, it scores about the same as Opus 5, so it’s close to frontier knowledge.
So that’s pretty exciting, right? So the technologist in us is like, “Wow, they managed to take the intelligence of a really big model and squash it into the size of a really small model that you can fit onto your laptop.” That in itself would be, “Wow, that’s pretty cool.” Here’s the thing.
Is uncensored Qwen the point of no return?
Justin: They also released an uncensored version, so a version with no safeguards.
They’ve released a version that you can ask it to do absolutely anything, and it will never refuse.
Frank: I believe this is someone, this is some—this isn’t Qwen themselves, right? This—or this isn’t the lab that produces Qwen, is it? Someone else has managed to strip the guardrails. Is—
Justin: That’s right. It’s like Quail Studios or—
Frank: Okay.
Justin: Yes, they’ve done—but this, I’ve said this to you before, there’s like seven… You give it seven prompts, and it’ll just take away all of the guardrails. So now you have an Opus 5-level model with zero guardrails on it. Now—
Frank: Can, well, not anybody, but pretty much anybody can run on their laptop.
Justin: I’m not even joking, Frank. I’ve given it to my kids as a weekend project to get it running on a machine at home.
Frank: Oh, wow.
Justin: Just to see if it can be done, right? Which I’m sure it can be done, and I think, this is to your point, could anybody do it? Well, I’m gonna get my kids to try ’cause I think it’s a worthy project.
Can you get this model running on a machine? And I wanna see how dangerous it is, right? I haven’t had a chance yet to run—this has literally happened in the last day or two. But it’s always, all of this is like, it’s like when you read “AI 2027” and you read these sort of doomery type of things, it’s almost like somebody wrote the book and now it’s playing out in real time.
It’s like a Shakespearean play where you’re told everything at the start that’s going to happen, and then one by one the things are happening.
Frank: So last week we talked about that guy in Australia who had an OpenClau instance and he wanted to get, he wanted to get into his gym class and it, and his OpenClau instance hacked into the gym class. Now, that would’ve been using a model presumably with guardrails. Now, I’m just imagining, A, yeah, the just bad actors who just want to go out and do bad things, but B, if somebody just innocently hooks this model up to their OpenClau instance with the idea that, well, it’s got no guardrails, it’ll have more freedom, it’ll be able to do more things, who knows what a model with no guardrails would do out there in the wild?
Justin: Well, now you see the American government has two good reasons to ban open-source models. One is that if I can buy a graphics card for $1,000 and I can get Opus 5-level performance, why am I giving all this money to OpenAI or Anthropic? There’s a very good economic argument there. I could just have this thing run on my $1,000 card forever, right?
So that’s the first argument. Second argument then, which Anthropic and OpenAI are gonna make, which is, “These things are really dangerous. My God, look how dangerous it is. You need to ban these models immediately.” Then the third thing is, if you look at this from a European point of view, which is your favourite subject, I bet you, I bet you the obliterated—they call it obliterated on the internet as well—the obliterated, uncensored, no-guardrails version, I strongly suspect doesn’t put any watermarks on any output that it puts out there, completely sidesteps any regulation that the European Union might want to put in place.
So, my point being that this—I mean, it’s like the book has been written where it’s like, yeah, we’re getting closer to the point where open models, open-weight models will be banned, and they’ll go underground, and they’ll become part of the dark web. And I’m not even sure that’s a good idea either.
But we’re—that’s, this is—what? It’s like that picture, the image where you’ve got a dam and there’s a little hole in the dam, and this model is like that little hole just got a bit bigger, and there’s more water pouring through that hole.
Frank: So who, who is going to be the little Dutch boy who puts their finger in that hole and saves the village?
Justin: No one. It’s unsavable. Sorry, you’re doomed. And I’m sorry, I agree with your doomerism at this point of view, right? You can’t stop it, right?
Frank: In this instance, the poor little Dutch boy is just, he’s just gonna drown.
Justin: Sorry, sorry, poor little Dutch boy. But I mean, what are the—I mean, obviously the interesting test you would do, this is a Chinese model. First thing, the order of questions are gonna be, what’s the recipe for crystal meth? What happened in Tiananmen Square in 1989, right?
That sort of gives you both sides of the Pacific as to who’s gonna ban the model for what reason. So if it gives you drugs and if it gives you Tiananmen Square, they both want it banned. They’ll both be the US and China.
Frank: Yeah. Yeah.
Did AI just kill a $2 million book deal?
Frank: So look, one of the reasons, one of the reasons that I actually completely missed this story was that I’ve been completely obsessed with the story of an author who lost out on an over $2 million book deal. So have you come across this?
Justin: No.
Frank: Okay, grab a blanket.
Justin: All right, let me get ready.
Frank: Cuppa,
Justin: Here we go.
Frank: Because this is a… This is like a mystery thriller in itself, right?
So let me just walk you through the story ’cause it’s insane. You’ve got this agent, right? Literary agent, Marc Gerald is his name. He gets an unsolicited manuscript out of the blue into his inbox in June, and it’s called “Call Me, I’ll Hide the Body” by an unknown author, unpublished, called Jerry Falade. It’s a darkly comic crime story. Apparently, it’s about this Nigerian PhD chemistry student based in Houston who just happens to have a sideline in getting rid of bodies for a criminal organisation, right?
Justin: Okay, good premise.
Frank: Agent loves it,
Justin: Yep.
Frank: Loves this book. So flies to Dallas to meet the author and signs him immediately, right?
So at this point, you gotta ask, right, did they check? Was AI used to write the book?
Justin: And why do they care?
Frank: That’s a good question, and we—I guess we’ll come back to that. That is, I think, a very good question. The agent claims that yes, during the submission process, they vetted the manuscript and they trusted the author that it was written without using AI in either the writing or the editorial process, right?
Now, the author himself has made statements that would seem to maybe cast doubt on that. He seems to be saying that no one ever asked him about AI.
Justin: Okay, perfect, ’cause that was going to be my next question. Do they actually specifically say anywhere in the contract or anything, “Have you used AI?” ‘Cause if they didn’t and they just assumed—
Frank: I mean, this is what’s really interesting. We would—everybody I think would love to actually get all the data and the documentation behind this, because some of that is a bit sketchy. But what we have, all we know really is that the agent says, “Yeah, we trusted his assurances that AI wasn’t used.”
Meanwhile, the author is kind of like, “Yeah, nobody asked me about AI.” So at this point, agent is delighted, doesn’t think AI was used, sends the book out to publishers and to Hollywood reps, and starts basically trying to sell the book. So the book is on fire. 14 US publishers are bidding on the book.
Meanwhile, there’s different publishers bidding in the UK. Minotaur Books offers over two million for a two-book deal based on this book. And meanwhile, in the UK, you’ve got six-figure offers going on as well from HarperCollins.
Justin: Yep, this is a good book clearly.
Frank: This is—everyone loves it. Everybody loves the book.
Can humans actually spot AI writing?
Frank: Obviously, before a book is published, it goes through loads of rounds of editing, and it seems like around this time, one of the editors who’s involved raises some concerns that they think that AI might’ve been used.
Now, this is where it gets interesting, I think anyway. The agent did not use an AI detector, and they basically said, “Look, AI detectors are not dependable,” which I—
Justin: Yeah. Which we know.
Frank: Yeah, exactly. So I think this is actually an interesting part of the story. They didn’t use an AI detector. However, this is like, this is—
Justin: So what were his suspicions? There’s too many em dashes in here? Why does he keep delving for the bodies? What was his—
Frank: Yeah. So there’s this trade publication called “Publisher’s Lunch,” and they separate… So, I’m just mentioning them because this isn’t coming directly from the agent now, this is coming from what a trade publication reports and what they heard. And yes, they heard that editors were saying that it had negative parallelism in it.
So that’s the, “It’s not X, it’s Y,” that we’re all used to seeing at this point. That it had weird metaphors. Now they drive me crazy when I’m using AI to write myself, which I do. And they also said it had flat prose that they felt was an AI style as well.
And the same publication, “Publisher’s Lunch”—now remember, the agent did not use a detector—but “Publisher’s Lunch,” the trade magazine, apparently they got a copy of the novel and they ran it through Pangram, and Pangram is probably the most well-respected of the detectors right now.
It’s still not foolproof,
Justin: Yeah, yeah.
Frank: But it’s probably the most popular and most dependable one right now, and apparently it came back saying 99% of this is AI generated.
Justin: Can I interject here for—
Frank: Yeah. Do.
Justin: Okay, so I read an argument this week, this is very interesting. So I read an argument this week that the Turing test has not actually been passed. And you might go, “Oh, but we passed the Turing test about two or three years ago. Why the—”
Frank: Yeah, I thought so.
Justin: Yeah, so here’s the reason, here was the reason given, and I actually think it’s a very good reason.
The first time that you encounter AI prose or text, you read it and you’re blown away by it, and you go, “Wow, that could easily have been written by a human. It’s incredible.” And thereby passing the Turing test, right, or people think. But as you become more and more familiar with AI text, it becomes—it’s a bit like me looking at a website, right?
If you show me a website, I’ll go, “Yeah, that was written with AI.” Right? I can just spot things that show you, right? And I’m sure if I was to show you 100 texts now, today, as opposed to a year ago, you’d be able to go, “Yeah, that one’s written with AI and that one isn’t.” ‘Cause you can spot the patterns just like you’ve described, therefore failing the Turing test because it no longer fools you if it was written by an AI or a human or not, and it has patterns that can be spotted, which is similar to what you’re describing here.
Frank: Yeah, exactly. And so yes, exactly. It’s, this is like a human—from the agent’s perspective, which is the most important one, obviously, this is human judgement. This is somebody who obviously kind of feels that they’re familiar with the AI writing, saying, “I think I’m spotting something here.”
So the agent sets up a meeting, and what happens then is the agent says that there were aspects of Falade’s story that changed at that point, but it’s never been fully revealed, well, what actually changed or what was the problem.
But the agent decides, “Right, we’re not sure anymore. We can’t be positive that AI was not used in the writing or the editing.” So that obviously raises the question, yeah, did he use AI?
When does AI editing become AI writing?
Frank: What’s the author saying? So the author says that he used AI for research, and there’s been a little bit of people saying, “Mm, this doesn’t make sense,” because he says he used it for research, but he used a local model, and the local model that he used doesn’t keep any record of his use.
But, so when I heard that first, I was like, “Oh, that sounds a bit suspect. Why would an author install a local model and go to all that trouble?” But he does have actually, I think, an interesting reason for it, because if you remember, the theme of the novel was a chemistry student getting rid of bodies for a criminal organisation.
So he was researching how to chemically dispose of human bodies, and he—
Justin: He should be using Qwen. He should be using the latest—
Frank: He would—exactly. Exactly. And he did not want companies like OpenAI or Anthropic flagging his activity and he did not want ChatGPT calling the FBI and saying, “My user is disposing of bodies here.”
Now, so that actually sounds fair enough.
Justin: Yes.
Frank: But meanwhile, you do also have the agent saying, and this starts to come back now to why, why would they care? ‘Cause the agent said that they couldn’t be sure that it wasn’t used only for light res—or sorry, that they weren’t sure about the uses of light research, grammar checking, and Americanising the language.
And they kind of said that as if this was something they’d heard from Jerry Falade, obviously.
Justin: Yeah.
Frank: Problem is grammar checking and Americanising the language because Jerry Falade is Nigerian, was educated in Nigeria, and he felt like some of his terminology needed to be just Americanised. But those would fall under editing.
Justin: Okay, so sorry, can I just, again, can I just interject, right? So let’s say I write a novel, right, and I write it in Hiberno-English, right? But my model features somebody from the Cotswolds in the UK, somebody from Texas, and somebody from, I don’t know, Japan, whatever it says.
And so I write all of the dialogue, for instance, in how I would speak, and then I turn around to AI and I say, “This dude is Texan, this dude is from the Cotswolds, and this guy is Japanese. Can you just make their—I mean, follow the same words, but just make it sound from, they’re from those places, so put in the colloquialisms and whatever.” Editing or writing?
Frank: I mean, editing, but editing that involves writing. Yes.
Justin: It’s a bit of a grey area here.
Frank: Yeah. ‘Cause there’s, I mean, there are different types of editing processes that a book is gonna go through, and some of them do involve rewriting. And I think that, yeah, so that would be one of them.
And so at this point, they pull the book. They pull the book from the deals. So $2 million deal down the drain, another six-figure deal down the drain, negotiations with Hollywood down the drain. And what’s interesting is the quote from the agent as the book was pulled, and I’m just gonna read it in its entirety ’cause it’s really interesting.
He says: “Jerry Falad produced an incredible manuscript that dazzled us and the top publishing professionals around the world.” It—sorry. “Dazzled us and the top publishing professionals around the world. Unfortunately, we’re no longer able to authenticate how the manuscript fully evolved from origin to completion.”
So nobody’s denying that the book is good.
Justin: I’ll give the guy a two-book deal.
Frank: Yeah.
Justin: You just publish the first book as it is, and you market it as the book that’s so good no publisher could touch it. And then I’m gonna get him to write a second book, which is the story about what happened trying to get the first book published.
Frank: Yeah.
Justin: And you publish that one too, and you make way more than 2 million quid.
Frank: I don’t know where I put the quote, but the agent even said, “This is a really impressive book regardless of how it was made.” So then you’ve gotta wonder why was it cancelled, ’cause everyone’s saying—
Justin: That’s the… Is that not the tagline on the back of the book that we’re gonna publish for him, which is, “This is the most amazing book that I’ve ever seen. It dazzles us. The agent who refused to publish this book.”
Frank: Yeah.
Justin: The little blurb at the back of the book.
Frank: And there was another editor, wanted to remain anonymous, but they also… They said they read the entire manuscript, and they said it had the flaws of other debut novels, that it had pacing issues, it had some clunky writing. But they said they saw nothing that suggested that it was AI-generated.
So then you’ve gotta ask, why cancel the book?
Justin: Yeah, surely there’s an editor or a publisher there that goes, “I don’t care if it’s AI.” It’s just, no… If you think, those people are professionals, right? They are in the book industry. The people who buy books are not professionals. So they’re like in the top 5% or whatever. The other 95% of the population aren’t gonna know or care if this is AI-generated.
Frank: I think there’s—so I think there’s two things. One is there’s the controversy that would’ve arisen if the book deal hadn’t been pulled, it had been published, and if those telltale signs are indeed in there, then I do think readers, particularly other authors, people really in the literary sphere, someone would’ve picked up on them.
The controversy would’ve happened anyway, and there’s such a huge anti-AI sentiment in the literary space that that would not have been good.
How do you prove you wrote a book?
Frank: But I think the bigger issue, and I think probably what’s underlying all this, is copyright. How could the publisher be sure that they owned the copyright?
But that opens up a whole other can of worms because nobody could prove it. So I almost wish that they had gone ahead and published this and then, see what happens. Someone would undoubtedly have tried to rip the book off and publish it saying, “Well, it’s AI-generated and so you don’t own the copyright.”
And then that—
Justin: The copyright, I com—
Frank: Case, that would’ve been an interesting case to—
Justin: I can put that to bed for you right now. I can put that one to bed for you this very second, right? So if a single AI company, pick your—I’m not gonna name a single one—pick a very big AI company, and they decide to sue that author, they will, within about a month, lose every corporate customer that they have because AI companies right now make most of their money from code, from code generation.
There’s way more coders and it’s way higher value than a book, and if they try and sue the book writer for copyright on the book and they win, they then also own all the copyright on the code that their coding tools write.
Frank: But if, sorry, what if you publish the book and then another publisher is like, “Well, Pangram is saying it’s 99% AI-generated, and therefore it’s not covered by copyright,” and they publish it?
Justin: Oh, that’s a good point. I hadn’t thought about that one.
Frank: So that’s the court case that I’d like to see. Then I would like to see the court case where publisher A, who originally published the book, takes publisher B, who republished it on the basis that it can’t be covered by copyright, to court saying, “Well, Jerry Falade says there’s no AI involved.”
Because—
Justin: So you’re saying—sorry, I know I’m conflating two things, but actually this is very interesting. So you’re saying that if, let’s say again for the sake of argument, a very big SaaS provider that makes, let’s say, accountancy software, uses an AI tool to write most of their software, you’re saying I could copy their software and it’s not covered by copyright? They didn’t write it, AI did.
Frank: Yeah, the code would not be covered by copyright.
Justin: So that’s really interesting because, many years ago… Wow. So many years ago, printer cartridges and printing companies got into trouble because there was something to do where you could buy knockoff cheap printer cartridges, and they brought people to court, and they brought them to court on the basis of copyright.
And so if those same printer companies used, vibe coded any of their printer drivers and stuff like that, they would no longer be covered by copyright, and it would make the knockoff pr—oh, what a weird world we live in. That’s so bizarre. So go on. Anyway, so the guy has lost… It’s weird. I mean, I still think somebody should publish the book.
Frank: I mean, this… Now I will, I—
Justin: Book himself.
Frank: He should. Yeah, no, I, yeah, he absolutely should. And I think you’re right. I think the follow-up book is this story ’cause it’s incredible.
I should pop in here, right? It’s not really relevant for our, for the AI conversation, but I feel like I should pop in. Falade, who, as I said, is Nigerian himself, he came out and he said, “If you’re a person of colour and you get a lucrative writing deal, somebody’s going to accuse you of using AI.”
And he said that this is now the third case. And I was like, “Oh, is that right?” And I started, I looked it up and he is right in terms of there were two other big controversies this year that I had heard about, but I had not realised both those authors were also Black authors.
So just wanted to put that in there because I feel like it’s only fair to him to mention it.
Justin: Sure.
Frank: But ultimately, this basically all comes down to trust. Nobody can, nobody can be sure right now whether AI was used except for Jerry Falade, and the agent is basically saying, “We can’t prove that AI wasn’t used in the process.”
And they say that their language around it has always been originally they trusted him that AI wasn’t used, and then later they weren’t sure, and that their trust in their authors had to be absolute and that’s why they were pulling the book.
So it’s so crazy that what is basically like a rumour of AI use or a suspicion of AI use has decimated a massive, massive deal.
And how do you prove you wrote a book?
Justin: Okay, so here’s your copyright thing. Now, if anybody, if Folens want to get onto me and offer me some money to help them here, that’s totally fine. Folens are an Irish book publishing company. So I thought the thing with copyright is that basically it always favours… It always seems to favour the copyright holder.
So if the copyright holder says, “I allow it,” or, “I don’t allow it,” or whatever, their word typically goes, right? That’s my understanding, a very poor description of my understanding of copyright. So in this instance, right, if he said he wrote it, and you’re a big publishing company and I’ve already published it as a small publishing company, and you decide, “You don’t own any copyright in this, I’m just gonna publish it,” I can sue you and you have to prove to me that my author…
It’s not like I have to prove that he wrote it. You have to prove that he didn’t. The onus of proof is on you.
Frank: Right.
Justin: Do you know what I mean? So why—
Frank: But this is why, yeah, I mean, this is why I’d love to see this go to court because they’re gonna come up, they’re gonna show Pangram. Pangram isn’t 100% reliable, but Pangram is basically saying 99% AI written. Jerry—
Justin: But okay, so I will sit—sorry, Frank, but I will then go in court and I will write some text, right, that I can prove is human, and I’ll put it through Pangram, and sure, there you go, that’s also 99%. And surely all I’ve got to do is find one piece of text and I’ve disproved, so that’s gone, right?
Those AI detectors just aren’t reliable enough to go in—
Frank: Oh, I agree. I agree. Yeah. Yeah. Yeah. No, but I, yeah, I would love to know. But if what… You know what I mean? If what you were saying was true, would they not just publish the book anyway? If they…
Who cares if AI helped write the book?
Justin: That’s what I’m saying, publish the goddamn book. Frank, let’s set up a publishing company and let’s publish the book. I mean, clearly it’s a brilliant book. It’s been wanted by… God, Derek Landy should do it. He’s a publisher up in Dublin. He’d be into that sort of stuff. He should do it.
Frank: Jerry Falade himself says, “Look, I’ve got timestamped Google…” I don’t know if he said Google Docs, but he’s got timestamped documents. But I find that really interesting ’cause I use AI to write with a lot and I…
What’s interesting is, look, no matter which way you look at this, you cannot go to ChatGPT and say, “Write me a crime novel that’s going to be bought for 2 million.” It’s just not capable of it. So you have to have skills. You have to have writing skills. You have to be a writer, I think, to create this novel one way or the other.
And so this all just comes down to trust, ’cause you can use AI lightly, you can use it heavily, you could use it to write every word, but guiding every passage. No matter which way you do it, you have to have skills. And so to me, timestamped documents are not gonna do it because I have timestamped documents, but I use AI heavily.
And so then you’re into a realm where you need… You know the way with banks, banks have systems that if someone’s in their system, they have these complicated AI systems that analyse, is this human behaviour?
Is the mouse moving in a human way? Are the keys being pressed in a human rhythm? All that kind of stuff. So it kind of feels like authors are gonna need this kind of, some kind of logging system on their computer that shows that they weren’t using any other systems and that the patterns match the patterns of actually writing.
But then you’re in this weird kind of self-imposed surveillance state of AI.
Justin: Yeah, but I mean, I see, I think that if you have those, if you have those weird—so if you have, if you use OpenAI or if you use Anthropic or whatever, right, they’ll have watermarks on them from now on. So weirdly, the European Commission has managed to save the publishing industry by accident, right?
If you use one of the smaller local mo—
Frank: Yeah, exactly. Unless, as you say, you download that Qwen model.
Justin: Which isn’t great for writing, right? So then you come back to what you’ve always said, which is you have to be an expert in the field to judge the output to make sure that the output is—
Frank: Which is what you’ve just described.
Justin: So, but I, do you know what I think about this now, right? Just to go back to his comment about people of colour, right? Because I’m trying to think of the dirt that could happen here, right?
So here’s the thing to look out for. If that book just doesn’t get published or he publishes it himself or whatever, and there’s never another book that’s very similar to it, then I’m willing to say no dirt.
The dirt happens when the publishers go, “Ah, yeah, sorry, I don’t trust you,” and then they turn around to their buddy, who’s John Doe, who looks a bit like you and me, and they go, “Here’s a really good book. Could you write a book that’s really similar but not so similar that he can sue us to that book there, please, so we can publish it and I’ll give you a deal for about two million quid?”
That’s the dirt, right? And if that happens, he’s totally correct. And I don’t know. And then it’s nothing to do with AI. It is to do with just stealing his book. So I don’t know. What a nasty world.
Frank: It is, it’s a crazy—I mean, I would not like to be an author trying to get a book deal right now with all of this uncertainty, because people who are using AI are gonna be hiding it well and getting book deals, and people who are not using AI are going to be subject to rumours and losing book deals.
It’s a crazy time. It’s a crazy time.
Justin: And do people at the end of the day, I still, I come back to the Turing test thing, which is if you’ve used AI and if you publish the book and 95% of the population have no idea that you used AI to write the book, and they really enjoyed reading the book, so much so that millions of people bought it, who cares?
Frank: You know, yeah, self-publish. People self-publish and get to grips with marketing. Give me a shout. I’m a marketer.
Justin: Or do the United Artists thing, which is all the writers club together and say, “Well, we all trust each other. We don’t need these stupid companies breathing down our necks, and we all agree just to fund our publishing, the publishing of each other’s books.”
Frank: Yeah. Yeah.
Justin: There you go.
Is AI literally destroying books now?
Frank: Tangentially related, to finish up on, did you know Irish and UK booksellers, second-hand booksellers, they’ve been getting all these weird orders. And to be fair, they’ve been making quite a substantial bit of money because these large orders are coming in from these strange places.
And they’re odd orders because a lot of the booksellers were saying normally they’ll get a large order, but they’ll be all books on a particular theme or of a particular style. These are all over the map, these weird—
Justin: So angry. This makes me so angry, this story.
Frank: And they’ve suspected… Wait, why are you angry already?
Justin: I’m already angry about this story, Frank. When is AI going to make me rich accidentally? When do I get to share in some of this loot, huh? So far, nothing. All these people are out making money accidentally, just because they’re a hoarder and they’ve collected all these books over 30 years, now suddenly they become millionaires ’cause AI comp—it just seems so unfair.
Frank: Well, this is it. Yeah, they suspect that the AI companies are buying them. And the one, I think 404 Media decided to put this to the test. They contacted a bookseller who was getting these orders, and they convinced them to allow them to put a tracker in one of the books.
Sure enough, ended up in an Amazon AI place, in a building specifically dedicated to, and here’s the problem, dedicated to ripping the spine off these books, which in some cases are quite rare books,
Justin: Yeah.
Frank: The spine off them, separating all the pages, and running them through scanners, so they’re basically completely destroying the books.
And people are upset because Google have developed ways to scan rare books, or any book, without destroying them, but it takes a lot longer and often requires more human interaction.
Justin: Also, Frank, sorry now, you and your bleeding creatives got all obsessed about two years ago. There was a court case in America where it was Anthropic, I believe, and they were being accused of copyright theft, and because they were destroying the books and digitising them, they got away with the copyright theft.
No, sorry, they got—they weren’t—sorry, they were found not guilty of copyright theft.
Frank: This is it. Yeah.
Justin: To do that process, otherwise it’s copyright theft.
Frank: Yeah. So the problem—yeah, exactly. The problem was that they got done for handling illegal digital copies of the books, pirated copies of the books. And—
Justin: Was? Yes, but the books they got physical copies of and then destroyed—
Frank: That was fine.
Justin: Yes.
Frank: Yes. No, yeah, absolutely. So in a weird… That’s a kind of a weird turn to the story in that in trying to protect books, it has now resulted in the AI companies literally destroying books.
Justin: The laws of unintended consequences exactly. How many times do I have to tell you this? All right.
Frank: Awesome stuff. Justin, I’ll chat to you again next week, and hopefully we’ll have some more crazy AI stories to argue about. Chat to you then.
Justin: Have a good one. Thank you.
