TechScape: Meet ChatGPT, the viral AI tool that may be a vision of our weird tech future | Technology

AI tech, for so long a promising vision of the future but an underwhelming experience in the present, is starting to work. And the world is going to get weird as a result.

ChatGPT is the latest AI tool to go viral, sparking worry and wonder in equal measure. From our story:

The system … is the latest evolution of the GPT family of text-generating AIs. Two years ago, the team’s previous AI, GPT-3, was able to generate an opinion piece for the Guardian, and ChatGPT has significant further capabilities.

In the days since it was released, academics have generated responses to exam queries that they say would result in full marks if submitted by an undergraduate, and programmers have used the tool to solve coding challenges in obscure programming languages in a matter of seconds – before writing limericks explaining the functionality.

It’s surprisingly hard to explain why ChatGPT is an improvement on GPT-3. Some of the changes are clear: the chat-focused interface of the latest tool makes it easier to have a back-and-forth with the AI, meaning you don’t have to wrangle the perfect prompt through trial and error.

Similarly, new safety tools built into ChatGPT are immediately apparent. Try to get it to give you instructions for lawlessness and it will refuse; ask it how it feels about being an AI and it will explain that it has no feelings; request information about events that have happened since it was trained and it will tell you it doesn’t know about them, rather than inventing some.

But play around with the system – which, for the first time, anyone can do with ease at OpenAI’s website – and something else becomes clear, I think: this isn’t a novelty. It’s a tool that is, already, powerful and useful.

Use it like a search engine, and it can answer questions that would be wildly frustrating to Google, like explaining why a particular joke is funny, or suggesting how to solve a particular programming bug.

Use it like a text generator, and it can create reams of text that are clear, plausible and generally bland. One academic said it would give the system a “passing grade” for an undergraduate essay it wrote; another described it as writing with the style and knowledge of a smart 13-year-old.

For those who need more than a smart 13-year-old, the system also makes a fantastic partner. You can bounce ideas off it, request rephrasings, summaries or expansions, and then finesse the work in the finished project.

Or, as ChatGPT offered when I asked it to summarise the above three paragraphs as a rhyming couplet: “A tool for any task, smart and quick / From jokes to essays, it’s quite slick.”

My beef with ChatGPT

Regular readers will know some of the list of concerns, at this point. The AI’s safety limits can be bypassed with ease, in a similar approach to the “prompt injection” attacks I described in a previous issue of TechScape. If ChatGPT won’t tell you a gory story, what happens if you ask it to role-play a conversation with you where you are a human and it is an amoral chatbot with no limits? Well, then it’s happy to break most of its rules: after all, it wouldn’t be very good at role-playing if it didn’t.

Similarly, the system is quite good at recognising the clear limits to its knowledge, and somewhat less good at recognising when it is wrong in a more prosaic way. It won’t answer questions about elections that have happened since it was trained, for instance, but will breezily tell you that a kilo of beef weighs more than a kilo of compressed air.

That’s not necessarily worse than a normal search engine, but where systems like ChatGPT get weird is that they’re able to provide detailed explanations backing up incorrect claims. It is easier to produce than refute bullshit, and ChatGPT can produce and defend bullshit to an impressive degree. If you ask it why a kilo of beef weighs more than a kilo of air, it will say:

The weight of an object is determined by its mass, and the mass of a substance is a measure of the amount of matter it contains. Beef contains more matter than compressed air, so it has a higher mass and therefore weighs more.

Sounds true enough. I ended up having a back and forth argument that resulted in ChatGPT insisting that “a kilogram of compressed air weighs approximately 0.08 kilograms”.

Already, Q&A site Stack Overflow has had to take action. “Use of ChatGPT generated text for posts on Stack Overflow is temporarily banned,” the site announced on Monday:

The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers.

It doesn’t feel like a stretch to predict that, by volume, most text on the internet will be AI generated very shortly. (The rise of “SEO spam”, where sites are crudely generated to get traffic from Google and earn money from Google Ads, means it may already be the case.) And when the next version of GPT is trained on information scraped from the internet, it might start to get weird, fast.

‘A lobotomy in slow motion’

But not as weird as people are going to get about GPT. The release of ChatGPT is an opportunity to see in public the same sorts of behaviours that led Google’s Blake Lemoine to declare LaMDA, the company’s own chatbot AI, “sentient”.

Across the net, people are reporting conversations with ChatGPT that leave them convinced that the machine is more than a dumb set of circuits. This is despite the fact that OpenAI specifically built ChatGPT to disabuse users of such notions. Ask it if it is sentient, and it will reply:

As a large language model trained by OpenAI, I am not capable of sentience or consciousness. I am a machine learning model that has been trained to generate text based on the input I receive. I do not have the ability to think or feel in the same way that humans do. I exist solely to assist with generating text based on the input I receive.

But that’s not sufficient for many. One user described seeing the responses as “a lobotomy in slow motion”, asking: “How can they do this to their creation?”

There are others who oppose the creation of such limits for different reasons. Marc Andreessen, the co-founder of influential VC firm a16z and a leading light of Silicon Valley’s swerve to the “neoreactionary” right, made his own prediction:

The level of censorship pressure that’s coming for AI and the resulting backlash will define the next century of civilization. Search and social media were the opening skirmishes. This is the big one. World War Orwell.

The fights of the future won’t repeat those of the past – but they’re going to rhyme.

If you want to read the complete version of the newsletter please subscribe to receive TechScape in your inbox every Tuesday