But GPT 4 is still assembling text or images or sound from probabilistic models generated through proximal relationships. It cannot know; there is no place where its knowing could take place.
The question I have: is knowing necessary? From what I understand about how these AIs are built and trained is they're basically black boxes – the engineers understand the principle, but don't know anything about how the software implements those principles. They can't know, yet not knowing doesn't prevent them from creating the program anyway.
In other words, to what extent is it necessary to understand your actions for those actions to be effective?
Very true. You could also say a 747 does not fly like a bird flapping its wings, but it still gets us from A to B in the air which is remarkable on its own.
I don't see LLMs as intelligence, but it's something significantly more than "just" auto complete as some say.
I just asked GPT, and it gave the right answer for the wrong reasons:
The ball broke the table because it was made of concrete.
The ball broke the table because it was made of cardboard.
Me:
What does "it" refer to in the above two sentences?
GPT:
In the first sentence, "it" refers to the ball, since the ball is the subject of the first clause ("The ball broke the table") and is the most recent singular noun preceding the pronoun "it".
In the second sentence, "it" refers to the table, since the table is the object of the second clause ("...because it was made of cardboard") and is the most recent singular noun preceding the pronoun "it".
We had a brief exchange on a previous thread but I find this one interesting because it’s the most fundamental disagreement I have with you, and with many other thinkers I respect deeply (and I think that about yourself) including Steven Pinker, Chomsky, Dennett, and many others. It would take at least a full essay, maybe even a damn book to lay out why I went from ‘AI is a joke’ to ‘holy shit we’re all replaceable’ after I first worked with machine learning in 2016, but I’ll throw one point out here.
Whether AI is conscious or sentient is at its roots a philosophical discussion, a fascinating one, but as a concrete impact to humans, sort of irrelevant. It doesn’t matter if it’s conscious, it matters if it’s competent. Get Stable Diffusion and a newer modelset, see the deepfake capabilities and ask how much you can trust the images you see on the internet. GPT-4 makes ChatGPT looks like a joke, and ChatGPT could easily replicate every opinion article written by a boilerplate left-liberal opinion writer, as long as you know prompt engineering. I’m always cheerful for discussions on this, even though I axed my plans for philosophy postgrad to go back into cybersecurity the intstant I realized the disaster this would cause the humanities.
Also, Eliezer has completely lost the plot. He’s had good points in his time about logistically how an AI could jump an air gap via deception, but in his 5,000 reasons for how something more intelligent could kill us all, in 10 years I haven’t seen him give a sensible reasons on exactly Why it would kill us all. The interviews with Sam Altman then him with Lex Fridman are worth watching, one can see Eliezer is truly losing it.
But here's the issue: the failure case of the chimpanzee example is possible because there's nothing in an LLM that can notice that a response doesn't make sense. All there is waste probabilistic strings of text. They're generated via an incredibly sophisticated system, but they can't apply tests of basic sense because they have no sense-generating instrument, only distributional semantic models that say that a given string of tokens is the most likely correct answer to a query. And so even if the failure rate is reduced dramatically, there's still this fundamental issue is lacking the reasoning and logic that make error apparent.
You’re wrong though. As they get larger, they do gain more ability to identify correct from incorrect. The same is true as they get more context. LLMs trained with images make fewer mistakes than ones that can’t for instance, because they’ve incorporated that visual information into their understanding
Correct and incorrect are not the question. The question is, can it reason based on using logic and a theory of the world that stems from embodied consciousness? It doesn't matter how sophisticated the models get; they're not even attempting that.
Wouldn't the model itself, as an abstract imprint of the data it's been trained on, be basically a "theory of the world" in much the same way that a memorized map is a theory of a location? And if that's the case, then what about a relational map where I can't paint an abstract layering but i know if I walk to one rock then turn I'll get where I'm going? Is that not also a theory of a location? I don't know much about the philosophy here, so I don't see why the answer to "where is its theory of the world" can't be it is itself the theory of the world.
What would it take for you accept that it was conscious? If GPT5 answers all your questions correctly and you have hours long deep philosophical discussion with it, would that be enough?
Freddie, where do you think consciousness resides? And before you say the brain, where specifically in the brain? (And I’m assuming you don’t believe in souls or some kind of immaterial consciousness?)
Be careful confusing consciousness and intelligence.
Consciousness isn’t measurable, nor is it remotely understood. We have absolutely no idea if these things are conscious and the safe bet is that they aren’t.
But they are intelligent, and they’re only getting smarter. That you can measure.
But that doesn’t matter for beating human capabilities in intellectual tasks, one by one. I was called a madman by artists over the last 7 years for saying no, machine learning could replicate digital art, but in March 2022, I, the crazy one, thought something like Stable Diffusion would be free in the 2030’s.
GPT-4 shatters benchmarks even on ‘common sense’. ChatGPT did the BAR and LSAT in the 10th percentile, a little over 3 months later, GPT-4 is in the 90th for both. That is a truly staggering rate of improvement, and these are both censored models. Microsoft’s researchers published a paper saying the uncensored GPT-4 they had access to had properties of AGI. But AGI, terminology, it isn’t really relevant to this. What’s relevant is disruption. I laid out to the head of a philosophy department how I could bypass both the 5 paragraph limit and reference blocking limitations on ChatGPT to write an A paper. Check Reddit for examples of this. My best friend’s an English teacher, it’s a running joke that the essays are fake.
Let’s say GPT-4 doesn’t exist, and everything halted at ChatGPT. The education system would still be in shambles, especially anything assigning take home essays. “Adjust it in the style of the New Yorker”.
Philosophy professors are already using AI to grade papers written by AI, so the humanities are a farce. This does not stop with education and digital artists. You’re in a good spot due to having a heterodox writing style and subject, but I worked in tech for a decade, including with machine learning, and I can lay out step for step how literally every white collar aspect of the business, of every business I worked in, will be chewed through by this, except those involving physical labor.
The bar exam and SAT examples are almost certainly memorized examples in the training data. We can't say for sure because "Open" AI has put a moat around their research methods and will no longer elaborate on even the broadest details of how their models work in their papers. Which, frankly, makes what they're doing more like pseudoscience than advancing a common understanding of what "intelligence" means.
Memorized examples is not how the LLM’s work. The parameter numbers for ChatGPT vs. GPT-4 are not that different, GPT-4 is trained for longer. Generative models do not memorize, that’s not how they work. Cleverbot could memorize the questions for an LSAT, that wouldn’t be impressive at all.
What’s different about these LLM’s in particular in generalization. Give GPT-4 a shot, it costs a bit a month but throw non-memorizable LSAT type questions at it, ones that have never been on a test. I guarantee it will do quite well. And the uncensored version only Sam Altman can play with? It’d probably get a perfect score.
Yes and the car massively disrupted human civilization. You don’t need to believe Sundar Pichai or Satya Nadella, or Sam Altman that this is a more important moment for the genus Homo than the discovery of fire, there is no need to compare machine learning to human intelligence, or the gibberish of the black box of machine learning to the processes of human learning. It doesn’t matter. Watch the Youtube videos or comments of artists at the existential crisis Stable Diffusion has caused them, read the threads on how many education assignments are now fake. My colleagues even in tech sales are composing most of their work with AI, and those who aren’t are falling behind and will get the axe in the tech layoffs.
This buzzsaw is going to chew through every single bit of knowledge work, the creatives first honestly. My Mom was an artist before her mind collapsed, even though I love machine learning it truly hurt seeing the devastation done to artists by this and reading about the drying up of commissions, but they never listened. You will be safe Freddie, but any white collar or creative working reading this, just be prepared, don’t be blindsided like the artists I’ve read having psychological breakdowns.
GPT-4 performs markedly differently on problems of the same difficulty class depending on whether those problems were online before or after the cutoff date for its training data.
"...we tested it on Codeforces problems from different times in 2021. We found that it could regularly solve problems in the easy category before September 5, but none of the problems after September 12.
In fact, we can definitively show that it has memorized problems in its training set: when prompted with the title of a Codeforces problem, GPT-4 includes a link to the exact contest where the problem appears (and the round number is almost correct: it is off by one). Note that GPT-4 cannot access the Internet, so memorization is the only explanation."
Yeah I worded that pretty poorly, since LLMs are predicting the next word (or similar) one after the other (the new ones do more than that but let’s stick with that for brevity), it’s definitely easier to predict the next term on something it’s already seen, and since it can tell you a famous quote verbatim, it can replicate text its seen precisely.
I just read that post though and while I don’t wholly disagree, I think it’s a non sequiter in a lot of ways, and also presumes that AI is written in amber when it’s the fastest progressing technology I’ve ever seen. I’m curious as to the authors predictions the author has made for when something like Stable Diffusion wouldn’t just be possible, but free and open source, or when GPT-4’s capabilities would be possible, I could stand corrected but I highly doubt he was predicting now. Free to use text to image, even most AI people I saw figured 2030’s, yet here we are. As for benchmarks, he makes good points, but as Freddie points out in the Cult of Smart, which I highly suggest, SAT/ACT and especially IQ tests are pretty damn good at measuring important human aptitudes for knowledge work tasks, flawed as they are when it comes to couting for past group injustices, I’m not a believer in The Bell Curve bullshit by any stretch, and Freddie tears that apart well in the book.
I’ve heard “AI can’t do that” so many times in the last many years though it’s become a huge headache. Every time the prediction is wrong, then the people who predicted wrong are still treated as if their next prediction is the one that’ll be right. The goalposts have been moved in this debate so many times they’ve left the stadium, went into the parking lot, and are now crossing the street into a nearby neighborhood.
If we value education, which is a big If...assessment will no longer be take home essays. Writing will be done in class because the assumption has to be that the take home product will be fake. More assessment will be on face to face interactions and in-class work. Good teachers will adjust. Good students who want to learn, will learn.
OK, this is such a good thread I should try to add some of my real world experience to it, see if it's useful. Again, thanks to JesterColin and everyone who wrote such thoughtful responses.
I've been an attorney for 20+ years. I have a lot of access to products being developed in the legal tech field - I'm not employed to work directly on the products themselves, but I'm close enough to the industry that I'm often involved once the need for practical legal knowledge goes beyond "law degree". I'm not claiming expertise on AI or how it works. But I HAVE had a lot of behind the scenes experience with machine learning and AI in the law.
I'm not losing sleep over AI's impact on attorneys, even though "law" is often discussed as one of the industries likely to be massively disrupted. The first problem is the obvious one - when I ask ChatGPT or the like a legal question it gives uselessly vague and incorrect answers, and if I ask it legal nonsense it makes something up. Fine, maybe refinement will improve this. But the main issue is more what Freddie outlined, and I don't see how it gets solved.
Practicing law isn't just knowledge of legal standards and drafting persuasive words; it's factual investigation, negotiation, discussion, synthesis, and knowledge of the particular persons involved that is needed in BEFORE you come up with legal strategy and draft those persuasive words. They're inseparable.
I can't see how it gets there. If (for the sake of argument) I agree that eventually we might be able to feed AI deposition testimony, witness statements, discovery responses, reams of document production, and privileged knowledge from a client and in response it could spit out... something. I actually don't even know what people think might happen. Motion papers, I guess? Even that seems incredibly unlikely, because it can't do the real work that I get paid to do - it didn't take the depositions, the witness statements, collect the discovery. Those things require Freddie's theory of mind - they require physical interactions, an ability to work with and understand reality, not just manipulate language. And they aren't tasks separate from drafting. They're necessary to the process.
There are areas of practice where I can see this being really useful, but my point is broader: I feel there's some Gell-Mann Amnesia going on. The media tells me that my industry is going to be turned on its head, that AI as able to write and research as well as lawyers; but I'm close to this stuff, and I'm confident it's not happening. Then I think: but sure, it could replace a doctor. It's probably better at diagnosis! Or that it could write good marketing copy, or create commercials, or write code, or write short stories. But it's the same media saying these things that told me we're on the brink of robot lawyers. So I'm starting to doubt that the impact is going to be as widespread as advertised.
Always good to read a thoughtful post! Hopefully this response doesn’t end up too long. Firstly, remember we’re in the infancy of these tools, they only became possible in the early 2010’s and didn’t really start working until 2020, there’s a pretty fun history on why if you become interested in this stuff. I got my first iPhone in 2008, early among people I know. I was the only one my friends who had one and when we needed to go anywhere they’d joke “ask your magic phone”.
I was flabbergasted that something like the iPhone existed even years into having one, the idea of a supercomputer in my pocket was Star Trek to me, but there it was. The gravity of the situations with smartphones didn’t sink into the general public until the mid 2010’s, and now we live in that future, for better or worse.
I don’t say all this to claim to he some future-predicting wizard regarding technology, I mean I missed social media being world changing, but even as a guy who followed machine learning all these years, as I mentioned not a single person I know, not one, even in tech thought LLMs like we have today would be available in the 2030’s. 95% believed that especially art could not be replicated period, that it was uniquely “human”, whatever the hell that means. That idea fell apart I figure in November when a usable Stable Diffusion UI came out.
Me, the delusional one, was so shocked when I first used ChatGPT that I dropped my damn phone. This wasn’t supposed possible, not for a decade, and I found it absolutely insane that it was free. Then I said “well no one will connect one of these things to the internet until like 2025, because that’d be insane”, 2 and a half months later, Microsoft does it with Bing. Combine those with the massive improvements from ChatGPT to GPT-4, and remember, this change to people’s reality has taken months, not years like the smartphone, or social media. No technology has been this impactful this fast in my lifetime, and those skeptical of its progress are the same people who literally didn’t think things like Stable Diffusion/Midjourney or GPT-4 could exist, even theoretically, and they believed that in 2022. I was seeing “lol but AI can’t do hands” even last month, right before Midjourney cracked that problem. Look at an AI picture from March of last year and March of this year, the difference is so mindblowing to me it’s hard to put words to. Check out AI music as well, I figure 2 years, that nee Taylor Swift album ain’t gonna be Taylor Swift.
The parts you mention in your career, definitely anything in person, those are safe from AI for a good while, (although I’m wrong by a factor of 10 often) I’m just not a huge believer in the progress of robots, or in person AI as compared the that in the cloud, but anything documents related, analyzing text? I worry I’ve already been too long winded here but there is no theoretical limit on it becoming better at that than humans, much better, and soon.
As for ChatGPT, never look at it as if you are bound by an Iron Law that you have to use the exact output, in fact that’s always a bad idea. I feel bad for the thing (no, I don’t think it’s alive) because the amount of guardrails slammed on it since the Day 1 version have basically destroyed its functionality, specifically around things like longform creative/analytics text creation. ChatGPT post-January is a shell of what it’s capabilities are with no guardrails.
Sorry for the massive wall of text, I really don’t get to have in depth written discussions with people, especially on my favorite topic, so I jump on the chances I can.
In contrast nothing about current AI capabilities shocks or wows me. It all seems like interesting parlor tricks. Which is to say somewhat useful in certain domains, maybe, but nothing world changing.
How subject is it to bad imput? What if there is something considered to be scientifically true, but the initial data is poor and hasn't been replicated properly? There are a number of articles lately about poor peer reviewed journal articles and outright faked data. Does AI just run with it?
I'm not actually as worried about creative endeavors because I think those will continue because people enjoy doing them. I write stories for fun. I'm never going to make any money at it, but I do it for fun. A few people like my writing and that's enough.
I think people will continue to paint and draw, because they enjoy doing so. I think people will continue to play music together for fun. The change may be in the monetary reward, or lack thereof.
I’ve been a software engineer for 20+ years and a lawyer for 12, your comment made me realize one aspect of the AI true believer thinking is to underestimate (or in some cases categorically misunderstand) what it is people in a given profession actually DO.
There's also the whole "internal model of the world" thing. GPT's context, though very large, isn't a real "model of the world". A lot of failure cases have to do with that lack of physical modeling. Y'know, putting X in Y, then Y in Z, then moving Z, those kinds of things.
There's also the question of memory and knowledge management when introduced to non-trained information. As I can personally attest from trying to run a D&D campaign with the thing in a homebrew world, the human still needs to handle the remembering of anything new introduced. (Otherwise, the damn thing falls back on cliches.)
There's some interesting work going on in both areas, but my money is that it'll take a fundamental change in AI architecture.
Maybe so, I’m not a computer scientist. Sounds reasonable to me though. My point, which I’m surprised Freddie doesn’t seem to agree with, is that if you accept materialist principles, then the difference between an AI and organic intelligence is a matter of degree, not kind. Consciousness, sentience, etc may be different, and I’m partial to the view that those are in fact physiologic process that silicone can’t truly recreate. But in terms of what we would broadly call intelligence, I’m not sure there is a big difference ultimately.
LOL I just looked up autocomplete. I find it a distraction. Something about it bothers me. But I'm old. I also turn my lights off and on by hand. I also cut my own wood with a chainsaw, haul it to the woodpile and heat my house with it in the winter.
Perhaps I'd have slightly more sympathy for Yudkowsky if he hadn't spent the past decade and a half persuading people previously interested in finding effective treatments for diseases and reducing global poverty that they should instead be researching the very AI systems he now thinks should be destroyed by airstrikes.
But instead of a reasonable conversation about likely risks informed by researchers who understand the current impacts of AI, we're debating how seriously we should take the night terrors of a bunch of people who read pseudo-Bayesian Harry Potter fan fiction. We are indeed well past the realm of reasonable discussion of any of this.
For the technology to be any good, I think we need to combine learning with human input.
Recently, I tried using AI to generate an image of a bowling ball. It doesn’t understand the number or placement of the holes, probably because photos of real bowling balls appear to have 0, 1, 2, or 3 depending on the angle. So it gives me balls with one big hole, or 4+ holes in a random pattern.
I just want to tell it “there are always three holes.” Just like I want to tell it “Humans never have more than two arms. Literally never. Maybe you thought you saw it, but that 3rd arm belonged to the person’s friend.”
Anyway, it’s very depressing to see shitty AI replace real content. The whole reason I was trying to make a bowling ball is because my app (Photoleap) replaced its stock photo collection with AI.
So, great…. no more licensing fees for Photoleap. I’m sure it will save them money in the long run. But I can’t even get a bowling ball, let alone a person who doesn’t look melted and deformed.
Maybe, or maybe “Theory of the world” is not as well defined as you make it out to be. Maybe it’s just an emergent property of sufficiently sophisticated statistical models.
The academics you’re sighting have consistently made predictions that have been proven wrong. Their ideas about intelligence are increasingly considered mistaken.
And you’ve misunderstood the significance of this moment. It’s not just that AI like ChatGPT are good autocompletes, they have fundamental properties that no predecessors had, and these properties appear to emerge from nothing more than increases in scale. This includes, in the case if GPT4, the emergence of agentic behavior, the ability to make and execute plans. Google put an LLM into a robot and it has the ability to perform novel tasks on which it wasn’t trained.
No, it’s not sentient. Maybe there won’t be a use for them that really transforms society. But you’re vastly underestimating what can be achieved with a simple input-output paradigm. The fact of the matter is that LLMs *are* something new, poorly understood, and potentially very powerful, unlike anything that has existed before.
I wish I had more time for a deeper response, but as someone working in AI, for all the hype, there’s something very real happening here.
It most certainly is very well defined. If you have specific test that a model can’t pass at one size, but then can when it is larger, then it is an emergent property. GPT2 could not do basic math. GPT3 could multiply 2 digit numbers, GPT4 can do calculus at the level of an average high schooler. These were all features that emerged on their own as a side effect of being larger.
Again: its mechanism is distributional semantics. The position of words near each other in language. This is entirely different that what we mean by knowing. If I had never had a cheeseburger but knew where the word cheeseburger was found in data sets, would you trust that I understood what a cheeseburger was?
At what point would you accept that it does know what a cheeseburger is? What if it could draw it? Create a 3D model of it? Design new recipes? Cook it itself as a robot? Where is the line here?
Also, LLMs can do what’s called zero shot learning, where they can complete a task they’ve never seen before using only a text description. Where does that fit in?
More to the point: How much does it have to be able to do before you’ll admit it’s actually intelligent? Where is the line where you would admit that it has something that could be called understanding? If the answer is “nothing”, then you may want to consider how scientific your stance really is.
Where in the brain does the consciousness reside? The prefrontal cortex? It's connections to the ventral tegmental area and nucleus accumbens that drive goal-directed behavior? The hippocampus where memories are stored? Layer III pyramidal neurons storing short term visuo-spatial memory? Fear processing in the amygdala? Or maybe the "consciousness" is an emergent property of many networks working together. If you meticulously measured every neuron in a human brain and coded its characteristics into a database, congratulations, you now have a conscious database. Not saying LLMs work exactly like a brain, but asking for the "conscious part" to be pointed out is a bit silly.
Can ChatGPT imagine? Can it speculate? Can it ask "WTF just happened?" and change its models, the way Rutherford did when he conducted the experiment that led to the discovery of the nucleus?
According the the then-current models of the atom, it was the equivalent of firing shells from an eighteen inch naval gun at a sheet of toilet paper and a *very* *small* *percentage* of those shells bouncing off the sheet of TP.
It can certainly change its models. As to the rest: who knows? I’m not arguing it’s conscious, or that it has experiences. I’m arguing it (or rather some near future version of it) is intelligent, that it has the capacity to understand and act towards goals.
That's a good question. Can it imagine? Can it come up with unique ideas.
I heard that this new tech is able to accelerate the ability to model the folding of proteins. This ability has tremendous potential. But could it come up with the idea of the importance in how proteins are folded in addition to their chemical bonds? Could it have originated that concept of folding proteins as an area to be explored and modeled?
I think that's an oversimplification of the mechanism.
Internally, systems like these aren't just representing the probabilities of words appearing in a given sequence. That's what a Markov model does, and it produces amusing gibberish.
These systems are building layers upon layers upon layers of connections between input tokens and output tokens. Represented in those middle layers are general patterns that aren't literally present in the inputs, but that have emerged during training because they're useful for linking a variety of inputs to their corresponding outputs -- in other words, *abstract concepts*. Those abstract concepts are inferred from the training data, and they can be expected to map to real-world concepts if the training data was written about real-world concepts.
Similarly, AI models that generate or recognize images tend to contain representations of the "concepts" that make up an image. For example, a system that recognizes dog photos will have components for recognizing paws, eyes, snouts, or tails; and those will have their own components for recognizing the curves and patterns that comprise them. You can see this in action with e.g. Google's DeepDream, which runs a recognizer backwards to amplify everything in an image that reminds it of something it could recognize, turning it into a nightmare of eyeballs and vaguely organic-looking shapes. Those components aren't hand-coded; they emerge during training.
So, for your questions about cheeseburgers and sushi: if you had read everything ever written about cheeseburgers, and demonstrated during your training that you could write a convincing article about cheeseburgers, I would suspect you had some internal representation of "cheeseburger" and weren't just throwing the word in because it seemed to fit.
If you had also read everything written about sushi, and written a convincing article about sushi, I'd suspect you had a concept of "sushi" too. And if you did that for a bunch of different foods, I'd suspect you also had more general concepts like "food" and "flavor" and "nutrition".
Now, would I take a recommendation for a sushi restaurant from someone who had never tasted sushi, but had read everything that's ever been written about sushi, *and* everything that's ever been written about nutrition, cooking, the perception of flavor, restaurant reviews, etc., etc.?
Someone in one of the AI Discords I follow, just a fellow experimenter in the prompting arts, said something that stuck with me. "LLMs won't change the world in the way that fire or electricity did. But they might in the way Photoshop did." I'm inclined to agree. At the very least, I'd argue this is within the realm of reasonable optimism.
All this hype is great for the big AI players' profits, not to mention an excuse for them to keep their secret sauce secret. (Because it's "dangerous", you see, and they're "worried about it". Y'know, precisely worried enough to keep things secret, but not so worried they'll refrain from releasing a potentially profitable product.)
As someone who believes in the more modest but very real potential of large language models as tools, I don't think all this hype is good for AI in general. It sucks oxygen away from interesting neurosymbolic work; the sort of thing that, even if doesn't give us AGI, will at least give us more trustworthy tools. And it sets up a backlash when the promises go unfulfilled and people see what we have right now as a "failure" because of what it isn't, rather than seeing what it is.
If I’m understanding correctly GPT2 couldn’t do addition. GPT3 can multiply 2 digit numbers. GPT4 can do calculus at a high school level. It was never programmed to do calculus. It read some books and websites about it and figured it out.
It seems like there is more going on than a fancy autocomplete.
The problem is not that machines will ever think like humans but that humans are coming more and more to think like machines. Is someone like Trump or DeSantis human anymore, or have they become transmuted into algorithm-driven robots? Machines can mimic human behavior and speech patterns; people like Trump and DeSantis don't even do that anymore.
The notion that LLMs are simply stochastic parrots is incredibly reductionist and cannot account for many of the emergent phenomena observed, like in-context learning. You can even give ChatGPT completely nonsense novel words it's never seen before and give it an assigned value or meaning, and it will dutifully use it to solve a problem. An example would be executing a breadth first search on an adjacency list where the graph nodes are made-up terms not in the training corpus. GPT is still able to recognize this and correctly solve the problem, despite it never having seen the problem before. While I don't believe ChatGPT is completely "sentient", I reject the notion that one must experience physical phenomena to be sentient. ChatGPT not having literally tasted peanut butter doesn't make its conceptual understanding of peanut butter invalid. Was Helen Keller not sentient because she could never see animals, her family, or the sky? Was her conception of the sky fake because she could never see it? Or her conception of cows incorrect because she could never see one or hear it moo? The ability to physically interact with things you have knowledge of was never a requirement for sentience. GPT-4 has been shown to be able to take images and say what the effect of various actions on objects in the image. (e.g. what will happen in this image if I cut the string? the ball will drop. It's clear GPT has a conceptual model of the mechanics of the universe that makes it able to solve problems and have a "common sense" that the many prior models have failed to produce.
An emergent phenomenon in the context of AI is an ability that a model can perform, either from descriptions or examples, despite the absence of any training (or re-training of the model) to do that specific ability. No simple stochastic parrot would be able to solve novel problems that contain novel tokens.
I absolutely would take sushi recommendations from someone who's never tried sushi. Someone could become an entire sushi historian and conduct polls on the general population on different flavor profiles and popularity to gain a rock solid understanding of what's popular and pairs well together, all without tasting a single bite. Fun fact: did you know one of the founders of Ben & Jerry's had no sense of smell or taste?
Someone named Daniel Nelson defines emergent properties as: "Emergent properties are properties that manifest themselves as the result of various system components working together, not as a property of any individual component. To put that another way, it is a property that a complex system or collection of system parts has, but which individual parts do not possess."
That doesn't seem *unscientific*. Does it meet your standard for "scientific"?
You're going to have as much trouble defining consciousness in scientific, testable terms as other people are going to have defining "emergent." Nobody has ever been able to observe a consciousness but their own, and we very much do not know where it resides or what are the conditions that give rise to one.
(Except that we suspect other humans have it, because they look like us and talk like us, and claim to have a consciousness which we also perceive ourselves as having.)
Particularly, if it is possible to create a conscious machine (which us physicalists suspect it is, at least in principle!), we will definitely not be able to tell scientifically that it has happened with our current knowledge of consciousness. We will either have to take the machine's word for it, or not.
Eloser is so predictable.
But GPT 4 is still assembling text or images or sound from probabilistic models generated through proximal relationships. It cannot know; there is no place where its knowing could take place.
The question I have: is knowing necessary? From what I understand about how these AIs are built and trained is they're basically black boxes – the engineers understand the principle, but don't know anything about how the software implements those principles. They can't know, yet not knowing doesn't prevent them from creating the program anyway.
In other words, to what extent is it necessary to understand your actions for those actions to be effective?
Not to use them, no! These tools are very useful, and that's enough. But we have to be clear about what they are and aren't.
To borrow from an earlier comment of yours: Define "knowing" in scientific terms.
Very true. You could also say a 747 does not fly like a bird flapping its wings, but it still gets us from A to B in the air which is remarkable on its own.
I don't see LLMs as intelligence, but it's something significantly more than "just" auto complete as some say.
I just asked GPT, and it gave the right answer for the wrong reasons:
The ball broke the table because it was made of concrete.
The ball broke the table because it was made of cardboard.
Me:
What does "it" refer to in the above two sentences?
GPT:
In the first sentence, "it" refers to the ball, since the ball is the subject of the first clause ("The ball broke the table") and is the most recent singular noun preceding the pronoun "it".
In the second sentence, "it" refers to the table, since the table is the object of the second clause ("...because it was made of cardboard") and is the most recent singular noun preceding the pronoun "it".
See my comment below: ChatGPT-4 gets the answer right, and for the right reasons.
We had a brief exchange on a previous thread but I find this one interesting because it’s the most fundamental disagreement I have with you, and with many other thinkers I respect deeply (and I think that about yourself) including Steven Pinker, Chomsky, Dennett, and many others. It would take at least a full essay, maybe even a damn book to lay out why I went from ‘AI is a joke’ to ‘holy shit we’re all replaceable’ after I first worked with machine learning in 2016, but I’ll throw one point out here.
Whether AI is conscious or sentient is at its roots a philosophical discussion, a fascinating one, but as a concrete impact to humans, sort of irrelevant. It doesn’t matter if it’s conscious, it matters if it’s competent. Get Stable Diffusion and a newer modelset, see the deepfake capabilities and ask how much you can trust the images you see on the internet. GPT-4 makes ChatGPT looks like a joke, and ChatGPT could easily replicate every opinion article written by a boilerplate left-liberal opinion writer, as long as you know prompt engineering. I’m always cheerful for discussions on this, even though I axed my plans for philosophy postgrad to go back into cybersecurity the intstant I realized the disaster this would cause the humanities.
Also, Eliezer has completely lost the plot. He’s had good points in his time about logistically how an AI could jump an air gap via deception, but in his 5,000 reasons for how something more intelligent could kill us all, in 10 years I haven’t seen him give a sensible reasons on exactly Why it would kill us all. The interviews with Sam Altman then him with Lex Fridman are worth watching, one can see Eliezer is truly losing it.
But here's the issue: the failure case of the chimpanzee example is possible because there's nothing in an LLM that can notice that a response doesn't make sense. All there is waste probabilistic strings of text. They're generated via an incredibly sophisticated system, but they can't apply tests of basic sense because they have no sense-generating instrument, only distributional semantic models that say that a given string of tokens is the most likely correct answer to a query. And so even if the failure rate is reduced dramatically, there's still this fundamental issue is lacking the reasoning and logic that make error apparent.
You’re wrong though. As they get larger, they do gain more ability to identify correct from incorrect. The same is true as they get more context. LLMs trained with images make fewer mistakes than ones that can’t for instance, because they’ve incorporated that visual information into their understanding
Correct and incorrect are not the question. The question is, can it reason based on using logic and a theory of the world that stems from embodied consciousness? It doesn't matter how sophisticated the models get; they're not even attempting that.
Of course it can be embodied: https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html?m=1
And “theory of the world” is not as rigid a concept as you make it sound. It’s spectrum.
Wouldn't the model itself, as an abstract imprint of the data it's been trained on, be basically a "theory of the world" in much the same way that a memorized map is a theory of a location? And if that's the case, then what about a relational map where I can't paint an abstract layering but i know if I walk to one rock then turn I'll get where I'm going? Is that not also a theory of a location? I don't know much about the philosophy here, so I don't see why the answer to "where is its theory of the world" can't be it is itself the theory of the world.
What would it take for you accept that it was conscious? If GPT5 answers all your questions correctly and you have hours long deep philosophical discussion with it, would that be enough?
Where does that consciousness reside? In a set of tables that define the relative positions of words in large data sets? How would that possibly work?
We have no idea how our brains manifest consciousness. Not knowing how something works doesn’t preclude it from working.
If a 1.4kg carbon based electrochemical computer can manifest consciousness…why can’t an electro silicon based computer do the same thing?
Freddie, where do you think consciousness resides? And before you say the brain, where specifically in the brain? (And I’m assuming you don’t believe in souls or some kind of immaterial consciousness?)
Be careful confusing consciousness and intelligence.
Consciousness isn’t measurable, nor is it remotely understood. We have absolutely no idea if these things are conscious and the safe bet is that they aren’t.
But they are intelligent, and they’re only getting smarter. That you can measure.
But that doesn’t matter for beating human capabilities in intellectual tasks, one by one. I was called a madman by artists over the last 7 years for saying no, machine learning could replicate digital art, but in March 2022, I, the crazy one, thought something like Stable Diffusion would be free in the 2030’s.
GPT-4 shatters benchmarks even on ‘common sense’. ChatGPT did the BAR and LSAT in the 10th percentile, a little over 3 months later, GPT-4 is in the 90th for both. That is a truly staggering rate of improvement, and these are both censored models. Microsoft’s researchers published a paper saying the uncensored GPT-4 they had access to had properties of AGI. But AGI, terminology, it isn’t really relevant to this. What’s relevant is disruption. I laid out to the head of a philosophy department how I could bypass both the 5 paragraph limit and reference blocking limitations on ChatGPT to write an A paper. Check Reddit for examples of this. My best friend’s an English teacher, it’s a running joke that the essays are fake.
Let’s say GPT-4 doesn’t exist, and everything halted at ChatGPT. The education system would still be in shambles, especially anything assigning take home essays. “Adjust it in the style of the New Yorker”.
Philosophy professors are already using AI to grade papers written by AI, so the humanities are a farce. This does not stop with education and digital artists. You’re in a good spot due to having a heterodox writing style and subject, but I worked in tech for a decade, including with machine learning, and I can lay out step for step how literally every white collar aspect of the business, of every business I worked in, will be chewed through by this, except those involving physical labor.
The bar exam and SAT examples are almost certainly memorized examples in the training data. We can't say for sure because "Open" AI has put a moat around their research methods and will no longer elaborate on even the broadest details of how their models work in their papers. Which, frankly, makes what they're doing more like pseudoscience than advancing a common understanding of what "intelligence" means.
Memorized examples is not how the LLM’s work. The parameter numbers for ChatGPT vs. GPT-4 are not that different, GPT-4 is trained for longer. Generative models do not memorize, that’s not how they work. Cleverbot could memorize the questions for an LSAT, that wouldn’t be impressive at all.
What’s different about these LLM’s in particular in generalization. Give GPT-4 a shot, it costs a bit a month but throw non-memorizable LSAT type questions at it, ones that have never been on a test. I guarantee it will do quite well. And the uncensored version only Sam Altman can play with? It’d probably get a perfect score.
Indeed, what it can do is very impressive. And my car goes faster than I can run.
Yes and the car massively disrupted human civilization. You don’t need to believe Sundar Pichai or Satya Nadella, or Sam Altman that this is a more important moment for the genus Homo than the discovery of fire, there is no need to compare machine learning to human intelligence, or the gibberish of the black box of machine learning to the processes of human learning. It doesn’t matter. Watch the Youtube videos or comments of artists at the existential crisis Stable Diffusion has caused them, read the threads on how many education assignments are now fake. My colleagues even in tech sales are composing most of their work with AI, and those who aren’t are falling behind and will get the axe in the tech layoffs.
This buzzsaw is going to chew through every single bit of knowledge work, the creatives first honestly. My Mom was an artist before her mind collapsed, even though I love machine learning it truly hurt seeing the devastation done to artists by this and reading about the drying up of commissions, but they never listened. You will be safe Freddie, but any white collar or creative working reading this, just be prepared, don’t be blindsided like the artists I’ve read having psychological breakdowns.
And the car put all the horses out of work. It didn’t walk or run or even have legs. Turns out that wasn’t the relevant question.
GPT-4 performs markedly differently on problems of the same difficulty class depending on whether those problems were online before or after the cutoff date for its training data.
"...we tested it on Codeforces problems from different times in 2021. We found that it could regularly solve problems in the easy category before September 5, but none of the problems after September 12.
In fact, we can definitively show that it has memorized problems in its training set: when prompted with the title of a Codeforces problem, GPT-4 includes a link to the exact contest where the problem appears (and the round number is almost correct: it is off by one). Note that GPT-4 cannot access the Internet, so memorization is the only explanation."
https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks
Yeah I worded that pretty poorly, since LLMs are predicting the next word (or similar) one after the other (the new ones do more than that but let’s stick with that for brevity), it’s definitely easier to predict the next term on something it’s already seen, and since it can tell you a famous quote verbatim, it can replicate text its seen precisely.
I just read that post though and while I don’t wholly disagree, I think it’s a non sequiter in a lot of ways, and also presumes that AI is written in amber when it’s the fastest progressing technology I’ve ever seen. I’m curious as to the authors predictions the author has made for when something like Stable Diffusion wouldn’t just be possible, but free and open source, or when GPT-4’s capabilities would be possible, I could stand corrected but I highly doubt he was predicting now. Free to use text to image, even most AI people I saw figured 2030’s, yet here we are. As for benchmarks, he makes good points, but as Freddie points out in the Cult of Smart, which I highly suggest, SAT/ACT and especially IQ tests are pretty damn good at measuring important human aptitudes for knowledge work tasks, flawed as they are when it comes to couting for past group injustices, I’m not a believer in The Bell Curve bullshit by any stretch, and Freddie tears that apart well in the book.
I’ve heard “AI can’t do that” so many times in the last many years though it’s become a huge headache. Every time the prediction is wrong, then the people who predicted wrong are still treated as if their next prediction is the one that’ll be right. The goalposts have been moved in this debate so many times they’ve left the stadium, went into the parking lot, and are now crossing the street into a nearby neighborhood.
If we value education, which is a big If...assessment will no longer be take home essays. Writing will be done in class because the assumption has to be that the take home product will be fake. More assessment will be on face to face interactions and in-class work. Good teachers will adjust. Good students who want to learn, will learn.
I had a ton of fun reading this thread. Thanks to everyone for writing such excellent comments.
OK, this is such a good thread I should try to add some of my real world experience to it, see if it's useful. Again, thanks to JesterColin and everyone who wrote such thoughtful responses.
I've been an attorney for 20+ years. I have a lot of access to products being developed in the legal tech field - I'm not employed to work directly on the products themselves, but I'm close enough to the industry that I'm often involved once the need for practical legal knowledge goes beyond "law degree". I'm not claiming expertise on AI or how it works. But I HAVE had a lot of behind the scenes experience with machine learning and AI in the law.
I'm not losing sleep over AI's impact on attorneys, even though "law" is often discussed as one of the industries likely to be massively disrupted. The first problem is the obvious one - when I ask ChatGPT or the like a legal question it gives uselessly vague and incorrect answers, and if I ask it legal nonsense it makes something up. Fine, maybe refinement will improve this. But the main issue is more what Freddie outlined, and I don't see how it gets solved.
Practicing law isn't just knowledge of legal standards and drafting persuasive words; it's factual investigation, negotiation, discussion, synthesis, and knowledge of the particular persons involved that is needed in BEFORE you come up with legal strategy and draft those persuasive words. They're inseparable.
I can't see how it gets there. If (for the sake of argument) I agree that eventually we might be able to feed AI deposition testimony, witness statements, discovery responses, reams of document production, and privileged knowledge from a client and in response it could spit out... something. I actually don't even know what people think might happen. Motion papers, I guess? Even that seems incredibly unlikely, because it can't do the real work that I get paid to do - it didn't take the depositions, the witness statements, collect the discovery. Those things require Freddie's theory of mind - they require physical interactions, an ability to work with and understand reality, not just manipulate language. And they aren't tasks separate from drafting. They're necessary to the process.
There are areas of practice where I can see this being really useful, but my point is broader: I feel there's some Gell-Mann Amnesia going on. The media tells me that my industry is going to be turned on its head, that AI as able to write and research as well as lawyers; but I'm close to this stuff, and I'm confident it's not happening. Then I think: but sure, it could replace a doctor. It's probably better at diagnosis! Or that it could write good marketing copy, or create commercials, or write code, or write short stories. But it's the same media saying these things that told me we're on the brink of robot lawyers. So I'm starting to doubt that the impact is going to be as widespread as advertised.
Always good to read a thoughtful post! Hopefully this response doesn’t end up too long. Firstly, remember we’re in the infancy of these tools, they only became possible in the early 2010’s and didn’t really start working until 2020, there’s a pretty fun history on why if you become interested in this stuff. I got my first iPhone in 2008, early among people I know. I was the only one my friends who had one and when we needed to go anywhere they’d joke “ask your magic phone”.
I was flabbergasted that something like the iPhone existed even years into having one, the idea of a supercomputer in my pocket was Star Trek to me, but there it was. The gravity of the situations with smartphones didn’t sink into the general public until the mid 2010’s, and now we live in that future, for better or worse.
I don’t say all this to claim to he some future-predicting wizard regarding technology, I mean I missed social media being world changing, but even as a guy who followed machine learning all these years, as I mentioned not a single person I know, not one, even in tech thought LLMs like we have today would be available in the 2030’s. 95% believed that especially art could not be replicated period, that it was uniquely “human”, whatever the hell that means. That idea fell apart I figure in November when a usable Stable Diffusion UI came out.
Me, the delusional one, was so shocked when I first used ChatGPT that I dropped my damn phone. This wasn’t supposed possible, not for a decade, and I found it absolutely insane that it was free. Then I said “well no one will connect one of these things to the internet until like 2025, because that’d be insane”, 2 and a half months later, Microsoft does it with Bing. Combine those with the massive improvements from ChatGPT to GPT-4, and remember, this change to people’s reality has taken months, not years like the smartphone, or social media. No technology has been this impactful this fast in my lifetime, and those skeptical of its progress are the same people who literally didn’t think things like Stable Diffusion/Midjourney or GPT-4 could exist, even theoretically, and they believed that in 2022. I was seeing “lol but AI can’t do hands” even last month, right before Midjourney cracked that problem. Look at an AI picture from March of last year and March of this year, the difference is so mindblowing to me it’s hard to put words to. Check out AI music as well, I figure 2 years, that nee Taylor Swift album ain’t gonna be Taylor Swift.
The parts you mention in your career, definitely anything in person, those are safe from AI for a good while, (although I’m wrong by a factor of 10 often) I’m just not a huge believer in the progress of robots, or in person AI as compared the that in the cloud, but anything documents related, analyzing text? I worry I’ve already been too long winded here but there is no theoretical limit on it becoming better at that than humans, much better, and soon.
As for ChatGPT, never look at it as if you are bound by an Iron Law that you have to use the exact output, in fact that’s always a bad idea. I feel bad for the thing (no, I don’t think it’s alive) because the amount of guardrails slammed on it since the Day 1 version have basically destroyed its functionality, specifically around things like longform creative/analytics text creation. ChatGPT post-January is a shell of what it’s capabilities are with no guardrails.
Sorry for the massive wall of text, I really don’t get to have in depth written discussions with people, especially on my favorite topic, so I jump on the chances I can.
In contrast nothing about current AI capabilities shocks or wows me. It all seems like interesting parlor tricks. Which is to say somewhat useful in certain domains, maybe, but nothing world changing.
What are the guardrails?
How subject is it to bad imput? What if there is something considered to be scientifically true, but the initial data is poor and hasn't been replicated properly? There are a number of articles lately about poor peer reviewed journal articles and outright faked data. Does AI just run with it?
I'm not actually as worried about creative endeavors because I think those will continue because people enjoy doing them. I write stories for fun. I'm never going to make any money at it, but I do it for fun. A few people like my writing and that's enough.
I think people will continue to paint and draw, because they enjoy doing so. I think people will continue to play music together for fun. The change may be in the monetary reward, or lack thereof.
I’ve been a software engineer for 20+ years and a lawyer for 12, your comment made me realize one aspect of the AI true believer thinking is to underestimate (or in some cases categorically misunderstand) what it is people in a given profession actually DO.
One aspect of being human is being an “incredibly sophisticated autocomplete”.
There's also the whole "internal model of the world" thing. GPT's context, though very large, isn't a real "model of the world". A lot of failure cases have to do with that lack of physical modeling. Y'know, putting X in Y, then Y in Z, then moving Z, those kinds of things.
There's also the question of memory and knowledge management when introduced to non-trained information. As I can personally attest from trying to run a D&D campaign with the thing in a homebrew world, the human still needs to handle the remembering of anything new introduced. (Otherwise, the damn thing falls back on cliches.)
There's some interesting work going on in both areas, but my money is that it'll take a fundamental change in AI architecture.
Maybe so, I’m not a computer scientist. Sounds reasonable to me though. My point, which I’m surprised Freddie doesn’t seem to agree with, is that if you accept materialist principles, then the difference between an AI and organic intelligence is a matter of degree, not kind. Consciousness, sentience, etc may be different, and I’m partial to the view that those are in fact physiologic process that silicone can’t truly recreate. But in terms of what we would broadly call intelligence, I’m not sure there is a big difference ultimately.
I can't help thinking about Asimov's Positronic Man. Was Andrew human?
LOL I just looked up autocomplete. I find it a distraction. Something about it bothers me. But I'm old. I also turn my lights off and on by hand. I also cut my own wood with a chainsaw, haul it to the woodpile and heat my house with it in the winter.
Perhaps I'd have slightly more sympathy for Yudkowsky if he hadn't spent the past decade and a half persuading people previously interested in finding effective treatments for diseases and reducing global poverty that they should instead be researching the very AI systems he now thinks should be destroyed by airstrikes.
But instead of a reasonable conversation about likely risks informed by researchers who understand the current impacts of AI, we're debating how seriously we should take the night terrors of a bunch of people who read pseudo-Bayesian Harry Potter fan fiction. We are indeed well past the realm of reasonable discussion of any of this.
For the technology to be any good, I think we need to combine learning with human input.
Recently, I tried using AI to generate an image of a bowling ball. It doesn’t understand the number or placement of the holes, probably because photos of real bowling balls appear to have 0, 1, 2, or 3 depending on the angle. So it gives me balls with one big hole, or 4+ holes in a random pattern.
I just want to tell it “there are always three holes.” Just like I want to tell it “Humans never have more than two arms. Literally never. Maybe you thought you saw it, but that 3rd arm belonged to the person’s friend.”
Anyway, it’s very depressing to see shitty AI replace real content. The whole reason I was trying to make a bowling ball is because my app (Photoleap) replaced its stock photo collection with AI.
So, great…. no more licensing fees for Photoleap. I’m sure it will save them money in the long run. But I can’t even get a bowling ball, let alone a person who doesn’t look melted and deformed.
Maybe, or maybe “Theory of the world” is not as well defined as you make it out to be. Maybe it’s just an emergent property of sufficiently sophisticated statistical models.
The academics you’re sighting have consistently made predictions that have been proven wrong. Their ideas about intelligence are increasingly considered mistaken.
And you’ve misunderstood the significance of this moment. It’s not just that AI like ChatGPT are good autocompletes, they have fundamental properties that no predecessors had, and these properties appear to emerge from nothing more than increases in scale. This includes, in the case if GPT4, the emergence of agentic behavior, the ability to make and execute plans. Google put an LLM into a robot and it has the ability to perform novel tasks on which it wasn’t trained.
No, it’s not sentient. Maybe there won’t be a use for them that really transforms society. But you’re vastly underestimating what can be achieved with a simple input-output paradigm. The fact of the matter is that LLMs *are* something new, poorly understood, and potentially very powerful, unlike anything that has existed before.
I wish I had more time for a deeper response, but as someone working in AI, for all the hype, there’s something very real happening here.
"Emergent property" is both unscientific and non-falsifiable. And you are free to leave links about how those guys are wrong.
It most certainly is very well defined. If you have specific test that a model can’t pass at one size, but then can when it is larger, then it is an emergent property. GPT2 could not do basic math. GPT3 could multiply 2 digit numbers, GPT4 can do calculus at the level of an average high schooler. These were all features that emerged on their own as a side effect of being larger.
Again: its mechanism is distributional semantics. The position of words near each other in language. This is entirely different that what we mean by knowing. If I had never had a cheeseburger but knew where the word cheeseburger was found in data sets, would you trust that I understood what a cheeseburger was?
At what point would you accept that it does know what a cheeseburger is? What if it could draw it? Create a 3D model of it? Design new recipes? Cook it itself as a robot? Where is the line here?
Also, LLMs can do what’s called zero shot learning, where they can complete a task they’ve never seen before using only a text description. Where does that fit in?
More to the point: How much does it have to be able to do before you’ll admit it’s actually intelligent? Where is the line where you would admit that it has something that could be called understanding? If the answer is “nothing”, then you may want to consider how scientific your stance really is.
Where in the code does consciousness reside?
I said nothing about consciousness. You can’t measure or observe consciousness. Consciousness has no effect on the outside world.
I’m talking about intelligence and understanding; The ability to meaningfully reason about and manipulate the world.
And I return to my question: what would make you admit these systems have genuine understanding?
Where in the brain does the consciousness reside? The prefrontal cortex? It's connections to the ventral tegmental area and nucleus accumbens that drive goal-directed behavior? The hippocampus where memories are stored? Layer III pyramidal neurons storing short term visuo-spatial memory? Fear processing in the amygdala? Or maybe the "consciousness" is an emergent property of many networks working together. If you meticulously measured every neuron in a human brain and coded its characteristics into a database, congratulations, you now have a conscious database. Not saying LLMs work exactly like a brain, but asking for the "conscious part" to be pointed out is a bit silly.
Can ChatGPT imagine? Can it speculate? Can it ask "WTF just happened?" and change its models, the way Rutherford did when he conducted the experiment that led to the discovery of the nucleus?
According the the then-current models of the atom, it was the equivalent of firing shells from an eighteen inch naval gun at a sheet of toilet paper and a *very* *small* *percentage* of those shells bouncing off the sheet of TP.
It can certainly change its models. As to the rest: who knows? I’m not arguing it’s conscious, or that it has experiences. I’m arguing it (or rather some near future version of it) is intelligent, that it has the capacity to understand and act towards goals.
That's a good question. Can it imagine? Can it come up with unique ideas.
I heard that this new tech is able to accelerate the ability to model the folding of proteins. This ability has tremendous potential. But could it come up with the idea of the importance in how proteins are folded in addition to their chemical bonds? Could it have originated that concept of folding proteins as an area to be explored and modeled?
I think that's an oversimplification of the mechanism.
Internally, systems like these aren't just representing the probabilities of words appearing in a given sequence. That's what a Markov model does, and it produces amusing gibberish.
These systems are building layers upon layers upon layers of connections between input tokens and output tokens. Represented in those middle layers are general patterns that aren't literally present in the inputs, but that have emerged during training because they're useful for linking a variety of inputs to their corresponding outputs -- in other words, *abstract concepts*. Those abstract concepts are inferred from the training data, and they can be expected to map to real-world concepts if the training data was written about real-world concepts.
Similarly, AI models that generate or recognize images tend to contain representations of the "concepts" that make up an image. For example, a system that recognizes dog photos will have components for recognizing paws, eyes, snouts, or tails; and those will have their own components for recognizing the curves and patterns that comprise them. You can see this in action with e.g. Google's DeepDream, which runs a recognizer backwards to amplify everything in an image that reminds it of something it could recognize, turning it into a nightmare of eyeballs and vaguely organic-looking shapes. Those components aren't hand-coded; they emerge during training.
So, for your questions about cheeseburgers and sushi: if you had read everything ever written about cheeseburgers, and demonstrated during your training that you could write a convincing article about cheeseburgers, I would suspect you had some internal representation of "cheeseburger" and weren't just throwing the word in because it seemed to fit.
If you had also read everything written about sushi, and written a convincing article about sushi, I'd suspect you had a concept of "sushi" too. And if you did that for a bunch of different foods, I'd suspect you also had more general concepts like "food" and "flavor" and "nutrition".
Now, would I take a recommendation for a sushi restaurant from someone who had never tasted sushi, but had read everything that's ever been written about sushi, *and* everything that's ever been written about nutrition, cooking, the perception of flavor, restaurant reviews, etc., etc.?
Personally, no. But that's because I hate sushi.
I for one welcome our new robot overlords.
I must be one of the 1%. I thought the table was made of concrete. Unreinforced concrete is brittle. A steel ball dropped on it could easily break it.
Someone in one of the AI Discords I follow, just a fellow experimenter in the prompting arts, said something that stuck with me. "LLMs won't change the world in the way that fire or electricity did. But they might in the way Photoshop did." I'm inclined to agree. At the very least, I'd argue this is within the realm of reasonable optimism.
All this hype is great for the big AI players' profits, not to mention an excuse for them to keep their secret sauce secret. (Because it's "dangerous", you see, and they're "worried about it". Y'know, precisely worried enough to keep things secret, but not so worried they'll refrain from releasing a potentially profitable product.)
As someone who believes in the more modest but very real potential of large language models as tools, I don't think all this hype is good for AI in general. It sucks oxygen away from interesting neurosymbolic work; the sort of thing that, even if doesn't give us AGI, will at least give us more trustworthy tools. And it sets up a backlash when the promises go unfulfilled and people see what we have right now as a "failure" because of what it isn't, rather than seeing what it is.
If I’m understanding correctly GPT2 couldn’t do addition. GPT3 can multiply 2 digit numbers. GPT4 can do calculus at a high school level. It was never programmed to do calculus. It read some books and websites about it and figured it out.
It seems like there is more going on than a fancy autocomplete.
The problem is not that machines will ever think like humans but that humans are coming more and more to think like machines. Is someone like Trump or DeSantis human anymore, or have they become transmuted into algorithm-driven robots? Machines can mimic human behavior and speech patterns; people like Trump and DeSantis don't even do that anymore.
The notion that LLMs are simply stochastic parrots is incredibly reductionist and cannot account for many of the emergent phenomena observed, like in-context learning. You can even give ChatGPT completely nonsense novel words it's never seen before and give it an assigned value or meaning, and it will dutifully use it to solve a problem. An example would be executing a breadth first search on an adjacency list where the graph nodes are made-up terms not in the training corpus. GPT is still able to recognize this and correctly solve the problem, despite it never having seen the problem before. While I don't believe ChatGPT is completely "sentient", I reject the notion that one must experience physical phenomena to be sentient. ChatGPT not having literally tasted peanut butter doesn't make its conceptual understanding of peanut butter invalid. Was Helen Keller not sentient because she could never see animals, her family, or the sky? Was her conception of the sky fake because she could never see it? Or her conception of cows incorrect because she could never see one or hear it moo? The ability to physically interact with things you have knowledge of was never a requirement for sentience. GPT-4 has been shown to be able to take images and say what the effect of various actions on objects in the image. (e.g. what will happen in this image if I cut the string? the ball will drop. It's clear GPT has a conceptual model of the mechanics of the universe that makes it able to solve problems and have a "common sense" that the many prior models have failed to produce.
https://www.erichgrunewald.com/posts/against-llm-reductionism/
And for what it's worth, GPT-4 answers your genus/species question correctly: https://i.imgur.com/YZLfYVj.png
Define "emergent" in scientific terms.
Would you take recommendations for a sushi restaurant from someone who had never had sushi?
An emergent phenomenon in the context of AI is an ability that a model can perform, either from descriptions or examples, despite the absence of any training (or re-training of the model) to do that specific ability. No simple stochastic parrot would be able to solve novel problems that contain novel tokens.
I absolutely would take sushi recommendations from someone who's never tried sushi. Someone could become an entire sushi historian and conduct polls on the general population on different flavor profiles and popularity to gain a rock solid understanding of what's popular and pairs well together, all without tasting a single bite. Fun fact: did you know one of the founders of Ben & Jerry's had no sense of smell or taste?
Someone named Daniel Nelson defines emergent properties as: "Emergent properties are properties that manifest themselves as the result of various system components working together, not as a property of any individual component. To put that another way, it is a property that a complex system or collection of system parts has, but which individual parts do not possess."
That doesn't seem *unscientific*. Does it meet your standard for "scientific"?
No, it doesn't sound unscientific. But it also doesn't explain how a database can spontaneously generate consciousness.
You're going to have as much trouble defining consciousness in scientific, testable terms as other people are going to have defining "emergent." Nobody has ever been able to observe a consciousness but their own, and we very much do not know where it resides or what are the conditions that give rise to one.
(Except that we suspect other humans have it, because they look like us and talk like us, and claim to have a consciousness which we also perceive ourselves as having.)
Particularly, if it is possible to create a conscious machine (which us physicalists suspect it is, at least in principle!), we will definitely not be able to tell scientifically that it has happened with our current knowledge of consciousness. We will either have to take the machine's word for it, or not.
Great post. I have been finding all of the breathless pieces about ChatGPT extremely annoying, and this does a good job of explaining why.
Was the trivia question in the header image inspired by today’s NYT crossword puzzle, perhaps?