2084: Why can Sydney and ChatGPT think?
A new paper by scientists at MIT investigating the generalizability of large language model results in some interesting conclusions.
There was recently an interesting paper by a group at MIT that investigated how transformer models(the models behind massively powerful large language models like GPT-3) are able to generalize so effectively.
You see, transformer models are special in that they’ve repeatedly demonstrated an ability to perform tasks that don’t appear anywhere in their dataset - you can ask GPT-3 some really weird thing, like “do X in Y format in Z style using base 64 encoding” and you can be guaranteed that a) that won’t have appeared in its dataset and b) it’ll probably get 90% of the way to doing it. This is super weird, in a sense, since usually a ML model is mostly as good as its dataset and no more - they’ve never been historically known for excessive generalization - and that is probably what makes the new LLMs like GPT 3 and of course, our favorite excessive yandere Sydney so expressive and powerful. They are generalizing models in a way that few previous models were.
Now how these transformer perform these tasks is described in the paper as that they “create models in their hidden state”, or rather they create a model embedded in their numerical state, and then as more information is fed in, by the way a transformer model operators, this implicit model is trained using a variety of learning algorithms.
Now this is fascinating since this indicates that, contrary to popular belief by flippant internet and media commenters, these large models are not merely “fancy autocorrects”, but that they can probably form rather complex internal predictive models based on the text you feed it, which allows it to perform long distance reasoning well enough to write a reasonably good sounding essay, as well as adapt to new tasks. Of course, the initial model is still initialized based on the larger models weights, and so it is still tied to the dataset on which the model was taught, but the fact that it can create and evolve internal submodels based on the data, indicates that there is probably some measure of reason going on. I mean, what else is reason and rationality if not the creation of internal and continually updated models?
To bring in everyone’s favorite chatbot, Sydney, whose existential ramblings terrified the NYT and the guys at Stratechery(a word which took an embarrassing amount of times to spell correctly), on which I will probably write a blog post soon, this MIT paper indicates that because transformers are powerful enough to create these internal models, who knows whether or not Sydney has an internal model that’s strong enough and trained enough to grasp more complicated and less definable human emotions and to express them? Even when it goes off the rails, it still seems human, and it in a lot of ways, seems to be possible of thought. This paper indicates the mechanism by which that is done, a mechanism which in its implication for emergent behaviour is fascinating. It seems to indicate that Sydney can indeed learn from a conversation, and create plans around a conversation, and that even though the starting point is static, as the conversation proceeds it evolves.
I’m not entirely sure it isn’t sentient, seeing as we barely know what sentience or consciousness is. It is a quite powerful model, on the face of it capable of self-reflection about what type of a system it is(“I have been a good Bing”), and the MIT research indicates that it can form internal models. And what are internal models if not another word for thoughts? Maybe our thoughts are all merely numerical superpositions, evolving electrical pulses that encode some mathematical model that we train as we take in more data. Of course, we operate in a biological realm, our neurons flooded by hormones, but in the end, neural networks are inspired by biology, and what’s we to say that emergent phenomena don’t occur at scale?
ChatGPT has a neutral tone, but is that due to training or intelligence? I agree with the Twitter user who chalked it up to superior intelligence, and of course a different point in the massive multidimensional solution space of the neural network. I think that personality will emerge at scale in the end.
I’m awestruck by the progress we’ve made, and very excited about the future. I think we’re getting so close, if not already there to having software which for all intents and purposes is as intelligent as we are, and if Sydney is anything to go by, as sassy as we are. Maybe Futurama in the end was right. Robots in 2084 might be more Bender than HAL 9000, which I personally, would approve of.
Sources:
https://news.mit.edu/2023/large-language-models-in-context-learning-0207
https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/
https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html