The breakthrough behind ChatGPT
What word comes after Good?
You might think Good Morning, or Good Bye. But you definitely wouldn’t say Good Loud. That just doesn’t make sense. For decades, computer scientists have been training AI to solve this exact problem.
Given a word, our AI predicts the next word. Do this several times, & you’ve generated a sentence.
This is how ChatGPT works.
Trained over the entire internet, ChatGPT has learned how to chat like a human. However, this immense feat was only made possible by a breakthrough in the late 2010s. A breakthrough underpinning ChatGPT & forever shaping the world we live in.
This is the story of an AI that read & learned from every book, tweet, & website across the entire internet. And how it was made possible.
Sentences are long.
When we move beyond a single word, next word prediction is a lot harder. Take this example.
In this context, it makes no sense to say I ate a good Morning. But our AI only looks at good, and spits out morning. In most cases even humans need many words to predict the next word. So an AI needs this extra information as well.
Our AI needs to read many words to predict the next word. ChatGPT can read more than 8,000 previous words at once. The natural way to do this would be to feed each word into the AI, one by one.
This is how AIs worked in the past. A Recurrent Neural Network (RNN) would take one word at…
Source link
