I listened to a podcast on the history and development of neural networks and algorithms that worked with language and speech. It started with studying the very structure of language, not any particular language, but any language, looking for various common features in order to identify patterns and implement them in algorithms for recognizing and generating speech. Then followed the simplest algorithms, for example, the well-known function for button phones “T9”. Then came the simplest machine learning, neural networks learned from huge arrays of text and were able to predict answers. For example, such an algorithm can solve the simplest arithmetic problem without knowing arithmetic at all, it just knows that if there are “2” “+” and “2” in the text, the answer according to the probability theory will be “4”. At this stage, the neural network could already be useful, but it is not perfect at all. Such systems did not look at the whole sentence, but checked one word at a time, so they could produce an incoherent set of words if the sentence was constructed in an atypical way.
In 2015, a company called Open AI appeared, which would later turn the idea of language models upside down, but in the beginning, they were losing the technology race to Google. In 2022, Open AI releases ChatGPT, which was a turning point in how people perceive language models. Users use ChatGPT for literally everything: It to help write an essay, find a bug in code, formulate the right Google query, now it can already draw pictures, and so on.
ChatGPT is a great example of how new technology can be used in real life by real people, and the best part is that it is actually being used.
