Post by account_disabled on Mar 10, 2024 22:39:14 GMT -5
Research in the field of Artificial Intelligence in recent years has been undergoing an acceleration which requires some attention if we want to understand its trajectory. Among the most active research centers is OpenAI , managed by the non-profit of the same name and also financed by Microsoft and Reid Hoffman (co-founder of LinkedIn). GPT-3, the third generation of their "Generative Pretrained Transformer", is causing a lot of discussion these days. A transformer is a neural network that uses Natural Language Processing techniques to perform a task. In other words, it is a linguistic computation model designed to generate sequences of words, code or other data, starting from an initial input.
The technique was introduced by Google in 2017 and used in machine translation to statistically India Mobile Number Data predict sequences of words. These statistical models need to train with large amounts of data to produce relevant results. The first GPT in 2018 used 110 million learning parameters (the values that a neural network tries to optimize during training). A year later GPT-2 reached the point of using 1.5 billion. Today GPT-3 uses as many as 175 billion. The current discussion started after some developers, who had access to the beta of the model, shared the amazing results. Manuel Araoz had GPT-3 write a blog post about GPT-3, starting with a short textual description.
Mario Klingemann managed to produce a fictional article on the importance of being on Twitter, in the style of 19th century writer Jerome K. Jerome. Sharif Shameem has posted a series of code generation experiments, for example the main page of Google, starting from simple instructions written in natural language. It must be said that, however extraordinary, these results are not even remotely close to the concept of general intelligence, so feared (primarily by Elon Musk who left OpenAI for this reason). Here the complex algorithm, thanks to the myriad of ingested information and statistical models, predicts the most probable sequence of terms, without understanding its meaning. Ultimately the model works on syntax, but not on semantics. This is why it could also give rise to the writing of racist texts, as Jerome Pesenti demonstrated, or meaningless ones, as in the examples of Kevin Lacker.
The technique was introduced by Google in 2017 and used in machine translation to statistically India Mobile Number Data predict sequences of words. These statistical models need to train with large amounts of data to produce relevant results. The first GPT in 2018 used 110 million learning parameters (the values that a neural network tries to optimize during training). A year later GPT-2 reached the point of using 1.5 billion. Today GPT-3 uses as many as 175 billion. The current discussion started after some developers, who had access to the beta of the model, shared the amazing results. Manuel Araoz had GPT-3 write a blog post about GPT-3, starting with a short textual description.
Mario Klingemann managed to produce a fictional article on the importance of being on Twitter, in the style of 19th century writer Jerome K. Jerome. Sharif Shameem has posted a series of code generation experiments, for example the main page of Google, starting from simple instructions written in natural language. It must be said that, however extraordinary, these results are not even remotely close to the concept of general intelligence, so feared (primarily by Elon Musk who left OpenAI for this reason). Here the complex algorithm, thanks to the myriad of ingested information and statistical models, predicts the most probable sequence of terms, without understanding its meaning. Ultimately the model works on syntax, but not on semantics. This is why it could also give rise to the writing of racist texts, as Jerome Pesenti demonstrated, or meaningless ones, as in the examples of Kevin Lacker.