ChatGPT with Rob Miles - Computerphile

I think it is still a problem that many people do not know how GPT really works. There is no part in GPT that can figure out something. So no intelligence and no learning of things. A good example is when you look at GPT writing poems. All poems that it writes are of the same structure and they are not very good. So it seems it does not know the difference between a sonnet and a limmerick. But what happens is, that there is a miss-alignment between writing a poem and making the user happy. GPT is trained that its highest goal is to make the user happy. GPT believes that users dont know what sonnets are anyway so i can get away with a rhyme that is not a sonnet. Here is a good video about that: ▶ this video

in “What GPT-4 Does Is Less Like “Figuring Out” and More Like “Already Knowing”” · View on HN

This video[1] cleared up a lot of questions for me as to the issues I've seen with ChatGPT. My current operating theory goes thusly: Think back to "This Person does not Exist" [2], a site which generates a simulated human face. This works by randomly picking a vector into the latent space of human faces from a trained model, and showing it to the user. When you're using ChatGPT, you're getting a simulated assistant, at random. The quality of the answer is highly dependent on how that portion of the latent space (and thus that particular simulated assistant) was trained. Thus, as you get a wide variety of faces from this-person-does-not-exist, you'll get a wide variety of simulated assistants from ChatGPT. For me, this explains the schizophrenic nature of ChatGPT. As for the Bullshit it seems to spew, it is working against a rating system (another AI) trained to act like a human rating text. If those humans didn't know something, they had no way to express it, they simply had "pick which is best", which removes all the other dimensions of consideration, and quashes it into a scalar. The optimum strategy for the rating system is to try to BS its way through things, which then teaches ChatGPT to BS its way through things as well. Much like rating systems on HN, and all social media, this destroys information and tends to dis-incentivize nuance. [1] ▶ this video [2] [link]

in “Ask HN: Are ChatGPT answers getting worse for anyone else?” · View on HN

It might help quite a bit to watch this overview of how ChatGPT works, and especially how it was trained .[1] Human attention was too expensive for the training, so they made another AI to simulate the trainers. Unfortunately, the trainer simulator training didn't have an expressive enough set of options for the actual humans, which kept a lot of quite valuable feedback out of the whole system by reducing things to single ranking. If the person didn't know the correct answer, they'd likely just go based on gut. This in turn lead the training simulator to do the same. This lead ChatGPT to just try to bullshit an answer rather than stating it didn't know, and getting downvoted. [1] ▶ this video

in “Princeton CS Prof: ChatGPT Is a Bullshit Generator (2022)” · View on HN

It depends on your definition of cheating, but this is definitely not "using humans to answer all questions one could ask". Rather it's a way to tune language models to be more like assistants rather than "most likely continuation machines". For context I recommend watching the discussion on what chatgpt is/does[0], and what the open assistant project's aim is[1]. [0]: ▶ this video [1]: [link]

in “Open Assistant: Conversational AI for Everyone” · View on HN

I’ve read that before, it is fascinating. GPT3 was just trained on source text, but ChatGPT has undergone an an additional very rigorous programme of intensive training bootstrapped by human feedback. This has evolved it in order to improve its performance as an information assistant, especially on specific tasks that have proved problematic in the past. This video explains the approach and some of its limitations. ▶ this video

in “The new Bing will happily give you citations for a pile of nonsense” · View on HN

Also it is important to note that GPT shows large miss-alignments. The problem comes from the fact, that it is hard or impossible to give an objective what GPT should be optimized to (nobody knows the truth), so proxys are used. One proxy is that it should make the user happy and the user should give many thumbs up. But this does not mean that it has to give the "correct" answer, which the user himself might not know in the first place. So it invents things because during the reinforcement learning users were happy with these answers. A funny example is the github co-pilot which writes buggy code, because it thinks this is what the user wants. Here is a video about that: ▶ this video

in “What GPT-4 Does Is Less Like “Figuring Out” and More Like “Already Knowing”” · View on HN