weinzierlJun 2023Recommends
Computerphile has an interesting video about Glitch Tokens, but as it goes with Computerphile videos it's also a good introduction to the topic. ▶ this video
in “Understanding GPT Tokenizers” · View on HN
Glitch Tokens - Computerphile
Glitch Tokens - Computerphile on HackerTube: recommended by 2 Hacker News commenters across 2 separate months, 2023.
Computerphile
Open on YouTubeexplainer · intro · dated
Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.
Why Hacker News recommends it
HN points to it as the accessible explanation of why glitch tokens exist whenever tokenizers or odd GPT behavior come up.
- people recommended it
- 2
- separate months
- 2
- strong recommendations
- 0
- discussed on HN
- 2023
“it's also a good introduction to the topic.”
What HN commenters said
7 Hacker News comments link this video. Each one links back to its thread.
non-Apr 2023Recommends
I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens". tldw: Certain junk data was thrown out post-tokenization e.g. the /r/counting[2] community data and debug logs from Rocket League some tokens specific to those contexts stuck around, however, and are now like "a color you've never seen before" as far as GPT-X models are concerned giving the model one of these "glitch" tokens causes it to kind of freak out and return gibberish or some completely random response because it has not encountered them during training, because they were removed when the data was cleaned. [1] ▶ this video [2] [link]
in “OpenAI Tokenizer” · View on HN
veidrJun 2023Cites it
That is indeed the funniest (and slightly horrifying) thing. Even if you understand "glitch tokens"[1] (which is BTW the best way I've seen to explain to nontechnicals (hi, Dad!!) and also some subset of technicals (hi various friends of mine!! +_+...) that LLM applied statistics models aren't anything like sentient AGI)... ...regardless, ChatGPT and its ilk nevertheless do often already outperform "random human being coworker or uncle" at recommending just about anything... which is like Ted Chiang said[2], “That’s more a statement about how much bullshit we are required to generate and deal with in our daily lives.” Although he was not referring to inputs from your (dumbass) coworkers/uncles, it still holds true. [1]: ▶ this video [2]: [link]
in “” · View on HN
dale_glassJun 2023Cites it
It is well known, yes. This was discussed a way back during the whole glitch tokens deal: ▶ this video A bunch of those, like " SolidGoldMagikarp" are Reddit users.
in “What Reddit Got Wrong” · View on HN
chaoraceMay 2023Cites it
This reminds me of the recent so-called "Glitch Token" phenomenon[1]. When GPT-3 was presented with reserved tokens it never encountered during training, it reacted in extremely unpredictable ways -- often with a simple "fuck you". For those unfamiliar with LLM architecture: "tokens" are the smallest unit of lexical information available to the model. Common words often have their own token (e.g.: Every word in the phrase "The quick brown fox jumped over the lazy dog" has a dedicated token), but this is a coincidence of compression and not how the model understands language (e.g.: GPT-3 understands "defenestration" even though it's composed of 4 apparently unrelated tokens: "def", "en", "est", "ration"). The actual mechanism of understanding is in learned associations between tokens. In other words: the model understands the meaning of "def","en","est","ration" because it learns through training that this cluster of tokens has something to do with the literary concept of violently removing a human via window. When a model encounters unexpected arrangements of tokens ("en","ration","est","def"), it behaves much like a human might: it infers the meaning through context or otherwise voices confusion (e.g.: "I'm sorry, what's 'enrationestdef'?"). This is distinctly different from what the model does when it encounters a completely alien form of stimulation like the aforementioned "Glitch Tokens". At the risk of anthropomorphizing, try imagining if you were having a conversation w
in “Google Translate “Get well [Swedish firstname]” translates to “fuck you”” · View on HN
Der_EinzigeApr 2023Cites it
Yup. There are so many reasons why BPE is an issue: 1. It really hurts the ability to generate creative writing/poetry (e.g. impossible for even ChatGPT-4 to fully understand syllable counts, leading to incorrect haikus and even poor rhyming), see [link] and [link] 2. It means that silly stuff like "Glitch Tokens" are a huge issue. i.e. whole tokens dedicated to weird usernames caused by people counting on a counting subreddit so much that their names got their own token. See ▶ this video 3. BPE has a lot of terrible vocabulary choices independent of Glitch Tokens. Lots of massive punctuation, garbage sequences, etc.
in “Semantic Tokenizer for Enhanced Natural Language Processing” · View on HN
Show 1 more comment
chiiMar 2023Cites it
The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contains in-context logic. The evidence is that there are these "glitched" tokens : ▶ this video if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training dataset has a bunch of "nonsense" (a large dump of the subreddit that just counts a number, as well as various other "data"). So my hypothesis is that if you attempted the in-context logic training dataset, it would produce large number of glitched tokens (different ones, but equally as "illogical" to the user).
in “Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook” · View on HN