The "platonic representation hypothesis" crowd can't stop winning.
Potentially useful for things like innate mathematical operation primitives. A major part of what makes it hard to imbue LLMs with better circuits is that we don't know how to connect them to the model internally, in a way that the model can learn to leverage.
Having an "in" on broadly compatible representations might make things like this easier to pull off.
You seem to be going off the title which is plainly incorrect and not what the paper says. The paper demonstrates HOW different models can learn similar representations due to "data, architecture, optimizer, and tokenizer".
"How Different Language Models Learn Similar Number Representations" (actual title) is distinctly different from "Different Language Models Learn Similar Number Representations" - the latter implying some immutable law of the universe.
Saw similar study comparing brain scans of person looking at image, to neural network capturing an image. And were very 'similar'. Similar enough to make you go 'hmmmm, those look a lot a like, could a Neural Net have a subjective experience?'
"using periodic features with dominant periods at T=2, 5, 10" seems inconsistent with "platonic representation" and more consistent with "specific patterns noticed in commonly-used human symbolic representations of numbers."
Edit: to be clear I think these patterns are real and meaningful, but only loosely connected to a platonic representation of the number concept.
(Pardon the self promotion) Libraries like turnstyle are taking advantage of shared representation across models. Neurosymbolic programming : https://github.com/jdonaldson/turnstyle
It's going to turn out that emergent states that are the same or similar in different learning systems fed roughly the same training data will be very common. Also predict it will explain much of what people today call "instinct" in animals (and the related behaviors in humans).
The "platonic representation hypothesis" crowd can't stop winning.
Potentially useful for things like innate mathematical operation primitives. A major part of what makes it hard to imbue LLMs with better circuits is that we don't know how to connect them to the model internally, in a way that the model can learn to leverage.
Having an "in" on broadly compatible representations might make things like this easier to pull off.
You seem to be going off the title which is plainly incorrect and not what the paper says. The paper demonstrates HOW different models can learn similar representations due to "data, architecture, optimizer, and tokenizer".
"How Different Language Models Learn Similar Number Representations" (actual title) is distinctly different from "Different Language Models Learn Similar Number Representations" - the latter implying some immutable law of the universe.
Same with images maybe?
Saw similar study comparing brain scans of person looking at image, to neural network capturing an image. And were very 'similar'. Similar enough to make you go 'hmmmm, those look a lot a like, could a Neural Net have a subjective experience?'
"using periodic features with dominant periods at T=2, 5, 10" seems inconsistent with "platonic representation" and more consistent with "specific patterns noticed in commonly-used human symbolic representations of numbers."
Edit: to be clear I think these patterns are real and meaningful, but only loosely connected to a platonic representation of the number concept.
Title is editorialized and needs to be fixed; the paper does not say what this title implies, nor is that the title of the paper.
(Pardon the self promotion) Libraries like turnstyle are taking advantage of shared representation across models. Neurosymbolic programming : https://github.com/jdonaldson/turnstyle
Curious if this similarity comes more from the training data or the model architecture itself. Did they look into that?
They describe that both are important, and researched in the paper, within the opening paragraph.
The eigenvalue distribution looks somewhat similar to Benford's Law - isn't that expected for a human-curated corpus?
It's going to turn out that emergent states that are the same or similar in different learning systems fed roughly the same training data will be very common. Also predict it will explain much of what people today call "instinct" in animals (and the related behaviors in humans).
Oh yeah, that's clever