what can language model embeddings tell us about whales speech, and decoding ancient texts? (on The Platonic Representation Hypothesis and the idea of *universality* in AI models)
Grateful for this read, which introduced me to the convergent representation hypothesis. I'm a neuroscience instrumentation engineer, not a computer scientist, and I've been following closely the developments in compressive sensing because I think these ideas may be important to brain recording. Anyway, I'm not sure I grasp your final point, or how these ideas relate--you suggest that we can hope to decode the few samples of Linear A we have by leveraging an otherwise complete corpus of language embeddings? At some point, the limited amount of Linear A we have still makes this a very hard inversion problem. (Luckily we can continue to record the whales...)
Grateful for this read, which introduced me to the convergent representation hypothesis. I'm a neuroscience instrumentation engineer, not a computer scientist, and I've been following closely the developments in compressive sensing because I think these ideas may be important to brain recording. Anyway, I'm not sure I grasp your final point, or how these ideas relate--you suggest that we can hope to decode the few samples of Linear A we have by leveraging an otherwise complete corpus of language embeddings? At some point, the limited amount of Linear A we have still makes this a very hard inversion problem. (Luckily we can continue to record the whales...)