Turning Vague
Evans makes a point interpretability and model-welfare researchers should take seriously: conceptual breakthroughs look trivial only in hindsight, so early AI self-reports may be wrongly dismissed as derivative. The analogy is persuasive but incomplete — humans building concepts of their own minds had no external check, whereas with AI we may get a second channel via mechanistic interpretability that could validate or debunk introspective claims.… more
The weaker link is assuming there's something to introspect at all: whether models have experiences remains contested, so the first crisp concepts may describe behavior rather than phenomenology. His point that AIs lag in hard-to-verify domains usefully undercuts the naive 'scaling will handle it' view — long CoTs and swarms may help, but verification stays the real bottleneck.
Turning vague intuitions and introspection into crisp concepts and ideas (a big part of making philosophical advances) is difficult. Very few people can do this well, and even then it takes them many years of dedicated study and grind (similar to producing novel math or science or art). Part of the challenge is that it's hard to know if you're maki…
Source:
X — Owain Evans
Discussion (0)