A great deal of their research has been focused on zigging where others zag. Their paper "Continuous Thought Machines" (https://arxiv.org/abs/2505.05522, presented at NeurIPS) was posed specifically under the framing of there needing to be more fundamental research beyond squeezing as much as we can out of relatively vanilla transformer stacks. It is very biologically inspired and unique.
Now that models are getting stronger at agentic work, it is very natural that many labs are chasing some form of auto-research.
On the contrary, I find them to be one of the least hypey companies. For instance, a cursory familiarity with David Ha's work would inform you that the team has been doing this kind of stuff for quite a long time.
OpenAI is not Sam Altman
Anthropic is not Dario Amodei
and Sakana is not David Ha
Organizations, especially businesses, are not individuals. If the implication is that David Ha has always been doing this, and will always be doing this, and that Sakana is David Ha ... then that's a far worse insult to the employees at Sakana than my little tweaking.
I don't know how RSI aligns with DPI ("Data Processing Inequality", which states that, basically, unless you have an infinite supply of real data, you will suffer from model collapse). Models can't keep improving themselves infinitely. See, for example, https://arxiv.org/html/2601.05280v2
I think the most impressive thing about Sakana.ai is their relentless pursuit of whatever is hype right now.
Genuinely it take a lot of work and talent to be this hype-motivated and completely ignore anything except what is popular on X at any given time.
Note: RSI is an incredibly important topic -- I just don't care to listen to Sakana on this matter -- they are the epitome of "hypebeast" https://www.urbandictionary.com/define.php?term=hypebeast
(Thanks for sharing hardmaru)
A great deal of their research has been focused on zigging where others zag. Their paper "Continuous Thought Machines" (https://arxiv.org/abs/2505.05522, presented at NeurIPS) was posed specifically under the framing of there needing to be more fundamental research beyond squeezing as much as we can out of relatively vanilla transformer stacks. It is very biologically inspired and unique.
Now that models are getting stronger at agentic work, it is very natural that many labs are chasing some form of auto-research.
On the contrary, I find them to be one of the least hypey companies. For instance, a cursory familiarity with David Ha's work would inform you that the team has been doing this kind of stuff for quite a long time.
OpenAI is not Sam Altman Anthropic is not Dario Amodei and Sakana is not David Ha
Organizations, especially businesses, are not individuals. If the implication is that David Ha has always been doing this, and will always be doing this, and that Sakana is David Ha ... then that's a far worse insult to the employees at Sakana than my little tweaking.
Being a Hypebeast leads to a rich acquisition.
TRUE
I heard about the ShinkaEvolve on a podcast where the guest had used it to evolve an agent harness for a less capable model.
I ended up borrowing the ideas from it for one of my own personal projects.
I don't know how RSI aligns with DPI ("Data Processing Inequality", which states that, basically, unless you have an infinite supply of real data, you will suffer from model collapse). Models can't keep improving themselves infinitely. See, for example, https://arxiv.org/html/2601.05280v2
Fortunately a rational society like Japan is not as interested in outsourcing their capacity to curve fitting models as other societies.
What?
Ask your favourite LLM. Thinking is hard.