From 300KB to 69KB per Token: How LLM Architectures Solve the KV Cache Problem

2 comments

• az09mugen an hour ago

  Unrelated, but 69KB is how much RAM Voyager 1 has.

  [-]
  • gregman1 7 minutes ago

    Voyager as a token of curiosity