Jack Clark wrote today "Picture two giants towering above your head and fighting one another - now imagine that each time they land a punch their fists erupt in gold coins that showers down on you and everyone else watching the fight. That’s what it feels like these days to watch the megacap technology companies duke it out for AI dominance as most of them are seeking to gain advantages by either a) undercutting eachother on pricing (see: all the price cuts across GPT, Claude, Gemini, etc), or b) commoditize their competitor and create more top-of-funnel customer acquisition by releasing openly accessible models (see: Mistral, Facebook’s LLaMa models, and now GEMMA)." https://importai.substack.com/p/import-ai-362-amazons-big-speech
The technical report certainly makes strong claims, and better yet people with preview access seem to be backing up the claims as well as the obviously unlocked new tech tree branches. The 99% recall on 1M tokens is really impressive. Given how current models struggle with recall on full contexts, I'm wondering how this G1.5 recall scales to the full 10M window.
> This immediately recalls Bill Gates apocryphal 1981 statement
Funny you mention this. The same quote came to mind, though my thought was more that 10M corresponds to a measly 10s of MBs. At current prices that's in the range of 50–100 USD/query. Latency is currently on the order of 60s, but the tech report says we should expect speedup. There's still plenty of slack for RAG to pick up on here.
Honestly, I'm not sure what to make of this on the border. The disruption to recalling details on small bodies of discovery documents, medium-sized codebases, and long videos is immediate. However, for exploratory research and problems where the answer isn't buried nicely in some known blob of data, the effects are all higher-order. Similarly, it'll be very interesting to see how this impacts higher-level analysis. Will we see a precipitous drop in hallucination? Or will the hallucinations just jump to higher analytical layers?
Jack Clark wrote today "Picture two giants towering above your head and fighting one another - now imagine that each time they land a punch their fists erupt in gold coins that showers down on you and everyone else watching the fight. That’s what it feels like these days to watch the megacap technology companies duke it out for AI dominance as most of them are seeking to gain advantages by either a) undercutting eachother on pricing (see: all the price cuts across GPT, Claude, Gemini, etc), or b) commoditize their competitor and create more top-of-funnel customer acquisition by releasing openly accessible models (see: Mistral, Facebook’s LLaMa models, and now GEMMA)." https://importai.substack.com/p/import-ai-362-amazons-big-speech
Yup. That is how it feels.
The technical report certainly makes strong claims, and better yet people with preview access seem to be backing up the claims as well as the obviously unlocked new tech tree branches. The 99% recall on 1M tokens is really impressive. Given how current models struggle with recall on full contexts, I'm wondering how this G1.5 recall scales to the full 10M window.
> This immediately recalls Bill Gates apocryphal 1981 statement
Funny you mention this. The same quote came to mind, though my thought was more that 10M corresponds to a measly 10s of MBs. At current prices that's in the range of 50–100 USD/query. Latency is currently on the order of 60s, but the tech report says we should expect speedup. There's still plenty of slack for RAG to pick up on here.
Honestly, I'm not sure what to make of this on the border. The disruption to recalling details on small bodies of discovery documents, medium-sized codebases, and long videos is immediate. However, for exploratory research and problems where the answer isn't buried nicely in some known blob of data, the effects are all higher-order. Similarly, it'll be very interesting to see how this impacts higher-level analysis. Will we see a precipitous drop in hallucination? Or will the hallucinations just jump to higher analytical layers?