What's interesting is how well the models know the exact wiki text. If the wiki excerpt is rewritten the pattern matching doesn't work quite as well. Gemini is only able to one-shot about 1/3rd of the time which is still very impressive considering not long ago LLMs couldn't count the letter 'r' in strawberry.
What's interesting is how well the models know the exact wiki text. If the wiki excerpt is rewritten the pattern matching doesn't work quite as well. Gemini is only able to one-shot about 1/3rd of the time which is still very impressive considering not long ago LLMs couldn't count the letter 'r' in strawberry.