Author here. We published working canary credentials in a public GitHub repo on purpose and logged everything for close to a week.
Unsurprisingly, Github detected the keys in seconds and notified providers. Other external actors followed soon after started testing the keys in minutes.
What surprised us -
- AWS attached a quarantine policy on the key but this still allowed AWS Secrets Manager reads. Someone pulled all our secrets from AWS about 5 minutes in.
- Anthropic never revoked its key. Unclear if they were even notified.
- The secrets exfiltrated from AWS Secrets Manager were never used. This kind of deep reconnaissance and analysis should be easy to automate with LLMs but it seems like this has not happened (yet).
Author here. We published working canary credentials in a public GitHub repo on purpose and logged everything for close to a week.
Unsurprisingly, Github detected the keys in seconds and notified providers. Other external actors followed soon after started testing the keys in minutes.
What surprised us -
- AWS attached a quarantine policy on the key but this still allowed AWS Secrets Manager reads. Someone pulled all our secrets from AWS about 5 minutes in.
- Anthropic never revoked its key. Unclear if they were even notified.
- The secrets exfiltrated from AWS Secrets Manager were never used. This kind of deep reconnaissance and analysis should be easy to automate with LLMs but it seems like this has not happened (yet).