63 comments

  • tristanj an hour ago

    Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating. The site is struggling to keep up. Github's COO confirms it here: https://x.com/kdaigle/status/2040164759836778878

    Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.)

    20% of all GitHub accounts were created in the past 6 months https://x.com/kdaigle/status/2082604368399159542

      evanelias 10 minutes ago

      My sense is that a lot of this activity isn't even legitimate use. As one example: my company has a GitHub app, and last night some bot added my app to 1700 repos. Then the bot immediately started rapid-firing commits which each affected hundreds of files, triggering a deluge of GitHub webhooks to my servers.

      The repo names all matched other GitHub apps, so my sense is this bot had added 1700 GitHub apps to 1700 repos, with that number rapidly increasing at the upper bound of whatever GitHub's rate limit is.

      My systems caught it quickly and auto-booted the bot, but the whole situation is ridiculous.

        digitalsushi a minute ago

        there's this attack on youtube where people who dont want their real names known will autoblock comments with their real names; bots will spam comments with every word and note the ones that were not visible.

        kinda reminds me of it. not my most valuable contribution to forensics, admittedly.

      arjie 38 minutes ago

      The obvious answer is to end free unlimited private repos. Source is no longer a specific thing. GitHub is just a free S3 or Azure provider.

        SoftTalker 31 minutes ago

        I think this would cut out a lot of the mindless commit activity.

        Charge $1/month per repo. If your project isn't worth that, run your own git or use another free forge.

          hei-lima 14 minutes ago

          GH isn't good enough to charge that much for its core services. 90% of its user base would just find better or cheaper alternatives, and the only ones remaining would be legacy enterprise codebases that are just impossible or too expensive to migrate.

          odo1242 25 minutes ago

          I think GitHub leans on this free repos thing a lot for getting new users though. Like I could see people just moving to GitLab

            hakunin 22 minutes ago

            …and encountering the same issues most likely.

              odo1242 20 minutes ago

              Fair, but the point is GitHub’s model is:

              Free Users use GitHub => when a company needs source control => all their coders use GitHub => company buys expensive enterprise contract.

              If there’s less of an advantage to using free GitHub then that funnel works less effectively.

      bhouston 42 minutes ago

      > Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating.

      Sure, but how is OpenAI, and Claude handling the increased usage? They are handling it a lot better than Github. Just because you have fast growth does not mean you can not handle it.

      Instead you have to realize that you are entering into a high growth phase and design around that.

      It has been clear that we are in a high growth phase for around a year now and Github still hasn't adapted to that properly.

      At some point you have to admit that Github ready for the moment.

        traviswingo 39 minutes ago

        GitHub is on Azure. Azure is notorious for being terrible. There was a post here recently from an ex-Azure engineer explaining all of this in a multi-part series.

        Edit: Found the thread: https://news.ycombinator.com/item?id=47616242

          Tiberium 35 minutes ago

          Isn't a lot of OpenAI infra also on Azure? They seem to be faring better than GitHub.

            bhouston 17 minutes ago

            I bet OpenAI has failovers to other data centers. I think they have a variety of data center provides all over the place.

        SamBam 32 minutes ago

        OpenAI and Anthropic are spending hundreds of billions of dollars that they may never recoup forcing the building massive new data centers in the face of widespread protest.

        You think GitHub ought to match that?

          bhouston 24 minutes ago

          > OpenAI and Anthropic are spending hundreds of billions of dollars that they may never recoup forcing the building massive new data centers in the face of widespread protest.

          So has Github's owner Microsoft. It's build out is on par with OpenAI and Anthropic.

          > You think GitHub ought to match that?

          Microsoft is.

          The issue is that the software stack is not stable. That is clearly the issue here.

          It isn't a hardware build out issue. It is related to the architecture of Github.

        remus 19 minutes ago

        > Sure, but how is OpenAI, and Claude handling the increased usage?

        By spending tens of billions in investor cash. It is not clear what a sustainable business model looks like for these companies at the moment.

        > They are handling it a lot better than Github.

        Are they? Anthropic has fairly frequent issues.

        There's high growth and then there's demand increasing 10x in a year. For a mature business like github that's a massive, and I can't think of many big businesses who would be able to scale smoothly to accommodate that kind of demand.

        theappsecguy 27 minutes ago

        Well they heavily limit free tier usage and for paid users they are charging a tiered pricing model that ramps up very aggressively. Github is a lot more generous in this sense and it has a lot more surface area in terms of the functionality it serves

        tekno45 36 minutes ago

        OpenAI and Claude don't accept commits and host PRs? How are they the same?

          bhouston 22 minutes ago

          I am pretty sure OpenAI and Claude actually have a higher hardware cost per request that Microsoft. Github is actually most just a standard REST API app, although with some caching and git specific operations in the background. It is not as hardware intensive as GPU-based inference per quest with huge contexts and low latency.

          Also Github's feature offering has been incredibly stable for the last 6 years. There isn't an excuse for it to be unstable like this.

            pixelesque 16 minutes ago

            Yes, but come on...

            That's due to the need for ultra-fast memory and inference compute, it's not the same thing.

          dumberquestions 33 minutes ago

          They're getting increasing demand for tokens at an accelerating pace.

            pixelesque 21 minutes ago

            Is that the same as storing state in a DB and on a filesystem though?

            Like, I get there are other features they have, and things like history, but I don't think it's quite the same.

            Isn't most (not all) of the LLM's resources stateless compute to a large degree?

            (Not defending MS at all though, I currently can't create a PR on the required branch for my work (won't load the branch list) and their availability is really bad recently).

          fragmede 22 minutes ago

          Fundamentally, there's incoming requests, an application server, and a database. The devil is in the nature of those requests, and how do you process those requests and stick them in the database. And then serve them back out again. Different application, different database, but fundamentally, it's the same problem. Google managed to do this very well. Microsoft has not.

          logicchains 33 minutes ago

          They're undergoing the same kind of load. Almost every AI commit that's putting pressure on GitHub's servers was written on OpenAI and Anthropic's servers.

            simonw 23 minutes ago

            Scaling ChatGPT and scaling GitHub are very different problems.

            For LLMs, prompt handling is effectively stateless. They do try to route follow-up prompts to the same cluster to benefit from prompt caching, but those can be effectively sharded. They also need to write results to storage but that's on a per-customer basis, so also easy to shard.

            GitHub is a shared space, where commits and issues and PRs need to become instantly available to all readers across all geographies. They have a very different shape of scaling challenge to the LLM vendors.

              bhouston 20 minutes ago

              > GitHub is a shared space, where commits and issues and PRs need to become instantly available to all readers across all geographies. They have a very different shape of scaling challenge to the LLM vendors.

              Github data is accessible by all (if open source), but they should be partition-able by individual repository (and their related forks.). Thus while there is more shared state across users, it isn't fully shared state.

              And they have been working on this semi-shared state design for over 10 years now.

      pluc an hour ago

      Nobody at Microsoft could've predicted AI was going to be so popular!

      fourseventy 36 minutes ago

      Github is owned by Microsoft, its not like its some boostrapped startup who has a closet of servers that are melting.

      adzuci 20 minutes ago

      I think that data is a bit dated and misleading, but I would love to see if AI slop is a contributing factor here once the dust settles and how Github will address it. Looking at the return to mean in https://areyougoingexponential.rhys.dev/kdaigle makes me think the COO spike could be a Karpathy wiki or an agent too.

      bluerooibos 36 minutes ago

      > Github is struggling because AI-boosted coding increased the number of commits 14x in the past year,

      OK so then use the additional revenue to add more resources and employees? What the f*k are they doing over there?

        flexagoon 35 minutes ago

        > OK so then use the additional revenue to add more resources and employees?

        What revenue? I'd assume the people pushing thousands of AI-generated commits are not the people paying for GitHub Enterprise.

        thevinter 35 minutes ago

        What revenue do AI-boosted commits provide? Am I missing something?

        pavel_lishin 35 minutes ago

        Is there additional revenue from a bunch of people pushing their vibe-coded stuff up to Github?

      0xbadcafebee 24 minutes ago

      If that's the superficial cause, then the root cause is a lack of limits. Any production system in the world will go down from too much load if you don't limit the load. You have to do performance testing to determine what the system's peak utilization is, and put limits in place to prevent exceeding that limit.

      There are multiple kinds of limits you need. First you need the raw i/o limits on things like networking, load balancer requests per second, etc so you don't slow to a crawl on high load; you can throttle max i/o so demanding users don't steal performance for others, or even throttle network traffic to the client IP and internal services. Then you need additional limits at app layer, like number of requests you can process that make DB calls (as most DBs have a connection and request limit), or limit API requests to X/minute. Then you need more business logic limits, like max number of concurrent logged-in users, max number of repos a user can have, max number of requests for free users, priority for paid requests, etc.

      When you hit a limit, you have a couple choices. You can shed load (dropping calls immediately, which results in 503 errors for users, or even network disconnects), but it makes a very poor user experience, and can cause larger issues/conflicts in a distributed system. You can instead keep a max number of active sessions, and adjust that number over time, to ensure that (on average) your users have a stable experience; this requires implementing a "waiting room" for new sessions when old ones expire or log out; still not great for users, but it prevents everyone from getting random errors, which means the majority of users have a good experience.

      There are way more kinds of limits to implement. Each business has to analyze its entire system to identify performance bottlenecks and implement limits to ensure the majority of users continue to have good service. Without those limits, you get a shit experience for everyone, your brand is tarnished, and customers run for the hills.

      Finally, the performance testing helps you plan how much infrastructure you need to serve a given number of users. If you plan properly, you know X infra allows for Y number of users. Once you exceed Y users, you need to start throttling or stopping new user signup (and requests per minute) until you can increase capacity. Continuing to allow signups and additional usage, without the capacity to support it, results in what we see today.

      But competent engineers know all this. If they had competent engineers, and it gets this bad anyway, the problem is management. Microsoft likely already made back what they acquired GitHub for. They probably figured they could keep riding the gravy train until the train went off the rails, and either eventually sell GitHub or write it off. Such is business.

      TZubiri an hour ago

      Are these disproportionately in the free tier? Or is the increased storage load similar across the paid tiers?

      Might be worth considering a /GB/MO fee. Let's face it, VCS is a (glorified) storage service.

        cute_boi 38 minutes ago

        I think Github should start charging who goes beyond fair usages.

        I just checked no 1 vibe coder https://github.com/steipete and he has like 279,440. Compared this to Taylor who works on huge project like Laravel and is only around 8k commits.

          ProofHouse 35 minutes ago

          The scam that anthropic pulled, is they remove free tier (bad idea for Github honestly), but blame it on AI vibe code abusers. Instead of for example finding the 1-5% of absolutely aggregous users like Steipete and restricting them.

            logicallee 27 minutes ago

            it's a bit of a stretch to call removing a free tier a scam. as you can see a lot of people in this thread are saying Github should start charging.

      cute_boi 42 minutes ago

      It is just an excuse. If they can't support paid member or don't have infra they should start charging these slop projects who are committing relentlessly.

  • herpdyderp an hour ago

    non-helpful answer: the "Microsoft Acquires GitHub" line in this graph answers all questions https://damrnelson.github.io/github-historical-uptime/

      flexagoon 32 minutes ago

      Note that most of the downtime in that graph comes from Github Actions, which simply didn't exist before the Microsoft acquisition.

      The downtime also starts happening in 2020, which is a while after the acquisition. 2020 was the year many people started learning to code while being stuck at home.

      joshdavham an hour ago

      Microsoft acquiring GitHub is not a valid root cause analysis explaining Github’s degraded SLA.

        cwnyth 40 minutes ago

        According to that graph, performance drops widely after Microsoft's acquisition before AI enters the scene. If not Microsoft, how else do you explain it, or is the graph somehow misleading?

          lordgilman 39 minutes ago

          You're the one making the argument! You need to show causality instead of correlation.

            cwnyth 11 minutes ago

            I've made no argument at all. I'm asking a question to a person who made the claim that "Microsoft acquiring GitHub is not a valid root cause analysis."

      traviswingo 38 minutes ago

      Microsoft acquires GitHub and subsequently moves all infrastructure to Azure

      ciupicri an hour ago

      On the other hand I think usage increased and GitHub operates now at a different scale.

      preciousoo 26 minutes ago

      No one hates msft more than me, but there’s no way you think this one data point is useful to explain anything at all

  • solfox an hour ago

    Many microsoft services are down/failing today, including sites hosted on Azure. I'm guessing it's a larger MS outage.

      joshdavham an hour ago

      So perhaps we should be asking Azure engineers, not Guthub engineers?

        adzuci 33 minutes ago

        Are they? https://azure.status.microsoft/en-us/status doesn't support this afaict.

          tux3 26 minutes ago

          The page shows all green, which coincidentally is also how it looks when Azure is down. That page is for SLA/billing purposes, it won't show live incidents as they occur.

          jdelsman 30 minutes ago

          How sure are we that their status page is accurate? AWS's status page often lags actual issues.

  • rep_movsd a few seconds ago

    If you are doing 1000 commits a day, whats the point of git?

    Does the AI ever look back at the shit trail it left behind?

    Diffs are no longer diffs, they look like largescale delete and rewrite

  • zeafoamrun 34 minutes ago

    So you're asking github employees to violate their NDA?

  • qrush 19 minutes ago

    I got an "it is unacceptable" from their CPO on 8/7, and that they are "working around the clock on it".

    https://x.com/mariorod1/status/2085800861469495465

    I really think something deeper is going wrong there, and they're not being honest with their paying customers (and enterprises) about it.

  • VCFundedGenYer 27 minutes ago

    I think it's two things:

    - GitHub attempting (and seemingly failing) to move to Azure infrastucture for its website backend

    - AI generated code wrecking the site due to the volume of activities.

  • toomuchtodo 43 minutes ago

    GitHub Will Prioritize Migrating to Azure Over Feature Development - https://news.ycombinator.com/item?id=45517173 - October 2025 (63 comments)

    HN Search: azure capacity - https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

    Azure Capacity Crunch Extends into 2026 Amid Data Center Constraints - https://windowsforum.com/threads/azure-capacity-crunch-exten... - October 9th, 2025

    https://news.ycombinator.com/item?id=47427779

  • Alien1Being 25 minutes ago

    GitHub is struggling because it is owned by Microsoft.

  • delduca an hour ago

    AI + Microsoft = kabum

  • pluc an hour ago

    Microsoft. Azure. AI.

    Pick one!

      lexoj 31 minutes ago

      Also the fact that its written in Ruby on Rails probably doesn’t make it easier, performance wise.

        t-writescode 23 minutes ago

        Honestly. They’re already performing miracles with RoR.

        How many non Ruby apps and all the databases and systems behind could climb their traffic 14x when they probably only perf-tested for a sustained 2-3x and, and paid for a profitable amount of rack space rather than a “lol, VC money, we’ll 10x later” amount of infra.