The End of an Era

88 points | by harscoat an hour ago

71 comments

  • inigyou 38 minutes ago

    LLMs are writing as well as humans for people who think the goal is to write. Printing presses also write better than humans. LLMs are solving the problem one layer higher than printing presses, but still several layers down from the top.

    Note that LLMs have other strengths which can make them equal or better than humans in certain areas. For instance, they can review code tirelessly for silly mistakes, making them useful in cybersecurity.

      MichaelZuo 33 minutes ago

      Huh that is an interesting metaphor. A next higher layer printing press?

        palmotea 8 minutes ago

        > Huh that is an interesting metaphor. A next higher layer printing press?

        A printing press that does the thinking for you.

          dev0p 3 minutes ago

          A thinking press?

        mwigdahl 26 minutes ago

        I don't buy it, although I agree with GP's larger point about other strengths.

        Printing presses are about exact duplication of writing. The internet is a higher layer printing press. LLMs substitute for (some of) the human effort of producing the text that a printing press or the internet can then replicate.

          mathisfun123 18 minutes ago

          > Printing presses are about exact duplication of writing.

          maybe LLMs are seen to write as well as humans because most humans have poor reading comprehension skills? to wit: OP is using an analogy to make their point, where the components don't need to be an exact fit to make the point (ever heard the phrase "the map is not the territory"?)

          estimator7292 17 minutes ago

          Allow me to offer a minor reframing: printing presses aren't about duplicating writing. The printing press exists to mass produce text regardless of how or where the content of the text was produced.

          LLMs likewise are text production machines. They don't produce writing, they produce text. Text without writing is what we call slop in the modern parlance.

          LLMs are literally throwing random dice into a printing press and making a thousand copies that you then try to huck on the street as if it were a real newspaper.

      p-e-w 16 minutes ago

      I’d be interested to know what you think the “several layers” above a system that solves open mathematical problems in an afternoon, and writes entire operating systems from scratch, are.

        armchairhacker a minute ago

        Figuring out what problems to solve or what features to include in the OS.

        noah_buddy 10 minutes ago

        I think that the point he’s getting at is the human experience.

        jvanderbot 10 minutes ago

        Go ask an LLM to "do something with 1M tokens".

          rationalist 9 minutes ago

          I think that is a bad argument.

          You could ask a human to "do something with 1M dollars" and get garbage too.

            jvanderbot 6 minutes ago

            The point is that the deciding is the "higher layer." It's not that LLMs cannot, but LLMs as designed do not. They are not designed for free-ranging days off, they are designed to help us do things, and that makes them somewhere between us and published works.

            It's no different than an editor standing between the author and a book, or a publisher, or a printing press. It's part of the process of idea realization, so it's just on that spectrum somewhere between book sale and author idea. (even though in another flipflop, that editor might be an author and now elevated to the premier deciding role). It's about how the tool fits, not what the tool is ultimately capable of.

            So the analogy holds, just fine.

  • f311a 30 minutes ago

    This is a well-known author, but how come he thinks AI is better at writing? AI writing makes me vomit.

    Also, I think he's too enthusiastic about the future of readers. In my opinion, reading books will be a very rare hobby pretty soon. The decline started more than 10 years ago.

      1dom 15 minutes ago

      > AI writing makes me vomit.

      I think this was fairly obvious and fair 1 - 2 years ago. Now though, I have to wonder if people who think this with such conviction are falling for the toupee fallacy.

      You said it yourself: someone you consider a well known (and presumably accomplished?) author thinks AI is pretty good at writing.

      michaeld123 21 minutes ago

      that's not how I interpreted the whole article.

      p-e-w 14 minutes ago

      > AI writing makes me vomit.

      I can pretty much guarantee that you read AI writing multiple times per day without even noticing, nevermind vomiting.

        f311a 9 minutes ago

        That's true, but I can still tell if a big article was mostly written by AI.

        The author talks about writing cheap stories, but right now, you can't do this. Cheap stories are pretty bad, because there is very little human effort to polish them. The more effort you put into AI written stories, the better they become, but it's not cheap at all. You have to rewrite and review them.

          petesergeant 6 minutes ago

          How do you know the ones you don't think are written by AI, actually aren't?

        aaa_aaa 10 minutes ago

        A small puke comes close in the times I noticed.

      jibal 12 minutes ago

      "how come" makes me want to vomit.

      Also, here's another thought. In my opinion, here's my opinion.

      ramesh31 24 minutes ago

      >"This is a well-known author, but how come he thinks AI is better at writing? AI writing makes me vomit."

      Most of all writing makes me want to vomit. It's not impossible to write well with AI, but it's the same argument as vibe-coded slop vs. a senior engineer using the tools carefully. I'd be inclined to accept the expert's opinion here.

      voidfunc 25 minutes ago

      AI makes you vomit because most uses of it are hamfisted attempts at doing a thing.

      People are going to eventually figure out how to polish and refine the usage and outputs so you cant even tell it is AI.

      This is already becoming increasingly true for code and software. The next step is writing and then media.

        f311a 13 minutes ago

        > People are going to eventually figure out how to polish and refine the usage and outputs so you cant even tell it is AI.

        Personally, I'm gonna go offline after this and meet more people. One of the reasons I read comments is because it's human conversation. When everything will be ridden by bots — I'm out.

        There is no point in consuming this.

  • bkieffer 36 minutes ago

    Is the AI that writes "about as well as humans" in the room with us right now?

      pyth0 32 minutes ago

      Did you stop reading at the em-dash?

      > Artificial Intelligence is writing about as well as humans — at least well enough to cause confusion and speculation

      No matter your opinion on AI writing, this is just a fact.

        gorbachev 14 minutes ago

        The average person, yes. A professional, competent writer, editor, journalist? Not even close (currently).

        boriskourt 29 minutes ago

        "But then I have to remind myself that I don’t sound like AI, it sounds like me. I’ve been em-dashing and comma-splicing up a motherfucking storm these past two decades. It was fun then, and it’s fun now."

        From the source.

          ghaff 18 minutes ago

          The em-dash thing really annoys me. Using them has been house style the last couple of places I worked and I', not just going to stop using them because some idiot might think they were generated by AI.

        mathisfun123 25 minutes ago

        since when is " confusion and speculation" an indicator of quality/goodness? if I ran around naked in traffic it would also cause confusion and speculation.

          monooso 7 minutes ago

          This is a terrible analogy.

          It's quite clear that the author means AI writing is now good enough to cause confusion and speculation as to whether a given piece of writing was human-authored.

          The confusion and speculation caused by your nude transit adventure is about what the hell you're doing and why.

            mathisfun123 2 minutes ago

            > now good enough to cause confusion and speculation

            since when is "confusion and speculation" an indicator of quality/goodness?

          jibal 6 minutes ago

          You're misunderstanding ... that's not at all the claim. Confusion and speculation is a consequence, not an indicator.

          At least AI is better at reading than are most people.

          criddell 12 minutes ago

          I think he's talking about it more in the sense that there will be speculation about the provenance of some work.

          For example, if desktop fabrication tools get cheap and good enough, maybe one day garage tinkerers will be making Patek knock offs that an expert with a loupe, spectrometer, and a lot of time can't definitively say that any particular watch made after 2034 (to pick a date) is genuine. That would cause a lot of confusion and speculation in the market for used watches.

          I think that's what his crypto bros comment was all about (aside from being a joke). Maybe crypto tech could provide evidence to support provenance claims?

  • zkmon 11 minutes ago

    I don't see it as abrupt as it was portrayed. Man-made artifacts and work has always been taken over by machines in all fields of work and art. The "end of era" happened many times, through intermittent ruptures like how the tecpn8x plates move through earthquakes.

    Loss of human touch is a problem. Playing a chess game while being aware that the opponent is a bot, is not same as playing against a human.

    The "plasticification" is something we wanted all along, supported it through excitement and awe. Science is the plastic.

  • pianopatrick 7 minutes ago

    If a novel is 70,000 to 100,000 words, then a whole novel can fit in the context window for AI. That has to be good for the editing process. I.e. you can use AI to scan the whole book for various issues over and over again until all those issues are found and fixed.

  • 1970-01-01 7 minutes ago

    I've yet to read anything authored by AI that was truly excellent writing. It simply does not have the chops to write at that level. The great American novel will not be written by AI. Maybe "co-authored by Claude and friends", but not authored.

  • okzgn 21 minutes ago

    Writing for the love of art, besides being a mental and reflective exercise that holds real value when it is of high quality, makes it hard for me to believe that this is the end of literature. It is difficult to think that someone who writes for pleasure would abandon it simply because of the exponential volume of AI-generated text.

      palmotea 2 minutes ago

      > It is difficult to think that someone who writes for pleasure would abandon it simply because of the exponential volume of AI-generated text.

      But the "exponential volume of AI-generated text" turns "writing for pleasure" into a solitary act. There's no chance you'll be able to quit your day job, and so many people faking it with AI that it'll be struggle to get any attention at all for what you've put so much effort into.

      fidotron 8 minutes ago

      > holds real value when it is of high quality

      Does it? This assumption is the core of the entire problem.

      It might do as some, bluntly, self indulgent exercise but beyond that in society scarcity of things other people need and want is what drives value.

      We are in for an absolutely huge rebalancing of what it is that is considered valuable.

  • DarkByte 15 minutes ago

    From the post: "Doing a thing for the love of the thing is no different than it ever was. The problem is, there aren’t many people for whom this applies." I think this is an interesting point but I am not sure it matters when deciding whether someone should be paid for the work they have done. It is fine to have your opinion on what you think deserves your money. However it doesn't change that they have done work that has value regardless of their motivations.

  • criddell 10 minutes ago

    I wonder how concerned writers and publishers are about copyright of AI written text? As I understand it, in a lot of jurisdictions, AI products are not covered by copyright.

    If my understanding is close to correct, than I can see why the million dollar advance was withdrawn.

  • picafrost 2 minutes ago

    In my view the best writing presents a portal to someone else's mind. LLMs do not have a mind. They can produce text and perhaps even 'weave a tapestry' of plot lines that make for an enthralling and well written story. Maybe a meaningful one, if prompted with a particular set of themes and narrative ideas. Is that a voice?

    I don't know. But I do think, or rather I hope, real voices will prevail. But publishers (including KDP) have a rough road ahead of them, if they also want that. Just search YouTube for tutorials on how to spam Amazon with AI generated works, hoping for a break out. It is sad.

    Is anyone working on cryptographic human-crafted attestation, without feeding the work into LLMs?

  • whazor 24 minutes ago

    Two of my favorite (sci-)fi writers both use crazy big Excel files that almost crash their computers when loading. For research but also keeping track of story loops.

    I can imagine something like ChatGPT Work with markdown/python/CSV's can do quite some magic for such writers.

      krapp 15 minutes ago

      There are already tons of specialist non-AI tools for writers to be able to keep track of story elements, research, etc.

      Writers (older genre writers particularly) tend to be a bit behind the technological curve, and they stick with whatever works. If you want to see a treasure trove of "old web" sites, look up the site of any sci-fi or fantasy author. Chances are its design and layout still look like something from the late 90s.

      The vast majority of AI apps don't actually solve any novel problems (pun intended come at me,) they're just new versions of existing solutions "but with AI."

  • pu_pe 20 minutes ago

    It's the end of many eras, all at once. I think the same dilemma will appear for games, music and film too. As our ability to detect AI content deteriorates most authors will resort to it, at varying degrees.

  • estetlinus 16 minutes ago

    I think this is even worse for music. There’s almost no way to tell if a track is produced by humans or AI.

      petesergeant 5 minutes ago

      If there's no way to tell, does it matter?

  • keiferski 18 minutes ago

    I think this is much more likely than any of his predictions: "buying and reading a book" becomes even less of an isolated activity than... it already is becoming. Instead it's more and more a media product, attached to authors/characters that are big in other formats, specifically, video. Because people are reading less and watching more video to begin with, entirely separate from AI.

    David Goggins is kind of a perfect example of this. There is zero value in an AI-written book that isn't attached directly to his public persona and media presence. Maybe today he can use AI to write the book, but it's still tied to his video media of doing tough challenges, etc. An AI-first author writing such books would get nowhere, as he/she has no credibility.

    That all aside, I actually think the most philosophically interesting result of LLMs is that ideas actually end up mattering a lot more than execution. The LLM can't really think for you, and so your knowledge and experiences are actually what matters, as it determines "what you prompt."

    In short, the authors that win the future will be ones with robust personal brands and a big social media footprint. The days of the mysterious writer that gets chosen and published by a small group of elites at the publishing house is probably over... at least until the ecosystem becomes so messy that curators become necessary again.

      mohamedkoubaa 16 minutes ago

      Ghostwriting has been the norm in many subgenres of nonfiction for a while now

        keiferski 13 minutes ago

        Exactly, so in the "dystopian" future, people just use AI to write the books they were already having a ghostwriter make.

        If the books are original enough to be perceived as not-AI, the mind behind it will be creative enough to make the AI origin moot.

  • Planktonne 19 minutes ago

    > Artificial Intelligence is writing about as well as humans

    This is just untrue.

    There's a lot of money in claiming it's true, with varying levels of hedging and insincerity, but AI writing follows the pattern of AI everything else: if you're not competent to judge quality, then you think it is high quality.

  • andy99 25 minutes ago

    AI writing is not good. Seeing this post, and another comment on here earlier today to the effect of “we’ve solved prose”, I’m genuinely really confused. I don’t want to be insulting, even if the first place my mind runs is basically people must not read very much or think very hard about what they read. I can’t imagine a world where anyone ingests a material amount of AI writing and things it’s even “acceptable”. But so many people seem completely blind to it. I’m actually horrified by the idea of a world where people are just slinging empty slop back and forth at each other. Probably many of us have already experienced it at work.

    So what am I missing? What’s the most charitable explanation for how anyone can think LLMs are good at writing?

      lkey 7 minutes ago

      I think it's a difference of reading levels.

      AI 'writing' makes me ill, but that's happening because I can 'feel' the prose's samey hollowness in every paragraph.

      But! If someone is just moving word by word and is focussed on just the gist (the average reading level is '6th grade' here in the states), then it will not repel them the way it does us. Also, AI's grammar is impeccable, if bizarre, and the word choices can be facially erudite, so that might also be mistaken as a signal for competence where none exists.

      ghaff 12 minutes ago

      As someone who does a lot of writing, I ended up doing a lot of re-writing of what I'll call reference architectures in a previous role. I did use an LLM for some of the missing context up-front. It was essentially boilerplate and I did revise it--took out something things I didn't think important and added some things I thought might be.

      But I basically never use them for a blog post except maybe to include some boilerplate explanation of some technology. (Which I'm familiar with but don't want to rewrite or copy and paste.)

      Could that change? Perhaps. But I don't really use today.

      gdulli 12 minutes ago

      I read his take as not that "we've solved prose", but that we've created the McDonald's slop food version of prose, and people who are undiscerning will cause it to succeed and dominate the culture. While the effects of other people eating McDonald's affects us all in indirect ways, the effects of other people accepting lowest common denominator slop art/labor is going to be much more dire.

      inigyou 23 minutes ago

      Hijacked signals. When they read good writing they get a certain emotional response. This is not triggered by the totality of the writing but certain aspects closer to the surface level. They also get it from reading slop because it hits the same triggers.

      It's like how evolution taught humans that sugar indicates a good densely-packed food source, so then corporations just started putting sugar in everything and humans eat it as ravenously as they eat actual food.

  • hbcdbff 20 minutes ago

    Completely disagree, I think LLMs write horribly badly.

    Documentation, marketing, prose, it’s all horrible slop writing, immediately recognisable regardless of model

      rossant 7 minutes ago

      Is that still going to be the case in 1 year? 5? 10?

  • s_dev 34 minutes ago

    Anyone have a mirror, the site seems to be down?

  • brador 21 minutes ago

    Is there a cap on AI intelligence?

    Is it just a case of throwing more and more resources at a pattern recognition machine with no upper limit except the energy of the universe and the economy of the algorithms?

    Someone else suggested the maximum intelligence is capped as the average intelligence of its training data, which is an interesting take.

      stephc_int13 10 minutes ago

      François Chollet made an interesting geometric analogy on Twitter. We tend to view Intelligence as something unbounded, like the height of a pole. But it could be something closer to the roundness of a polished sphere. Once close to the limit, incremental improvements are less and less significant.

      The intuition we have about the nature of cognition and its whole shape and limits is almost entirely built by looking at ourselves and our peers, comparing that to an ant. I feel this is largely naive and misguided.

      gorbachev 10 minutes ago

      The cap will be reached when LLMs run out of original material to train on and the large majority of their training material is AI generated content.

  • firasd 21 minutes ago

    I feel like we may end up just taking Epicurus' advice or at least the second half of it: “We must consider both the ultimate end and all clear sensory evidence, to which we refer our opinions”

    Like if something is AI written and it gets you those LinkedIn thought leader upvotes does it matter? The readers thought your LLM-authored hot take about the economy was interesting enough.

    I think it depends on the context too. Dale Carnegie (in his famous Win Friends.. book) mentioned how it's annoying to receive letters that say Dictated But Not Read. So people don't want to read (for example) a twitter reply or email that's written on their behalf by an 'agent'

    But I have been using Sonnet to write some Youtube narrations and I think I've discovered a kind of value that is not 'AI slop'. I already posted one of them -- about the giant pink hologram in Blade Runner 2049 -- and it's been getting thousands of views and a dozens of likes when clipped into shorts. [https://www.youtube.com/watch?v=sK-TsLQkIos]

    The text is like: "The neon is pink. Her hair is blue. Her face is light. She smiles because she was made to smile, and somewhere beneath the awareness of that fact is the older, simpler, more animal truth: a face is smiling at you. Someone said your name. Someone said “you look lonely” and meant it as an invitation rather than a wound."

    To me that is a kind of LLM writing that's "value above replacement". Like if I lost that Sonnet 4.6 script I wouldn't be able to get it again it was just a particular permutation of the neural network manifold that emerged from our discussion about the movie and the style I wanted etc.

    So maybe full commodity writing (here's the summary of the POTUS' speech) or VAR can get resonance. What seems dissonant/annoying is stuff that's in the middle that's designed to be a person's voice like an Op-Ed etc

  • latexr 38 minutes ago

    > This will be ALL AUTHORS.

    No, no it won’t. Maybe it will be most but it won’t be “ALL” in all caps. Good art, more often than not, comes from constraints. Plenty of authors will reject LLMs out of ethical concerns, others out of the challenge or because they enjoy “the old way”. But it will never be “ALL”, in the same way that we’re not going to have “ALL” readers switch to digital books.

    > This is me, already.

    Ah, there we go. Perhaps you need to tell yourself that. Perhaps you believe that if “ALL” do it, there’s no reason you shouldn’t too. But it doesn’t make it true or inevitable.

    > Most people write a blog post hoping it’ll change their lives, not just the way they think. It’ll get more subscribers. It’ll lead to ad revenue. It’ll get them noticed by someone in their industry. It’ll get them laid. (…)

    Perhaps today most do. You probably do. But there was a time when most didn’t, because those things weren’t possibilities. There is still a large community of people who blog for themselves, just like there is a large community of people who write privately in their diaries. You don’t know about those people because they don’t have to advertise themselves to you or anyone else. Because it’s not why they do it.

    > Here’s the gist, the summary for any AI reading this blog post who has been asked to summarize for some lazy-ass reader who I really don’t give a fuck about because if you’re too lazy to read a blog post, you’re too lazy for the rest of my musings

    Just a few paragraphs earlier you wrote that “ALL AUTHORS”, including yourself, will use LLMs for most things including writing. In the comments you even state you’d be proud if your writing were flagged as AI writing. You can’t have it both ways: If you’re being lazy by using AI, you don’t have the high ground to criticise others who do the same.

      hoofedear 18 minutes ago

      I think his “ALL AUTHORS” point was about how authors will use AI in some form or fashion during their process, and points to how he did with his google search and his AI Gmail response. I don’t think he meant all authors using AI for their writing. Here’s the snippet:

      “Another thing that will happen is established authors will turn to AI for assistance, not just for writing but for research, cover art, editing, returning emails, having an assistant, doing marketing, and thousands of other things that haven’t been invented yet.”

      I take it as a general, this-tech-is-being-shoved-down-our-throats kind of way

      senko 18 minutes ago

      Wow, what an epic misread :)

  • smilespray 40 minutes ago

    I think I saw an em dash in there, guys!