75 comments

  • llmslave2 18 hours ago

    X needs to be held liable in a criminal court for this. They should not be allowed to hide behind an AI and claim they are not at fault or whatever. They provide a service where users can request images, and the service generated illegal imagery. I will die on the hill that engineers and execs should be heavily fined or going to jail for this.

      devilsdata 18 hours ago

      Hear, hear. There should always be steep consequences for taking advantage of the vulnerable.

      ares623 18 hours ago

      I know Musk will be untouchable but if X has offices around the world can the people there be held accountable?

        ronsor 18 hours ago

        X will simply close those offices, as they've done in the past.

      s5300 18 hours ago

      [dead]

      dmitrygr 18 hours ago

      > service generated illegal imagery

      For my own curiosity, could you please elaborate which law was broken? And if you do find one, elaborate how is it not unconstitutional, given that “put X into a bikini” is constitutionally protected speech, and the output fails the Roth test for obscenity and is thus protected just the same. Would not every other law claiming to be relevant be null and void w.r.t. stoping this in this particular case? SCOTUS was quite clear in Roth v. United States. The later Miller case is also of no help, since nowadays appearing publicly in a bikini is considered quite normal and not at all obscene or sexual.

      I get it. If this happened to a member of my family, I'd be pissed too. But, as logical people is it not our job to NOT simply leap from "I'm very angry" to "It must be illegal", but to be calm and rational.

        soraminazuki 13 hours ago

        Grok is generating content of real woman and children being undressed, raped, humiliated, hurt, or killed. I wonder what's driving some people to downplay and defend these monstrous actions. "Calm down! Not only is this quite normal, it's a constitutional act." The reality distortion field is very strong this past year or two when it comes to AI-related or wrongfully flagged threads.

        - https://futurism.com/future-society/grok-violence-women

        - https://www.reuters.com/legal/litigation/grok-says-safeguard...

        - https://www.rollingstone.com/culture/culture-news/elon-musk-...

        text0404 14 hours ago

        > “put X into a bikini” is constitutionally protected speech

        Unfortunately if "put X into a bikini" is a command to an AI to generate CSAM of an underage child, that's no longer "constitutionally protected" territory. Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US.

          ndsipa_pomu 5 hours ago

          > Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US

          It's peculiar to me how people in the U.S. seem to prioritise "freedom of speech" and de-prioritise freedom from being harassed online. In general, I support free speech, but I don't take an absolutist position as there can be a lot of harm created by allowing lies to propagate unrestricted (c.f. Brexit in the UK). Meanwhile, there's direct emotional harm being caused by these AI images being posted and also a speech chilling effect by making people, especially women, afraid of posting on contentious topics.

        labrador 16 hours ago

        It's illegal in California.

          kemayo 13 hours ago

          Not just California. Non-consensual deepfake porn is explicitly federally illegal now.

          Ted Cruz's "Take It Down" Act passed last year: https://www.congress.gov/bill/119th-congress/senate-bill/146

          (This may or may not be constitutional in the end, but it's certainly current law on the books.)

          dmitrygr 16 hours ago

          > It's illegal in California.

          1. What specifically is? Posting twitter comments? Prompting an ML model? Generating images based on existing images?

          2. Even if so, states are not allowed to make unconstitutional laws. 1stA covers this, as far as I can tell. Do you read Roth or Miller differently?

            labrador 14 hours ago

            "is posting nudes of women without their consent legal in california"

            https://www.google.com/search?q=is+posting+nudes+of+women+wi...

              xigoi 8 hours ago

              Posting such images is illegal, i.e. it’s the user who has violated the law.

                Timon3 7 hours ago

                But the user isn't posting the image, Grok is. The user can't even delete the image if Grok generated something the user doesn't want.

                  xigoi 7 hours ago

                  Ah, that’s a terrible implementation then.

            kccoder 7 hours ago

            It’s seems as though you’re being deliberately obtuse. It’s obvious why this is illegal.

      TrainedMonkey 18 hours ago

      While we are at it, let's also get all the weapon manufacturers. How dare they hide behind "we are just making a tool" and "we are not responsible for what our users do". And don't get me started on people using cars to cause harm to others, why is nobody going after big auto?

        janice1999 17 hours ago

        What an unserious response.

        Gun manufacturer accountability (or lack thereof) is a complex topic and one with ongoing lawsuits and evolving legal arguments. (In the USA see the Bush-era NRA-backed Protection of Lawful Commerce in Arms Act in 2005 and recent Smith & Wesson Brands, Inc. v. Mexico arguments).

        Personally if a gun manufacturer markets a gun with the primary feature being finger print resistant (yes, that's real) and being easy to carry concealed, I think lawmakers should investigate. Likewise f someone makes a big CSAM generator button and puts it in front of millions of users, it also deserves legal attention.

          TrainedMonkey 17 hours ago

          It was an unserious response to an absolutist statement. I pointing out where such absolutism would lead if applied to other areas.

          I don't think fingerprint scanner on guns will be effective as it tracks ownership and not legality of usage. However, a number of modern vehicles do have capabilities to perform autonomous actions, including overriding user input.

            llmslave2 17 hours ago

            It's not an absolutist statement and it doesn't lead to any of the insane conclusions you came up with. Holding the people running a model liable for what it generates has nothing to do with making tool makers liable.

          xigoi 8 hours ago

          It’s not a CSAM generation button, it’s a general content generation button that someone used to generate CSAM, and it’s that someone who is responsible for it.

        llmslave2 18 hours ago

        No, because Grok or ChatGPT etc are not tools, they are services. It doesn't matter if Grok uses an LLM or a bunch of people using Photoshop, if they are outputting illegal content the consequences should be the same.

        People don't go after big auto because they sell cars that other people drive. If they suddenly started offering a service where they would drive people around in their cars, and they started crashing into other people, of course they would be responsible for that.

        arcatech 18 hours ago

        Flattening the discussion to “everything is actually exactly the same” isn’t helpful. This situation is not the same as a person driving a vehicle.

  • labrador 18 hours ago

    This shouldn't be flagged because this is a very interesting AI problem. Grok was trained on X content which contains a lot of porn. Other image generation models aren't trained on porn so they don't know how to produce it. It appears to be very difficult to stop Grok from making porn since it has been trained on it. Is the ony workable solution to not train Grok 5 on porn?

      janice1999 17 hours ago

      They would never do that. They have an anime girlfriend app. They've picked their target audience, let's if it was worth it.

  • afavour 18 hours ago

    I'm curious to see how long this lasts on the front page before being flagged off, like all the others have. Ask yourself: why is that?

    EDIT: what do you know, flagged

      FloorEgg 18 hours ago

      Maybe start by reviewing the HN guidelines and asking yourself whether or not this is on-topic for HN.

        monsieurgaufre 18 hours ago

        The guidelines are bent all the time.

        This article is about tech used in illegal ways. How is that not following the guidelines?

          FloorEgg 17 hours ago

          It sure seems to me like you haven't read the guidelines, because if you had you would have made a different case for why this post should be on-topic, since the guidelines use "crime" as an example of what is off-topic.

          The spirit of the guidelines is:

          On-topic: fosters intellectual curiosity

          Off-topic: derails intellectual curiosity

            monsieurgaufre 16 hours ago

            Your own submission do not follow these guidelines.

              FloorEgg 16 hours ago

              I've only made one submission in 2024 and yes it's true it was not appropriate for HN. Since then I've read the guidelines multiple times and learned how to follow them better.

              I don't see how a small mistake I made in 2024 changes the fact that the HN guidelines seem to align with a decision to flag this submission.

            mariusor 6 hours ago

            I think it's a gross omission on the part of whomever wrote the guidelines if discussions about ethics are deemed off topic.

            Because many of these events where tech is being used, or abused, in a way that's morally challenging lead to those discussions, and I for one, am very curious why the tech literate crowd of HackerNews is uncomfortable in talking about them, and presents their amorality as "it's against the guidelines".

        UncleMeat 14 hours ago

        I really don't think I can imagine a more on-topic thing than a story that combines the richest tech entrepreneur in the world (musk), a major tech company (twitter), and the most important technical development in ages (ai).

          an hour ago
          [deleted]
          FloorEgg 13 hours ago

          On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.

          Off-Topic: Most stories about politics, or crime, or sports, or celebrities, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably off-topic.

          I can think of many way more on-topic submissions. They are the ones where no one is upset and everyone is nerding out about an obscure topic.

          Let me be clear, classifying this submission as off topic is not cut and dry. My point was there is clearly good justification for it in the official guidelines.

          - it's not a topic that fosters intellectual curiosity because it is somewhat rage inducing. It's outrageous. It's frustrating and it's scary. It's very likely to upset people.

          - it's also related to a controversial celebrity

          - it's also about crime

          - it's also something that surely will be covered by TV news

          These things are laid out right there clearly in the guidelines.

          I am choosing to give the moderators the benefit of the doubt because I believe they are doing their best to try and foster good intellectually curious dialog and minimize emotionally charged heel digging disrespectful debates and flame wars.

            panda-giddiness 4 hours ago

            Clearly the "on-topic" and "off-topic" sets have a non-empty intersection, so the guidelines are inherently ambiguous.

            UncleMeat 13 hours ago

            > it's not a topic that fosters intellectual curiosity because it is somewhat rage inducing. It's outrageous. It's frustrating and it's scary. It's very likely to upset people.

            Frustrating, outrageous, upsetting, and scary things are worth discussing. These are important topics.

              FloorEgg 13 hours ago

              Are you debating my interpretation of the HN guidelines or the HN guidelines themselves?

              It seems a bit like you are shifting the goalpost here. I was clarifying how the decision to flag this submission can be explained by the HN guidelines.

              I agree with your statement in principle, but it's irrelevant to my point or yours about this submission being on-topic.

              I regularly discuss these kinds topics in groups with people that have diverse viewpoints, but also can maintain civil dialog. The problem is that anonymous people on the internet usually can't discuss these topics and stay civil without devolving into echo chambers. I assume this is why HN made a decision to call them out of scope. Personally I think it was a good decision and it's part of what makes HN unique and valuable. It's a kind of sanctuary.

              If you want to discuss these topics there are lots of other places to go.

        afavour 18 hours ago

        I have, and it is. In what world would it not be?

          FloorEgg 18 hours ago

          Interesting, I'm surprised you would think that.

          Given the very first part about what's on-topic and off-topic, how would you categorize this post?

            afavour 17 hours ago

            > On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.

            Deepfakes and their implications are interesting, they’re technical, they’re of interest to hackers. A tech company operating globally and violating the laws in multiple jurisdictions at once? An interesting topic worth of discussion.

            Your attempt at gatekeeping here would be more believable if we weren’t all able to look at other submissions on the front page. A government site called “eat real food” has been on the front page for nearly a day now. It’s very convenient to selectively apply guidelines but it’s also transparently dishonest.

              FloorEgg 17 hours ago

              Why didn't you also include the off-topic portion of the guidelines and make a case why this post doesn't classify as off-topic?

              In what way am I gatekeeping? You were asking for an explanation, and I was attempting to show you one. I sat back and wondered (without any emotional stake in this) "hmm, what is afavour implying here and why would this be flagged?" And then re-read the guidelines myself, and it was pretty clear how an interpretation of this post would classify as off-topic.

              I also don't understand your point about the front page or the "eat real food" post. What are you implying exactly?

              What this situation looks like to me:

              - you are outraged

              - you are making assumptions about political bias in the moderators choices to flag this post

              - you assume that I am siding with the moderators, rather than just pointing out how the moderators choices to flag this topic seems to align with the official guidelines.

              - you assume I fall into some political or ideological bucket, but I almost certainly don't.

              - the HN guidelines are there to foster intellectual curiosity, and part of that is acknowledging that topics that trigger outrage derail intellectual curiosity and are therefore off-topic. This post being flagged seems to be reasonably in line with this.

              - I misinterpreted your original comment as genuine curiosity. I conflated your "I'm curious how long this will take to be flagged" with the "ask yourself why", and assumed you would be open to an explanation for the "why". Clearly you were not. You've made up your mind already about the matter. You aren't curious about this at all, so maybe then we shouldn't be having a discussion about it on HN?

                cholantesh 17 hours ago

                >you assume I fall into some political bucket, when I almost certainly don't (I am not American and I don't follow American media). I don't use any social media other than HN and a couple private group chats.

                This is pretty meaningless; one can share an ideology with an official they did not or even cannot vote for, and can do so even if they don't have Twitter.

                  FloorEgg 17 hours ago

                  Are you implying that because I am pointing out how the HN guidelines state crime and other topics that trample curiosity are off-topic, I share some ideology with some American official that condones child porn?

                    cholantesh 16 hours ago

                    Whatever the guidelines state, those other topics are frequently discussed without much rancor which you refuse to confront. But anyway, all I was saying is that that specific line item is a non-sequitur.

                      FloorEgg 16 hours ago

                      > without much rancor which you refuse to confront

                      I was responding to the "why is this post flagged?", and you changed scope to "justify all moderator decisions across the platform". Consider that you may have a sensitivity to certain topics that I do not. It seems like you see something I don't.

                      I'm not condoning all moderator decisions. I'm not remotely familiar with all posts on HN. I'm only considering this one post and acknowledging how this topic is clearly trampling curiosity and that it seems reasonable to me that it's been flagged given the spirit and word of the HN guidelines.

                      Of the posts I am familiar with, there have been several cases when a thread was flagged that at first seemed on-topic but after reflecting I realized the comments had gone off the rails and that's why it was flagged. This decision seems very much in line with other cases I've seen, but I can't justify all decisions nor am I position to even judge them.

                      > But anyway, all I was saying is that that specific line item is a non-sequitur.

                      It actually wasn't, though I could have been more clear about why it was relevant. Where people get information and what tribe they feel they belong to changes how they interpret new information. The reason I brought it up is because I was getting the sense from multiple replies to my comments that some people were assuming I (and the moderators) wanted to censor the underlying concern about Grok as if I was a part of the American culture/political war. I was just trying to clarify I am not a part of that and want nothing to do with it.

                        cholantesh 13 hours ago

                        >I was responding to the "why is this post flagged?", and you changed scope to "justify all moderator decisions across the platform".

                        The actual question in the top-level comment you replied to is "why is this post, and other posts that discuss potential malfeasance from X/Grok regularly flagged?" It is implicit from follow-ups that appealing to the guidelines does not wash except as a thought-terminating cliche. In that sense I guess you are right, the topic can derail intellectual curiosity.

                        > I was getting the sense from multiple replies to my comments that some people were assuming I (and the moderators) wanted to censor the underlying concern about Grok as if I was a part of the American culture/political war.

                        The current iteration of techno-optimism characteristic of a lot of this site's userbase may have originated in SV, but it isn't the exclusive province of Californians. That's most likely what is being referred to.

                    UncleMeat 14 hours ago

                    This is sealioning.

                      FloorEgg 14 hours ago

                      No it's not...

                      The GP's accusations of gatekeeping and dishonesty led me to deduce he was making assumptions about political leaning rather than a more respectful interpretation being to protect intellectually curious dialog.

                      Then parent made a comment about how political categorization doesn't matter when I can share an ideology with a political official out side of my jurisdiction.

                      So I think I reasonably asked a clarifying question about whether I was being assumed an ideology for my stance of the interpretation of HN guidelines. I asked this because I couldn't see any other reason why parent would have brought up ideology or made that comment.

                      This is certainly not sealioning. I am not trolling, I am not harassing, I am not demanding evidence or attempting to disrupt dialog. I was only trying to clarify parents point and reason for introducing ideology into the dialog.

                        UncleMeat 14 hours ago

                        I interpret the conversation totally differently. This person was extremely clear with you.

                        This post is bullseye dead center in bounds for a HN post. The idea that it does not foster curiosity comes from a very particular place.

                          FloorEgg 14 hours ago

                          > This person

                          It's two different people.

                          If I zoom out and look at the whole thread, I can take a more generous view of your criticism and acknowledge that I could have been more direct rather than leading through questions. I still don't think that's sealioning, but I wasn't as clear and patient as I could have been or want to show up here. I am doing my best and trying to get better.

                          The issue I think is that the HN guidelines have two different sets of criteria for on-topic and off-topic. With a bit more time to reflect, it seems like the people who think this submission is on-topic are only looking at the on-topic criteria set and not really considering the off-topic criteria set.

                          They also aren't considering the spirit of the guidelines to foster intellectually curious dialog and avoid topics that derail it.

                          The comment two up from the one you replied to was not my best and does warrant some criticism, and I think that's the one that probably triggered you to make the sealioning accusation, even though I disagree with that particular classification.

                          > The idea that it does not foster curiosity comes from a very particular place

                          Spell it out for me please. What exactly are you accusing me of? This entire thread has been hostile to me but keeps beating around the bush. Are you saying that HN mods and I are protecting Grok in a biased way?

                          > This post is bullseye dead center in bounds for a HN post.

                          It definitely isn't, as it hits multiple criteria for off-topic as stated clearly in the HN guidelines.

                            UncleMeat 13 hours ago

                            > They also aren't considering the spirit of the guidelines to foster intellectually curious dialog and avoid topics that derail it.

                            Yes they are. We considered it and came to a different conclusion than you. We believe that this topic does foster intellectually curious dialog and your insistence that we don't is frustrating.

                            > Not if the topic consistently devolves away from intellectually curious dialog. There are clearly a lot of charged emotions and strong opinions across most of these comments.

                            IMO, this is the core problem. Charged emotions and strong opinions are not the opposite of intellectually curious dialog. An insistence that conversation play out like an abstract game separated from all feeling is anathema to useful communication, in my opinion. People are not better, smarter, or more interesting by being dispassionate. Especially when the topic is wide scale mass abuse and harassment. Feelings are not bad.

                              FloorEgg 13 hours ago

                              Okay, well my goal here was to communicate a benefit of the doubt most respectful interpretation of the moderators decision, and how it can be justified by the official HN guidelines.

                              While I could have been a lot more clear from the start, I think I've done my part. You can disagree with me and assume bad faith if you want. That's your choice.

                                soraminazuki 11 hours ago

                                It's revealing that you're repeatedly framing your decision as someone else's. Did you not see the flag button that you yourself pressed? And I'm assuming you did press it, because you so vehemently argue in favor of flagging this submission.

                                  FloorEgg 10 hours ago

                                  > It's revealing that you're repeatedly framing your decision as someone else's

                                  Yes it's revealing that I was confused about how HN works.

                                  I did flag it, and I don't deny that. I explained in another comment how I thought it was 70/30 off/on topic. However my confusion stems from the fact that I thought moderators made the final decision to flag a submission. I just read about it and now realize that users drive flagging which alerts moderators and then they choose whether to set it as dead. At least now that's how I think it works.

                                  So yes, a lot of my comments didn't make any sense. All of my interpretation of the guidelines I stand by, but my misunderstanding about the flagging mechanism totally skewed a bunch of my comments and I no longer agree with them. If I could, I would edit them.

                                    UncleMeat 3 hours ago

                                    [flagged] [dead] is also not set by mods. It is automatic. It is only when you see [dead] without [flagged] that the mods have done it.

                                UncleMeat 12 hours ago

                                The flags don't come from moderators. They come from users.

                                  FloorEgg 9 hours ago

                                  Thank you for pointing this out to me. I didn't understand how that worked and it very much changed my perspective on this whole thread.

            square_usual 18 hours ago

            You don't find it interesting that a major social media company enabled generating deepfake porn, and now faces global legal action for it?

            (I know for a fact you and people like you will say No, so posts about deepfakes and their legal consequences have been extremely relevant for HN; you just need one search to prove this: https://hn.algolia.com/?q=deepfakes)

              FloorEgg 17 hours ago

              > I know for a fact you and people like you will say No

              You "know for a fact" what about me exactly?

              And what do you mean by "people like me"?

              I think AI deep fakes were an interesting new phenomenon at one point but that time has past. I think this post falls under crime and outrage, and I would categorize it maybe 70/30 as off-topic / on-topic. It's not cut and dry for me, but it does skew more as off-topic than on-topic.

              The comment I was replying to was acting as if there was no explanation, yet the explanation is clearly laid out right in the top of the HN guidelines.

              There are so many places to go on the Internet if you want to be outraged or spread outrage about things that outrage you, HN is one of the few places meant to harbour intellectual curiosity, and outrage and intellectual curiosity unfortunately don't mix well, hence the guidelines.

        square_usual 18 hours ago

        How is this not on topic for HN?

          groby_b 18 hours ago

          Given the guideline is "anything that gratifies one's intellectual curiosity." one could certainly make the case it's not a topic that's fit.

          "Large company does horrible things, occasionally an investigation happens" is not satisfying curiosity on an intellectual level. What, specifically, are we learning?

          Yes, we can debate if it's relevant to the audience anyways, or if it's healthy for HN to have that guideline, but those are different criteria & topics.

            frm88 8 hours ago

            "Large company does horrible things, occasionally an investigation happens" is not satisfying curiosity on an intellectual level. What, specifically, are we learning?

            By your metric most of the topics concerning Google or Microsoft in the past months should not have staid on the front page, let alone most of the American politics topics. And yet, here we are: anything critical of any of Musk's companies gets flagged, American politics stay.

            @dang: how does this flagging thingy work? How many user flags are necessary until a topic appears as [flagged]? Thank you in advance for an explanation.

              3 hours ago
              [deleted]
            danudey 17 hours ago

            > What, specifically, are we learning?

            That some countries are willing to hold tech companies accountable (or claim their intent to hold tech companies accountable) for breaking the laws. In turn, this could lead to court cases deciding if companies can hide behind "It wasn't us, our AI did it!" or if they have to take responsibility for the thing they made.

      awnird 14 hours ago

      Garry Tan, the owner of this site, is friends with Musk (a pedophile with ties to Epstein), and a vocal supporter of trump (a pedophile with ties to Epstein).

      Pedophilia is one of his most consistent values, and that’s reflected in the values of this site and its moderation.

      pepperball 18 hours ago

      [flagged]

  • queenkjuul 14 hours ago

    Oh wow somebody decided news about an international tech giant is suddenly irrelevant on a site about tech news, crazy how that happens with one particular category of tech news...

  • kyleee 12 hours ago

    Where are the examples? How do we know this story is true?

  • soraminazuki 13 hours ago

    It's hardly the first time, but we have yet again confirmed that the people who say they flag these posts because it's not related to tech are outright lying.

  • huflungdung 18 hours ago

    [dead]

  • llmslave2 18 hours ago

    [flagged]