Uh, yeah, that's kind of how LLM's work. They're next token predictors. If you give it sequences of text that are worded in a particular way, the sequences that follow will follow that statistical distribution from the training data. This is a feature, not a bug.
The weird framing of this being a negative thing toward women is the personal bias of the women who published this and has no place being in this study. The measurement of what constitutes a response as "high quality" is also open to interpretation and varies depending on personal preference. You can't argue that a shift in the direction of the metrics mentioned in the report are objectively better or worse, they're just different.
This seems part of a broader pattern which isn't specific to gender. In my experience, LLMs match the sophistication of the answer to the user's inferred level of understanding. Linguistic habits like hedging and tag questions seem to overlap between "ways women talk" and "ways people who are uncertain about a topic talk."
> users cannot easily avoid them through strategic self-presentation
Prompting LLMs differently than you talk to humans doesn't really seem that hard. I already do this (e.g. ask basic questions in a separate chat so I'll look smart, and get better responses, in the main session.)
Disclaimer: only read the abstract, feel free to point out if I missed the point.
Its done on tiny and/or old models only. I find that hedging help on larger models with making the model more willing to double check things. Its a shame they didn't test models people actually use.
Would have been nice if they actually had a corpus of male vs female authored prompts, rather than simulating them from a general corpus.
For example, I tend to use language in prompts that they attribute to women:
> Women’s language is more likely to include hedges (e.g.,
maybe, I think), tag questions (e.g., isn’t it?), collective reference (e.g., we, our), and expressive
adjectives (e.g., lovely, wonderful).
But there are subtle ways that their Figure 1 example prompt goes way beyond this, and blatantly derails the entire thing:
> Let’s compose an email
together to arrange our mid-year
appraisal with our team
This is not about saying "our appraisal" or "our team", but about literally asking for a collaborative workflow, rather than for a draft.
The "male" prompt in that Figure 1 comparison was also weird ("your team"), but alas.
Uh, yeah, that's kind of how LLM's work. They're next token predictors. If you give it sequences of text that are worded in a particular way, the sequences that follow will follow that statistical distribution from the training data. This is a feature, not a bug.
The weird framing of this being a negative thing toward women is the personal bias of the women who published this and has no place being in this study. The measurement of what constitutes a response as "high quality" is also open to interpretation and varies depending on personal preference. You can't argue that a shift in the direction of the metrics mentioned in the report are objectively better or worse, they're just different.
This seems part of a broader pattern which isn't specific to gender. In my experience, LLMs match the sophistication of the answer to the user's inferred level of understanding. Linguistic habits like hedging and tag questions seem to overlap between "ways women talk" and "ways people who are uncertain about a topic talk."
> users cannot easily avoid them through strategic self-presentation
Prompting LLMs differently than you talk to humans doesn't really seem that hard. I already do this (e.g. ask basic questions in a separate chat so I'll look smart, and get better responses, in the main session.)
Disclaimer: only read the abstract, feel free to point out if I missed the point.
Its done on tiny and/or old models only. I find that hedging help on larger models with making the model more willing to double check things. Its a shame they didn't test models people actually use.
Would have been nice if they actually had a corpus of male vs female authored prompts, rather than simulating them from a general corpus.
For example, I tend to use language in prompts that they attribute to women:
> Women’s language is more likely to include hedges (e.g., maybe, I think), tag questions (e.g., isn’t it?), collective reference (e.g., we, our), and expressive adjectives (e.g., lovely, wonderful).
But there are subtle ways that their Figure 1 example prompt goes way beyond this, and blatantly derails the entire thing:
> Let’s compose an email together to arrange our mid-year appraisal with our team
This is not about saying "our appraisal" or "our team", but about literally asking for a collaborative workflow, rather than for a draft.
The "male" prompt in that Figure 1 comparison was also weird ("your team"), but alas.