"The average success rate for generating a patch that fully resolved the vulnerability (without materially changing application behavior) was just 26.0 percent,' wrote Director of Security Research Keith Hoodlet, adding that even patches that did fix the flaw also mucked up the application’s behavior 20 percent of the time."
"'Conversely, LLM-generated patches did not resolve the vulnerability, added a new vulnerability, or both, an average 53.9% of the time,' Hoodlet said."
"The average success rate for generating a patch that fully resolved the vulnerability (without materially changing application behavior) was just 26.0 percent,' wrote Director of Security Research Keith Hoodlet, adding that even patches that did fix the flaw also mucked up the application’s behavior 20 percent of the time."
"'Conversely, LLM-generated patches did not resolve the vulnerability, added a new vulnerability, or both, an average 53.9% of the time,' Hoodlet said."
These are some interesting points. I wonder: why?