Python is OK, documents are dangerous... MS study reveals AI limitations

An experiment evaluating how well 19 large-scale language models (LLMs) performed complex multi-step tasks showed that these models were highly error-prone and unreliable in many cases. These results were included in a pre-released paper ‘LLMs Corrupt Your Documents When You Delegate’ co-authored by Microsoft researchers Philip Laban, Tobias Schnabel, and Jennifer Neville. The researchers simulated the actual work flow of knowledge workers through a benchmark called ‘DELEGATE-52’, and the paper is currently undergoing peer review.