Since its introduction to the public in November 2022, ChatGPT, an artificial intelligence system, has substantially grown in use, creating written stories, graphics, art and more with just a short prompt from the user. But when it comes to scientific, peer-reviewed research, could the tool be useful?
The researchers took three different topics—fractures and the nervous system, Alzheimer’s disease and bone health and COVID-19 and bone health—and prompted the subscription version of ChatGPT ($20/month) to create scientific articles about them. The researchers took 3 different approaches for the original draft of the articles—all human, all ChatGPT or a combination. The study is published in a compilation of 12 articles in a new, special edition of Current Osteoporosis Reports.
In the articles written only by ChatGPT, up to 70% of the references were wrong. But when using an AI-assisted approach with more human involvement, they saw more plagiarism, especially when giving the tool more references up front. Overall, the use of AI decreased time spent to write the article, but required more extensive fact checking.
Another concern is with the writing style used by ChatGPT. Even though the tool was prompted to use a higher level of scientific writing, the words and phrases were not necessarily written at the level someone would expect to see from a researcher.
