AI in Research: Progress and Challenges
This week, researchers at the University of Florida found that while AI can be a valuable assistant, it falls short of replacing human scientists in many critical areas.
The researchers tested how well popular generative AI models including OpenAI’s ChatGPT, Microsoft’s Copilot, and Google’s Gemini could handle various stages of the research process. They put these AI systems through six stages of academic research – ideation, literature review, research design, documenting results, extending the research, and final manuscript production – while limiting any human intervention. What they discovered was a mixed bag of capabilities and limitations.
Still, Japanese company Sakana announced this month that a paper written by its “AI Scientist” passed the peer review process at a top machine learning conference workshop, possibly the first time a fully AI-generated paper has passed the peer review process.
The company said: ‘We believe that the next generation of AI scientists will usher in a new era of science. The fact that AI can generate entire papers that pass peer review is a sure sign of progress to come.’
Comparing AI science with human science is not the ultimate goal, says Sakana. ‘What is most important is that discoveries made by human and AI science will contribute to human prosperity, such as leading to the treatment of diseases.’
However, concerns remain regarding the proliferation of AI-generated papers. Karin Verspoor, Dean of the School of Computing Technologies at RMIT University, noted the risks of AI flooding scientific literature, thus potentially making future AI systems less effective at innovation.
A review by Miryam Naddaf in Nature last week highlights the growing use of AI in the peer review process. Carl Bergstrom, an evolutionary biologist at the University of Washington, emphasized that if reviewers rely too much on AI, it risks leading to ‘shallow analysis.’ He said, ‘Writing is thinking.’
Overall, as AI continues to evolve, its integration into research processes presents both opportunities and challenges for the scientific community.
- 0 Comments
- Academic Collaboration
- Peer Review
- Scientific Research