IMPORTANT NOTE: The advent of ChatGPT and other forms of Artificial Intelligence appears to have sparked an existential debate within the realm of higher education. What will this tool mean for students, faculty, administrators, and the very future of education? Please read this page from UNF’s Office of Faculty Excellence to learn more.
As with any tool of this nature, decisions about academic integrity should not be based on a single data point, but rather an indicator that further review may be needed. As we learn more about AI-Detection, we will continue to update this knowledge base and blog.
Turnitin – the anti-plagiarism LTI installed in UNF’s instance of Canvas – recently added an AI-Detection feature described by the vendor as having the ability to “evaluate student writing for the presence of AI-generated text.”
UNF has not turned this feature on because, while the creators of various AI-Detection tools assert that they can accurately identify AI-generated content with 98% to 99% accuracy, third-party evaluations reveal higher rates of false positives associated with these tools. CIRT’s own internal testing also revealed high rates of false negatives (instances in which tools fail to detect the use of artificial intelligence altogether).
Here's What We Know
We want to use this blog as an opportunity to share some of the news concerning AI and AI-detection, as well as statements made by Turnitin and OpenAI (the creators of ChatGPT). We hope you will find the information below helpful as you consider using AI in your classroom.
Turnitin’s New AI-Detection Only Works with ChatGPT
Currently, Turnitin’s AI-Detection only works with ChatGPT versions 3 and 3.5 (it may not work with other generative text tools like Google Bard, Bing Chat, AI-Writer, or Ryter). OpenAI already provides paid access to GPT-4, and Turnitin claims that future updates to their detection software will address this version as well.
Turnitin’s Confidence in AI-Detection
On April 4, Turnitin released their AI-detection feature. While Turnitin claimed that their tool was more reliable than others on the market, some AI-professionals raised concerns about the tool’s accuracy and usefulness.
By June 2023, Turnitin publicly admitted that their AI-detection tool had a higher false positive rate than they had previously claimed. To their credit – unlike many other tools that claim high accuracy in detection – Turnitin released a blog in which they stated: “We cannot mitigate the risk of false positives completely given the nature of AI writing and analysis…”
OpenAI’s Confidence in AI-Detection
On July 25th of this year, OpenAI (the creators of ChatGPT), admitted in a blog post that they themselves were unable to reliably detect the use of their own software. After decommissioning their own internal detection tool, OpenAI said they are still in the process of developing “mechanisms that enable users to understand if audio or visual content is AI-generated,” though they gave no indication of when such a tool would be ready for release. A recent Insider article, which highlights a guide released by OpenAI on how to use ChatGPT in the classroom, further elaborates on why AI-detection tools are unreliable.
Our Current Stance
Given the above, UNF has decided that current AI-detection software isn’t reliable enough to use without a significant risk of false positives or false negatives, which can lead to problems for both students and faculty. We believe that the data doesn’t support using any AI-Detection tool at this time and therefore, we cannot endorse or support such tools.
Instead of relying on AI-Detection, CIRT is working closely with several units on campus to consider a response to the advent of AI and AI-detections tools in the classroom and to develop guidance for UNF instructors. These units include ITS, the Dean of Students, the Office of Faculty Excellence, the Writing Program & Center, and CIRT’s Instructional Design team.
In the meantime, CIRT is happy to provide several resources to help navigate this technology. First, you may find it useful to review the resources available from the University of Missouri, which has been a forerunner on this subject. We also recommend reaching out to your department’s Instructional Designer Liaison to discuss new assessment strategies that account for student/AI collaboration.
Finally, if you believe a student has committed Academic Misconduct by violating a class or University policy, please contact the Dean of Students.
