AI Tools Documentation
Submitter: Theresa (Terri) Senft, Macquarie U
——————————————————
The experiment:
This Template can be used to document use of AI tools during assessment preparation. It was trialed during a Social Media Glossary assessment, which tasked students to teach one theoretical concept to high schoolers using social media examples. For our class, the template was ungraded but mandatory for students choosing to use AI. In class, students used a bot I developed to evaluate their concept definitions for ‘high school friendliness.’ Using Chat PDF, they located quotes connected from class readings to connect to specific concepts, evaluating these for quality (e.g. making arguments and/or voicing strong sentiments.) Using ChatGPT, they prompted hypothetical scenarios where a particular class concept might shift due to changes (technological, financial, political, etc.) on the horizon.
Page One of the Template considers AI tool use in time blocks called ‘sessions.’ For each, students were asked to give prompts they used, evaluate information given by the tool for usefulness, justify what they did (use all, use part, discard, etc.) and explain how accuracy was checked. Page Two explains why students are being asked for this and offers sample language to use.
Results:
While our university establishes an official stance on AI for assessments, instructors can develop unit-specific policies. Because this unit is called “Social Media,” I tried to expose students to debates about AI, train them to evaluate material produced by AI, and emphasize the importance of fact-checking and decision-making transparency while using AI tools. Although no student was obligated to use AI during class assessments, I did have a rule that nobody was allowed to treat other humans as AI ‘idea bots’ (e.g., saying in group discussions or office hours, “I don’t have any ideas—what should I do?” without putting in work alone.) To get them comfortable filling it out, we spent a great deal of time going over the Documentation Template. Pedagogically, I think it accomplished its biggest aim, which as to underscore why documentation matters in AI tool use. This is especially true in the creative and communication industries, where the question is not
if the tools get used (if they are perceived as money-saving, they will) but how they get used, and whose livelihood is impacted when they are used without oversight. Far too often, the person fired for AI-generated errors is the person with the least time on the job, or the least power there, or both. In other words: recent graduates. During class, I spent some time talking about recent examples of AI use gone wrong in advertising copywriting, public relations, and even some NGO studies. So often, these tools are perceived as somehow all-knowing, or else mysterious and impossible to understand, when neither of these is the truth. In each horror story, I tried to point out how best practices requested on our Documentation Template might have saved a career.
Relevant resources:
Leave a Reply