Professor Bot: An Exercise in Algorithmic Accountability

Submitter: Jentery Sayers, U of Victoria; Vee, Laquintano, and Schnitzler; eds.

——————————————————

The experiment:

I wrote a low-tech, tool-agnostic, small-stakes assignment that prompted students to attend to issues of power and governance in artificial intelligence (AI), with an emphasis on what students do not know and may thus want to learn about algorithmic decision-making. Students first considered a hypothetical scenario where AI is assessing university entrance essays. They then consulted publications on “algorithmic accountability” to articulate questions they would want to ask key decision-makers about the AI decision-making process. They concluded the exercise by reflecting on what they learned about algorithmic accountability, transparency, and social responsibility.

Results:

The most common student response to this assignment was a palpable sense of curiosity when they learned that Canada already wrote a directive on automated decision-making. The existence of this directive meant students did not need to start from scratch when addressing power and governance in AI. More interesting, students recommended a variety of social actions when they reflected on the process of an “algorithm audit.” Although I did not quantitatively track their responses, I found that many of them deemed governance to be a technical matter: that is, they rendered transparency in the social process tantamount to transparency in data and the inner workings of AI, where social responsibility implies being responsible for the recipe of AI but not necessarily its uses or effects. Students thus frequently found black-boxed AI to be unfair to them yet held post-secondary institutions or governments rather than private corporations accountable for the integration of AI with decision-making in education. Regardless of their position on accountability, students tended to be productively surprised when they learned how much they did not know about AI decision-making even beyond the technical particulars, and a common student suggestion was improving AI literacy and including within it more education about audits, transparency, regulation, and policy-making.

Relevant resources: https://wac.colostate.edu/repository/collections/textgened/ethical-considerations/professor-bot-an-exercise-in-algorithmic-accountability/

Contact:

• Website: https://jntry.work/
• Email: jentery[AT]uvic[DOT]ca