Exploring Speculative Landscapes with Multimodal AI: Visualizing the Plan of St. Gall

Submitter: Daniel Hutchinson, Belmont Abbey C

——————————————————

The experiment:

In an introductory history course students utilized multimodal AI to explore speculative historical spaces. This assignment was designed as part of a unit exploring the social history of monastic communities in medieval Europe. One of the key sources we studied is the Plan of St. Gall, a detailed architectural plan of a Benedictine monastery from the ninth century that was never actually constructed. This assignment was guided by the question: “What might have the actual St. Gall have looked like?” We answered this by creating an illustrated guide of the Plan of St. Gall using OpenAI’s GPT-4V model. This multimodal AI possesses capacities to ‘see’ uploaded images of the Plan of St. Gall. With this ability the model generated explanatory text based on specific regions of Plan based on accompanying historical sources. GPT-4V then used DALL-E 3 to create images of these regions to provide a more immersive exploration of the Plan of St. Gall. This guide was then developed into a web app for broader educational use. In completing this assignment students not only engaged with course content but also honed digital literacies on the effective use and limits of multimodal AI.

Results:

This assignment effectively engaged students with the course content and broader implications of AI in education. One student remarked, “I find the results to be very interesting and enlightening. I think this tool is valuable because it helps to picture the lives of the people we are studying.” Students appreciated the AI’s capacity for enriching historical exploration, but also recognized the importance of scrutinizing its outputs. As another student noted, “I like that AI can turn words into images, and it is helpful for understanding the ‘vibe’ of the places/spaces depicted. It isn’t always super accurate but it doesn’t necessarily need to be to give a general understanding of concepts.” For future iterations of this assignment, I’m interested in creating illustrated guides of other speculative spaces informed by historic and literary sources.

Relevant resources:

• Daniel Hutchinson, “Visualizing the St. Gall Plan”
• Christine Axen, “What the St Gall Plan tells us about Medieval Monasteries,” Medievalists.net
• Medieval Academy of America, “St. Gall Project”

Contact:

• Email: danielhutchinson[AT]bac[DOT]edu
• Twitter: @d__hutchinson
• Website: http://danielhutchinson.org/