Tell Me Without Telling Me

Submitter: Mark Marino, U of Southern California, and Jeremy Douglass, U of California, Santa Barbara

——————————————————

The experiment:

LLMs follow models, including prompts. That means a person entering a prompt is also modeling the way they want the LLM to produce the language. In this exercise, developed in consultation with Jeremy Douglass, students enter a prompt in the style of the output they want produced. Demonstrating with some obvious examples (pirate talk or verbose language) can help point students in the direction of writing styles and away from types of non-standard English associated with cultural groups. However, if students experiment with those forms (as opposed to stereotyping them) they can test out the tendency of LLMs to push toward the homogenization of language.

Results:

This exercise goes a long way to help students understand how the LLM functions and moves them away from the twin tendencies to personify LLMs and to expect themT merely to choose a suitable form of writing. Anecdotal reports of LLMs producing better results when the prompt is “pollite,” is much more likely the LLM producing the kind of prose that that the prompter is modeling for it. Whether that output registers as “polite” or not, more likely reflects the cultural norms of the person doing the prompting.

Contact:

• @markcmarino
• https://markcmarino.com
• markcmarino[AT]gmail[DOT]com