Solving the Clever Hans Phenomenon: The Power of Infinity
Brent Sullivan
Founder
Solving the Clever Hans Phenomenon: The Power of Infinity
TimeValuePrep is experimenting with ways to improve learning.
Clever Hans
Clever Hans was a horse famous in the early 20th century for his ability to answer questions correctly. However, it was later discovered that Hans needed to answer the questions independently. Instead, he could read the subtle cues in the body language of his trainer, Wilhelm von Osten. Hans would watch von Osten's face for any indication of whether the answer he was giving was correct or incorrect. When Hans saw the slightest sign of approval, he would stop tapping his hoof.
The Clever Hans phenomenon is a reminder that even animals can be fooled by unconscious cues from humans. In the case of Hans, he was able to learn to associate specific body language cues with correct answers. As a result, he could give the impression that he was answering questions correctly, even though he was not actually thinking about the answers.
See a concise write-up from the NIH here.
How Clever Hans Manifests in Standardized Exam Prep
The Clever Hans phenomenon can also manifest in standardized exam prep. For example, students may memorize the answer letter or the answer sentence structure of practice questions rather than learning the material. As a result, they may get a good score on the practice questions, but they will not be able to perform as well on the exam.
In the science of learning, this is called poor "transfer." See this article for some background.
Solving the Clever Hans Effect with Generative AI
Generative AI can be used to solve the Clever Hans effect by creating infinitely varied tasks for students to complete. This will help students learn the material rather than simply memorizing the answer letter or sentence structure.
Infinitely Varied Explanations
Generative AI can be used to create infinitely varied explanations for concepts.
Here is an explanation of universal life insurance told in a Shakespeare sonnet (but thankfully in plain English)
In life's uncertain play, what role to choose? One where your loved ones bear no heavy loss. Universal life, a path that you can use, A flexible shield against financial cross.
Premiums adjust as life ebbs and flows, Cash value grows with interest at its side. A death benefit to ease the deepest throes, In this insurance plan, you can confide.
But be aware, for market risk does loom, On cash value's growth, it leaves its mark. Yet, in the shadow of the financial tomb, This plan provides a light within the dark.
Through life's rough storm, let not your heart be vexed, With Universal Life, you're well-protected.
Infinitely Varied Flashcards
Generative AI can also be used to create infinitely varied flashcards. This will help students to memorize the material to maximize transfer.
For example, if one is learning about universal life insurance, the key things to remember are...
Flexibility: Universal life insurance allows adjustable premium payments.
Cash Value: It has a cash value component that can accumulate over time.
Death Benefit: Provides a death benefit to the beneficiaries.
Market Risk: The cash value component is subject to market risk and can fluctuate.
Here are five identical but differently-phrased obverses for a single universal life insurance flash card.
Flashcard 1: Universal life insurance is characterized by flexible premiums and death benefits, a cash value component that grows tax-deferred, and exposure to investment risk based on the performance of the cash value investments.
Flashcard 2: The key features of universal life insurance include adjustable premium payments and death benefit amounts, a potential for cash value accumulation on a tax-deferred basis, and a level of investment risk tied to the cash value's performance.
Flashcard 3: Universal life insurance stands out due to its adaptability in premiums and death benefits, tax-deferred cash value growth potential, and investment risk based on cash value investment returns.
Flashcard 4: Highlighting flexibility, universal life insurance allows adjustments to premiums and death benefits, offers a tax-deferred avenue for cash value growth, and carries investment risk associated with the cash value component.
Flashcard 5: The defining attributes of universal life insurance encompass flexible premium and death benefit terms, tax-deferred cash value appreciation potential, and investment risk related to the cash value investments' performance.
Infinitely Varied Test Questions
Generative AI can also be used to create infinitely varied test questions. This will help to ensure that students cannot memorize the answer letter or the sentence structure.
Open-ended questions
We can ask students open-ended questions and have an LLM grade the response. This is very similar to the "Little Feynman's," approach suggested by Scott Young.
Conclusion
The Clever Hans phenomenon is a reminder that subtle cues can undermine learning. We are experimenting with ways to make learning more active using the power of LLMs. We hope this will improve transfer and test performance. All of our content is expert-reviewed, but we see a future where expert-reviewed content powers fine-tuned LLMs to fuel infinite variation.