Abstract: The Big Picture

Why do some sentences make our brains ache?

This presentation explores Dependency Locality Theory (DLT), which argues that processing difficulty increases when related words are far apart.

We'll see how this principle connects human cognition to the behavior of Large Language Models (LLMs), suggesting efficiency is a universal rule of thinking.

The Core Question

Why do some sentences tire the brain?

Have you ever read a long sentence and realized you forgot how it began? That's your brain's short-term memory protesting.

The Answer: Dependency Locality Theory (DLT)

DLT suggests a simple answer:

The farther related words are in a sentence, the harder it becomes to process.

Our minds work like short-term memory buffers. When related words are far apart, the brain must hold incomplete pieces longer, increasing effort.

Let's Feel the Strain: Example 1 (Easy)

This sentence feels relatively easy to process:

Let's Feel the Strain: Example 2 (Hard)

This sentence is correct, but feels "heavier":

Why? The dependency is non-local.

Why is it "Heavier"?

In the "hard" sentence:

"The reporter who the senator attacked..."

• Your brain has to hold onto the word "who" (an incomplete subject-object dependency)...
• ...past the intervening words "the senator"...
• ...until it finally reaches the verb "attacked" to resolve it.

This "holding" action is what increases the mental effort.

The "Locality Revolution": Measuring Effort

Linguists proposed two main types of mental effort that increase with distance:

Two Types of Effort: 1. Integration Cost

Integration Cost:

How hard is it to connect a new word (like a verb) to its related subject or object from earlier in the sentence?

Longer distance = Higher integration cost.
(It's harder to "plug in" the new word.)

Two Types of Effort: 2. Memory Cost

Memory Cost:

How many incomplete dependencies (or "open expectations") must your brain hold onto at one time?

More incomplete parts = Higher memory cost.
(Your brain's "RAM" is getting full.)

Example: The "Memory Cost" Nightmare

This is why "tongue-twisters" or complex legal text is so confusing:

"The administrator who the intern who the nurse supervised had bothered lost the reports."

By the time you reach "had bothered", your brain is juggling multiple open connections:

• (who... bothered?) → administrator
• (who... supervised?) → intern

No wonder it feels confusing!

A Global Habit: Smarter, Not Harder

This preference for "local" connections isn't just an English quirk. It's a global human tendency.

• Across Languages (Hindi, Japanese, Turkish): Speakers unconsciously organize sentences to keep related words close. (e.g., placing shorter phrases before the verb in Verb-Final languages).
• Good Enough: Our brains aim for "good enough" efficiency, not perfection. We naturally simplify our speech.
• Example: We prefer "She explained the problem patiently" over "She patiently explained the problem" to keep the verb and its object together.

A Global Habit: The Hindi Example

DLT also explains *how* speakers structure conversations.

In Hindi conversations, speakers make it easier for the listener by following a "Discourse Context" principle:

Enter the Machines: Do LLMs Think Like Us?

Large Language Models (like GPT) don't have brains or biological memory limits...

...yet they often show human-like behavior.

• Natural Preference: Even though they *can* connect words over long distances, they find short dependencies more "natural."
• Predicting "Surprise": LLMs assign higher "surprise" (lower probability) to the same long, complex sentences that slow humans down.

How? They Absorbed Our Habits

LLMs reflect our cognitive biases because they were trained on billions of human-written sentences.

They didn't "learn" DLT as a rule; they statistically absorbed our natural tendency to follow DLT principles for efficient communication.

So, while machines don't "struggle" with memory like us, their language generation mirrors our struggle.

The Big Lesson: A Universal Principle

A pattern emerges across both humans and AI:

Efficiency and locality are universal principles of thinking.

When we code, write, or teach, clarity comes from reducing unnecessary distance between related ideas.

Conclusion: A Shared Principle

DLT bridges cognitive science and artificial intelligence by showing that both thrive on locality.

"Simplicity isn't a limitation - it's intelligence in its most elegant form."

Future Work & Discussion

This field of study opens several avenues for further exploration:

• Refining DLT Metrics: How can we better quantify processing costs across diverse language structures?
• LLM Architectures & DLT: Can we design new AI architectures that are more (or less) sensitive to DLT principles?
• Broader Applications: How can DLT improve educational tools, writing aids, or even cross-modal communication (like visual stories)?
• Cognitive Universals: What other hidden human cognitive biases (like "thinking locally") have LLMs learned from our data?