Speech Perception - Will Styler

# Speech Perception
### Sixth College Seminar - Will Styler

---

### My goal today is to convince you that speech perception is magic

---

## Today's Plan

- What is speech?

- What is speech perception?

- Some unexpected phenomena

- Why is it difficult?

---

# What is speech?

---

### The Speech Process

* Flapping bits of meat inside your head while blowing out air

* This creates vibrations in the air you're expelling

* The ear picks these up, and inteprets them as speech.

* This process is studied in the Linguistic subfield of **Phonetics**

---

### The Lungs

---

### Flapping bits of meat ("articulation")

---

### Let's do an experiment

---

> The North Wind and the Sun were disputing which was the stronger, when a traveler came along wrapped in a warm cloak.

---

### Speech is absolutely insane

- It's a series of fluid and overlapping gestures

- It's amazingly complex

- ... and it's nothing like we think it is

---

### ... but we can think about speech as a sequence of vowels and consonants

- Individual speech sounds

- A series of articulatory targets...

- Not the same ones as your writing system

- With hard-to-identify boundaries between them

- Adjacent sounds affect each other

- Which is broadcasted to the world acoustically

---

### So, what do these vibrations look like?

---

---

---

### Sound is compression and rarefaction in a medium

- Sound needs something to travel in (like air or water)

- This is why a clap or gun-shot are so loud

- But these compressions and rarefactions travel

---

### Different speech gestures create different sounds

- Each element of a speech articulation has acoustical consequences

- Combinations of different frequencies at different amplitudes over time

- Today, we'll sort of gloss over this complexity and think in theory

---

---

### How do humans pick them up?

---

---

---

### "Oh, he's talking about that thing up there!"

---

# What is Speech Perception?

---

### Well, what *is* speech perception?

Thoughts?

---

## Speech Perception

The process of turning acoustic speech signals back into linguistic content

---

### We go from this...

---

### To this...

---

## What do we want from a theory of speech perception?

- We want it to explain how we go from the sounds we hear to language

- We want it to handle the dynamic nature of sounds

- We want it to be flexible for the variability in speech

- We want it to explain our accuracy at this *super* hard task

---

... but the thing is

- ### Speech perception is really weird

---

# Stupid Speech Perception Tricks

---

## Speaker Variation

---

---

### Every single person you meet sounds different from every other person

- Yet *you have never given even the smallest of damns*

---

## Categorical Perception

---

### Vowels are very *gradient*

- There are many possible gestures to make a vowel

- There are many positions between the tongue position for 'date' and 'debt'

- So, we should feel a very smooth progression between them?

- Right?
    
---

### Date vs. Debt

---

# Date

---

# Debt

---

## ?

---

## ??

---

## ???

---

### Let's do an experiment!

---

## ????
<audio controls>
<source src="phonmedia/date_debt_11_steps.mp3" type="audio/mp3">
</audio>

---

### Categorical Perception

* We use our knowledge of the categories to make strong decisions about which sounds are which

- Even when it's completely gradient!

---

### Degraded Signals

---

### Let's listen to some sounds

---
### Let's listen to some sounds

### Now let's play all three at once!

---

### Let's listen to some sounds

### Now let's play all three at once!

### Does this help?

---

### Sine Wave Speech

- Recodes complex speech signals as the sum of three moving sine waves

- Sine Wave Speech demonstrates how good we are at recovering speech in the world

---

## Multi-modal perception

---

### The McGurk Effect (Part 1)

---

### The McGurk Effect (Part 2)

---

### They're the same video!

- <huge>😇</huge>

---

### Gick and Derrick 2009: "Aero-tactile integration in speech perception"

Nature. 2009 November 26; 462(7272): 502–504. doi:10.1038/nature08572.

---

> Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as aspirated (for example, causing participants to mishear ‘b’ as ‘p’). These results demonstrate that perceivers integrate event-relevant tactile information in auditory perception in much the same way as they do visual information.

(from Gick and Derrick 2009: "Aero-tactile integration in speech perception")

---

## Huh.  That was weird.

---

# The Big Questions

---

### We have more questions than answers in speech perception

- But these are the three big ones

---

### How are we perceiving speech?

- Are we listening directly to the acoustics?

- "General Auditory" theory

- Are we trying to figure out 'what they did' in terms of gestures using our own speech system?

- Motor Theory

- Are we building a mental model of the mechanism that must have produced that sound?

- 'Direct Realism' or Direct Perception

---

### What are we storing in our mind?

- Abstractions (e.g. /nɔɪz/)

- 'Averages' from many productions

- Individual productions of words

---

### Is speech special?

- Do we process speech in the same way that we process every other acoustic stimulus?

- Do we process speech using a special language-specific or tongue/motor localized processes?

---

### So, Speech perception?

- <img class="r-stretch" src="img/magic.jpg" alt="$txt">

---

"The Large Hadron Collider is the most complex endeavor humans have ever undertaken."

---

---

### **Speech** is the most complex endeavor humans have ever undertaken.

- ... and speech perception is just a small part of the complexity of human language

---

### In Summary...

---

# Speech perception is Magic

- ... and that makes you all wizards

---

---

<huge>Thank you!</huge>

Questions? <wstyler@ucsd.edu>