MICHALE FEE: So today, we're going
to introduce a new topic, which is related
to the idea of fine-tuning curves,
and that is the notion of receptive fields.
So most of you have probably been,
at least those of you who've taken 9.01 or 9.00 maybe,
have been exposed to the idea of what a receptive field is.
The idea is basically that in sensory systems
neurons receive input from the sensory periphery,
and neurons generally have some kind of sensory stimulus
that causes them to spike.
And so one of the classic examples
of how to find receptive fields comes from the work
of Huble and Wiesel.
So I'll show you some movies made
from early experiments of Huble-Wiesel
where they are recording in the visual cortex of the cat.
So they place a fine metal electrode
into a primary visual cortex, and they present.
So then they anesthetize the cat so the cat can't move.
They open the eye, and the cat's now
looking at a screen that looks like this, where
they play a visual stimulus.
And they actually did this with essentially a slide projector
that they could put a card in front
of that had a little hole in it, for example,
that allowed a spot of light to project onto the screen.
And then they can move that spot of light
around while they record from neurons in visual cortex
and present different visual stimuli to the retina.
So here's what one of those movies looks like.
So you're hearing the actual potential
of a neuron visual cortex.
So you can see the neuron generates
lots of spikes when you turn a spot of light
on a particular part of the visual field.
So they will basically play around with spots of light
or bars of light and see where the neuron spikes a light,
and then they would draw on the screen--
I think they're going to draw in a moment here--
what they would call the receptive field of the neuron.
So you can see that this neuron responds with high firing rate
when you turn on a stimulus in that small region there.
Notice that the cell also responds when you turn off
a stimulus that is right in the area
surrounding that small receptor field.
So the neuron has parts of its receptive field that
respond with increased firing when you apply light,
and they also have parts of the receptive field that
respond with higher firing rate when you remove light.
That was actually a cell, I should have said,
that's in the thalamus that projects to visual cortex.
So that was a thalamic neuron.
Here's what a neuron in cortex might look like.
So they started recording in the thalamus.
They saw that those neurons responded to spots of light
in small parts of the visual field.
They were actually recording from neurons
in the visual cortex.
They got kind of-- they couldn't really
figure out what the neurons were doing,
and they pulled the slide out of the projector, which
made an edge of light moving across the visual field.
And the neuron they were recording from at that moment
responded robustly when they pulled the slide out.
And they realized, oh, maybe it's
an edge that the neuron is responding to.
And so then they started doing experiments with bars of light.
Here's an example.
So you can see the neuron responds when you turn
a light on in this area here.
But is responds when you turn light off in this area here.
And so you can see they're marking
a symbol for positive responses, positive responses to light
on here, and negative responses or increased firing
when you turn the light off.
So there's different parts of the receptive
field that have positive and negative components.
But you can see that the general picture here
is that the process of finding receptive fields
at this early stage was kind of random.
You just tried different things and hoped
to make the neurons spike.
And we're going to come back to this idea of finding
receptive fields by trying random things,
but in a more systematic way, at the end of the lecture today.
So here's what we're going to be talking about.
So you can see that Hubel and Wiesel were
able to describe that receptive field by finding
positive and negative parts and writing symbols
down on a screen.
We're going to take a more mathematical approach
and think about what that means in a quantitative model of how
neurons respond to stimuli.
And the basic model that we'll be talking about
is called an LN model, linear/nonlinear model.
And we're going to describe neural responses
as a linear filter that acts on the sensory stimulus followed
by a nonlinear function that just says neurons can only
fire at positive rates.
So we're going to have our neurons spike when that filter
output is positive, but not when the filter output is negative.
And we're going to describe spatial receptive fields
as a correlation of the receptive
field with the stimulus.
And we're also going to talk about the idea
of temporal receptive fields, which
will be a convolution of a temporal receptive
field with the stimulus.
So the firing rate will be a convolution
of a receptive field with the temporal structure
of the stimulus.
We're going to then turn to the--
combine these things into the concept
of a spatial temporal receptive field that simultaneously
describes the spatial sensitivity
and the temporal sensitivity of a neuron, as an STRF,
as it's called.
And we'll talk about the concept of separability.
And finally, we're going to talk about the idea
of using random noise to try to drive neurons,
to drive activity in neurons, and using what's
called a spike-triggered average to extract the stimulus that
makes a neuron spike.
And we're going to use that to compute--
we're going to see how to use that to compute
a spatial temporal receptive field in the visual system
or a spectral temporal receptive field in the auditory system.
So let's start with this.
What are spatial and temporal receptive fields?
So we just saw how you can think of a region of the visual space
that makes a neuron spike when you turn light on or makes
a neuron spike when you turn light off.
And at the simplest level, you can
think of that in the visual system
as just a part of the visual field
that a neuron will respond to.
So if you flash of light over here, the neuron might respond.
If you flash of light over here, it won't respond.
So there's this region of the visual field where
neurons respond, but it's more than just a region.
There's actually a pattern of features within that area that
will make a neuron spike, and other patterns
will keep the neuron from spiking.
And so we can think of a neuron as having some spatial filter
that has positive parts, and I'll
use green throughout my lecture today
for positive parts of a receptive
field, and negative parts.
And this is a classic organization
of receptive fields, let's say, in the retina
or in the thalamus, where you have an excitatory central part
of a receptive field and an inhibitory surround
of the receptive field.
So we can think of this as a filter that
acts on the sensory input.
And the better the stimulus overlaps with that filter,
the more the neuron will spike.
So let's formalize this a little bit into a model.
So let's say we have some visual stimulus that
is an intensity as a function of position x and y.
We have sum filter, G, that filters that stimulus.
So we put the stimulus into this filter.
This filter, in this case, just looks like, in this case,
an excitatory surround in an inhibitory center.
That filter has an output, L, which
is the response of the filter.
Then we have some nonlinearity.
So we take the response of the filter, L,
we add it to some spontaneous firing rate,
and we take the positive part of that sum
and call that our firing rate.
So that would be a typical output nonlinearity.
It's called "a threshold nonlinearity,"
where if the sum of the filter output
and the spontaneous firing rate is greater than 0,
then that corresponds to the firing rate of the neuron.
So in this case, you can see that as a function of L--
I should have labeled that axis L--
as a function of L, when L is 0, you
can see the neuron has some spontaneous firing
rate, R naught.
And if L is positive, the rate goes up.
If L is negative, the rate goes down until the rate hits 0.
And then once the neuron stops firing, it can't go negative.
So the neuron firing rate stays at 0.
And then once you have this firing rate of the neuron, what
is it that actually determines whether the neuron will spike?
So in most models like this, there
is a probabilistic spike generator
that is a function of the rate output of this nonlinear
output.
It's basically a random process that generates spikes at a rate
corresponding to this R.
And in the next lecture, we're going
to come back and talk a lot more about what spike trains look
like, how to characterize their randomness,
and what different kinds of random
processes you actually see in neurons.
A very common one is the Poisson process,
where there's an equal probability per unit
time of a neuron generating a spike,
and that probability is controlled by the firing rate.
We'll come back to that and discuss it more.
Any questions?
Yes, [INAUDIBLE]
AUDIENCE: Is something biologically [INAUDIBLE]
something like if the overlap [INAUDIBLE] it's just it's
more excitatory?
MICHALE FEE: Yeah.
So we're going to come back in, I think,
a couple lectures where we're going
to talk about exactly how you would build a filter like this
in a simple feed forward network.
So at the simplest level, you can just
imagine you have a sensory periphery that
has neurons in it that detect, let's say,
light at different positions.
Those neurons send axons that then impinge on,
let's say, the neuron that we're modeling right here.
And the pattern of those projections,
both excitatory and inhibitory projections from the periphery,
would give you this linear filter.
And then this nonlinearity would be a property of this neuron
that we're modeling.
Yes, Jasmine?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Great, exactly.
So it's going to turn out that we're
going to treat this as a linear filter.
The output of this filter will be calculated
for a spatial receptive field as the correlation of this filter
with the stimulus.
But in the time domain, when we calculate a temporal receptive
field, we're going to use a convolution.
And we'll get to that in a minute.
That's the very next thing.
Great question.
Anything else?
So that's called a LN model.
I should have put that on the slide--
linear/nonlinear model.
So let's describe that mathematically.
So let's say we have a two-dimensional receptive
field.
We're going to call that G of x and y.
So remember, we had intensity as a function of x and y.
There's our stimulus input.
And we're going to ask, how well does that stimulus overlap
with this receptive field?
And we're going to describe the receptive field as a function
on this space, x and y.
And our linear model is going to be
how well the stimulus matches or overlaps
with the receptive field.
And we do that just by multiplying the receptive
field times the stimulus and integrating over x and y.
x and y, just think of it as a position on the retina.
So let's look at this in one dimension.
So remember, this was a receptive
field that has a positive central region
and an inhibitory surround.
So if we just take a slice through that
and plot G as a function of x, you
can see that there is a positive central lobe and inhibitory
surround, inhibitory side load.
That's a very, very common receptive field early
in the visual system, in the retina
and in the lateral geniculate nucleus.
So in one dimension, we just take this receptive field, G,
multiply it by the stimulus pattern,
and integrate over position.
That is L. That's the output of the linear filter.
We're going to add that to a spontaneous firing rate,
and that gives us the firing rate of our neuron.
And you can see that that's like--
that this product, an integral over x,
is just like a correlation--
G of I times intensity of I summed over I.
So let's walk through what that looks like.
So here's G of x, the receptive field.
Let's say that's our intensity profile.
So we're going to have a bright spot of light surrounded
by a darker side lobe.
So the way to think about this is,
in visual neuroscience experiments, usually
the background is kind of gray.
And you'll have bright spots, like here, and dark spots,
like there.
And the rest will just be gray.
So that's how you get positive and negative intensities here,
because they're relative to some kind of gray background.
And so now we can just multiply those two together.
And you can see that when you multiply positive times
positive, you get positive.
And when you multiply negative times negative,
you get positive.
And when you integrate over position x,
you get a big number.
You get some positive number.
So that neuron, that stimulus, would
make this a neuron with this receptive
field likely to spike.
Let's consider this case.
Now, instead of a small spot of light centered
over the excitatory lobe of the receptive field,
you have a broad spot of light that
covers both the excitatory and inhibitory lobes
of the receptive field.
What's going to happen?
Yeah?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
You're going to get a positive times of positive here,
and a negative times a positive.
And so you're going to get negative contribution
from the side lobes, and those things
can exactly cancel out when you integrate over position.
And so you can get a very small response.
If you have--
I'm not going to go through this example,
but it's pretty obvious.
If you were to have light here, and then dark
here, and then light there, you would
have only these negative side lobes activated.
You would have no contribution from this excitatory lobe,
and the integral would actually be negative.
And so the firing rate of the neuron would go down.
Do you remember seeing those different points
in that first movie where you saw
the donut of light turning on?
The neuron kind of shuts off when you turn that light on.
Yes?
AUDIENCE: So I'm just curious what
we're signing up 0 [INAUDIBLE].
MICHALE FEE: Yeah.
So 0 is just this gray background.
It's some intermediate level of light intensity--
pretty straightforward.
So that's a spatial receptive field right there.
We refer to this correlation process, this linear filter
as linear, because you can see that if you
put in half the light intensity, let's say,
you get half the product.
And when you integrate, you get half the neural response.
If you take the stimulus and you cut it in half so that you only
apply light and dark to half the receptive field,
then you'll also get half the response of the neuron.
Because this will contribute to the integral and this
won't, and so you'll get a neural response
that's half as big.
So in this model, the response varies linearly
with this overlap of the receptive field
and the intensity.
So that's where the term linear comes from.
Any question about that?
Correlation is a linear operation.
So the next thing we're going to talk about
is temporal receptive field.
So we just talked about spatial receptive fields.
Neurons are also very sensitive to how things vary in time.
So we're going to take the same concept.
Instead of a stimulus that's a function of position
on the retina, let's say, we're going
to take a stimulus that's a function of time.
And we're going to operate on that temporal stimulus
with a filter that [INAUDIBLE] a temporal sensitivity.
We're going to get the output of that filter,
add it to a spontaneous firing rate,
and we're going to get a time-dependent firing rate.
So let me just show you what this looks like.
So let's say that you have a stimulus that's
fluctuating in time.
So imagine that you have a neuron that
has a spatial receptive field that's just a blob,
and you shine just a positive bump.
And you shine light on it, and the intensity of that light
varies.
And this is the intensity that you
apply to that spatial receptive field as a function of time.
And again, 0 is some kind of average gray level.
And so you can go dark, dark, or bright around that.
So now, neurons generally have receptive fields in time,
and this is what a typical receptive field might
look like, a temporal receptive field might look like,
for a neuron.
Neurons are often particularly driven
by stimuli that go dark briefly and then go
bright very suddenly, and that causes a neuron to spike.
So we can imagine that temporal receptive field,
and sliding it across the stimulus,
and measuring the overlap of that temporal receptive field
with the stimulus at each time.
Does that makes sense?
And so you can see that most of the time
that negative bump and positive bump
are just going to be overlapping with just
lots of random wiggles.
But let's say that the stimulus has
a negative part and then a positive part,
dark, then bright.
You can see that at this point that filter
will have a strong overlap with the stimulus.
Why?
Because the negative overlaps with the negative, and that
product is positive.
Positive overlaps with positive, that product is positive.
And so when you integrate over time you get peak.
Does that make sense?
This is, what I'm plotting here, is the product of these two
functions at each different time step
as I slide this temporal receptive
field over the stimulus.
Does that make sense?
Any questions about that?
I'm going to go through that.
We're going to work on that idea little bit more,
but if you have any questions, now's a good time.
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So you're asking, why
is it correlation in the spatial domain?
Because-- well, let me answer that question
after we define what this is mathematically.
So what is this mathematically?
AUDIENCE: A convolution.
MICHALE FEE: It's a convolution.
It's exactly what we talked--
it's a lot like what we talked about when we talked synapses.
In that case, we had some delta functions here
corresponding to spikes coming in,
and the synaptic response was like some decaying exponential.
And we slid that over the stimulus.
In this case, we have this fluctuating sensory input,
this light intensity, and we're sliding that linear response
of the neuron over and measuring the overlap
as a function of position, and that's a convolution.
Mathematically, what we're doing is
we're taking this linear kernel, this linear filter,
sliding it over the stimulus, using this variable t,
and we're integrating over this variable tau.
So we have a kernel, D, multiplied by the stimulus
at different times shifts.
We integrate over tau, and that's
the output of our temporal receptive field.
That's the linear output of our receptive field.
And we add that to spontaneous firing right,
and that gives us a time-dependent firing
rate of the neuron.
Yes?
AUDIENCE: So is tau how much we [INAUDIBLE]??
MICHALE FEE: Great question.
t is the location of this kernel as we're sliding it along.
Tau is the variable that we're integrating over
after we multiply them.
Does that make sense?
So we're going to pick a t, place the kernel down
at that time, multiply this.
And remember, this is 0 everywhere outside of here.
And so we're going to multiply the stimulus by this kernel.
It's going to be-- that product is going to be 0 everywhere
except right in here.
You're going to get a positive bump when
you multiply these two, a positive bump when
you multiply those two.
And the integral over tau gives us this positive peak here.
If we picked a slightly different t
so that this thing was lined up with this positive peak here,
then you'd see that you'd have positive times negative.
That gives you a negative.
When you integrate over tau, that gives you
this negative peak here.
Does that makes sense?
So let's just go back to the math.
So you can see that we're integrating over tau,
but we're sliding the relative position of D and S
with this variable t.
Yes?
AUDIENCE: Is that the [INAUDIBLE]??
MICHALE FEE: Yes.
S is the stimulus.
AUDIENCE: Oh.
And D is the kernel?
MICHALE FEE: D is the linear kernel.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So nature chooses the shape of the kernel for us.
So that is the receptive field of neurons.
Now, I just made this up to demonstrate
what this process looks like.
But in real life, this is the property of a neuron,
and we're going to figure out how
to extract this property from neurons using
a technique called spike-triggered average, which
we'll get to later.
But for now, what I'm trying to convey
is, if we knew this temporal receptive field of a neuron,
then we could predict the firing rate of the neuron
to a time varying stimulus.
That was a very important question.
Does everyone understand that?
Because it's one of those cases where once you see it
it's pretty obvious, but sometimes I
don't explain it well enough.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yes.
So I've already flipped it, and sometimes you'll see.
So this is all going this way-- positive tau.
I've flipped it for you already.
Sometimes you'll see it plotted the other way with tau
going positive to the right, but I've
plotted it this way already.
Any questions?
Oh, and so that was actually the very next question.
You might normally-- you might sometimes
see temporal receptive fields plotted
this way with positive tau going to the right.
And kind of meant--
I always just flip it back over.
Because in this view, you see that what the neuron responds
to is dark followed by light, and then right there
is when you have a peak spiking probability.
Peak firing rate happens right here.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So we're going to get to that.
But typically, neurons in the retina--
I'll show you an example in the retina.
A typical time scale here might be tens to 100 milliseconds,
so pretty fast.
So that's called the temporal kernel
or the temporal receptive field.
And again, it's linear in the sense
that if you, for example, had a stimulus intensity that just
had this positive bump without the negative bump, then
the response would be lower just by the ratio of areas.
So if you got rid of this big negative bump here,
then the response would be, I don't know, a third as big.
It would be linear in the area.
Let's push on.
So now, let's extend this.
So we've been talking about spatial receptive fields
and temporal receptive fields.
But in reality, you can combine those things together
into a single concept, called a "spatial temporal receptive
field," and that's usually referred to as an STRF.
If you're working in the auditory system, STRF,
it's the same acronym, but it just
means spectral temporal receptive field,
because it's sensitive to the spectral content
of the sounds, not the spatial structure
of the visual stimulus.
So in general, when you have a visual stimulus,
it actually depends on x- and y-coordinates
in the retina and time.
So just I of x and y, which would be like a still image
presented to you.
I of x, y, and t is--
any movie can be written like that.
Your favorite movie is just some function of I of x, y, and t.
And so we're going to now present to our retina,
and we're going to simplify this by considering
just one spatial dimension.
So we're going to take your favorite movie
and just collapse it into intensity
as a function of position.
It's probably not nearly as interesting,
but it's much easier to analyze.
So we're going to write the firing rate
as a function of time as a spontaneous firing
rate plus a filter, D, which is a spatial temporal receptive
field acting on that intensity.
And you can see that we're doing stuff in here that
looks like a convolution integrating over tau,
and we're also doing stuff that looks like a correlation when
we integrate over x.
So there's the convolution integrating over tau.
What I've done is I've pulled out the D tau,
because we can consider--
I've just written this as two separate integrals.
So we have an integral over tau that looks like a convolution.
And we have an integral over x that looks like a correlation.
So what is separability mean?
So separability is just a particularly--
if a receptive field is separable,
it means that you can write down a spatial receptive field
and a temporal receptive field separately.
And that looks like this.
So I imagine that if you have a spatial temporal receptive
field, D, that's a function of position and time.
But you can see that you can just
write it as a product of the spatial part
and the temporal part.
So here, you have a temporal receptive field
that looks like this, a positive lobe here and a negative lobe
there, a spatial receptive field that looks like this, just
a positive lobe.
And if you multiply this function
of x by this function of t, you can
see that you get a function of x and t
that looks like this, where at any position
the function of time just looks like this-- scaled.
And at any time, the spatial receptive field
just looks like this.
Does that make sense?
Other receptive fields are not separable.
You can see that you can't write this receptive
field as a product of a temporal receptive field
and a spatial receptive field.
Does that make sense?
Is that clear why that is?
So basically, you can see that if you take a slice here
at a particular position, you can
see that the temporal pattern here
looks very different than the temporal pattern here.
And so you can't write this simply
as a product of a spatial and a temporal receptive field--
separable, inseparable.
So let's take a look at what happens when you have
a separable receptive field.
Things kind of become very simple.
We can now write our spatial temporal receptive field
as a spatial receptive field, which
is a function of position, times a temporal receptive field
that's a function of time.
And when you put that into this integral, what you find
is that you can pull that spatial part of the receptive
field out of the temporal integral.
So basically, the way you think about this
is that you find the correlation of the spatial receptive field
with the stimulus, and that gives you
a time-dependent stimulus, a stimulus that's
just a function of time.
Then you can convolve the temporal receptive field
with that time-dependent stimulus.
So you can really just treat it as two separate processes,
which can be kind of convenient just
for thinking about how a neuron will
respond to different stimuli.
So let's just think about, develop some intuition about,
how neurons with a particular receptive field
will respond to a particular stimulus.
So here's what I've done.
I've taken a spatial temporal receptive field here.
This is a function of position and time,
and we're going to figure out how that neuron responds
to this stimulus.
So this stimulus is also a function of space and time.
It's one-dimensional in space.
So what does this look like?
This looks like a bar of light that extends from position 2,
let's say, 2 millimeters to 4 millimeters on our screen.
And it turns on at time point 1, stays on,
and turns off at time point 6.
Let's say 1 second to 6 seconds.
Does that make sense?
So just imagine we have a 1D screen, just a bar,
and we turn on light that's a bar between 2 and 4.
So we turn on a bar of light.
We turn it on at time 1, and we turn it off at time 6.
It's just a very simple case.
We flash of light on at a particular position,
and then we turn it off.
So let's see how this neuron responds.
So what we're going to do is we're going to slide--
remember, in the 1D case where we had the temporal receptive
field, we just slid it across the stimulus.
So we're going to do the same thing here.
We're going to take that spatial temporal receptive field,
and we're going to slide it across the stimulus.
And we're going to integrate, we're
going to take the product, and we're going to integrate.
And the integral plus the spontaneous rate
is going to be the firing rate of our neuron.
So what is the integral right there?
The product is--
AUDIENCE: 0.
MICHALE FEE: --0.
where? integrate, it's 0.
So we're going to add a spontaneous firing rate, which
will be right there.
So that will be our firing rate.
Now, let's slide the stimulus a little further.
That means that this time we're asking,
what is the firing rate of that neuron?
So what is it going to look like?
AUDIENCE: Go up a bit.
MICHALE FEE: It's going to go up a little bit,
because we have a positive part of the receptive field.
Green is positive in our pictures here.
It's going to overlap with this bar of light,
because that neuron is sensitive to light between, let's say,
1 and 4, positions 1 and 4 on the screen.
And so the light is falling within the positive part
of that receptive field, and so the neuron's
going to increase its firing rate.
So now what's going to happen?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: It's going to cancel.
You're going to get a positive contribution to the firing rate
here--
whoops-- and a negative contribution here.
And those two are going to add up.
You're going to multiply that times that.
That gives you a plus.
That times that gives you-- sorry.
That times the light that's shining on it is negative.
Add those up, and it's going to cancel.
And the firing rate's going to go back to baseline.
Now, the light in this receptive field,
we're continuing to slide it in time over our stimulus.
What happens here?
AUDIENCE: Same thing.
MICHALE FEE: Same, good.
How about here?
It's going to go--
AUDIENCE: Down.
MICHALE FEE: --down.
It's going to dip down, that's right.
And then?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
By 0, you mean the spontaneous firing.
Yeah, exactly.
Cool.
Yes?
AUDIENCE: [INAUDIBLE] the rate of response because the slope
of the line [INAUDIBLE]?
MICHALE FEE: So you should think about this thing sliding
over the stimulus in real time.
So if these are units of seconds,
then this thing is sliding across the stimulus
at 1 second per second sliding across.
And so that is firing rate as a function
of time in those units.
Does that make sense?
AUDIENCE: Yeah.
But why [INAUDIBLE]?
MICHALE FEE: Oh, OK.
Like why doesn't this go up to here?
So what's the answer to that?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
So how would I make that steeper?
How would I make that go up to here?
AUDIENCE: [INAUDIBLE] the light.
MICHALE FEE: What's that?
AUDIENCE: You'd turn the light up.
MICHALE FEE: Yeah, you'd turn the light up, that's right.
This is the receptive field of a neuron,
so we generally can't control that.
So if we wanted to make this neuron respond more,
we'd turn the light up to a higher intensity.
Any other questions?
So neurons often have more complex receptive fields.
So here's an example.
What is this going to do as we slide this across the stimulus?
What's that?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
It's not going to--
the neuron isn't going to respond at all.
Because as soon as it overlaps, it has a positive contribution.
The light activates these lobes of the receptive field,
but inhibits these lobes of the receptive field.
And the net result is that when you integrate over the product,
you're going to get 0.
Does anyone have any idea what kind of stimulus
might make this neuron respond?
This is a very special kind of receptive field.
Yes, [INAUDIBLE]
AUDIENCE: The light goes from [INAUDIBLE]
MICHALE FEE: Yeah.
What is that called?
AUDIENCE: I'm not sure.
MICHALE FEE: It's called a stimulus that moves.
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Moves-- a moving stimulus.
Good.
So that's a receptive field that response to a moving stimulus.
So let's take a look at that.
So here we go.
Anybody want to take a guess at what this stimulus will
do to this neuron?
Can you visualize sliding it across?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: And then what?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
You can see that this--
so let's describe what this [AUDIO OUT]..
So we've turned a bar of light on here between 0 and 2,
and then we slide it up over the course of a few seconds.
So we've turned a spot of light on, and then we move it up--
off.
So it's a spot of light that turns on, moves, and then
disappears.
So let's walk through it.
So there's a little bit of overlap
there, so the neuron's firing rate
is going to start going up.
But then as it goes further, this light
is now activating those inhibitory lobes,
which is going to have a negative contribution.
So when you take the product, you're
going to get lots of negatives there, very little contribution
from the positive lobes, and so the firing rate's
going to go down.
And what happens is it goes down,
and once the firing rate hits 0, it can't go any more negative.
So the firing rate is just going to sit at 0
until this stimulus moves out of the temporal receptor
of this neuron.
And then what?
AUDIENCE: Back up.
MICHALE FEE: It's going to go back up
to the spontaneous rate.
So what kind of stimulus will activate this neuron?
A stimulus that moves from top down.
So let's take a look at that.
So here's our stimulus.
You see that it's going to just hit that inhibitory lobe,
go down a little bit.
And then the excitatory lobes of the receptive field
are going to overlap with the stimulus.
You're going to get a big positive peak,
and then the stimulus will move out of the receptive field,
and the firing rate will go back down to baseline.
Any questions about that?
So that's very common, in both the visual system
and in the auditory system, to have
neurons that are responsive to moving stimuli.
What does moving stimulus mean in the auditory system?
AUDIENCE: Changing pitch.
MICHALE FEE: Right, changing pitch.
So [WHISTLE], like that.
That activated a gazillion neurons
in your brain that are sensitive to upward-going pitches
that you have structure like this.
Isn't that crazy?
[WHISTLE]
I can control all the neurons in your brain--
[WHISTLE]
[LAUGHS]
--at least the ones that respond to whistles.
So now that we've seen mathematically
how to think about what a receptive field is
and how it interacts with a sensory stimulus,
how do you actually discover what the receptive
field of a neuron is?
That turns out to actually be a very challenging problem.
So in very early parts of the visual
and the auditory system, like in the retina and the LGN,
and as far as, let's say, V1 in visual cortex,
it's been possible to find receptive fields of neurons
by basically just randomly flashing bars and dots of light
and just hoping to get lucky and find what the response is.
It turns out that that's generally a very--
it can be a very time-consuming process.
And so people have worked out ways
of discovering the receptive fields of neurons
in a much more systematic way.
And that's what we're going to talk about next-- the idea
of a spike-triggered average.
So here's the idea.
We're going to take a stimulus, and we're going to basically--
we're basically just going to make noise,
just a very noisy stimulus.
So we're going to take, let's say, an intensity, a light,
a spot of light, and we're going to fluctuate the intensity
of that light very rapidly.
And we're going to do that basically with a computer.
We just take a computer, make a random number generator,
hook that up to, let's say, a light source
that we can control the brightness of with a voltage.
And then have the computer generate--
put out that random number sequence,
control the light level, and then play that to our,
let's say, our visual neuron.
And that neuron is going to spike.
And now, what we can do is take the times of those spikes
and go back figure out basically what
made the neuron fire post-hoc.
So if we do that here, you can basically take the spike times.
Now, you know that whatever made the neuron spike happened
before the spike.
It didn't happen after the spike.
So you can basically ignore whatever
happened after the spike and just consider the stimulus that
came in prior to the spike.
So we're just going to take a little block of the stimulus
prior to the spike, and we're going
to do that for every spike that the neuron generates,
and we're going to pile those up and take an average--
spike-triggered.
That's it.
And that is going to be--
what's really cool is that you can show that
under some conditions that spike-triggered average is
actually just the receptive field of the neuron.
It's the linear receptive field of the neuron.
So let's think-- and you can write that down as follows.
We're going to add a stimulus.
We're going to write down the times at which all these spikes
occur, t sub i, or the times in the stimulus
at which the spikes occur.
We're going to take the stimulus at those times minus some tau,
and we're going to average them over all the spikes, all the n
spikes, that we've measured.
And that K of tau is going to be the spike-triggered average,
and in many cases, it's actually the linear kernel.
Now, let's think for a moment about what the conditions are.
What kind of stimulus do you have
to use in order to get the spike-triggered average
to actually be the linear kernel of the neuron, that receptive
field of the neuron?
Any guesses?
Let me give you a hint.
What happens if I take a stimulus that
varies very slowly?
So instead of having these wiggles,
it just goes like this.
It has very slow, random wiggles.
Will that be a good stimulus for extracting the receptive field?
Why is that?
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
So I think what you're saying is that that stimulus is very
slow, and it doesn't actually have
the fast fluctuations in it that makes the neuron spike.
If the stimulus varies very slowly, then it--
see, this neuron likes to have this very fast wiggle,
this negative followed by a positive.
But if the stimulus you put in just varies slowly,
then that stimulus doesn't actually
have the kind of signal that's needed to activate this neuron.
Yes?
AUDIENCE: [INAUDIBLE] the stimulus [INAUDIBLE]
smaller than tau?
MICHALE FEE: Well, tau is just the variable that describes
the temporal receptive field.
But I think what you're saying is that the stimulus varies
more slowly than the temporal structure
in the receptive field, in the temporal receptive field.
That's right.
Tau is just this variable that we define the receptive field
on.
Yes, [INAUDIBLE]
AUDIENCE: So when we add up an average,
are we actually adding up from everything
before the spike, everything before the spike?
MICHALE FEE: So great question.
So how far back do you think you would need to average?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Maybe.
I mean, in principle, you could have spikes happening
very fast, and you could have signal
that affects the response of a neuron from even
before the last spike.
But in general, what do you think the answer
to that question is?
Brainstorm some more ideas.
So let's say that you were recording in the retina,
and you knew that neurons tend to respond to visual stimuli
only for--
that temporal receptive fields in the retina
never extend back more than 100 milliseconds.
Then how would you choose that window?
You would just choose this window to be 100 milliseconds,
and that would be it.
If you're recording in a brain area
that you really have no idea, then you
have to actually try different things.
So you can try a window that goes back 100 milliseconds.
And if when you do the spike-triggered average,
it hasn't gone to 0 yet, then you need to take more window.
So you can figure it out.
You can create a short window, and it only
takes-- like you change one number in your Matlab code,
and hit Run again, and do it again.
It's pretty simple.
Yes?
AUDIENCE: So when you've got like [INAUDIBLE]..
MICHALE FEE: Yes.
AUDIENCE: Wouldn't that depend [INAUDIBLE]
what kind of filter [INAUDIBLE]
MICHALE FEE: Yeah.
So you're saying that the stimulus that you choose
actually depends on the kinds of filters that the neurons are,
right?
Actually, the right answer, it depends.
AUDIENCE: And so we have--
[INAUDIBLE]
MICHALE FEE: Yeah.
So generally, the statement is that the stimulus you use
has to have fluctuations in it that are faster
than the fluctuations in the kernel
that you're trying to measure.
And so most people choose what's called a "white noise
stimulus."
And white noise stimulus comes from the idea
that when you take the spectrum--
and we're going to get into spectra next week.
But when you look at the spectrum of the stimulus,
and you take the Fourier transform of it
and look at how much power there is as a function of frequency,
the spectrum is flat.
And just like in colors, white light has a flat spectrum.
And so the term evolved to a noise that has a flat spectrum
white noise.
And that's what people generally refer to when they
do spike-triggered averages.
They use noise that has a flat spectrum.
And so you'll often refer to people
saying that they've used a white noise stimulus to extract
a spike-triggered average.
Now, of course, you can't ever make a noise
that truly has a flat spectrum.
You have to-- you can only make things
fluctuate as fast as your experimental setup can
make them fluctuate.
So things eventually fall off.
Fortunately, neurons tend to have receptive fields that
only have fluctuations that are on the scale of 10
milliseconds, or maybe a millisecond in extreme cases.
Maybe in the auditory system you might get a little fluctuation
for a millisecond, and that's generally pretty
slow for an experimental setup.
So you choose a white noise stimulus,
where white noise means it's got fluctuations
faster than the fastest fluctuations
in the temporal receptive field, and that for real neurons tends
to be, even in early sensory areas,
tends to be on the scale of millisecond fluctuations.
And in higher brain areas, they would
have even slower fluctuations.
Now, let me just say one word about spike-triggered average.
It works really well in lower sensor areas.
But once you get up out of primary sensory areas,
this method doesn't work any more.
Neurons don't actually have simple receptive fields.
And this method starts not working so well outside
of primary sensory areas.
But before I get too much into the limitations of this,
let me just show you some examples where it
works really beautifully well.
So this is--
I'll show you some slides that I got from Marcus Meister, who
studies--
he's at Caltech.
He used to be at Harvard, and he studies the retina.
And so he developed this setup for extracting receptive fields
of retinal neurons.
And here's the idea.
So here's a piece of retina, thats a representation
of the circuitry in the cells within a piece of retina.
You extract the retina, and you place it
on a dish, a special dish, that has electrodes embedded,
metal electrodes embedded, in the glass,
sort of on the surface of the glass.
You take the retina out.
You press it down onto the glass.
So now the electrodes are sensing the spiking activity
of these neurons down here in the retinal ganglion cell
layer.
These are the photoreceptors up here.
And then what he does is he has a computer monitor that's
generating random patterns of visual stimuli,
and you project that using a lens down
onto the photoreceptors of the retina.
And those neurons now make lots of spikes,
and you can extract those spikes using the methods that you
saw in the video from Tuesday.
So here's what those signals look like.
This just shows the signal on four different electrodes
that happen to be right near each other,
like four adjacent electrodes on this electrode array.
And you can see that you get spike wave forms on all
these different electrodes.
You can see that you get--
that you see what looks like lots of cells on these four
electrodes.
One really interesting thing to note
is that these electrodes are actually placed close enough
together that multiple electrodes detect
the spike signal from a single cell.
So you can see right here, here's a spike.
It exactly lines up with the spike on this other electrode.
So here is a spike on one electrode that lines up
with a spike on another electrode.
You can see there's a little blip there
and a little blip there.
All of those spikes are actually from a single cell
whose electrical activity is picked up
on four adjacent electrodes.
David?
AUDIENCE: Is this the raw data?
MICHALE FEE: Yeah.
This is the raw voltage data coming out of those electrodes.
And you can see here's at a different cell right here.
You can see that this cell has a peak of voltage fluctuation
on electrode two.
You see a little blip there, and a little blip there,
and nothing there.
And here is yet another cell that
has a big peak on electrode three, essentially nothing,
maybe a small bump there.
So you can actually extract many different cells
looking at the patterns of activity that
appear on nearby electrodes.
And it turns out that this multi-electrode array system
is actually very powerful for extracting
many different cells, the spiking
activity of many different cells,
out of a piece of tissue.
So what you can do is you put this through what's called
a "spike-sorting algorithm," which uses these different
spike wave forms on these different electrodes to pull
out a spike train.
And the spike train is now going to be a delta function
for each different neuron that you've
identified in this data set.
So even though different neurons appear
on these different electrodes, you're
eventually going to extract this now so that you
have a spike train for one neuron,
a spike train for another neuron,
a spike train for a third neuron, and so on.
And then you can plot the firing rate
of those different neurons.
This is actually a histogram, a peristimulus histogram,
of the activity of a bunch of different neurons
to a movie being played to this piece of retina in the dish.
And that's just literally a movie from a forest with trees
swaying around in the breeze.
So you have all these different neurons.
And you can see that each neuron responds to a different feature
of that movie.
And that's because each neuron has a receptive field that's
in a slightly different location,
has a slightly different spatial and temporal receptive field,
that it allows it to pick out different features
of the visual stimulus.
And there are about a million of these neurons that project
on the back of the retina.
Actually, I should be careful.
It's actually the-- it's the back of the retina.
Because the light goes through the ganglion cells
through the photo receptors, which are actually--
sorry.
Photoreceptors are actually on the backside of the retina.
Ganglion cells are on the front, and light
goes through the ganglion cells to get to the photoreceptors.
And there are million of those retinal ganglion
cells that then project up through the optic nerve
to the thalamus.
So how do we figure out what each of those neurons
is actually responding to in this movie?
So what we can do is--
you could imagine doing a spike-triggered average
of these neurons to the movie that's
playing the trees swaying in the breeze.
But why would you not want to do that?
Why would that be a bad idea?
What is it that we just decided is
the best kind of stimulus to use to extract receptive fields?
This is a highly structured stimulus
that's got particular patterns in the stimulus
and both in space and in time.
So it's really not an optimal stimulus
for finding the receptive fields in neurons.
What we want to do is to make a very noisy stimulus
that we can play, and that's what they did.
So then they make this, what they
call in the visual system, a "random flicker stimulus."
So it's basically a movie where you randomly choose that
the pixel values, both in R, G, and B-- red, blue, and green--
for each stimulus at each time step.
And here's what that looks like.
So now, you play that movie to the retina,
and you record the spike trains.
So there's the neurons spiking.
And now what you do is you--
because this is now a two-dimensional stimulus, what
you do is you have to collect the samples of the movie
at a bunch of time steps prior to the neurons spiking.
Does that make sense?
Now, you do that for each spike that occurs.
And now, you average those all together
to produce a little movie of what
happened on average before each spike of the neuron.
And here's what that looks like.
So this is for two different neurons.
So what is that?
So this is time across the top.
So it starts at minus half a second.
So what did that look like?
What was it that made that neurons spike?
What was it that happened right before that neuron spiked?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah, a dark spot.
So that neuron was excited by a spot of light,
sorry, by a stimulus that looked like a darkness right
in that location right there.
So that neuron is essentially being inhibited by light
at that location.
And when the light at that location goes away,
boom, the neuron is released from inhibition and spikes.
Here's another cell.
So that neuron responded to a spot of light
right there in that location.
And that's because that neuron gets excitatory input
from bipolar cells that are located
in the retina at that location.
And those bipolar cells respond to input
from the photoreceptors at that location.
That's called an on cell.
That's called an off cell.
So there are many different kinds of neurons in the retina.
There's something like--
I forget the latest count--
40 or 50 different types of retinal ganglion cells that
have very specific responses to visual stimuli.
So now let's break that down into a spatial and temporal
receptive field.
Most-- I probably shouldn't say most--
but many retinal ganglion cells are separable in the sense
that they have a spatial receptive field
and a temporal receptive field that are just
a product of each other.
The STRF is a product of a spatial and temporal component.
So here you can see as a function of time
before the spike.
So this is the stimulus at the time of the spike.
This neuron responds with a spike
to a spot of light that happened about 150 milliseconds earlier.
And here's what that stimulus looked like as a function
of space on the retina.
So that's the spatial receptive field.
Sorry, that's the spatial temporal receptive field--
a spatial stimulus as a function of time.
You can write that as a product of a spatial receptive field
and a temporal receptive field.
So here's what the spatial receptive field
looks like, and here's the temporal receptive field.
You can see that this neuron, just like the example that we
talked about earlier, this neuron
likes to respond when there's a darkening in the central area,
followed by a bright spot.
You can see that little bit of darkening right here.
So the response when this goes dark and then bright.
So that's the visual system.
Let's take a look at auditory - you use this same method
for finding receptive fields in the auditory system.
So we're going to talk briefly.
We're going to come back to spectral analysis
and spectral processing of signals in a couple
lectures, but let me just introduce
some of the basic ideas.
So we're going to talk about the idea
of a spectral representation of a sound.
So this is a microphone signal of--
let me see if you can guess what it is--
of a creature.
There are parts of this stimulus that have high frequency.
So this is a microphone signal.
It fluctuates due to fluctuations in air pressure
when you hear something.
Parts of that signal have high-frequency fluctuations.
Parts of that signal have low-frequency fluctuations.
You can compute a Fourier transform--
which we'll talk about more later--
as a function of time stimulus and see what
the spectral components are.
So this is a spectrogram of the sound that I just-- right now.
But you can see frequency as a function of time,
and the intensity, or in this case the darkness on the plot,
shows you how much energy there is
at a particular frequency at a particular time,
so frequency as a function of time.
And now, neurons respond to stimuli like this.
It's a canary song.
And neurons respond to different sounds.
And so you can discover what sounds activate neurons
by doing the same trick.
So I'll show you.
This is from a paper from Michael Merzenich's lab.
This was worked on by Christophe deCharms
who was a post-doc in the Merzenich lab.
And basically, what you can do is--
OK.
So this is for calculating a visual receptive field.
For calculating an auditory receptive field, what
you can do is you can basically play noisy stimuli
in auditory space.
So what you can do is present random patterns of tones.
So this is frequency, and this is time.
And so what you can do is you can
make a little chords of tone, tone, tone that last,
let's say, 20 milliseconds.
And then you make a different random combination of tones,
and then a different random combination of tones.
And this sounds like a very scrambled, noisy stimulus.
And you play this to the animal while you're recording
a neuron in auditory cortex.
The neuron spikes.
And then what you can do is just do exactly the same trick.
You can look at the stimulus that
occurred before each spike, pile up those columns.
There's a little spectrum temporal pattern
of stimuli that the bird--
in this case, a monkey heard right
before that neuron spiked.
And you can do the same thing.
You can take that little snapshot of that sound
and average them together.
And here are the kinds of things you see.
So that's a spectro-temporal receptive field.
You can see I plotted it.
It's plotted in a way that this is the stimulus that
occurs with the spike.
So this is like the D plotted already flipped.
And you can see that-- how would you describe
what this neuron responds to?
How would you describe that?
So this is frequently.
Sorry, this is frequency in kilohertz.
And that's time in milliseconds.
So what do you think this neuron responds to?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: What's that?
AUDIENCE: It responds [INAUDIBLE]..
MICHALE FEE: It responds maximally, actually,
to a very short tone at 4 kilohertz.
You see how it kind of has some inhibition there?
See how it's kind of darker right there?
So this neuron actually will respond better
to a tone that only lasts 20 milliseconds than it will
to a tone that lasts a long time.
So this is a neuron that responds to a short tone pulse.
What happens if we play a stimulus
to this neuron that's broad?
That instead of just being a tone, [WHISTLE],,
is broad, like [STATIC]?
The noise that's at 4 kilohertz here
will tend to excite the neuron, but the noise
that's over here at 5 kilohertz or 3 kilohertz
will tend to inhibit the neuron.
So the best response, the best stimulus
to make this neuron respond, is a pure tone
at 4 kilohertz that lasts about 20-ish milliseconds.
How about this?
Let's take a look at this neuron right here.
What about that neuron?
What does that neuron want to respond to?
What does it like to hear?
I'm anthropomorphizing shamelessly.
You're not supposed to do that.
What kind of stimulus drives this neuron?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Good, a downward sweep, tone sweep,
that goes from about 4 kilohertz to 3 kilohertz.
In how long?
AUDIENCE: 100 [INAUDIBLE].
MICHALE FEE: In about 100 milliseconds, that's right.
Maybe 50 would do, [WHISTLE],, like that.
How about this?
It's kind of messy, right?
So you can see that neurons have receptive fields that
can be very complex in space, or in this case,
in frequency and time.
They are very selective to particular patterns
in the stimulus.
So we talked about a mathematical version
of receptive fields, which are essentially
describing patterns of sensory inputs
that make a neuron spike.
And we've talked about a very specific model, called
a linear/nonlinear model, that describes
how neurons respond to stimuli or become selective to stimuli.
We've talked about a spatial receptive field, described
the response of a neuron as a correlation
between the spatial receptive field and the stimulus.
Temporal receptive fields, where we've used convolution
to predict the response of a neuron to a temporal--
to a stimulus.
We've talked about the idea of a spatio-
or spectro-temporal receptive field,
and we've talked about how to use a spike-triggered average
to extract the spectro-temporal or spatio-temporal receptive
field of a neuron using white noise or random stimuli.