Motor Representation in Joint Action

Press the right key for the next slide (or swipe left)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (no equivalent if you don't have a keyboard)

Press m or double tap to see a menu of slides

\title {Motor Representation in Joint Action: Talk Notes }

\maketitle

\def \ititle {Motor Representation in Joint Action}

\begin{center}

{\Large

\textbf{\ititle}

}

\iemail %

\end{center}

I want to start with a challenge ...

challenge

Explain the emergence of sophisticated human activities including referential communication and mindreading.

The challenge is to explain the emergence, in evolution or development, of sophisticated forms of human activity including referential communication and mindreading.

A number of researchers have suggested that meeting this challenge requires us to invoke some kind of social interaction ...

According to what Moll and Tomasello call the Vygotskian Intelligence Hypothesis,

‘participation in cooperative ... interactions … leads children to construct uniquely powerful forms of cognitive representation.’

(Moll & Tomasello 2007)

\citep{Moll:2007gu}

‘perception, action, and cognition are grounded in social interaction’

(Knoblich & Sebanz 2006)

\citep[p.\ 103]{Knoblich:2006bn}

‘human cognitive abilities … [are] built upon social interaction’

(Sinigaglia and Sparaci 2008)

\citep{sinigaglia:2008_roots}

I'm going to assume that they are right.

If we take these ideas seriously, the first question we need to ask is, What kinds of social interaction matters for the emergence of sophisticated human activities?

What kinds of social interaction? Joint action!

There seems to be some consensus on the idea that joint action is particularly important.

But what is joint action?

A paradigm case of joint action would be two sisters cycling to school together.

By contrast, two strangers cycling side-by-side are performing parallel but merely individual actions.

Or, to take another paradigm case,

when members of a flash mob in the Central Cafe respond to a pre-arranged cue by noisily opening their newspapers, they exercise collective agency.

But when others happen to noisily open their newspapers in response to the same cue, they do not \citep{Searle:1990em}.

challenge

Explain the emergence of sophisticated human activities including referential communication and mindreading.

So the challenge was to explain the emergence of sophisticated human activities including referential communication and mindreading.

conjecture

Joint action plays a role in explaining how sophisticated human activities emerge.

The conjecture I want to consider, borrowed from a variety of researchers, is that joint action plays a role in explaining how sophisticated human activities emerge.

In developing this conjecture, one potentially useful task is to provide a framework for thinking about joint action ...

Can we say more carefully what joint action is?

A basic question about joint action is, In virtue of what are joint actions <span class="collectively">collectively</span> directed to outcomes?

In virtue of what are joint actions collectively directed to outcomes?

Let me explain ...

In virtue of what are joint actions collectively directed to outcomes?

Some joint actions are purposive in the sense that

In virtue of what are joint actions collectively directed to outcomes?

among all their actual and possible consequences,

In virtue of what are joint actions collectively directed to outcomes?

there are outcomes to which they are directed

In virtue of what are joint actions collectively directed to outcomes?

and it is the joint action as a whole that is directed to this outcome

In virtue of what are joint actions collectively directed to outcomes?

so it is not just a matter of each agent’s individual actions being directed to this outcome.

In virtue of what are joint actions collectively directed to outcomes?

Rather, it is the joint action as a whole that is directed to the outcome.

In such cases we shall say that the joint action is \emph{collectively} directed to this outcome.%

\footnote{\label{fn:collective}

In using the term `collective' here we are borrowing from literature on plural quantification.

On a widely accepted view, the predication in `The goal of their actions was to lift this block' could be interpreted either distributively and collectively. On the distributive reading, the truth of the sentence is entailed by the truth of `For each of their actions, the goal of that action was to lift this block'. On the collective reading this entailment does not necessarily hold.

See further the overview in \citet{Linnebo:2005ig}.

}

In virtue of what are joint actions collectively directed to outcomes?

So the question is, In virtue of what are joint actions <span class="collectively">collectively</span> directed to outcomes?

To illustrate, suppose that Ayesha puts slugs in Zach’s shoes and Beatrice puts worms in his hat.

Ayesha might say, truthfully, ‘The goal of our action was not to harm these innocent creatures but to freak Zach out.’ \

Suppose this statement is true not because, or not only because, the goals of Ayesha’s and Beatrice’s individual actions was to freak Zach out.

(Perhaps they each know that Zach is so robust that finding either slugs or worms in his clothing would barely perturb him.)

What then could make Ayesha’s statement true?

In virtue of what are joint actions collectively directed to outcomes?

One quite standard way to answer this question is by appeal to a notion of shared (or 'collective') intention.

In virtue of what are joint actions collectively directed to outcomes?

Suppose Ayesha and Beatrice have a shared intention that they freak Zach out.

Then, on many accounts of shared intention,%

the shared intention involves each of them intending that they freak Zach out, or each of them being in some other state which picks out this outcome.

The shared intention also provides for the coordination of their actions

(so that, for example, they don’t both attempt to fill Zach’s shoes, leaving his hat empty).

And coordination of this type would normally facilitate occurrences of the type of outcome intended.

In this way we can explain what it is for a joint action to be collectively directed to an outcome by appeal to a shared intention.

All we need now is to understand what shared intention is.

The first thing to note is that, on almost any account, shared intentions are neither shared nor intentions.

Thank you philosophers for that useful piece of terminology.

Now we are fortunate in that there is a philosophical account of shared intention

... and less fortunate in that there are many slightly different, mutually incompatible accounts.

For today I am simply going to focus on Michael Bratman's, which is the best developed and most influential account of shared intention.

shared intention

Bratman says that shared intentional agency consists at bottom in the interconnected planning agency of the participants.

So Bratman's view hinges on the notion of interconnected planning.

To say that our plans are interconnected is to say that facts about your plans feature in mind, and conversely.

So if we are tasked with moving a table, it is not just facts about the table's size and weight that feature in my plans.

No, my plans must also concern facts about your plans.

But making plans about plans requires sophisticated insights into other minds, and is likely to be cognitively and conceptually demanding.

We can get a sense of how demanding it is by considering the conditions Bratman offers as sufficient for there to be shared intention.

The conditions require not just that we intend the joint action, but that we intend it because of each other's intentions, where this is common knowledge.

So we need not just intentions about intentions

But also you need to know things about my knowledge of your intentions concerning my intentions.

This indicates that, in general, having shared intentions requires mindreading at close to (or perhaps just beyond) the limits of most adult humans' abilities.

And this is a problem for us.

challenge

Explain the emergence of sophisticated human activities including referential communication and mindreading.

conjecture

Joint action plays a role in explaining how sophisticated human activities emerge.

This is a problem because our conjecture was that joint action plays a role in explaining how sophisticated human activities emerge.

objection

Joint action presupposes mindreading at the limits of human abilities.

But if joint action presupposes mindreading at close to the limits of human abilities,

and if mindreading abilities are a paradigm case of humans' cognitive sophistication,

then we must reject the conjecture.

For in appealing to joint action we would be presupposing what was supposed to be explained.

In what follows I want to defend the conjecture by identifying a way around the objection.

In virtue of what are joint actions collectively directed to outcomes?

We got into trouble by appealing to shared intention in trying to explain joint action.

To find a way around the objection, we need some way of explaining joint action without invoking the notion of shared intention.

motor representation

Fortunately there has been much work recently on the role of motor representations in interpersonal coordination.

I want to see whether reflection on discoveries about how motor representation play a role in coordinating agents' actions might also provide us with an alternative way of thinking about joint action, one that doesn't require shared intention.

Let’s step back and consider an individual action.

An agent moves a mug from one place to another, passing in from her left hand to her right hand half way [*demonstrate].

It’s a familiar idea that motor representations of outcomes resemble intentions in that they can trigger processes which are like planning in some respects.

These processes are like planning in that they involve starting with representations of relatively distal outcomes and gradually filling in details, resulting in a structure of motor representations that can be hierarchically arranged by the means-end relation \citep{bekkering:2000_imitation,grafton:2007_evidence}.

Processes triggered by motor representations of outcomes are also planning-like in that they involve selecting means for actions to be performed now in ways that anticipate future actions \citep{jeannerod_motor_2006,zhang:2007_planning,rosenbaum:2012_cognition}.

Now in this action of moving a mug, there is a need, even for the single agent, to coordinate the exchange between her two hands.

(If her action is fluid,

she may proactively adjust her left hand in advance of the mug’s being lifted by her right hand \citep[compare][]{diedrichsen:2003_anticipatory,hugon:1982_anticipatory, lum:1992_feedforward}.)

How could such tight coordination be achieved?

Part of the answer involves the fact that motor representations and processes concerning the actions involving each hand are not entirely independent of each other.

Rather there is a plan-like structure of motor representation for the whole action and motor representations concerning actions involving each hand are components of this larger plan-like structure.

It is in part because they are components of a larger plan-like structure that the movements of one hand constrain and are constrained by the movements of the other hand.

But how is any of this relevant to the case of joint action?

Outcomes are represented motorically not only in performing an action but sometimes also in observing another agent perform that action.

in observing others act

motor planning* for others’ actions occurs

This can happen not only when observing a single agent acting alone but even when observing several agents performing a joint action \citep{manera:2013_time}.

Manera et al (2013)

--- even when several others act jointly ---

These motor representations trigger planning-like processes in the observer much like those that would occur if the observer were actually acting,

and thereby sometimes enable you to anticipate the targets of others’ actions, so that it is almost as if you were covertly anticipating their actions by planning how someone in their situation would proceed \citep{ambrosini:2011_grasping,Costantini:2012fk}.

and enables us to anticipate their actions.

Costantini et al (2013)

Consider one agent who is engaged in joint action with another:

they are moving a mug from one place to another, passing it between their hands half-way through.

Motor representations concerning outcomes to which the other’s actions are directed,

and the associated planning-like processes,

can occur in these agents \citep{kourtis:2010_favoritism,kourtis:2012_predictive,meyer:2011_joint}.

Now these agents engaged in joint action are performing complementary actions.

In general, performing one action while observing (or even imagining) a complementary action produces interference,

and at least some of this interference is probably caused by motor representations related to the observed action \citep{kilner:2003_interference,ramsey:2010_incongruent_}.

Given this, we might guess that

when motor representations concerning the other’s actions occur in joint actions involving complementary actions,

these representations would interfere with the agents’ performance and so impair coordination.

In joint action

Kourtis et al (2010; 2012)

it can also occur

However, it turns out that the opposite is true, at least in some cases.

Motor representations concerning another’s actions can facilitate interpersonal coordination in joint action.

(This is true for a variety of joint actions including passing an object, ballistic actions and playing a piano duet. (\citealp[p.\ 9]{kourtis:2012_predictive}; \citealp{loehr:2011_temporal}; \citealp{novembre:2013_motor}; \citealp{vesper:2012_jumping}; \citealp{vesper:2013_our}).

But how might motor representations concerning another’s actions facilitate performance in joint action?

Vesper et al (2012);

and informs planning* for our own actions.

Novembre et al (2013)

At this point it may be natural to suppose that in joint action

there are two (or more) separate sets of motor representations and processes:

one for your own actions, another for the other’s (or others’)

Like this [image of two separate plans].

This is indeed how things are in some competitive actions.

In competitive actions there are sometimes planning-like motor processes concerning an opponent’s action as well as concerning your own \citep{sartori:2011_simulation}.

In competitive action

we sometimes plan* the others’ action and then our own.

Sartori et al (2011)

But there is evidence that in some joint actions, a single outcome to which the agents’ actions are collectively directed is represented motorically by each agent \citep{loehr:2013_monitoring,Menoret:2013fk,tsai:2011_groop_effect}.

But in joint action, sometimes

Tsai et al (2011);

there is one goal that we each represent

Loehr et al (2013); Ménoret (submitted); Vesper et al (in prep.)

There is also evidence that when agents are engaged in joint action,

they sometimes take into account future actions to be performed by others when choosing how to act now, and do so in much the way they would if they were performing the whole action alone \citep{meyer:2013_higher-order}.

Taken together, this evidence suggests that when an agent is involved in joint action, there is sometimes a single plan-like structure of motor representations concerning both her own and the others’ actions.

\textbf{In joint action, it is sometimes almost as if we each engage in motor planning for all of our actions.}

and we each make a single plan* for both of our actions

Meyer et al 2013

This idea---that there may sometimes be a single plan-like structure of motor representations concerning both your own and others’ actions---can easily seem incoherent. This is because it involves, or at least flirts with, the idea that in joint action, each agent plans others’ actions as well as her own.

But how is this possible without irrationality or ignorance?

How could one agent rationally plan another's actions knowing that she can't control them?%

\footnote{

This objection is due to Wolfgang Prinz (personal communication).

}

agent-neutral

Here it is helpful to appeal to the possibility of agent-neutrality.

By saying that something is \emph{agent-neutral} we mean that it does not involve identifying any particular agent or agents.

Agent-neutral planning is a familiar feature of everyday life and not unique to motor representation.

For example, some housemates who have decided to bake a pizza might sit down together and make a plan without yet assigning roles to particular individuals.

Someone needs to prepare the dough,

another should prepare toppings,

while a third might mix some salad.

In so planning, each housemate is thinking about what

agents in their situations should do

and not what she herself or anyone in particular will do.

It is in this sense that one agent can rationally make plans for another’s actions even without any expectation of controlling them.

Indeed, an agent might not know who will perform the actions in advance of planning them.

Since it is plausible that some planning-like motor processes involve agent-neutral representations of outcomes \citep{ramsey:2010_understanding},

\footnote{\citep[p.\ 1146]{ramsey:2010_understanding}:

"different, although complementary, brain networks process actor-related and object-goal-related aspects of observed actions."

}

there is nothing incoherent in the idea that in each agent involved in a joint action there might be a plan-like structure of motor representations concerning all of the agents’ actions.

So some evidence indicates that in joint action we may each represent motorically an outcome to which our actions are collectively directed,

and there may sometimes be a single plan-like structure of motor representations concerning both my own and your actions.

\textbf{But how, if at all, could the existence of such planning-like structures facilitate coordination?}

Earlier we considered what is involved in performing an ordinary, individual action, where an agent moves a mug from one place to another passing it between her hands half-way.

Compare this individual action with the same action performed by two agents as a joint action.

One agent takes the mug and passes it to the other, who then places it.

The joint action is like the individual action in several respects.

First, the goal to which the joint action is directed is the same, namely to move the mug from here to there.

Second, there is a similar coordination problem---the agents’ two hands have to meet.

And, third, the evidence we have mentioned suggests that in joint action, motor representations and processes occur in each agent much like those that would occur if this agent were performing the whole action alone.

Why would this be helpful?

Suppose the agents' planning-like motor processes are similar enough that, in this context, they will reliably produce approximately the same plan-like structures of motor representations.

Then having a single planning-like motor process for the whole joint action in each agent means that

\begin{enumerate}

\item in each agent there is a plan-like structure of motor representations concerning each of the others’ actions,

\item each agent's plan-like structure concerning another's actions is approximately the same as any other agent's plan-like structure concerning those actions,

\item each agent's plan-like structure concerning her own actions is constrained by her plan-like structures concerning the other’s actions.

\end{enumerate}

So each agent’s plan-like structure of motor representations concerning her own actions is indirectly constrained by the other agents' plan-like structures concerning their own actions

by virtue of being directly constrained by her plan-like structures concerning their actions.

In this way it is possible to use ordinary planning-like motor processes to achieve coordination in joint action.

What enables the two or more agents' plan-like structures of motor representations to mesh is not that they represent each other's plans but that they processes motorically each other's actions and their own as parts of a single action.

So how does the joint action differ from the corresponding individual action?

There are at least two differences.

First, we now have two plan-like structures of motor representations because in each agent there is a planning-like motor process concerning the whole action.

These two structures of motor representations have to be identical or similar enough that the differences don’t matter for the coordination of the agents’ actions---let us abbreviate this by saying that they have to \emph{match}.

The need for matching planning-like structures is not specific to joint action;

it is also required where one agent observing another is able to predict her actions thanks to planning-like motor processes concerning the other’s actions (we mentioned evidence that this occurs above).

A second difference between the joint action and the individual action is this.

In joint action there are planning-like motor processes in each agent concerning some actions which she will not eventually perform.

There must therefore be something that prevents part but not all of the planning-like motor process leading all the way to action.

Exactly how this selective prevention works is an open question.

We expect bodily and environmental constraints are often relevant.

There may also be differences in how others’ actions are processed motorically \citep[compare][]{novembre:2012_distinguishing}.

\footnote{\citep[p.\ 2901]{novembre:2012_distinguishing}: 'in the context of a joint action—the motor control system is particularly sensitive to the identity of the agent (self or other) of a represented action and that (social) contextual information is one means for achieving this distinction'}

And inhibition could be involved too \citep[compare][]{sebanz:2006_twin_peaks}.

Our proposal, then, is this.

Some joint actions are collectively directed to an outcome which each agent represents motorically.

These representations trigger planning-like processes in each agent concerning all of the agents’ actions.

Thanks to environmental constraints and similarities between the agents,

the agents’ plan-like structures of motor representations will match in a limited but useful range of cases.

This matching ensures that in each agent, plan-like structure of motor representations concerning her own actions are indirectly constrained by the other agents' plan-like structures of motor representations concerning their actions.

Thus agents are able to achieve coordination for joint action not by representing each others’ plans but by treating motorically each other's actions and their own as if they were parts of a single action

Now we can see how joint action looks different if we approach it by thinking about motor representation rather than propositional attitudes like intention ...

Earlier I mentioned that Bratman thinks that joint action depends on interconnected planning, that is, planning in which your plans feature not only facts about the table to be moved and your environment but also facts about my plans.

It is the insistence on interconnected planning that leads to views on which joint action demands sophisticated insight into others' minds.

But reflection on the role of motor representation in coordinating joint action points to a different kind of planning, parallel planning. When we engage in parallel planning, you plan my actions as well as yours. So parallel planning is just ordinary planning. The only special thing is that some of the actions you plan will eventually be performed by me.

Making a single plan for your own and the other's actions is a way of using your planning abilities to meet relational constraints on our actions.

But this does not involve thinking about my plans. In core cases of parallel planning there is no mindreading, just planning.

In virtue of what are joint actions collectively directed to outcomes?

Let us return to the question we started with.

In virtue of what are any joint actions collectively directed to outcomes?

We started by reprising a standard way of answering this question by appeal to collective (or ‘shared’) intention.

The argument we have just presented does not imply that this standard way of answering this question is wrong;

but it does imply that it is not the whole story.

In virtue of what are joint actions collectively directed to outcomes?

For we can answer the question not only by appeal to collective intention but also by appeal to structures of motor representations.

Suppose there is a single outcome two or more agents each have a motor representation of,

and that these motor representations are causally responsible for the occurrence, in each agent, of agent-neutral motor planning-like processes concerning all of the agents’ actions.

Then these structures of motor representations provide for the agents’ actions to be collectively directed to this outcome, and they do so in just the way that collective intentions are supposed to.

For these structures of motor representations

involve a representation, on the part of each agent, of a single outcome;

and they coordinate the agents’ actions

where coordination of this type would normally facilitate occurrences of the type of outcome represented.

\textbf{So joint action is not only a matter of what we intend: sometimes it constitutively involves structures of motor representation.}

challenge

Explain the emergence of sophisticated human activities including referential communication and mindreading.

conjecture

Joint action plays a role in explaining how sophisticated human activities emerge.

objection

Joint action presupposes mindreading at the limits of human abilities.

This matters for the objection I mentioned earlier.

If we characterise joint action in the standard way, by invoking shared intention, then I think it's reasonably clear that performing a joint action will require sophisticated cognitive abilities including mindreading.

As we saw, this is fatal for the conjecture that joint action plays a role in explaining how sophisticated human activities emerge.

But we have just seen that we can avoid this objection by recognising that there is a constitutive role for motor representation in characterising joint action.

One amazing thing about the recent work on motor representation in joint action

is that not only tells us about mechanisms of coordination:

it also provides us with a new way of thinking about what joint action is.

And one benefit of this is that it allows us to hold on to the conjecture that joint action is a core form of social intelligence,

one that may well play a role in explaining how sophisticated human abilities like mindreading and referential communication emerge in evolution or development.