Lecture 3

Amit Agarwal - Vikas Bansal

Date: August 02 2002

Contents

Nash Equilibrium: In this lecture we consider several examples of strategic games and try to find Nash Equlibria for them and answer the following questions regarding Nash Equilibrium for a strategic game.

• Does Nash Equilibrium always exist for a game ?
• If it exists how to compute it and is it always possible to compute it ?

We also analyze a general constant sum game and Nash Equilibria for it.

Strategic Games

A STRATEGIC GAME is a model of interacting decision makers referred to as players. More formally, a strategic game consists of

• a set of players
• for each player, a set of actions/strategies
• for each player, preferences over the set of action profiles.

Definition: A strategy profile $(s_1^*,s_2^*, s_3^*,. ..,s_n^*)$ for a $n$ player game is a Nash Equilibrium iff
$\forall i, \hspace{0.2cm} \forall s_i,\hspace{0.2cm} u_i(s_1^*,s_2^*,s_3^*,...... ...)\hspace{0.2cm} \geq \hspace{0.2cm}u_i(s_1^*,s_2^*,...,s_{i-1}^*,s_i,...,s_n^*)$, where $u_{i}$ is the utility of player $i$.

The above definition neither implies that a strategic game has a Nash equilibrium, nor that it has only one. Examples in the next section show that some games have a single Nash equilibrium, some possess no Nash equilibria and others have many Nash equilibria.

Choice of Strategies

• Pure Strategies: A pure strategy is one where players deterministically choose their moves.
• Mixed Strategies: A mixed strategy is one where players randomly choose one out of many different strategies. For example players can choose a probability distribution over the the set of possible strategies and randomly pick one before playing the game.

The best strategy for a player in a game may be a mixed one. In some games, however, it is possible for a pure strategy to be optimal.

Examples of Nash equilibrium in some games

Prisoner's Dilemma

Two suspects in a crime are held in separate cells. There is enough evidence to convict each of them of a minor offense, but not enough evidence to convict either of them of the major crime unless one of them acts as an informer against the other. If they both stay quiet, each will be convicted of the minor crime and spend only one year in prison. If one and only one of them confesses, she will be freed and the other person will be convicted of the major crime and spend 10 years in jail. If they both confess, each will spend five years in prison.
This situation may be modeled as a strategic game with :
Players: The two suspects.
Actions: Each player's set of actions is {Confess, Deny}.
We can represent the suspects' preference orderings with a payoff function and represent the game compactly with the payoff matrix:

	C	D
C	5,5	0,10
D	10,0	1,1

By examining the four possible pairs of actions in this game, one can see that the action pair (Confess,Confess) is a pure strategy Nash Equilibrium because if player 2 chooses Confess,player 1 is better off choosing Confess than Deny. Similarly given that player 1 chooses to Confess, player 2 is better off choosing Confess than Deny.

Battle of the Sexes: Cricket or Movie

Two people (a boy and his girlfriend for example) wish to go out together.The boy prefers to go for the cricket match whilst his girlfriend prefers watching the movie. If they go out together for the movie the girl is very happy while the boy is happy but not so much, while if they go out together for the cricket match the boy is very happy but the girl is not very happy. But if they go out separately they are both equally unhappy. This situation can be modeled as the two-player strategic game with the payoff matrix as shown below, in which the boy who prefers cricket chooses a row while the girl chooses a column.

	C	M
C	10,5	0,0
M	0,0	5,10

In this case, {Cricket,Cricket} and {Movie,Movie} are the two pure strategy Nash Equilibria, i.e if in every encounter, both players choose to watch a Movie, then no player has an incentive to deviate; if, in each encounter both choose to watch cricket, then again no player has an incentive to deviate. Moreover (1/3, 2/3) is a mixed strategy Nash Equilibrium.

Matching Pennies

In this game two people choose, simultaneously, whether to show the Head or the Tail of a coin. If they show the same side, person 2 pays person 1 a rupee; if they show different sides, person 1 pays person 2 one rupee. A strategic game form that models this situation is shown the figure. In this representation of the players' preferences, the payoffs are equal to the amounts of payoff involved. In this game the player's interests are completely opposite, whereas player one wants to take same action as player two while player two benefits when he takes the opposite action to player one. This game is also an example of a zero-sum game where the sum of the payoffs for the two players for each choice is zero.

	H	T
H	1,-1	-1,1
T	1,-1	1,-1

By checking each of the four pair of actions in this game, one can see that this game has no pure strategy Nash Equilibrium. Since for the pair of choices (T,T) and (H,H), player two is better off deviating, while for the pair of actions (H,T) and (T,H), player 1 is better off deviating. (1/2, 1/2) is a mixed strategy Nash Equilibrium here.

Iterated Deletion to compute Nash Equilibrium.

Definition : Strategy $i$ dominates strategy $i'$ for row player iff

$\forall j,\hspace{0.2cm} u_r(i,j)\geq u_r(i',j)$

Consider the example of Prisoner's Dilemma:

C	5	0	$\leftarrow$Dominates
D	10	1

Note: Here the payoffs are in the negative sense.


Definition: Strategy $j$ dominates strategy $j'$ for column player iff

$\forall i,\hspace{0.2cm} u_r(i,j)\geq u_r(i,j')$

Definition : Strategy $s_i^*$ is a dominant strategy for player i iff

$\forall s_1,s_2,...,s_n \hspace{0.3cm} u_i(s_1,s_2,...,s_i^*,s_{i+1},...,s_n)\geq u_i(s_1,s_2,...,s_n)$

One can delete rows or columns which are dominated by other rows or columns interactively to identify the Nash equilibria. Consider an example for which the payoff matrix is as below:

I $\backslash$II	A	B	C	D
A	5,2	2,6	1,4	0,4
B	0,0	3,2	2,1	1,1
C	7,0	2,2	1,5	5,1
D	9,5	1,3	0,2	4,8

In the above matrix column D dominates column A for column player, hence column A can be deleted to get a reduced matrix. Now in the reduced matrix, row B dominates row A and also row C dominates row D hence rows A and D can be deleted. Furthermore (in the reduced matrix) column C dominates column D for column player, hence the metric reduces to a 2 x 2 matrix, where row B dominates row C. Finally column B dominates column C leaving (3,2) as the unique Nash Equilibrium.
This strategy however does not always succeed in giving the Nash Equilibrium. We might reach a stage from which we cannot delete further rows/columns to obtain the Nash Equilibrium.
For example, in the battle of sexes game described previously, this method does not give the Nash Equilibria since no column or row is dominated by any other column or row respectively.

A general two player constant sum game

Since the game is a constant sum game,

$\forall i,j, a_{ij}+b_{ij}= constant= c.$

A general two-player constant sum game can be represented as a pair of payoff matrices. Consider a general 2 player game in which the row player has $m$ choices of strategy and the column player has $n$ choices. The payoff matrix for this game would be a $m$ x $n$ matrix.

		$a_{ij},b_{ij}$

In this game, $a_{ij}$ is the payoff to the row player if the row player plays strategy $i$ and the column player chooses strategy $j$, and $b_{ij}$ is the payoff to the column player, where $a_{ij}$ $+$ $b_{ij}$ is constant. A mixed strategy of the row player is represented by a $m$-tuple of probabilities p,

where $p_{i}$ $\geq$ 0, 1 $\leq$ $i$ $\leq m$

and

$\sum p_i = 1.$

Similarly, the mixed strategy of the column player is represented as a $n$-tuple, q where $\sum q_i = 1.$ When row player plays the mixed strategy p and the column player plays the mixed strategy q, the payoff to the row player is $u_r(\underline{p},\underline{q})=\sum_j\sum_ip_iq_ja_{ij} = p^TAq$


Similarly, the payoff to the column player is $u_c(\underline{p},\underline{q})=\sum_i\sum_jp_iq_jb_{ij} = p^TBq.$

Definition : $\mathbf{(p^*,q^*)}$ is a Nash Equilibrium (for mixed strategy) iff,

For row player A,

$\bf {{\forall p, {p^*}^TAq^* \geq p^TAq^*}}$

and for column player B,

$\bf{ {\forall q, {p^*}^TBq^* \geq {p^*}^TBq}}$

Saddle Points

Definition : For a Matrix A, $a_{ij}$ is a saddle point of A if it is simultaneously a minimum in its row and a maximum in its column, i.e

$\forall k, a_{ij}\geq a_{kj}$
$\forall l, a_{ij}\leq a_{il}$

$\mathbf{Theorem}$ : $a_{ij}$ is a saddle point $\iff$ (i,j) is a Nash Equilibrium.

Proof:( $\Rightarrow$) Since $a_{ij}$ is a saddle point, $a_{ij}$ is maximum in its column hence the row player cannot increase his payoff given that column player has chosen column $j$. Similarly since it is minimum in its row and the payoff of the column player is ( $c - a_{ij}$), hence he cannot increase his payoff by changing his strategy given that the row player has chosen row $i$. Hence $(i, j)$ is a Nash Equilibrium.


( $\Leftarrow$)If $(i, j)$ is a Nash Equilibrium, obviously $a_{ij}$ is the maximum value in its column(it is the payoff to the column player). Similarly for the row player, it is the minimum in its row since (c - $a_{ij}$) is the payoff to the row player. Hence, ($i$,$j$) is a saddle point.

$\mathbf{Theorem}$ : If $a_{ij}$ is a saddle point and $a_{mn}$ is also a saddle point, then $a_{in}$ and $a_{mj}$ are also saddle points and

$a_{ij}$ $=$ $a_{mn}$ $=$ $a_{in}$ $=$ $a_{nj}$.

$\mathbf{ Proof}$: Since by definition, saddle points are minima in their rows and maxima in their columns, $a_{ij}\leq a_{in}\leq a_{mn}$ and $a_{mn}\leq a_{in}\leq a_{ij}$,

Hence,

$a_{ij}=a_{mj}=a_{mn}=a_{in}.$

Also, $a_{in}$ is a saddle point since $a_{in}$ $=$ $a_{ij}$ is a minimum in its row i, and $a_{in}$ $=$ $a_{mn}$ is a maximum in its column $n$. Similarly, $a_{mj}$ is a saddle point. Definition: Let $a_{i}$ $=$ $\min_{j}\hspace{0.1cm}{a_{ij}}$ be the guaranteed payoff to row player if he chooses row $i$. Let

$u_{r}$ $=$ $\max_{i}$ $a_{i}$ $=$ $\max_{i}$ $\min_{j}\hspace{0.1cm} \bf {{a_{ij}}}$
$u_{c}$ $=$ $\min_{j}$ ${\max_{i}\hspace{0.1cm} \bf {{a_{ij}}}}$

Lemma: For any matrix A,

$u_{c}$ $\geq$ $u_{r}$.

Proof: We have, $a_{ij} \geq \min_{k}$ $a_{ik}$ $\forall i,j$. Hence $\max_{i}\hspace{0.2cm}a_{ij}$ $\geq$ $\max_{i}$ $\min_{k}$ $a_{ik}$, $\forall j$.
This implies $\max_{i}\hspace{0.2cm}a_{ij}$ $\geq$ $u_{r}$, $\forall j$. Therefore $\min_{j}$ $\max_{i}\hspace{0.2cm}a_{ij}$ $\geq$ $u_{r}$

$\Rightarrow$ $u_c$ $\geq$ $u_r$
The above lemma also holds for the case of mixed strategies. The proof is given later

Theorem: Matrix A has a saddle point $\iff$ $u_r$ $=$ $u_c$.

Proof: ( $\Rightarrow$) Let $a_{ij}$ be a saddle point of A. By definition, $a_{ij}$ $=$ $\min_{l}$ $a_{il}$.
Also, $u_r \geq$ $a_{ij}$ and $a_{ij}$ $=$ $max_{k}$ $a_{kj}$. Hence, $u_c \leq a_{ij}.$
Combining these two, $u_c \leq a_{ij} \leq u_r$. But from the previous lemma $u_r \leq u_c$.
Hence $u_r$ $=$ $u_c$.
( $\Leftarrow$) Choose $i$ s.t., $\min_{k}$ $a_{ik}$ $=$ $u_r$.
Now choose $l$ s.t. $a_{il}$ $=$ $\min_{p}$ $a_{ip}$ $=$ $u_c$ $=$ $u_r$.
Since $a_{il}$ is the minimum in row $i$ and $\exists$ column $q$ such that $\max_{q}$ $a_{qj}$ $=$ $u_c$.

Thus $a_{il}$ $=$ $u_c$ $=$ $\max_{q}$ $a_{qj}$ $\geq$ $a_{ij}$. Since $a_{il}$ is a minimum in its row, $a_{il}$ $=$ $a_{ij}$.
Thus, $a_{ij}$ is also a minimum its row.
$\Rightarrow$ $a_{ij} = a_{il} = \max_{q}a_{qj}$
which proves that $a_{ij}$ is a saddle point of A.

Minimax Theorem

Definition : Row value for a mixed strategy is defined as $v_r =$ $\max_{p}$ $\min_{q}$ $p^{T}Aq$

Similarly column value is defined as, $v_c =$ $\min_{q}$ $\max_{p}$ $p^{T}Aq$

In other words, $v_r$ is the amount of payoff that the row player is guaranteed to win on the average, assuming that he plays rationally.

Lemma: For any matrix A, $v_{c}$ $\geq$ $v_{r}$.

Proof : We observe that $\mathbf{p}^{T}\mathbf{A}\mathbf{q} \geq \min_{\mathbf{q}} \mathbf{p}^{T}\mathbf{A}\mathbf{q} \forall \mathbf{p,q}$. Taking the maximum over all $\mathbf{p}$ on both sides, $\max_{\mathbf{p}}\: \mathbf{p}^{T}\mathbf{A}\mathbf{q} \geq \max_{\mathbf{p}} \min_{\mathbf{q}}\: \mathbf{p}^{T}\mathbf{A}\mathbf{q} \forall \mathbf{q}$. The RHS is $v_{r}$, thus the previous equation can be re-written as $\max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q} \geq v_{r}, \forall \mathbf{q}$. Therefore $\min_{\mathbf{q}} \max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q} \geq v_{r}, \forall \mathbf{q}$. This proves $v_{c} \geq v_{c}$.

$\mathbf{Theorem}$ : $\mathbf{p*,q*}$ is a Nash Equilibrium iff $v_{c} = v_{r} = \mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$.

Proof : ( $\Rightarrow$) As $\mathbf{p*,q*}$ is a Nash Equilibrium of A, $\mathbf{p*}^{T}\mathbf{A}\mathbf{q*} = \min_{\mathbf{q}}\mathbf{p*}^{T}\mathbf{A}\mathbf{q}$. Also, $v_{r} \geq \mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$ and $\mathbf{p*}^{T}\mathbf{A}\mathbf{q*} = \max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q*}$. Hence, $v_{c} \leq \mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$. Combining these two we get, $v_{r} \geq v_{c}$. But from previous lemma $v_{c} \geq v_{r}$. This proves $v_{c} = v_{r}$.
( $\Leftarrow$) Choose $\mathbf{p*}$ s.t., $\min_{\mathbf{q}}\mathbf{p*}^{T}\mathbf{A}\mathbf{q} = v_{r}$. Now choose $\mathbf{q'}$ s.t. $\mathbf{p*}^{T}\mathbf{A}\mathbf{q'} = \min_{\mathbf{q}} \mathbf{p'}^{T}\mathbf{A}\mathbf{q} = v_{c} = v_{r}$. Since $\mathbf{p'}^{T}\mathbf{A}\mathbf{q'}$ is the minimum over all $\mathbf{p}$ and $\exists$ strategy $\mathbf{q*}$ s.t.
$\max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q*} = v_{c}$.
Thus $\mathbf{p'}^{T}\mathbf{A}\mathbf{q'} = v_{c} = \max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q*} \geq \mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$. Since $\mathbf{p*}^{T}\mathbf{A}\mathbf{q'}$ is the minimum over all $\mathbf{p}$, $\mathbf{p*}^{T}\mathbf{A}\mathbf{q'} = \mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$.
Thus $\mathbf{p*}^{T}\mathbf{A}\mathbf{q*}$ is also minimum over all $\mathbf{p}$ for the same $\mathbf{q}$. $\Rightarrow \mathbf{p*}^{T}\mathbf{A}\mathbf{q*} = \mathbf{p*}^{T}\mathbf{A}\mathbf{q'} = \max_{\mathbf{p}} \mathbf{p}^{T}\mathbf{A}\mathbf{q*}$. Thus $\mathbf{p*,q*}$ is a Nash Equilibrium.


$\mathbf{ [Von Neumanns' Minimax Theorem]}$ For any two person zero-sum game specified by matrix $\mathbf{A}$, optimal mixed strategies exist for both players. Moreover the row and column values are equal. In other words,

$$\max_{\mathbf{p}} \min_{\mathbf{q}}\: \mathbf{p}^{T}\mathbf{A}\... ... = \min_{\mathbf{q}} \max_{\mathbf{p}}\: \mathbf{p}^{T}\mathbf{A}\mathbf{q}$$ (1)

or $v_r = v_c$. Also if $p^*$ and $q^*$ denote the optimal strategies for the row and the column player respectively, then

1. $v_r = v_c = p^{*T}Aq^*$
2. ($p^*$,$q^*$) is a Nash Equilibrium for this two player game.

The optimal strategy for row player will yield the same payoff as the optimal strategy for Column player! If either the row or the column player plays her optimal strategy, the opponent cannot improve the expected payoff. Thus once a player has publicly committed to play the optimal strategy, it is possible for the other player to play the game with a pure strategy and still receive the optimal expected payoff.

Proof of Minimax Theorem

This proof requires the duality theorem, a well known result in linear programming. A linear programming problem can be defined in terms of constraints $\mathbf{Ax} \geq \mathbf{b}$ and $\mathbf{x} \geq 0$ , and a cost vector $\mathbf{C}$ . The goal is to minimize the cost $\mathbf{C^{T}x}$ subject to the constraints and given a cost vector. This is called the primal problem. Associated with every primal problem is a dual problem stated as follows. The constraints now become $\mathbf{A^{T}y} \leq\mathbf{C}$ and $\mathbf{y} \geq 0$, the new cost vector is $\mathbf{b}$ and the goal is to maximize $\mathbf{b^{T}y}$.
$\mathbf{ [Duality Theorem]}$ If either problem (primal or dual) has a best vector (called $\mathbf{x^{*}}$ or $\mathbf{y^{*}}$), then so does the other. The minimum $\mathbf{C^{T}x^{*}}$ equals the maximum $\mathbf{y^{*T}b}$

$$\mathbf{C^{T}x^{*}} = \mathbf{b^{T}y^{*}}$$ (2)

In terms of $P$(the row player) and $Q$(column player), we want to minimize (primal) $\mathbf{c^{T}p}$ (called $\overline{p^{*}}$) subject to the constraints $\mathbf{p^{T}A} \geq \mathbf{b}$ and $\mathbf{p} \geq0$. We also want to maximize (dual) $\mathbf{b^{T}q}$ (called $\overline{q^{*}}$) subject to the constraints $\mathbf{Aq} \leq\mathbf{c}$ and $\mathbf{q} \geq0$, where $\mathbf{c}$ and $\mathbf{b}$ are unit vectors. In light of this formulation and the duality theorem, we can state

$$\mathbf{c^{T}\overline{p^{*}}} = \mathbf{b^{T}\overline{q^{*}}} = \theta$$ (3)

Therefore, our probability distributions that correspond to our optimal vectors $\mathbf{p^{*}}$ and $\mathbf{q^{*}}$ are obtained by setting $\mathbf{p^{*}} = \mathbf{\overline{p^{*}}} / \theta$ and $\mathbf{q^{*}} = \mathbf{\overline{q^{*}}} / \theta$.

[Proof of Von Neumanns' Minimax Theorem] Since $\mathbf{\overline{p^{*}}^{T}A} \geq\mathbf{b}$, $\forall \mathbf{q}, \; \mathbf{\overline{p^{*}}^{T}Aq} \geq\mathbf{b^{T}q}$. And since $\mathbf{b^{T}q} = 1$, this implies that $\mathbf{p^{*T}Aq} \geq1 / \theta$. This gives a lower bound on how much $P$ is winning ( $1 / \theta$). Similarly, $\mathbf{\overline{p^{*}}^{T}Aq} \geq\mathbf{b^{T}q} = 1$ implies that $\mathbf{p^{T}Aq^{*}} \leq1 / \theta$ and that $1 / \theta$ is an upper bound on $Q$'s loss. Therefore,

$$\mathbf{p^{*T}Mq^{*}}$$ $$=$$ $$\frac{1}{\theta}$$
$$\mathbf{p^{T}Mq^{*}}$$ $$\leq$$ $$\mathbf{p^{*T}Mq^{*}} = \frac{1}{\theta}$$
$$\mathbf{p^{*T}Mq^{*}}$$ $$\leq$$ $$\mathbf{p^{*T}Mq}$$