# Lecture 3

Amit Agarwal - Vikas Bansal

Date: August 02 2002

Contents

Nash Equilibrium: In this lecture we consider several examples of strategic games and try to find Nash Equlibria for them and answer the following questions regarding Nash Equilibrium for a strategic game.
• Does Nash Equilibrium always exist for a game ?
• If it exists how to compute it and is it always possible to compute it ?
We also analyze a general constant sum game and Nash Equilibria for it.

## Strategic Games

A STRATEGIC GAME is a model of interacting decision makers referred to as players. More formally, a strategic game consists of
• a set of players
• for each player, a set of actions/strategies
• for each player, preferences over the set of action profiles.
Definition: A strategy profile for a player game is a Nash Equilibrium iff
, where is the utility of player .

The above definition neither implies that a strategic game has a Nash equilibrium, nor that it has only one. Examples in the next section show that some games have a single Nash equilibrium, some possess no Nash equilibria and others have many Nash equilibria.

#### Choice of Strategies

• Pure Strategies: A pure strategy is one where players deterministically choose their moves.
• Mixed Strategies: A mixed strategy is one where players randomly choose one out of many different strategies. For example players can choose a probability distribution over the the set of possible strategies and randomly pick one before playing the game.
The best strategy for a player in a game may be a mixed one. In some games, however, it is possible for a pure strategy to be optimal.

## Examples of Nash equilibrium in some games

### Prisoner's Dilemma

Two suspects in a crime are held in separate cells. There is enough evidence to convict each of them of a minor offense, but not enough evidence to convict either of them of the major crime unless one of them acts as an informer against the other. If they both stay quiet, each will be convicted of the minor crime and spend only one year in prison. If one and only one of them confesses, she will be freed and the other person will be convicted of the major crime and spend 10 years in jail. If they both confess, each will spend five years in prison.
This situation may be modeled as a strategic game with :
Players: The two suspects.
Actions: Each player's set of actions is {Confess, Deny}.
We can represent the suspects' preference orderings with a payoff function and represent the game compactly with the payoff matrix:
 C D C 5,5 0,10 D 10,0 1,1
By examining the four possible pairs of actions in this game, one can see that the action pair (Confess,Confess) is a pure strategy Nash Equilibrium because if player 2 chooses Confess,player 1 is better off choosing Confess than Deny. Similarly given that player 1 chooses to Confess, player 2 is better off choosing Confess than Deny.

### Battle of the Sexes: Cricket or Movie

Two people (a boy and his girlfriend for example) wish to go out together.The boy prefers to go for the cricket match whilst his girlfriend prefers watching the movie. If they go out together for the movie the girl is very happy while the boy is happy but not so much, while if they go out together for the cricket match the boy is very happy but the girl is not very happy. But if they go out separately they are both equally unhappy. This situation can be modeled as the two-player strategic game with the payoff matrix as shown below, in which the boy who prefers cricket chooses a row while the girl chooses a column.
 C M C 10,5 0,0 M 0,0 5,10
In this case, {Cricket,Cricket} and {Movie,Movie} are the two pure strategy Nash Equilibria, i.e if in every encounter, both players choose to watch a Movie, then no player has an incentive to deviate; if, in each encounter both choose to watch cricket, then again no player has an incentive to deviate. Moreover (1/3, 2/3) is a mixed strategy Nash Equilibrium.

### Matching Pennies

In this game two people choose, simultaneously, whether to show the Head or the Tail of a coin. If they show the same side, person 2 pays person 1 a rupee; if they show different sides, person 1 pays person 2 one rupee. A strategic game form that models this situation is shown the figure. In this representation of the players' preferences, the payoffs are equal to the amounts of payoff involved. In this game the player's interests are completely opposite, whereas player one wants to take same action as player two while player two benefits when he takes the opposite action to player one. This game is also an example of a zero-sum game where the sum of the payoffs for the two players for each choice is zero.
 H T H 1,-1 -1,1 T 1,-1 1,-1
By checking each of the four pair of actions in this game, one can see that this game has no pure strategy Nash Equilibrium. Since for the pair of choices (T,T) and (H,H), player two is better off deviating, while for the pair of actions (H,T) and (T,H), player 1 is better off deviating. (1/2, 1/2) is a mixed strategy Nash Equilibrium here.

## Iterated Deletion to compute Nash Equilibrium.

Definition : Strategy dominates strategy for row player iff
Consider the example of Prisoner's Dilemma:

 C 5 0 Dominates D 10 1

Note: Here the payoffs are in the negative sense.

Definition: Strategy dominates strategy for column player iff
Definition : Strategy is a dominant strategy for player i iff
One can delete rows or columns which are dominated by other rows or columns interactively to identify the Nash equilibria. Consider an example for which the payoff matrix is as below:
 I II A B C D A 5,2 2,6 1,4 0,4 B 0,0 3,2 2,1 1,1 C 7,0 2,2 1,5 5,1 D 9,5 1,3 0,2 4,8
In the above matrix column D dominates column A for column player, hence column A can be deleted to get a reduced matrix. Now in the reduced matrix, row B dominates row A and also row C dominates row D hence rows A and D can be deleted. Furthermore (in the reduced matrix) column C dominates column D for column player, hence the metric reduces to a 2 x 2 matrix, where row B dominates row C. Finally column B dominates column C leaving (3,2) as the unique Nash Equilibrium.
This strategy however does not always succeed in giving the Nash Equilibrium. We might reach a stage from which we cannot delete further rows/columns to obtain the Nash Equilibrium.
For example, in the battle of sexes game described previously, this method does not give the Nash Equilibria since no column or row is dominated by any other column or row respectively.

## A general two player constant sum game

Since the game is a constant sum game,
A general two-player constant sum game can be represented as a pair of payoff matrices. Consider a general 2 player game in which the row player has choices of strategy and the column player has choices. The payoff matrix for this game would be a x matrix.
In this game, is the payoff to the row player if the row player plays strategy and the column player chooses strategy , and is the payoff to the column player, where is constant. A mixed strategy of the row player is represented by a -tuple of probabilities p,

where 0, 1

and

Similarly, the mixed strategy of the column player is represented as a -tuple, q where When row player plays the mixed strategy p and the column player plays the mixed strategy q, the payoff to the row player is

Similarly, the payoff to the column player is

Definition : is a Nash Equilibrium (for mixed strategy) iff,
For row player A,
and for column player B,

Definition : For a Matrix A, is a saddle point of A if it is simultaneously a minimum in its row and a maximum in its column, i.e

: is a saddle point (i,j) is a Nash Equilibrium.
Proof:( ) Since is a saddle point, is maximum in its column hence the row player cannot increase his payoff given that column player has chosen column . Similarly since it is minimum in its row and the payoff of the column player is ( ), hence he cannot increase his payoff by changing his strategy given that the row player has chosen row . Hence is a Nash Equilibrium.

( )If is a Nash Equilibrium, obviously is the maximum value in its column(it is the payoff to the column player). Similarly for the row player, it is the minimum in its row since (c - ) is the payoff to the row player. Hence, (,) is a saddle point.
: If is a saddle point and is also a saddle point, then and are also saddle points and
.
: Since by definition, saddle points are minima in their rows and maxima in their columns, and ,
Hence,
Also, is a saddle point since is a minimum in its row i, and is a maximum in its column . Similarly, is a saddle point. Definition: Let be the guaranteed payoff to row player if he chooses row . Let

Lemma: For any matrix A,
.

Proof: We have, . Hence , .
This implies , . Therefore

The above lemma also holds for the case of mixed strategies. The proof is given later

### Theorem: Matrix A has a saddle point .

Proof: ( ) Let be a saddle point of A. By definition, .
Also, and . Hence,
Combining these two, . But from the previous lemma .
Hence .
( ) Choose s.t., .
Now choose s.t. .
Since is the minimum in row and column such that .
Thus . Since is a minimum in its row, .
Thus, is also a minimum its row.

which proves that is a saddle point of A.

## Minimax Theorem

Definition : Row value for a mixed strategy is defined as
Similarly column value is defined as,
In other words, is the amount of payoff that the row player is guaranteed to win on the average, assuming that he plays rationally.

Lemma: For any matrix A, .

Proof : We observe that . Taking the maximum over all on both sides, . The RHS is , thus the previous equation can be re-written as . Therefore . This proves .

: is a Nash Equilibrium iff .

Proof : ( ) As is a Nash Equilibrium of A, . Also, and . Hence, . Combining these two we get, . But from previous lemma . This proves .
( ) Choose s.t., . Now choose s.t. . Since is the minimum over all and strategy s.t.
.
Thus . Since is the minimum over all , .
Thus is also minimum over all for the same . . Thus is a Nash Equilibrium.

For any two person zero-sum game specified by matrix , optimal mixed strategies exist for both players. Moreover the row and column values are equal. In other words,

 (1)

or . Also if and denote the optimal strategies for the row and the column player respectively, then
1. (,) is a Nash Equilibrium for this two player game.
The optimal strategy for row player will yield the same payoff as the optimal strategy for Column player! If either the row or the column player plays her optimal strategy, the opponent cannot improve the expected payoff. Thus once a player has publicly committed to play the optimal strategy, it is possible for the other player to play the game with a pure strategy and still receive the optimal expected payoff.

## Proof of Minimax Theorem

This proof requires the duality theorem, a well known result in linear programming. A linear programming problem can be defined in terms of constraints and , and a cost vector . The goal is to minimize the cost subject to the constraints and given a cost vector. This is called the primal problem. Associated with every primal problem is a dual problem stated as follows. The constraints now become and , the new cost vector is and the goal is to maximize .
If either problem (primal or dual) has a best vector (called or ), then so does the other. The minimum equals the maximum

 (2)

In terms of (the row player) and (column player), we want to minimize (primal) (called ) subject to the constraints and . We also want to maximize (dual) (called ) subject to the constraints and , where and are unit vectors. In light of this formulation and the duality theorem, we can state

 (3)

Therefore, our probability distributions that correspond to our optimal vectors and are obtained by setting and .

[Proof of Von Neumanns' Minimax Theorem] Since , . And since , this implies that . This gives a lower bound on how much is winning ( ). Similarly, implies that and that is an upper bound on 's loss. Therefore,