We have a Bellman equation and first we want to know if there exists a value function that satisfies the equation and second we want to know the properties of such a solution. In order to answer the question we will define a mapping which maps a function to another function, and a fixed point of the mapping is to be a solution. The mapping we discussed is a mapping on the set of functions, which is a bit abstract. So today we will look at the math review.
So first we consider a set, , For us what it will be relevant to describe a sort of distance between any two points in a set. We will use the concept of a metric.
A metric is a function with the properties that it is non-negative, , symmetric, , and satisfies the triangle inequality,,
A common metric is euclidean distance, , Another is ,
A space, is a set of objects equipped with some general properties and structure
We may be interested in a metric space, a space with a metric such as, where is the set of all bounded rational functions, and is some distance function. Once we have a metric space we can discuss convergence and continuity.
A sequence, , converges to , , if s.t. for ,
A sequence , , is called a Cauchy sequence if for ,
Question: does every Cauchy sequence converge?
The metric space, is complete if every Cauchy sequence converges.
examples of completeness
- is complete.
- is not complete. Proof: let , So os Cauchy, but does not converge to a point in our set ,
- is complete. Are all closed sets complete? A closed subspace of a complete space is complete.
- is complete.
A mappting is a contraction mapping on a metric space, , if such that , Sometimes we write instead of ,
This means that any two points in our set, , are mapped such that after the mapping the distance between the points shrinks.
examples of contraction mapping
- is a contraction mapping on ,
Now we state the contraction mapping theorem.
Contraction mapping theorem
If is complete and is a contraction mapping, then with ,
We will prove this theorem for a general metric space later on. However, we must remember that it is necessary for this proof that the space be complete.
Let us now look at a criteria to verify that a mapping is a contraction mapping.
Contraction Mapping criteria
For and , Let satisfy the following two conditions:
- (M, monotonic condition) and , and , if then ,
- (D, discout condition) , for , .
Then is a contraction mapping.