19th Ave New York, NY 95822, USA

数学代写| ST202-12 Stochastic Processes代写| assignment 4

Throughout the 2020-21 academic year, we will be adapting the way we teach and assess your modules in line with government guidance on social distancing and other protective measures in response to Coronavirus. Teaching will vary between online and on-campus delivery through the year, and you should read the additional information linked on the right hand side of this page for details of how this will work for this module. The contact hours shown in the module information below are superseded by the additional information.You can find out more about the University’s overall response to Coronavirus at: https://warwick.ac.uk/coronavirus.
All dates for assessments for Statistics modules, including coursework and examinations, can be found in the Statistics Assessment Handbook at http://go.warwick.ac.uk/STassessmenthandbook

ST202-12 Stochastic Processes

Academic year
Undergraduate Level 2
Module leader
Nicholas Tawn 
Credit value
Module duration
10 weeks
Study location
University of Warwick main campus, Coventry
Introductory description

This modules runs in Term 2.
This module is core for students with their home department in Statistics.
It is available as an option or unusual option for other students.
Statistics Students: ST115 Introduction to Probability AND MA137 Mathematical Analysis
Non-Statistics Students: ST111 Probability A AND ST112 Probability B AND (MA131 Analysis I OR MA137 Mathematical Analysis)
Leads to: ST333 Applied Stochastic Processes and ST406 Applied Stochastic Processes with Advanced Topics.
Module web page

Module aims
Loosely speaking, a stochastic or random process is any measurable phenomenon which develops randomly in time. Only the simplest models will be considered in this course, namely those where the process moves by a sequence of jumps in discrete time steps. We will discuss: Markov chains, which use the idea of conditional probability to provide a flexible and widely applicable family of random processes; random walks, which serve as fundamental building blocks for constructing other processes as well as being important in their own right; and renewal theory, which studies processes which occasionally “begin all over again.” Such processes are common tools in economics, biology, psychology and operations research, so they are very useful as well as attractive and interesting theories.
The aims of this module are to introduce the idea of a stochastic process, and to show how simple probability and matrix theory can be used to build this notion into a beautiful and useful piece of applied mathematics.
Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

  1. Brief review of fundamental probability notions.
  2. Introduction to Markov processes (Definitions, Chapman-Kolmogorov equations, notions of recurrence, transience, positive recurrence, transition probability matrices,
  3. Long-run behaviour of Markov Chains, (equilibirum distributions, convergence to equilibrium)
  4. Some applications.
  5. Discussion of extensions to continuous settings and if time permits to non-Markov settings.
Learning outcomes

By the end of the module, students should be able to:

  • Understand the notion of a Markov chain, and how simple ideas of conditional probability and matrices can be used to give a thorough and effective account of discrete-time Markov chains.
  • Understand notions of long-time behaviour including transience, recurrence, and equilibrium.
  • Be able to apply these ideas to answer basic questions in several applied situations including genetics, branching processes and random walks.
Indicative reading list

S.M. Ross, Introduction to Probability Models
G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes
P.W. Jones and P. Smith, Stochastic Processes
J.R. Norris, Markov Chains
View reading list on Talis Aspire

Subject specific skills


Transferable skills



问题 1. Will these become extinct:
Let $\left(X_{n}\right)_{n \geq 0}$ be a branching process (as described in lectures) with $X_{0}=1$. The following gives 3 different scenarios for the offspring distribution, $Z .$ In each case determine if the process is going to become extinct with certainty or not. Full credit for elear and concise working as well as the correct classification:
(i) $Z \sim$ Uniform \{0,1,2\}
(ii) $Z \sim W+V$ where $W \sim \operatorname{Geom}(3 / 5)$ and $V \sim \operatorname{Geom}(7 / 12)$
(iii) $Z \sim M$ where $M$ has probability mass function:
(1-q)^{m / 2} q & \text { if } 0 \leq m \text { is even } \\
0 & \text { otherwise }
where we set $q=2 / 3$.

问题 2. Bacteria Breakup
Suppose that $\left(X_{n}\right)_{n \geq 0}$ represents the popuLation size of a colony of bacteria where $X_{0}=1 .$ Over each time step each bacterium (independently of all others in the population) can only do one of three things; it either dies out, remains the as an individual or divides in to two with probabilities $a, b$ and $c$ respectively.
(i) Find an expression for $G(s)$ and hence find the extinction probability for the process. Hint: Your solution will have 2 answers which are associated with conditions, that you should state explicitly, depending on the unknown offspring distribution constants.
(ii) Herein for the rest of this question suppose that $a=1 / 4, b=1 / 4$ and $c=1 / 2 .$ Determine the probability of extinction.
(iii) Explain in one sentence why the maximum population at time $n$ is given by $2^{n}$.
(iv) Derive the function $f(n)$ and constant $d$ for the following expression:

问题 3. More Branching Processes. The aim of this question is to slightly modify the definition of the branching process from the lectures and check how this affects the results of the final two theorems in Chapter $3 .$
Let $\left(X_{n}\right)_{n \geq 0}$ represent the number of individuals in a population at time $n$. At each generation each individual (independently of all others) gives birth to a random number of offspring that forms the next generation but now the offspring distribution is determined by whether the generation index is odd or even! In fact, the offspring distribution alternates between consecutive generations.
To put this mathematically, with $X_{0}=1,$ then
X_{n+1}=\sum_{k=1}^{X_{n}} Z_{k}^{n}
where all $Z_{k}^{n}$ are independent and
Z_{k}^{n} \sim\left\{\begin{array}{l}
Z_{1} \text { if } n \text { is odd } \\
Z_{2} \text { if } n \text { is even }
where $Z_{1}, Z_{2}$ are RVs on $\mathbb{N}$ such that $\mathbb{E}\left(Z_{1}\right)=\mu_{1}<\infty$ and $\mathbb{E}\left(Z_{2}\right)=\mu_{2}<\infty$. Also, assume that $0<\mathbb{P}\left(Z_{i}=0\right)<1$ for $i=1,2$
(i) Define $F_{n}(s):=\mathbb{E}_{1}\left[s^{X_{n}}\right]$ and $G_{i}(s):=\mathbb{E}\left[s^{Z_{i}}\right] .$ Using the same methodology as in lectures (or otherwise) show that for $0 \leq s<1$
F_{2 n}(s)=G_{2}\left(G_{1}\left(F_{2 n-2}(s)\right)\right)
(ii) By either differentiating wrt $s$ (using chain rule), OR carefully using Tower property, derive an expression of
\mathbb{E}_{1}\left(X_{2 n}\right)
(iii) Adjust a proof in lectures to prove that the probability of extinction of this process is given by $\alpha$ which is the minimal non-neg solution to:
(iv) Again by replicating and adjusting a proof from the lectures; show that if $\mu_{1} \mu_{2} \leq 1$ then the process becomes extinct with probability 1 whereas if $\mu_{1} \mu_{2}>1$ then extinction is not certain.
(v) Suppose that $Z_{1} \sim \operatorname{Po}(5)$ and $Z_{2} \sim \operatorname{Po}\left(\mu_{2}\right) .$ Give a range of values for $\mu_{2}$ that guarantee extinction for the population.

问题 4. To infinity and beyond: This question is all about showing that an asymmetric RW will “drift” off in the direction of the bias, This is done without needing SLL.N or even WLLN. It simply uses careful application of the Gambler’s ruin result from lectures.
Consider a Simple Random Walk, $\left(X_{n}\right)_{n \geq 0},$ on $Z$ where the dynamics are given by the following:
(P)_{i j}=\left\{\begin{array}{ll}
p & \text { if } j=i+1 \\
q=(1-p) & \text { if } j=i-1 \\
0 & \text { otherwise. }
Now suppose that $1>p>q>0 .$ Hence, we know (from lectures) that the process $\left(X_{m}\right)_{n} \geq 0$ is transient for all states $i \in \mathbb{Z}$
(i) For $m \in I,$ let $H^{m}:=\inf \left\{n \geq 0: X_{n}=m\right\}$. Suppose that $m<0$. Then show that for $m<i \in I$
h_{i}:=\mathbb{P}_{i}\left(H^{m}<\infty\right)=a^{f(i, m)}
where $a$ is a constant and $f(i, m)$ is a function of $i$ and $m$ that you should display clearly at the end of your solution. Hint: Just adapt the Gambler’s ruin proof from lectures.
(ii) Define the event:
E:=\left\{\omega \in \Omega: \exists N=N(\omega) \in \mathbb{N} \forall n \geq N, \quad X_{n}(\omega)>-N\right\}
So $E$ is the event that the chain has a finite minimal point for any given “trajectory”. We want to show that under the law $\mathbb{P}$ o this event has probability $1 .$ Take a look at how we wrote the extinction event in a branehing process as a union of increasing events. The aim now is to do something similar for $E$.
On the event that $X_{0}=0,$ write an identity for the event $E$ in terms of a countable union of events involving the random variables $H^{-m}$ for $m \in \mathbb{N}$. Hint: $E$ could be read as “eventually we can find some $N \in \mathbb{N}$ such that the chain does not hit $-N$ (and therefore any state below that)”.
(iii) Justify that
(iv) Using the fact that we know 0 is a transient state and the state space is irreducible, show that for any $K>0$ the Markov chain eventually leaves the interval $[-K, K]$ If done correctly this should take no more than 5 concise statements to prove this and so long unclear answers will be lose credit. Hint: One approach is to assume there is no final visit to the set and prove a contradiction.
(v) Combine the results from (iii) and (iv) to show that for any fixed $K \in \mathbb{N}$ the Markov Chain will eventually exceed and then remain above the value $K$

E-mail: [email protected]  微信:shuxuejun4



Leave a comment