Dynamic discrete choice (Ofer Abarbanel online library)

Dynamic discrete choice (DDC) models, also known as discrete choice models of dynamic programming, model an agent’s choices over discrete options that have future implications. Rather than assuming observed choices are the result of static utility maximization, observed choices in DDC models are assumed to result from an agent’s maximization of the present value of utility, generalizing the utility theory upon which discrete choice models are based.[1]

The goal of DDC methods is to estimate the structural parameters of the agent’s decision process. Once these parameters are known, the researcher can then use the estimates to simulate how the agent would behave in a counterfactual state of the world. (For example, how a prospective college student’s enrollment decision would change in response to a tuition increase.)


Estimation of dynamic discrete choice models is particularly challenging, due to the fact that the researcher must solve the backwards recursion problem for each guess of the structural parameters.

The most common methods used to estimate the structural parameters are maximum likelihood estimation and method of simulated moments.

Aside from estimation methods, there are also solution methods. Different solution methods can be employed due to complexity of the problem. These can be divided into full-solution methods and non-solution methods.

Full-solution methods

The foremost example of a full-solution method is the nested fixed point (NFXP) algorithm developed by John Rust in 1987.[2] The NFXP algorithm is described in great detail in its documentation manual.[3]

A recent work by Che-Lin Su and Kenneth Judd in 2012[4] implements another approach (dismissed as intractable by Rust in 1987), which uses constrained optimization of the likelihood function, a special case of mathematical programming with equilibrium constraints (MPEC). Specifically, the likelihood function is maximized subject to the constraints imposed by the model, and expressed in terms of the additional variables that describe the model’s structure. This approach requires powerful optimization software such as Artelys Knitro because of the high dimensionality of the optimization problem. Once it is solved, both the structural parameters that maximize the likelihood, and the solution of the model are found.

In the later article[5] Rust and coauthors show that the speed advantage of MPEC compared to NFXP is not significant. Yet, because the computations required by MPEC do not rely on the structure of the model, its implementation is much less labor intensive.

Despite numerous contenders, the NFXP maximum likelihood estimator remains the leading estimation method for Markov decision models.[5]

Non-solution methods

An alternative to full-solution methods is non-solution methods. In this case, the researcher can estimate the structural parameters without having to fully solve the backwards recursion problem for each parameter guess. Non-solution methods are typically faster while requiring more assumptions, but the additional assumptions are in many cases realistic.

The leading non-solution method is conditional choice probabilities, developed by V. Joseph Hotz and Robert A. Miller.[6]


  1. ^Keane & Wolpin 2009.
  2. ^Rust 1987.
  3. ^Rust, John (2008). “Nested fixed point algorithm documentation manual”. Unpublished.
  4. ^ Jump up to:ab c Su, Che-Lin; Judd, Kenneth L. (2012). “Constrained Optimization Approaches to Estimation of Structural Models”. Econometrica. 80 (5): 2213–2230. doi:10.3982/ECTA7925. hdl:10419/59626. ISSN 1468-0262.
  5. ^ Jump up to:ab c d Iskhakov, Fedor; Lee, Jinhyuk; Rust, John; Schjerning, Bertel; Seo, Kyoungwon (2016). “Comment on “constrained optimization approaches to estimation of structural models””. Econometrica. 84 (1): 365–370. doi:10.3982/ECTA12605. ISSN 0012-9682.
  6. ^Hotz, V. Joseph; Miller, Robert A. (1993). “Conditional Choice Probabilities and the Estimation of Dynamic Models”. Review of Economic Studies. 60 (3): 497–529. doi:10.2307/2298122. JSTOR 2298122.
  7. ^Aguirregabiria & Mira 2010.
  8. ^Hotz, V. J.; Miller, R. A.; Sanders, S.; Smith, J. (1994-04-01). “A Simulation Estimator for Dynamic Models of Discrete Choice”. The Review of Economic Studies. Oxford University Press (OUP). 61 (2): 265–289. doi:10.2307/2297981. ISSN 0034-6527. JSTOR 2297981.


Ofer Abarbanel online library