This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.
This paper deals with the output feedback H∞ control problem for a class of nonlinear stochastic systems. Based on the latest developed theory of stochastic dissipation, a notable result about the nonlinear H∞ output feedback control of deterministic system is generalized to the stochastic case. Finally, in the cases of state feedback and output feedback, two families of controllers are provided respectively.
In this paper, we develop a new mathematical model for the mammalian circadian clock, which incorporates both transcriptional/translational feedback loops （TTFLs） and a cAMP-mediated feedback loop. The model shows that TTFLs and cAMP signalling cooperatively drive the circadian rhythms. It reproduces typical experimental observations with qualitative similarities, e.g. circadian oscillations in constant darkness and entrainment to light dark cycles. In addition, it can explain the phenotypes of cAMP-mutant and Rev-erba^-/^- -mutant mice, and help us make an experimentally-testable prediction： oscillations may be rescued when arrhythmic mice with constitutively low concentrations of cAMP are crossed with Rev-erba^-/- mutant mice. The model enhances our understanding of the mammalian circadian clockwork from the viewpoint of the entire cell.
