Sztochasztika

1. Fundamentals of probability theory:
Kolmogorov's axioms, measure theory. Borel-Cantelli lemmas. Convergence types and their criteria. Uniform integrability. Dunford and Pettis' theorem. Kolmogorov's extension theorem. Kolmogorov's criterion for the continuity of a stochastic process. Conditional expectation, regular conditional distribution. Probability measures in metric spaces, weak convergence, compactness criterion (Prohorov's theorem). Measures on the function spaces C[0,1] and D[0,1], criteria for tightness.
Literature:
Alfréd Rényi: Probability. Tankönyvkiadó, Bp. 1972.
John Lamperti: Probability. A survey of the mathematical theory. Wiley, New York, 1996.
Richard Durrett: Probability: theory and examples. Duxbury Press, Belmont, CA, 1996.
Patrick Billingsley: Convergence of probability measures. Wiley, New York, 1999.

2. Martingales and limit theorems:
Martingale inequalities, stopping times, optional stopping. Submartingales, convergence theorem, predictable quadratic variation process (angle bracket process). Kolmogorov's 0-1 law, laws of large numbers, law of iterated iterated logarithm. Central limit theorem (global and local form). Theory of large deviations, Cramer's theorem.
Literature:
Richard Durrett: Probability: theory and examples. Duxbury Press, Belmont, CA, 1996.
David Williams: Probability with martingales. Cambridge University Press, Cambridge, 1991
Amir Dembo, Ofer Zeitouni: Large deviations techniques and applications. Springer-Verlag, New York, 1998

3. Random walks and Markov chains:
The problem of Recurrence vs transience, Pólya's theorems. Reflection principle and its applications, arcsin theorems. Potential theory. Classification of the states of a Markov chain, recurrence concepts (positive vs null). Stationary measure, ergodic theorems. Reversed Markov chain, reversibility. Renewal processes. Branching processes. Birth-death processes. Queueing theory. Markov fields, Gibbs measures, Ising model.
Literature:
William Feller: An Introduction to Probability Theory and its Applications I. Wiley, New York 1968.
Frank Spitzer: Principles of random walks. Springer, New York, 1976.
Samuel Karlin, Howard Taylor: Stochastic Processes. Gondolat Publishing, Bp. 1985

4. Stationary processes, ergodic theory:
L2 theory, spectral measure, spectral characterization. Gaussian processes, moving average, autoregressive, ARMA processes. Ornstein-Uhlenbeck process. Basic examples of ergodic theory: rotation of the circle, torus shift, baker's map, Arnold's CAT map, Gauss map. Neumann and Birkhoff-Khinchin ergodic theorems. Kingman's subadditive ergodic theorem. Ergodic and mixing maps, entropy theory (Kolmogorov-Sinai theorem). Symbolic dynamics, Bernoulli automorphisms.
Literature:
John Lamperti: Stochastic Processes – a Survey of the Mathematical Theory. Springer 1977.
Peter Walters: An introduction to ergodic theory. Springer-Verlag, New York- Berlin, 1982.
Marianna Bolla and Tamás Szabados. Multidimensional Stationary Time Series: Dimension Reduction and Prediction. Chapman and Hall/CRC, 2021.

5. Stochastic analysis:
Brownian motion: construction and basic properties. Donsker's (weak) invariance principle. Poisson process. Itô integral with respect to Brownian motion. Notable stochastic differential equations (Ornstein-Uhlenbeck process, Bessel processes). Diffusion processes. Cameron-Martin-Girsanov theorem, Feynman-Kac formula. Application in financial mathematics: Black-Scholes formula.
Literature:
Daniel Revuz, Marc Yor: Continuous martingales and Brownian motion. Third edition. Springer-Verlag, Berlin, 1999
Kai-Lai Chung, Ruth Williams: Introduction to Stochastic integration. Second Edition. Birkhäuser 1990.
H. P. McKean: Stochastic Integrals. Academic Press, New York, 1969

6. Information Theory:
Information measures, method of types. Source coding with fixed and variable-length codewords. Shannon code, Huffman code, arithmetic coding. Universal coding for memoryless, Markov, and finite-state sources; Lempel–Ziv coding and its variants. Coding theorems for memoryless channels, linear codes. Information-theoretic methods in statistics.
Literature:
Imre Csiszár and János Körner. 2015. Information Theory: Coding Theorems for Discrete Memoryless Systems (2nd. ed.). Cambridge University Press, USA.
Thomas Cover, Joy Thomas: Elements of information theory. John Wiley & Sons, Inc., New York, 1991.

7. Fundamentals of Statistics and estimation theory:
Empirical distribution, Glivenko–Cantelli theorem. Theory of ordered samples. Kolmogorov–Smirnov theorems. Weak convergence of empirical distribution functions to the Wiener bridge. Concepts of sufficient and complete statistics, Neyman–Fisher factorization, exponential families. Point estimates: unbiasedness, efficiency, consistency. Rao–Blackwell–Kolmogorov theorem, Cramér–Rao-type inequalities. Estimation methods: maximum likelihood principle, method of moments, Bayesian estimations and properties of these estimations. Interval estimates, properties of Student's t and chi-square distributions.
Literature:
Borovkov, A. A.: Mathematical Statistics. CRC Press, Boca Raton, FL, 1998.
Johnson, R.A.,Bhattacharyya, G.K.: Statistics. Principles and Methods. Wiley, New York, 1992.
Kendall, M.G., Stuart, A.: The Theory of Advanced Statistics I-II. Griffin, London, 1966.
Lehman, E. L.: Theory of Point Estimation. Wiley, New York, 1983.

8. Hypothesis testing:
Fundamental concepts of hypothesis testing, randomized tests. Neyman–Pearson lemma and its extensions for testing composite hypotheses in families with monotone likelihood ratio. Uniformly most powerful and unbiased tests, applications of large deviation theorems in statistics. One-sided and two-sided alternative hypotheses; likelihood ratio test, asymptotic distribution of test statistics. Optimality of classical parametric and nonparametric tests. Kolmogorov–Smirnov tests. Wald’s sequential analysis, Wald–Wolfowitz theorem.
Literature:
Borovkov, A. A.: Mathematical statistics. Typotex, Budapest, 1999.
Johnson, R.A., Bhattacharyya, G.K.: Statistics. Principles and Methods. Wiley, New York, 1992.
Kendall, M.G., Stuart, A.: The Theory of Advanced Statistics II-III. Griffin, London, 1966.
Lehman, E. L.: Testing Statistical Hypotheses. Wiley, New York, 1959.

9. Multivariate statistics:
Multivariate normal distribution, Wishart distribution. Maximum likelihood estimation of parameters of the multivariate normal distribution, hypothesis tests. Multivariate linear regression, linear models, linear estimators, Gauss–Markov theorem, statistical tests in linear models. Analysis of variance and covariance, Fisher–Cochran theorem. Principal component analysis, factor analysis, canonical correlation analysis. Classification methods, pattern recognition: cluster analysis, discriminant analysis. Multidimensional scaling. Analysis of contingency tables: correspondence analysis, log-linear models. Multivariate threshold models, probit and logit analysis, Kaplan–Meier estimator for censored data. EM algorithm for incomplete data, ACE (Alternating Conditional Expectation) algorithm for generalized regression tasks.
Literature:
Anderson, T.W.: An Introduction to Multivariate Statistical Analysis. Wiley, New York, 1949.
Lawley, D.N., Maxwell, A.E.: Factor Analysis as a Statistical Method. Butterworths, London, 1971.
Johnson, R.A., Wichern, D.W.: Applied Multivariate Statistical Analysis. Pearson, 6th edition, 2007.
Rao, C.R.: Linear Statistical Inference and Its Applications. Wiley, New York, 1965.
Bolla, M.: Lecture notes on Multivariate Mathematical Statistics. BME Mathematics Institute, Department of Stochastics, available on Prof. Bolla’s homepage.