This dissertation is focused on the theory and applications of central limit theorems (CLTs) and non-central limit theorems (NCLTs). Part I has preliminary character. In Part II, we deal with the limit behavior of random sums. We prove a NCLT and rates of convergence. In this part, our primary emphasis is the new method of proof we propose. In Part III, we study the limit behavior of semimartingales for small times. We prove CLTs and extend these to functional CLTs on the process level. Subsequently, we show applications in mathematical finance. Part II and Part III are independent of each other. Part I, Preliminaries: In Chapter 1, which is the foundation of Part II and Part III, we recapitulate classical CLTs. In Chapters 2 and 3, we introduce the reader to the rudiments of Stein's method and review parts of the asymptotic theory of random sums. Chapter 4 deals with the concept of size biasing. The knowledge of Chapters 2 to 4 is essential for Part II. Chapter 5 motivates the research which is carried out in Part II. In Chapter 6 we change the subject and give a heuristic argument in favor of a smalltime CLT for a class of continuous semimartingales. Chapter 7 seizes this idea and further motivates the research in Part III. Part II, Analysis of Poisson Mixture Sums via Stein's Method: By using Stein's method, we study the Wasserstein, as well as the Kolmogorov distances of Poisson mixture sums and their limit distributions. The primary focus is laid on how Stein's method is applied. By stochastic conditioning, it is possible to work with Stein's equation of the Gaussian distribution instead of a more complex Stein equation. Part III, Small-Time Central Limit Theorems for Semimartingales: We prove a CLT, as well as a functional CLT on the process level for continuous semimartingales for small times. These results are extended to semimartingales with jumps. As an application to mathematical finance, we discuss the pricing of at-the-money digital options with short maturities and the asymptotics of at-the-money short time implied volatility skews.