In cryptography, scalar multiplication with large numbers is a frequently used operation. It is usually implemented using a digit representation of the scalar. The choice of the representation influences the running time of this scalar multiplication. Thus it is one possibility to optimize cryptographic algorithms and increase the security level. More specifically, the running time of the multiplication depends on certain parameters of the digit representation. Such parameters can be the sum of digits, or the number of non-zero digits. To give precise estimates of the running time of an algorithm, a precise asymptotic analysis of these parameters of the chosen digit representation is of interest. For syntactically defined digit representations, these parameters can be computed as the output sum of finite state machines. In this thesis, the output sum of an arbitrary finite state machine is analyzed asymptotically. This includes expectation, variance, covariance and central limit theorem for a sequence defined as the output sum of a finite state machine. For the moments, not only the main terms are provided, but also the existence of a continuous, nowhere differentiable, periodic function as the second order term is proved and a formula for the coefficients of its Fourier series is given. This setting of sequences which can be defined as the output sum of finite state machines includes many well-studied sequences, like q-additive functions, digital sequences and the sum-of-digits function and the Hamming weight of syntactically defined digit representations. Thus, the results in this work generalize well known facts about the asymptotic behavior of all these sequences. Also, sequences defined by certain recursions fit into this setting. The output sum of a finite state machine is in general asymptotically normally distributed. However, this is not guaranteed: It may happen that the output sum degenerates. In this work, this case is characterized algebraically and combinatorially. It turns out that this degeneration only happens in trivial cases and the conditions can be checked easily by inspecting a representation of the finite state machine as a graph. These results are applied to asymptotically analyze several sequences in this thesis. This includes the sequence of carries occurring when performing addition of two numbers given in a syntactically defined digit representation. Furthermore, the number of iterations of von Neumann's addition algorithm is analyzed by using automata. However, this number of iterations is not asymptotically normally distributed, but converges to a double exponential distribution. All the steps to obtain an asymptotic analysis of a given sequence, starting with explicitly constructing the corresponding finite state machine and stopping finally with the computation of the Fourier coefficients, can be performed automatically in the mathematical software system Sage. To do this, the new finite state machine package, developed in the framework of this thesis, is presented. In the meanwhile, this package is included in the mathematical software system Sage. This package was developed to conveniently work with automata and transducers. Thus, all results of this thesis are implemented as methods of this finite state machine package and the examples are computed by using them accordingly.