Nonlinear Mixed Effects (NLME) modelling has for a long time been used for modelling individuals of a population that behave by the same qualitative mechanisms but with quantitative differences between individuals. This kind of modelling increases for example knowledge on the effects of drugs on the body, which is important in drug development when deciding dosing regiments. An important part of NLME modelling involves estimating the parameters of the model for a given dataset. This is often done using gradient based methods, where the gradients are traditionally approximated using finite differences. This approximation might cause longer execution times and numerical problems leading to failure at estimating parameters. This thesis investigates the robustness of parameter estimation using the method S-FOCE, where gradients of the optimisation algorithm are computed exactly instead of using numerical approximations. This is done for NLME models with deterministic dynamics. The comparison was performed by estimating parameters of simulated data from pharmacokinetic and pharmacodynamic models using both a parameter estimation program that uses the exact gradients and an industry standard parameter estimation software, NONMEM, that partly uses finite difference approximations to compute the gradients. This thesis also shows how the S-FOCE method could be extended for a general NLME model with stochastic dynamics. This involves deriving the first and second order sensitivities of the Extended Kalman Filter. The results show that for a simple model with no failure in parameter estimation using finite differences, the S-FOCE method performs equally as well. However, models where the finite difference method had lower success frequency, the S-FOCE method suggests significant improvement in robustness in terms of the frequency of successful estimates of parameters and their uncertainties as well as in terms of the quality of the uncertainty estimates.