In this thesis, a robust visual tracking method is proposed by casting tracking as an estimation problem in a joint space of non-rigid appearance model and state (JISAT). Accuracy of conventional trackers using non-rigid appearance models has been degraded due to incorrectly labeled local models. To alleviate this problem, multiple non-rigid appearance models were proposed and the probabilities from them are effectively marginalized by employing Particle Markov chain Monte Carlo (PMCMC). PMCMC draws samples from joint space of state and appearance models so that achieves exact approximation of joint density and estimate an accurate state. It also provides theoretical evidences of exact approximation and efficiency using marginalization. In the sampling step, an appearance model plays a role of the particle in PMCMC and represents a specific segmentation result combining multiple regression results from local regressors of different types of features. The proposed method was compared to nine state-of-the-art trackers in various sequences and analyzed both qualitatively and quantitatively.