Active Appearance Models (AAM) are a well-established method for facial landmark detection and face tracking. Due to their widespread use, several additions to the original AAM algorithms have been proposed in recent years. Two previously proposed improvements that address different shortcomings are using robust statistics for occlusion handling and adding feature descriptors for improved landmark fitting performance. In this paper, we show that a combination of both methods is possible and provide a feasible and effective way to improve robustness and precision of the AAM fitting process. We describe how robust cost functions can be incorporated into the feature-based fitting procedure and evaluate our approach. We apply our method to the challenging 300-videos-in-the-wild dataset and show that our approach allows robust face tracking even under severe occlusions.