In recent years, user personality has been recognized as valuable info to build more personalized recommender systems. However, the effort of explicitly acquiring users’ personality traits via psychological questionnaire is unavoidably high, which may impede the application of personality-based recommenders in real life. In this paper, we focus on deriving users’ personality from their implicit behavior in movie domain and hence enabling the generation of recommendations without involving users’ efforts. Concretely, we identify a set of behavioral features through experimental validation, and develop inference model based on Gaussian Process to unify these features for determining users’ big-five personality traits. We then test the model in a collaborative filtering based recommending framework on two real-life movie datasets, which demonstrates that our implicit personality based recommending algorithm significantly outperforms related methods in terms of both rating prediction and ranking accuracy. The experimental results point out an effective solution to boost the applicability of personality-based recommender systems in online environment.