Wi-Fi based sensing becomes more and more popular in ubiquitous computing with prevalence of Wi-Fi devices. In this paper, we propose a new task, indoor trajectory sensing, based on Received Signal Strength Indicator (RSSI) of Wi-Fi. Traditional distance measure based methods, like Dynamic Time Warping (DTW) based Nearest-Neighbor (1NN) method, have poor performance in indoor trajectory sensing due to that RSSI of Wi-Fi in indoor environment is fluctuating, partially missing, time-varying and device-dependent. Recently, Recurrent Neural Networks (RNN) and its variants have strong abilities in learning the temporal dependency of sequence data since it can extract more meaningful features. Consequently, it is necessary to design an RNN model for the indoor trajectory sensing problem with relatively small size data. We adopt a passive way to collect Wi-Fi signals from the smart phone to ensure more data collected and generate multiple time series for each trajectory. For the recurrent neural network training, RNN and its variants are applied into our sequence data to find more meaningful patterns especially for different environment and devices. Series of real-world experiments have been conducted in our test bed and the results show that the deep based approach can achieve better performance than traditional methods with challenging environment and device factors.