In this paper, we propose a new approach for object matching in videos. By applying our novel descriptor on points of interest we obtain point descriptors or signatures. Points of interest are extracted from the object using a simple color Harris detector. This novel descriptor is issued from a rotating filtering stage. The rotating filtering stage is made of oriented anisotropic half-Gaussian smoothing convolution kernels. Further, the dimension of our descriptor can be controlled by varying the angle of the rotating filter by small steps. Our descriptor with a dimension as small as 36 can give a matching performance similar to that of the well-known SIFT descriptor. The small dimension of our descriptor is the main motivation for extending the matching process to videos. By construction, our descriptor is not Euclidean invariant, hence we achieve Euclidean invariance by FFT correlation between the two signatures. Moderate deformation invariance is achieved using Dynamic Time Warping (DTW). Then, using a cascade verification scheme we improve the robustness of our matching method. Eventually, our method is illumination invariant, rotation invariant, moderately deformation invariant and partially scale invariant.