The widely adopted image matching approach remains dependent on exhaustive matching of local features across images. We challenge this and investigate enhanc- ing matching efficiency by not approximating nearest neighbors but using a hierarchical approach. We hypothesize that efficiently identifying sufficiently similar geometrically meaningful feature matches, rather than the most similar but geometrically random ones, can improve or maintain matching performance, with lower computational complexity. We propose a novel method named group-guided nearest neighbors, matching groups of features as one and then matching individual features across matched groups only. Inspired by concepts from hyperdimensional computing and group testing, the hierarchical pipeline reduces the time complexity of feature matching from n squared to n times the square root of n. Empirical results on homography and pose estimation indicate that our method outperforms the standard nearest neighbors algorithm and achieves the performance level of other methods. We formulate the proposed method as a general framework that offers a continuum of methods with varying levels of computational cost. Additionally, we intro- duce a linear-time matching algorithm which first tests memberships of the most distinct features to feature groups of the other image, then matches these distinct features only with the members of the matched groups. Experiments show that this algorithm performs better than linear-time adaptations of quadratic-time algorithms. We also propose techniques for generating better synthetic image pair datasets for homography estimation and faster evaluation of image matching pipelines. These contributions result in an image matching framework with efficient matchers, realistic datasets, and fast evaluation