In this paper, we attempt to tackle the MediaEval 2013 Retrieving Diverse Social Images challenge, which is a filter and refinement problem on a Flickr-based ranked set of social images. We developed three different approaches, using visual data, textual data and a combination thereof, respectively. Hierarchical clustering on highly relevant images, combined with a greedy approach to complement the ranking, forms the basis of our approach.