We study two overdetermined elliptic boundary value problems on exterior domains (the complement of a ball and the complement of a solid cylinder in R-3 respectively). The Neumann condition is non-constant and involves the mean curvature of the boundary. We show there exists a family of bifurcation branches of domains which are small deformations of the complement of a ball and of the complement of a solid cylinder, respectively, and which support the solution of the overdetermined boundary value problem.