Let M be a complete Riemannian manifold and F⊂M a set with a nonempty interior. For every x∈M , let Dx denote the function on F×F defined by Dx(y,z)=d(x,y)−d(x,z) where d is the geodesic distance in M. The map x↦Dx from M to the space of continuous functions on F×F , denoted by DF , is called a distance difference representation of M. This representation, introduced recently by Lassas and Saksala, is motivated by geophysical imaging among other things. We prove that the distance difference representation DF is a locally bi-Lipschitz homeomorphism onto its image DF(M) and that for every open set U⊂M the set DF(U) uniquely determines the Riemannian metric on U. Furthermore the determination of M from DF(M) is stable if M has a priori bounds on its diameter, curvature, and injectivity radius. This extends and strengthens earlier results by Lassas and Saksala.