R, 32 bytes
function(x)split(x,(x-seq(x))^2)
(51 bytes if we want to include the empty lists for displacements that don't occur:
function(x)Map(function(n)x[abs(x-s)==n],s<-seq(x)), painfully long because of the double "function" keyword, although this could be replaced by "\" in R≥4.1 for 37 bytes)