We show that the motive of the Hilbert scheme of length-$n$ subschemes on a K3 surface or on an abelian surface admits a decomposition similar to the decomposition of the motive of an abelian variety obtained by Shermenev, Beauville, and Deninger and Murre.