Your First Moment Earns You More Moments