Machine learning dark matter halo formation