Victoria Public Market in the former Hudson's Bay department store will close due to persistent high vacancies.