If your girls are sporty, they might like the Grouse Grind. Maybe the Capilano suspension bridge? The aquarium and Stanley park are nice too. If you're there at the beginning of July, there's a huge Latin carnival called Carnaval del Sol. Not sure of the exact dates this year... Restaurants I like are Burgoo, Phnom Penh, Ramen Butcher, and there are lots of nice brewery/restaurants in Yaletown.
I don't know that I'd come over to the island just for Butchart Gardens, and if you only have 5 days in Vancouver, between the sights and the shopping you'll probably find lots to do.