For years, mainlanders dismissed the city on Vancouver Island’s south coast as a picturesque haven for honeymooners and retirees, “the newlywed and the nearly dead.” But lately, Victoria has developed a hipper side. Tech companies now occupy brick warehouses, craft distilleries share streetfronts with traditional tea rooms, and marijuana dispensaries are popping up among the old Edwardian mansions. At times it feels like a Portland North, set amid the architectural glories of a one-time imperial outpost: a mashup of traditional and alternative, with a sneak-up-sideways charm.

The historic heart of Victoria’s walkable downtown is Bastion Square, where the city was born as a Hudson’s Bay Company trading post in 1843. Now lined with pubs and home to a popular Sunday farmers market, the square’s oldest buildings date to the 1860s, after the discovery of gold on the Fraser River transformed little Fort Victoria into a boomtown thronged by American miners and outfitters.

Prepping at Agrius. PHOTO: RUSH JAGOE

The Cantonese migrants who followed them from San Francisco founded Canada’s first Chinatown, and on Fisgard Street, the neon signs of the Don Mee Restaurant and the Fantan Café continue to lure patrons to old-school feasts of Szechuan seafood and sweet-and-sour pork. Fan Tan Alley, whose three-foot-wide entrance forced the local constabulary to enter single file when raiding its louche gambling and opium dens, now houses vendors of used records, handmade chocolates and Dr. Martens boots.

The streets around Lower Johnson, a strip where prospectors once spent their gold in saloons, brothels, and outfitters’ shops, have undergone a similar transformation. In the district, redubbed LoJo, brick facades are now home to chic boutiques and specialty shops like Silk Road Tea, a mix of day spa and high-end tea room. Off Yates Street, Little Jumbo restaurant channels the district’s past with a down-the-hall entrance and speakeasy vibe, where an aperol-and-absinthe cocktail makes a bracing prelude to delicate local oysters and lightly grilled lingcod.

Change has come even to the venerable Fairmont Empress hotel. After a two-year, $40-million-plus renovation, the trademark ivy has been stripped from the facade—the family of raccoons who called it home had to be relocated—transforming its former flag deck into a terrace with a privileged harbor view. A rooftop garden now yields the herbs and edible flowers on the menu at Q at the Empress, and four beehives on the grounds hone the honey served with scones and clotted cream at high tea.

There is still plenty of old Victoria to savor. The hotel’s unapologetically colonialist Bengal Lounge has been left untouched: punkah fans still sway from the mahogany-inlaid ceiling over murals of dhows and elephant-borne rajahs on the Ganges. And the corridors of the sixth floor—where guests continue to report sightings of the ghost of a chambermaid who plunged to her death in the 1930s—are still as crooked and atmospheric as ever.

The Milkman’s Daughter, a décor shop in Victoria’s Chinatown. PHOTO: JANIS NICOLAY

By contrast, Victoria’s alternative side thrives in the funky Fernwood neighborhood east of downtown. In the last decade, neighborhood associations have spearheaded the transformation of the buildings around Fernwood Square. An old Methodist church became the multi-staged Belfry Theatre; the Fernwood Inn, a former dive bar, was reborn as a pub serving local ales and ciders. The once-rundown building across from the Inn now houses the Crossroads, an espresso bar that has become an unofficial community meeting place, and the relaxed Stage Wine Bar, where one can dine on small plates of cauliflower pakora, Salt Spring Island mussels, and gnocchi.

A stroll in the surrounding streets takes visitors past book exchange boxes outside century-old Arts-and-Crafts bungalows, a community garden on the former grounds of a schoolhouse, and an ice cream parlor called Cold Comfort offering “Hoyne’s Dark Matter,” an improbable but successful combination of vanilla ice cream and brown ale.

A 10-minute walk south of downtown is the tranquil James Bay neighborhood, site of the childhood home of Emily Carr. It has been turned into a museum commemorating the life of the author and painter, who returned from a trip to France to create striking, Post-Impressionist takes on northern Pacific landscapes. View Carr’s deeply Canadian canvases of listing Haida and Gitxsan totem poles in the nearby Royal B.C. Museum.

In the Rockland neighborhood, perched on the highest point within the city limits is Craigdarroch, a late-19th-century castle built for Scottish railway-and-coal baron Robert Dunsmuir. The Downton Abbey-style splendor of its parlors, paneled with Spanish mahogany and Hawaiian koa, give way on upper floors to unadorned walls and bare floor boards. Dubbed “King Grab” by a local newspaper, which cast him as more robber baron than genuine royalty, Mr. Dunsmuir died a few months before the castle was completed, and never lived there.

The fifth-floor turret, with a view of ocean breakers and snow-capped Mount Baker, looks out onto what is probably Victoria’s greatest urban asset: the tip of Vancouver Island on which it perches. Its location in the rain shadow of the Olympic Mountains means sunny and dry summers (windmill palm trees grow in local front yards), and access to spectacular north Pacific ecosystems start right from downtown.

Victoria Distillers’s gin-based Empress & Tonic.

An excellent way to explore those outdoor wonders is by rented bicycle. Heading west out of downtown on the Galloping Goose and the Lochside Trail, well-marked and level bike routes follow old railway right-of-ways. Along the way the outdoor terrace of Sea Cider provides a stop for a light lunch, with high-octane ciders produced from the 50 kinds of heirloom apples that grow in the surrounding orchards. Farther down the end of the trail is Victoria Distillers, a craft distillery in Sidney, 15 miles north of downtown. A tasting of Empress 1908 gin, infused with dried flowers from Thailand that make it turn from deep indigo to pink when tonic is added, is a welcome reward for a long afternoon of pedaling.