Unfortunately I am only a regular visitor at the airport, but Confiserie Spruengli is always worth a visit. Pretty much anything they make is first rate, with the truffes du jour a definite must.
Look for the pockets of not-so-elusive Swiss junkies hanging out in various corners of the city. More interesting IMO than taunting the Forellen in the Limmat.