candy man (plural candy men) (chiefly US, euphemistic, slang) A drug dealer. quotations % (dated) A person who sells sweets, especially a sidewalk or street vendor.

The Candyman of the song is a gambler, a drinker, a musician, and a ladies' man that much is certain. He is also, likely, from context and from the traditional use of the moniker, a drug dealer. So he is on the wrong side of the law, as is the case with many of the narrator characters in Grateful Dead songs.