I have recently been experimenting with unicode characters on this blog. You would have noticed them next to links, at the end of the footnotes and very recently, the bullet icon for the label under the title. I use three glyphs at the moment, the outbound arrow (↗) to symbolise external links that open in a new window/tab, the hooked left arrow (↩) for the link back to the text and the star (★) which just looks good next to the labels. If you can't see them and/or the characters in the brackets, you should get a browser that does. The perfect option is Firefox which has really good support for unicode characters.
I took some inspiration from John Gruber's Daring Fireball for my choice of characters (the hooked arrow and star), and realised that they actually work better than little icons which people go to the first chance they get. The main advantage is that modern browsers can render it choosing a font for unicode characters quite easily, which means you don't 'need' icons when a good looking character will do! There tons of glyphs to choose from. You could refer to the list at Wikibooks for a complete chart of the characters at your disposal (there are many, you'll need patience).
The occasional problem of cross browser rendering will surface. Every browser chooses different fonts to show their unicode characters. Which is why it is good to check how certain characters are rendered. One workaround for any problem could be to specify your own list of fonts to try and pick for a unicode character containing element. On this blog, I use Tahoma as the main font, with a font size of '7.5em' I've been told off for this many times! I just can't get myself to have a bigger font size! It just looks too ugly to me!, but when a unicode character is rendered at that size, it becomes too small. Hence, I've had to manually increase the font size for them to '8.5em'. 'Tahoma' is an almost complete typeset, and a modern browser is mostly able to 'fill-in' empty characters by finding a font which has the characters. You will have to do some testing to see which one works for you. You can do this by using 'class > element' selector. This will not work in Internet Explorer, but the same goes for the character itself. I doubt unicode characters show up at all in Internet Explorer. My solution for that is to get a 'real' browser! As simple as that!
As a designer, it helps me remove unwanted graphics from the page, and keep things simple. Especially if one uses a monochrome colour combination for the text (white and dark gray for this blog), it works best to keep it looking uniform and clean! Go ahead, give it a whirl!