Yes, this is often a problem, as people on flickr will tag their photos rather loosely...

Try and be a bit more specific in your tags, putting several separated by commas.

http://flickholdr.com/250/185/happy,office for example, might be more appropriate.

Disclaimer: I'm the developper behind flickholdr