Thanks for emphasizing this, and I agree I need to qualify my statement. I haven't completely understood the restrictions yet, but the effectiveness of this device has to be conditional on the fact that the IR laser is a laser, or otherwise is in some low-entropy coherent state. Otherwise, it could be used to transform part of a uniform IR bath into high-frequency photons, passively generating a temperature gradient from nothing.

The incoherent case is the relevant situation when you're talking about passive glasses for viewing a scene in ambient IR light.

Here's one place where this is mentioned:

http://physics.stackexchange.com/questions/98790/what-low-le...

Still looking for an authoritative treatment...