The defect rate is not 4 times as high, it is actually higher than that. If there is some probability x that a single pixel is not defective, the defect-free probability for a display is x^(width * height).

If we choose x to be 99.99999% that a single pixel is not defective, the probability of generating a defect-free 1080p (19201080) display is 81.27%, and the probability of generating a defect-free 4k (38402160) display is only 43.63%. This means that in order to produce one defect-free 1080p display, on average 1.23 displays need to be produced, and on average to produce one defect-free 4k display 2.29 displays must be produced. In pixels, this translates to 2.55e6 and 1.90e7 respectively.

The difference between the two is the number of pixels comprising 7.9377 1080p displays, about 6.4 of which would be expected to be defect free. This should indicate that the raw cost of a defect-free 4k display should be at least 6.4 times the raw cost of a defect-free 1080p display, assuming that a full display is manufactured all-at-once, and there is no interim testing to discard bad panels early, and that the cost-per-pixel of manufacture is a constant. In reality, I would expect the defect-tolerance and the pre-defect cost per pixel of the 4k display to be higher than a 1080p display. There probably is a small early-adopter penalty, but the cost isn't very out of line.