Low appearance value is eliminated by evolution, and adults with high appearance value choose the result
Recently, a paper in progress of science pointed out that the gene named baz1b is related to facial development, which supports the human 'self domestication' hypothesis. The hypothesis is that our ancestors would choose people with less aggressive faces and other features as partners, so they look milder than ancient humans in appearance.