I like the advantages of earth-sheltered construction and I don't want to build any other way. There are some disadvantages, however, and they should be considered.
In order to take advantage of the sun's heat and to mitigate the effect of cold winter winds, all the glass should be on the south face of the building. The window overhangs should be designed so that the inside is shaded in summer and gets full sun in winter. The problem here is that the best scenery and view might not be to the south. Also, building on the north side of hilly land might not give you enough sunlight. And the constraints of efficient passive solar design definitely restricts the architectural design. The WOW factor can become BOW WOW.
Most people have programmable thermostats to control temperatures in their homes. Above-ground homes are subject to wide swings in temperature, but underground homes keep a constant temperature because of the thermal mass of the structure. That flywheel effect makes it very difficult to change the inside temperature of an underground house. So it's best to pick one temperature and stick with it summer and winter.
The biggest disadvantage of earth-sheltered construction is cost. It is THE reason why above-ground homes are still built. As a rough estimate, I'd say that underground homes cost 50% more than above-ground homes. The beefy structure is one reason, but the biggest wallet-shocking surprise comes from excavation. Moving dirt is hugely expensive.
Oh yeah, and cell phones don't work underground. I have to walk outside to use mine.