China's carbon peak and neutrality targets make higher demands for building energy efficiency and the building envelope, as a bridge connecting indoor and outdoor, plays an important role in building energy consumption. It is of great significance to study the thermophysical properties of building envelopes to reduce this energy consumption. In this paper, employing inverse problem thought and particle swarm optimization algorithm, the ideal thermal conductivity distribution and energy saving potential of seven representative cities in five climatic regions in China were calculated. The results showed: (1) the ideal thermal conductivity distributions in different cities were different, but most of them reached the maximum value at 18~24°C; (2) the thermal conductivity in various temperature ranges made a huge difference in the influence of the annual energy consumption of a room; (3) applying the ideal thermal conductivity, Guangzhou had the highest energy saving in refrigeration. Urumqi had the highest energy saving in heating and total energy consumption; (4) Urumqi had the largest energy saving potential, and Kunming had the smallest energy saving potential.