Water reaches earth via precipitation. Before it hits the ground, it is very soft: hardness of rainwater is close to zero.
Then it seeps away and finally collects as ground water. On its way through different layers of earth and rock, it dissolves various mineral salts. The composition of these rock layers determines the hardness of the water. If a large amount of calcium and magnesium salts gets into the water, it becomes hard. This is particularly the case in areas with a lot of limestone or chalk.