Ahmad won 34 of the 54 votes in the election, which was held in Ethiopia‘s capital Addis Ababa on Thursday.
The result means a change in leadership for the first time since Cameroonian Hayatou took charge in 1988.
Ahmad, who appeared very emotional, becomes only the seventh Caf president in the body’s 60-year history.
The 57-year-old, who goes by a single name, told BBC Sport afterwards: “I can’t talk at this time. Only one thing – I thank God, I thank my team.
“We worked hard but we won. That was the first step. The second step is to develop African football.
“Some days I thought I would win – today I didn’t (think I would win).”
The result was greeted by cheers at the Caf congress as Ahmad was carried on the shoulders of jubilant supporters to the podium after the result was announced.
In contrast, Hayatou was led from the auditorium by aides but he told reporters as he departed: “It is not that bad.”
Read more at: BBC Sport