Ivic, who has signed a contract for a year and a half, took charge at Watford in August 2020 in the English second tier following relegation from the Premier League in the previous season, but was relieved of his duties four months later.
Jardim, whose contract with Al Ain was due to expire at the end of the revamped Club World Cup in July this year, joined the club in November following spells in Saudi Arabia and Qatar.
The decision to change the coach came a day after the holders' 2-1 defeat by Al Rayyan in the Asian Champions League, which meant they crashed out of the competition without registering a single win.
The team also had only one win from their last five league games.
Al Ain have two points from seven games in the Asian Champions League and are fourth in the UAE Pro League standings after 14 games.