In the 60's 70's and 80's nobody really gave a rats arse about celebrating Australia Day.
It didn't really become popular until the neo-liberals worked out that it was an excellent means by which to divide and conquer the population through good old fashioned Nazi style nationalism.
Enter John Winston Howard - he loved Australia Day as it allowed him to promote his white supremacist ideology.
Massive push for Australia Day from 1996 on wards. Plutocratic owned media machine also got on board as well to promote the very worst of Australia against minorities.

When should we celebrate Australia Day ? Never. It's nationalistic white supremacist bullshit and the sooner it's forgotten the better.