A below-par Millers went down 2-1 to leave them trailing league leaders West Didsbury & Chorlton by three points.
But the joint Millers boss knows there is a long way to go just yet.
“There are some good teams up there who can beat each other,” said Birch. “There is half a season to go and the Isle and Man game doesn't define our season.
“It is three points and everyone still has to pay each other. There is a lot of football to be played and there will be ups and downs to come.”
Buxton manager Jamie Vermiglio happy with opening day point at Peterborough Sports
Rebuild will take time says New Mills joint manager Dave Birch after defeat double
Buxton are held by 10-men Peterborough Sports in historic first game at National League North level
Buxton beef up with new quartet - and more could be on the way
GET ACTIVE COLUMN: Advice on muscle spasms
Birch was left disappointed by the narrow defeat on the island after his side had impressed in the match before in a 1-0 win at Abbey Hulton United.
“We went to Abbey Hulton with players missing and put in a fantastic performance,” he said.
“The lads that came in did great and it was a real battle. It should have been far more comfortable in the end.
“We are disappointed we couldn't build on it.
“We just didn't perform on the day, it's as black and white as that.
“We had more than enough to go there and get three points, but we didn’t turn up.
“We gave them two goals and just didn't get going - the mistakes cost us. We had too many lads that had off days and the best team won to be honest.”