It comes up every now and then, and, although all the descriptions get the same result, the rules are more complex than they need to be.
If you are counting objects, use “fewer”. If you are measuring it (by eye, ruler, scale, balance, measuring cup, micrometer…) use “less”.
There is less sand in pile A. We are measuring by eye.
There are fewer grains of sand in pile A. Yes, you are an idiot to count each grain, but you could.
The same goes for time and distance.
There are fewer hours in an Earth day than a day on Titan. There is less time in an Earth day.
It takes less time to drive to the store than to the cottage. There is less distance, but fewer miles.