Of course numbers matter, if you train but don't eat enough calories you will lose more muscle than you had to start with. I understand what you're saying but you have to keep track.
More important to know your body than to know numbers, you cant have an app on a phone getting it right for 1000's of people, we are individuals with individual needs, use the numbers as a rough guide dont live by them