I'm amazed my last neighbours didn't have something to say. 4 years of noise and my garage wall joined on to their living room.
Now I've moved my garage only connects to my house, and next doors garage, their house is a bit higher, and they've never said anything.
They're also my landlords so, if I was causing problems I'm sure they'd say.
To the OP. when you see any of your neighbours apologise if you've been annoying them with the noise, the person who comes across as awkwardly saying, "oh you don't" is the person who reported it.