It's really more about signal-to-noise ratio I'd say, if there are often loud noises then your brain will have to attend to them, if there is constantly a fan noise, for example, then your brain will actually focus since there is no signal--but I've definitely found it's best to get rid of both in almost all domains.