The problem is that remote solutions to the privacy problem inevitably involve my trusting someone else's privacy assurances. On-device processing reduces the trust required, although of course it doesn't eliminate it.
The common response to this is to point out that I'm trusting Apple or Google anyway just by using their phones, which is true, but (1) since phones are directly accessible to security researchers, there's more potential to find out about malfeasance if data is exfiltrated from the phone in a way that it shouldn't be, and (2) even if I have complete trust in Apple or Google, I also have to trust that they'll be able to protect my data from malicious actors, and, the less data I give them, the less I have to worry when they fail.
The common response to this is to point out that I'm trusting Apple or Google anyway just by using their phones, which is true, but (1) since phones are directly accessible to security researchers, there's more potential to find out about malfeasance if data is exfiltrated from the phone in a way that it shouldn't be, and (2) even if I have complete trust in Apple or Google, I also have to trust that they'll be able to protect my data from malicious actors, and, the less data I give them, the less I have to worry when they fail.