I work on UX, coming from an engineering background. It means that everyday I work close to product management and engineering.
The trend over the last 10 years is to collect tons of data to improve the product. Some PMs and UXrs believe that they’ll get a magic insight from the data, and the skeptics do it anyways because is another data point to have. For engineering, services like GA are cheap and easy to integrate.
Nobody has a bad intention. But, we are distracted by the next product release to see the long term consequences for the society.
The reality is that some data is useful, but most of it is BS. To measure adoption and engagement you can do a pilot and then deactivate data collection. Big app errors are reported soon after a release, and you don’t need to continue collecting that for a long time.
To improve the UX you can do research with less data points, and smaller groups. The irony: I wish to have data to prove it, my hypothesis is based on my experience. I got more actionable insights from qualitative research, self-reported metrics, or quantitative data focused in certain aspects (instead of collecting all just in case). Some times having nice reports based on tons of data is more useful as an argument for corporate politics rather than to improve the product, but users doesn’t need to pay the consequences of your company stupidity (I’m looking at you MS telemetry ;-) )
There is a simple thing that we can do to change this trend. Ask yourself: What is the goal of collecting the data? What product hypothesis you want to prove? Can you get insights from a small group?
If you don’t know.... hold on your data collection desires.
For those cases the app can collect the exceptions only (as many apps and OS do).
I worked on a desktop product with this type of data collection. Usually what happens is that after a new release you may see new errors coming up, and then they start to repeat. The data collection becomes a burden, new reports of the same error type doesn’t give you more information.
It’s a good opportunity for a good UX, e.g point the user to the relevant support info to solve the problem.
For support cases you may be able to ask for diagnostics on demand. The app can collect it internally without sharing and send part of it when an exception occurs and the user accepts to send it.
The trend over the last 10 years is to collect tons of data to improve the product. Some PMs and UXrs believe that they’ll get a magic insight from the data, and the skeptics do it anyways because is another data point to have. For engineering, services like GA are cheap and easy to integrate.
Nobody has a bad intention. But, we are distracted by the next product release to see the long term consequences for the society.
The reality is that some data is useful, but most of it is BS. To measure adoption and engagement you can do a pilot and then deactivate data collection. Big app errors are reported soon after a release, and you don’t need to continue collecting that for a long time.
To improve the UX you can do research with less data points, and smaller groups. The irony: I wish to have data to prove it, my hypothesis is based on my experience. I got more actionable insights from qualitative research, self-reported metrics, or quantitative data focused in certain aspects (instead of collecting all just in case). Some times having nice reports based on tons of data is more useful as an argument for corporate politics rather than to improve the product, but users doesn’t need to pay the consequences of your company stupidity (I’m looking at you MS telemetry ;-) )
There is a simple thing that we can do to change this trend. Ask yourself: What is the goal of collecting the data? What product hypothesis you want to prove? Can you get insights from a small group? If you don’t know.... hold on your data collection desires.