Surveys

I don't trust any quoted statistics by default because I've (anecdotally) realised that pretty much every study is sloppily done or misinterpreted when quoted. The only time I trust a statistical analysis is when I have personally looked into it and seen that they did things right. I'm not even a stats pro or anything, the bar is in hell and any AI model, when given sufficient data and reasonably good prompting can debunk things pretty well.The other day, I spoke to a data scientist who worked on surveys. He spoke about how a reputed survey firm they hired that's been around for over half a century that can't do the most basic things right. The kind of problems are unfathomably bad—the class of errors is stuff like asking the questions in a language that's not of a specific town, finishing the questions in impossibly fast times, not actually going to any household, etc.In this case, this person had to throw close to half of the data points away because they were suspect. This just…