SOCIETY
Consequences of inappropriate detection and removal of outliers in statistics
In statistics, the presence of outliers in the data set could wrongly distort the estimation of the mean. In addition, the extreme values increase the variability and consequently, the power of the statistical methods decreases. However, there are disagreements in the literature both about what the nature of outliers is, and about how to deal with them when doing further statistical analyses. A lot of conventional procedures for both detecting and dealing with outliers are discussed. The effect of increase the probability of error of the first order type is demonstrated with two simple simulations. The general conclusion is that an information outside of the data set is necessary for a correct decision. This information could come only from the human expertise of the researchers of the specific domain of in terest. The importance of the topic for outliers is discussed, the need of deeper analyses, accompanied with many simulation studies, is argued.