Change analysis

We compare measurements at two different times for each attribute. Then, we calculate the upper and lower thresholds based on the most significant absolute changes between these two timestamps. Any change values lying outside the range of threshold values are marked as outliers.

We determine the upper and lower thresholds by looking at the top ten absolute changes. If the combined contribution of a measure at the first or second timestamp exceeds 50%, we stop iterating. The largest negative change becomes the lower threshold, while the smallest positive change becomes the upper threshold.

In the example above, based on the absolute change values, the upper threshold is calculated as 708.09K, and the lower threshold value is calculated as -606.58K. Attribute values lying on or outside the threshold values are marked as outliers.

While working with rations, ThoughtSpot can’t perform the difference analysis as with the Type 1 algorithm. This limitation arises because ratio values become exceedingly large whenever the denominator approaches zero. To address this concern, we adopt an alternative approach: for each attribute value, we compute a hypothetical percentage change. This calculation determines the overall percentage change in the measured value assuming that specific value remained constant between the two reference points. A smaller hypothetical percentage change for an attribute value indicates that this value significantly contributes to explaining the overall change. This is because if that attribute had not changed, the overall change in the measure would have been smaller.

For example, if we do change analysis on the average discount monthly for July 1998 and August 1998, the insight says that "burnished(Color)" is a significant change contributor. If the average discount for burnished(Color) had not changed, the increase would have been 0.56% instead of 7.23%. This means that the overall average profit increased by 7.23% from July 1998 to August 1998, but if we disregard the change in the average discount of burnished(Color), the overall average profit would have increased by just 0.56%.

We start by comparing measurements at two different times for each attribute. Next, we treat the changes as a normal distribution and calculate the values of upper and lower thresholds using the Z-score calculation. Any change values lying outside the range of threshold values are marked as outliers.

A Z-score represents the number of standard deviations from the mean a data point is. Z-scores are a way to compare results from a test to a "normal" population. The basic Z-score formula for a sample is: z = (x – μ) / σ where σ is the standard deviation, μ is the mean, x is the data value. Upper threshold is assigned as μ + N * σ and lower threshold is assigned as μ - N * σ. So any value with a Z-score less than or greater than N is an anomaly.

The minimum value of N is 2.0, which covers 95.45% of points in a normal distribution. The maximum is 5.0, which covers 99.99994% of points in a normal distribution. The value of N is decided according to the table provided below:

In the example above, based on the absolute change values, the upper threshold is calculated as 1.29 and the lower threshold is calculated as -3.26. Attribute values lying on or outside the threshold values are marked as outliers.