Type Members

1.  final case class Threshold(value: Long) extends HotKeyMethod with Product with Serializable

   keys with more appearances than the threshold value will be considered hot.

   keys with more appearances than the threshold value will be considered hot. Some runners have inefficient GroupByKey implementation for groups with more than 10K values. Thus it is recommended to set the threshold value to below 10K, keep upper estimation error in mind. If you sample input via sampleFraction make sure to adjust threshold value accordingly.

2.  final case class TopN(value: Int) extends HotKeyMethod with Product with Serializable

   top N keys that appear most often are considered hot.

   top N keys that appear most often are considered hot. **Warning**: The effect is that a top-N CMS has an ordering bias (with regard to hot keys) when merging instances. This means merging hot keys across CMS instances may lead to incorrect, biased results: the outcome is biased by the order in which CMS instances / hot keys are being merged, with the rule of thumb being that the earlier a set of hot keys is being merged, the more likely is the end result biased towards these hot keys.

3.  final case class TopPercentage(value: Double) extends HotKeyMethod with Product with Serializable

   keys that appear more that the percentage * total will be considered hot.

   keys that appear more that the percentage * total will be considered hot. This also means that this parameter is an upper bound on the number of hot keys that will be tracked: the set of heavy hitters contains at most 1 / percentage elements. For example, if percentage=0.01, then at most 1 / 0.01 = 100 items (or if percentage=0.25, then at most 1 / 0.25 = 4 items) will be tracked/returned as hot keys. This parameter can thus control the memory footprint required for tracking top keys.