Packages

c

com.spotify.scio.hash

ApproxFilterSCollection

implicit final class ApproxFilterSCollection[T] extends AnyVal

Source
package.scala
Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ApproxFilterSCollection
  2. AnyVal
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new ApproxFilterSCollection(self: SCollection[T])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    Any
  2. final def ##: Int
    Definition Classes
    Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    Any
  4. def asApproxFilter[C <: ApproxFilterCompanion](c: C, expectedInsertions: Long, fpp: Double)(implicit hash: asApproxFilter.C.Hash[T]): SCollection[asApproxFilter.C.Filter[T]]

    Creates an ApproxFilter from this SCollection with the expected number of insertions and expected false positive probability.

    Creates an ApproxFilter from this SCollection with the expected number of insertions and expected false positive probability.

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    c

    companion object of the ApproxFilter implementation, e.g. BloomFilter.

  5. def asApproxFilter[C <: ApproxFilterCompanion](c: C, expectedInsertions: Long)(implicit hash: asApproxFilter.C.Hash[T]): SCollection[asApproxFilter.C.Filter[T]]

    Creates an ApproxFilter from this SCollection with the expected number of insertions and default fpp of 0.03.

    Creates an ApproxFilter from this SCollection with the expected number of insertions and default fpp of 0.03.

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    c

    companion object of the ApproxFilter implementation, e.g. BloomFilter.

  6. def asApproxFilter[C <: ApproxFilterCompanion](c: C)(implicit hash: asApproxFilter.C.Hash[T]): SCollection[asApproxFilter.C.Filter[T]]

    Creates an ApproxFilter from this SCollection with the collection size as expectedInsertions and default fpp of 0.03.

    Creates an ApproxFilter from this SCollection with the collection size as expectedInsertions and default fpp of 0.03.

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    c

    companion object of the ApproxFilter implementation, e.g. BloomFilter.

  7. def asApproxFilterSideInput[C <: ApproxFilterCompanion](c: C, expectedInsertions: Long, fpp: Double)(implicit hash: asApproxFilterSideInput.C.Hash[T]): SideInput[asApproxFilterSideInput.C.Filter[T]]

    Creates a SideInput[ApproxFilter] from an SCollection with the expected number of insertions and expected false positive probability.

    Creates a SideInput[ApproxFilter] from an SCollection with the expected number of insertions and expected false positive probability.

    The expectedInsertions should be approximately the number of unique elements in the SCollection.

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    Since this results in one filter as a SideInput care should be taken that the size of the filter does not exceed the runner recommended max size of Side Inputs (100 MB for Dataflow) This implies that expectedInsertions should not exceed 112 Million with a fp of 0.03 on Dataflow.

  8. def asApproxFilterSideInput[C <: ApproxFilterCompanion](c: C, expectedInsertions: Long)(implicit hash: asApproxFilterSideInput.C.Hash[T]): SideInput[asApproxFilterSideInput.C.Filter[T]]

    Creates a SideInput[ApproxFilter] from an SCollection with the expected number of insertions and expected false positive probability.

    Creates a SideInput[ApproxFilter] from an SCollection with the expected number of insertions and expected false positive probability.

    The expectedInsertions should be approximately the number of unique elements in the SCollection.

    The default false positive probability is 0.03

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    Since this results in one filter as a SideInput care should be taken that the size of the filter does not exceed the runner recommended max size of Side Inputs (100 MB for Dataflow) This implies that expectedInsertions should not exceed 112 Million with a fp of 0.03 on Dataflow.

  9. def asApproxFilterSideInput[C <: ApproxFilterCompanion](c: C)(implicit hash: asApproxFilterSideInput.C.Hash[T]): SideInput[asApproxFilterSideInput.C.Filter[T]]

    Creates a SideInput[ApproxFilter] from an SCollection with the collection size as expectedInsertions and false positive probability of 0.03.

    Creates a SideInput[ApproxFilter] from an SCollection with the collection size as expectedInsertions and false positive probability of 0.03.

    Note that overflowing an ApproxFilter with significantly more elements than specified, will result in its saturation, and a sharp deterioration of its false positive probability.

    Since this results in one filter as a SideInput care should be taken that the size of the filter does not exceed the runner recommended max size of Side Inputs (100 MB for Dataflow) This implies that expectedInsertions should not exceed 112 Million with a fp of 0.03 on Dataflow.

  10. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  11. def getClass(): Class[_ <: AnyVal]
    Definition Classes
    AnyVal → Any
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. def toString(): String
    Definition Classes
    Any

Inherited from AnyVal

Inherited from Any

Ungrouped