trait PipelineTestUtils extends AnyRef
Trait with utility methods for unit testing pipelines.
- Alphabetic
- By Inheritance
- PipelineTestUtils
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def runWithContext[T](fn: (ScioContext) => T): ScioExecutionContext
Test pipeline components with a ScioContext.
Test pipeline components with a ScioContext.
- fn
code that tests the components and verifies the result
runWithContext { sc => sc.parallelize(Seq(1, 2, 3)).sum should containSingleValue (6) }
- def runWithData[T1, T2, T3, T4, U](data1: Iterable[T1], data2: Iterable[T2], data3: Iterable[T3], data4: Iterable[T4])(fn: (SCollection[T1], SCollection[T2], SCollection[T3], SCollection[T4]) => SCollection[U])(implicit arg0: Coder[T1], arg1: Coder[T2], arg2: Coder[T3], arg3: Coder[T4], arg4: Coder[U]): Seq[U]
Test pipeline components with in-memory data.
Test pipeline components with in-memory data.
Input data is passed to
fn
as SCollection s and the result SCollection fromfn
is extracted and to be verified.- data1
input data
- data2
input data
- data3
input data
- data4
input data
- fn
transform to be tested
- returns
output data
- def runWithData[T1, T2, T3, U](data1: Iterable[T1], data2: Iterable[T2], data3: Iterable[T3])(fn: (SCollection[T1], SCollection[T2], SCollection[T3]) => SCollection[U])(implicit arg0: Coder[T1], arg1: Coder[T2], arg2: Coder[T3], arg3: Coder[U]): Seq[U]
Test pipeline components with in-memory data.
Test pipeline components with in-memory data.
Input data is passed to
fn
as SCollection s and the result SCollection fromfn
is extracted and to be verified.- data1
input data
- data2
input data
- data3
input data
- fn
transform to be tested
- returns
output data
- def runWithData[T1, T2, U](data1: Iterable[T1], data2: Iterable[T2])(fn: (SCollection[T1], SCollection[T2]) => SCollection[U])(implicit arg0: Coder[T1], arg1: Coder[T2], arg2: Coder[U]): Seq[U]
Test pipeline components with in-memory data.
Test pipeline components with in-memory data.
Input data is passed to
fn
as SCollection s and the result SCollection fromfn
is extracted and to be verified.- data1
input data
- data2
input data
- fn
transform to be tested
- returns
output data
- def runWithData[T, U](data: Iterable[T])(fn: (SCollection[T]) => SCollection[U])(implicit arg0: Coder[T], arg1: Coder[U]): Seq[U]
Test pipeline components with in-memory data.
Test pipeline components with in-memory data.
Input data is passed to
fn
as an SCollection and the result SCollection fromfn
is extracted and to be verified.- data
input data
- fn
transform to be tested
- returns
output data
runWithData(Seq(1, 2, 3)) { p => p.sum } shouldBe Seq(6)
- def runWithLocalOutput[U](fn: (ScioContext) => SCollection[U]): (ScioResult, Seq[U])
Test pipeline components with a ScioContext and materialized resulting collection.
Test pipeline components with a ScioContext and materialized resulting collection.
The result SCollection from
fn
is extracted and to be verified.- fn
transform to be tested
- returns
a tuple containing the ScioResult and the materialized result of fn as a Seq
- def runWithRealContext[T](options: PipelineOptions)(fn: (ScioContext) => T): ScioExecutionContext
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()