package testing
- Source
- package.scala
- Alphabetic
- By Inheritance
- testing
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- trait BigtableMatchers extends SCollectionMatchers
Trait with ScalaTest Matcher s for SCollection s specific to Bigtable output.
- case class DistCacheIO[T](uri: String) extends Product with Serializable
- trait EqInstances extends FallbackEqInstances
- class EqualNamePTransformMatcher extends PTransformMatcher
Matches a PTransform with exactly name
name
. - sealed trait FallbackEqInstances extends AnyRef
- trait PipelineSpec extends AnyFlatSpec with Matchers with SCollectionMatchers with PipelineTestUtils with RunEnforcementJobTest
Trait for unit testing pipelines.
Trait for unit testing pipelines.
A simple test might look like this:
class SimplePipelineTest extends PipelineSpec { "A simple pipeline" should "sum integers" in { runWithContext { sc => sc.parallelize(Seq(1, 2, 3)).sum should containSingleValue (6) } } }
- trait PipelineTestUtils extends AnyRef
Trait with utility methods for unit testing pipelines.
- trait RunEnforcementJobTest extends AnyFlatSpec
Trait that enforces JobTest.Builder.run is called.
- trait SCollectionMatchers extends EqInstances
Trait with ScalaTest Matcher s for SCollection s.
- trait ScioIOSpec extends AnyFlatSpec with PipelineSpec
Trait for unit testing ScioIO.
- implicit final class TestStreamScioContext extends AnyVal
Enhanced version of ScioContext with streaming methods.
- class TransformFinder extends Defaults
Pipeline visitor collection all matched PTransform.
Pipeline visitor collection all matched PTransform. This can be used in test to make sure the underlying transforms are properly configured
Value Members
- def testStreamOf[T](implicit arg0: Coder[T]): Builder[T]
Create a
TestStream.Builder
instance. - object ApproximationAssertions
- object CoderAssertions
- object DistCacheIO extends Serializable
- object EqInstances extends EqInstances
- object JobTest
Set up a Scio job for end-to-end unit testing.
Set up a Scio job for end-to-end unit testing. To be used in a PipelineSpec. For example:
import com.spotify.scio.testing._ class WordCountTest extends PipelineSpec { // Mock input data, mock distributed cache and expected result val inData = Seq("a b c d e", "a b a b") val distCache = Map(1 -> "Jan", 2 -> "Feb", 3 -> "Mar") val expected = Seq("a: 3", "b: 3", "c: 1", "d: 1", "e: 1") // Test specification "WordCount" should "work" in { JobTest("com.spotify.scio.examples.WordCount") // Or the type safe version // JobTest[com.spotify.scio.examples.WordCount.type] // Command line arguments .args("--input=in.txt", "--output=out.txt") // Mock input data .input(TextIO("in.txt"), inData) // Mock distributed cache .distCache(DistCacheIO("gs://dataflow-samples/samples/misc/months.txt"), distCache) // Verify output .output(TextIO("out.txt")) { actual => actual should containInAnyOrder (expected) } // Run job test .run() } }
- object Pretty
- object PrettySysProps extends SysProps
- object TransformOverride