Packages

package testing

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. testing
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Type Members

  1. trait BigtableMatchers extends SCollectionMatchers

    Trait with ScalaTest Matchers for SCollections specific to Bigtable output.

  2. case class DistCacheIO[T](uri: String) extends Product with Serializable
  3. trait EqInstances extends FallbackEqInstances
  4. sealed trait FallbackEqInstances extends AnyRef
  5. trait PipelineSpec extends AnyFlatSpec with Matchers with SCollectionMatchers with PipelineTestUtils with RunEnforcementJobTest

    Trait for unit testing pipelines.

    Trait for unit testing pipelines.

    A simple test might look like this:

    class SimplePipelineTest extends PipelineSpec {
      "A simple pipeline" should "sum integers" in {
        runWithContext { sc =>
          sc.parallelize(Seq(1, 2, 3)).sum should containSingleValue (6)
        }
      }
    }
  6. trait PipelineTestUtils extends AnyRef

    Trait with utility methods for unit testing pipelines.

  7. trait RunEnforcementJobTest extends AnyFlatSpec

    Trait that enforces JobTest.Builder.run is called.

  8. trait SCollectionMatchers extends EqInstances

    Trait with ScalaTest Matchers for SCollections.

  9. trait ScioIOSpec extends AnyFlatSpec with PipelineSpec

    Trait for unit testing ScioIO.

  10. implicit final class TestStreamScioContext extends AnyVal

    Enhanced version of ScioContext with streaming methods.

Value Members

  1. def testStreamOf[T](implicit arg0: Coder[T]): Builder[T]

    Create a TestStream.Builder instance.

  2. object ApproximationAssertions
  3. object CoderAssertions
  4. object DistCacheIO extends Serializable
  5. object EqInstances extends EqInstances
  6. object JobTest

    Set up a Scio job for end-to-end unit testing.

    Set up a Scio job for end-to-end unit testing. To be used in a PipelineSpec. For example:

    import com.spotify.scio.testing._
    
    class WordCountTest extends PipelineSpec {
    
      // Mock input data, mock distributed cache and expected result
      val inData = Seq("a b c d e", "a b a b")
      val distCache = Map(1 -> "Jan", 2 -> "Feb", 3 -> "Mar")
      val expected = Seq("a: 3", "b: 3", "c: 1", "d: 1", "e: 1")
    
      // Test specification
      "WordCount" should "work" in {
        JobTest("com.spotify.scio.examples.WordCount")
        // Or the type safe version
        // JobTest[com.spotify.scio.examples.WordCount.type]
    
          // Command line arguments
          .args("--input=in.txt", "--output=out.txt")
    
          // Mock input data
          .input(TextIO("in.txt"), inData)
    
          // Mock distributed cache
          .distCache(DistCacheIO("gs://dataflow-samples/samples/misc/months.txt"), distCache)
    
          // Verify output
          .output(TextIO("out.txt")) { actual => actual should containInAnyOrder (expected) }
    
          // Run job test
          .run()
      }
    }
  7. object Pretty
  8. object PrettySysProps
    Annotations
    @registerSysProps()

Inherited from AnyRef

Inherited from Any

Ungrouped