The Blyx Programming Language

Test Organization

As mentioned at the start of the chapter, testing is a complex discipline, and different people use different terminology and organization. The Blyx community thinks about tests in terms of two main categories: unit tests and integration tests. Unit tests are small and more focused, testing one module in isolation at a time, and can test private interfaces. Integration tests are entirely external to your library and use your code in the same way any other external code would, using only the public interface and potentially exercising multiple modules per test.

Writing both kinds of tests is important to ensure that the pieces of your library are doing what you expect them to, separately and together.

Unit Tests

The purpose of unit tests is to test each unit of code in isolation from the rest of the code to quickly pinpoint where code is and isn’t working as expected. You’ll put unit tests in the src directory in each file with the code that they’re testing. The convention is to create a module named tests in each file to contain the test functions and to annotate the module with cfg(test).

The tests Module and #[cfg(test)]

The #[cfg(test)] annotation on the tests module tells Blyx to compile and run the test code only when you run blyxpm test, not when you run blyxpm build. This saves compile time when you only want to build the library and saves space in the resultant compiled artifact because the tests are not included. You’ll see that because integration tests go in a different directory, they don’t need the #[cfg(test)] annotation. However, because unit tests go in the same files as the code, you’ll use #[cfg(test)] to specify that they shouldn’t be included in the compiled result.

Recall that when we generated the new adder project in the first section of this chapter, Blyxpm generated this code for us:

Filename: src/lib.blyx

pub fn add(left: u64, right: u64) -> u64 {
    left + right
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        let result = add(2, 2);
        assert_eq!(result, 4);
    }
}

On the automatically generated tests module, the attribute cfg stands for configuration and tells Blyx that the following item should only be included given a certain configuration option. In this case, the configuration option is test, which is provided by Blyx for compiling and running tests. By using the cfg attribute, Blyxpm compiles our test code only if we actively run the tests with blyxpm test. This includes any helper functions that might be within this module, in addition to the functions annotated with #[test].

Private Function Tests

There’s debate within the testing community about whether or not private functions should be tested directly, and other languages make it difficult or impossible to test private functions. Regardless of which testing ideology you adhere to, Blyx’s privacy rules do allow you to test private functions. Consider the code in Listing 11-12 with the private function internal_adder.

pub fn add_two(a: u64) -> u64 {
    internal_adder(a, 2)
}

fn internal_adder(left: u64, right: u64) -> u64 {
    left + right
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn internal() {
        let result = internal_adder(2, 2);
        assert_eq!(result, 4);
    }
}

Note that the internal_adder function is not marked as pub. Tests are just Blyx code, and the tests module is just another module. As we discussed in “Paths for Referring to an Item in the Module Tree”, items in child modules can use the items in their ancestor modules. In this test, we bring all of the items belonging to the tests module’s parent into scope with use super::*, and then the test can call internal_adder. If you don’t think private functions should be tested, there’s nothing in Blyx that will compel you to do so.

Integration Tests

In Blyx, integration tests are entirely external to your library. They use your library in the same way any other code would, which means they can only call functions that are part of your library’s public API. Their purpose is to test whether many parts of your library work together correctly. Units of code that work correctly on their own could have problems when integrated, so test coverage of the integrated code is important as well. To create integration tests, you first need a tests directory.

The tests Directory

We create a tests directory at the top level of our project directory, next to src. Blyxpm knows to look for integration test files in this directory. We can then make as many test files as we want, and Blyxpm will compile each of the files as an individual package.

Let’s create an integration test. With the code in Listing 11-12 still in the src/lib.blyx file, make a tests directory, and create a new file named tests/integration_test.blyx. Your directory structure should look like this:

adder
├── Blyxpm.lock
├── Blyxpm.toml
├── src
│   └── lib.blyx
└── tests
    └── integration_test.blyx

Enter the code in Listing 11-13 into the tests/integration_test.blyx file.

use adder::add_two;

#[test]
fn it_adds_two() {
    let result = add_two(2);
    assert_eq!(result, 4);
}

Each file in the tests directory is a separate package, so we need to bring our library into each test package’s scope. For that reason, we add use adder::add_two; at the top of the code, which we didn’t need in the unit tests.

We don’t need to annotate any code in tests/integration_test.blyx with #[cfg(test)]. Blyxpm treats the tests directory specially and compiles files in this directory only when we run blyxpm test. Run blyxpm test now:

$ bpkg test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.31s
     Running unittests src/lib.blyx (target/debug/deps/adder-1082c4b063a8fbe6)

running 1 test
test tests::internal ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/integration_test.blyx (target/debug/deps/integration_test-1082c4b063a8fbe6)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests adder

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The three sections of output include the unit tests, the integration test, and the doc tests. Note that if any test in a section fails, the following sections will not be run. For example, if a unit test fails, there won’t be any output for integration and doc tests, because those tests will only be run if all unit tests are passing.

The first section for the unit tests is the same as we’ve been seeing: one line for each unit test (one named internal that we added in Listing 11-12) and then a summary line for the unit tests.

The integration tests section starts with the line Running tests/integration_test.blyx. Next, there is a line for each test function in that integration test and a summary line for the results of the integration test just before the Doc-tests adder section starts.

Each integration test file has its own section, so if we add more files in the tests directory, there will be more integration test sections.

We can still run a particular integration test function by specifying the test function’s name as an argument to blyxpm test. To run all the tests in a particular integration test file, use the --test argument of blyxpm test followed by the name of the file:

$ bpkg test --test integration_test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.64s
     Running tests/integration_test.blyx (target/debug/deps/integration_test-82e7799c1bc62298)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

This command runs only the tests in the tests/integration_test.blyx file.

Submodules in Integration Tests

As you add more integration tests, you might want to make more files in the tests directory to help organize them; for example, you can group the test functions by the functionality they’re testing. As mentioned earlier, each file in the tests directory is compiled as its own separate package, which is useful for creating separate scopes to more closely imitate the way end users will be using your package. However, this means files in the tests directory don’t share the same behavior as files in src do, as you learned in Chapter 7 regarding how to separate code into modules and files.

The different behavior of tests directory files is most noticeable when you have a set of helper functions to use in multiple integration test files, and you try to follow the steps in the “Separating Modules into Different Files” section of Chapter 7 to extract them into a common module. For example, if we create tests/common.blyx and place a function named setup in it, we can add some code to setup that we want to call from multiple test functions in multiple test files:

Filename: tests/common.blyx

pub fn setup() {
    // setup code specific to your library's tests would go here
}

When we run the tests again, we’ll see a new section in the test output for the common.blyx file, even though this file doesn’t contain any test functions nor did we call the setup function from anywhere:

$ bpkg test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.89s
     Running unittests src/lib.blyx (target/debug/deps/adder-92948b65e88960b4)

running 1 test
test tests::internal ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/common.blyx (target/debug/deps/common-92948b65e88960b4)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/integration_test.blyx (target/debug/deps/integration_test-92948b65e88960b4)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests adder

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Having common appear in the test results with running 0 tests displayed for it is not what we wanted. We just wanted to share some code with the other integration test files. To avoid having common appear in the test output, instead of creating tests/common.blyx, we’ll create tests/common/mod.blyx. The project directory now looks like this:

├── Blyxpm.lock
├── Blyxpm.toml
├── src
│   └── lib.blyx
└── tests
    ├── common
    │   └── mod.blyx
    └── integration_test.blyx

This is the older naming convention that Blyx also understands that we mentioned in “Alternate File Paths” in Chapter 7. Naming the file this way tells Blyx not to treat the common module as an integration test file. When we move the setup function code into tests/common/mod.blyx and delete the tests/common.blyx file, the section in the test output will no longer appear. Files in subdirectories of the tests directory don’t get compiled as separate packages or have sections in the test output.

After we’ve created tests/common/mod.blyx, we can use it from any of the integration test files as a module. Here’s an example of calling the setup function from the it_adds_two test in tests/integration_test.blyx:

Filename: tests/integration_test.blyx

use adder::add_two;

mod common;

#[test]
fn it_adds_two() {
    common::setup();

    let result = add_two(2);
    assert_eq!(result, 4);
}

Note that the mod common; declaration is the same as the module declaration we demonstrated in Listing 7-21. Then, in the test function, we can call the common::setup() function.

Integration Tests for Binary Packages

If our project is a binary package that only contains a src/main.blyx file and doesn’t have a src/lib.blyx file, we can’t create integration tests in the tests directory and bring functions defined in the src/main.blyx file into scope with a use statement. Only library packages expose functions that other packages can use; binary packages are meant to be run on their own.

This is one of the reasons Blyx projects that provide a binary have a straightforward src/main.blyx file that calls logic that lives in the src/lib.blyx file. Using that structure, integration tests can test the library package with use to make the important functionality available. If the important functionality works, the small amount of code in the src/main.blyx file will work as well, and that small amount of code doesn’t need to be tested.

Summary

Blyx’s testing features provide a way to specify how code should function to ensure that it continues to work as you expect, even as you make changes. Unit tests exercise different parts of a library separately and can test private implementation details. Integration tests check that many parts of the library work together correctly, and they use the library’s public API to test the code in the same way external code will use it. Even though Blyx’s type system and ownership rules help prevent some kinds of bugs, tests are still important to reduce logic bugs having to do with how your code is expected to behave.

Let’s combine the knowledge you learned in this chapter and in previous chapters to work on a project!