scalingAcceptance

#+TITLE: Scaling acceptance tests

This section follows the Scaling acceptance tests chapter of the
[[https://quii.gitbook.io/learn-go-with-tests/testing-fundamentals/scaling-acceptance-tests][Learn Go with Tests]].

Acceptance tests are essential, and they directly impact your ability to
confidently evolve your system over time, with a reasonable cost of change.

They're also a fantastic tool to help you work with legacy code. When faced with
a poor codebase without any tests, please resist the temptation to start
refactoring. Instead, write some acceptance tests to give you a safety net to
freely change the system's internals without affecting its functional external
behaviour. ATs need not be concerned with internal quality, so they're a great
fit in these situations.

After reading this, you'll appreciate that acceptance tests are useful for
verification and can also be used in the development process by helping us
change our system more deliberately and methodically, reducing wasted effort.

Someday you will recognize the need for acceptance tests; some way to test a
system from a user's point of view and to verify it works how it's intended, but
almost without exception, the cost of these tests became a real problem for the
team.
- Slow to run.
- Brittle.
- Flaky.
- Expensive to maintain, and seem to make changing the software harder than it
  ought to be.
- Can only run in a particular environment, causing slow and poor feedback
  loops.

Let's say you intend to write an acceptance test around a website you're
building. You decide to use a headless web browser (like [[https://www.selenium.dev/][Selenium]]) to simulate a
user clicking buttons on your website to verify it does what it needs to do.

Over time, your website's markup has to change as new features are discovered,
and engineers bike-shed over whether something should be an ~<article>~ or a
~<section>~ for the billionth time.

Even though your team are only making minor changes to the system, barely
noticeable to the actual user, you find yourself wasting lots of time updating
your acceptance tests.

Think about what prompts acceptance tests to change:
- An external behaviour change. If you want to change what the system does,
  changing the acceptance test suite seems reasonable, if not desirable.
- An implementation detail change / refactoring. Ideally, this shouldn't prompt
  a change, or if it does, a minor one.

Too often, though, the latter is the reason acceptance tests have to change. To
the point where engineers even become reluctant to change their system because
of the perceived effort of updating tests!

These problems stem from not applying well-established and practised engineering
habits written by the authors mentioned above. You can't write acceptance tests
like unit tests; they require more thought and different practices.

* Anatomy of good acceptance tests
  If we want acceptance tests that only change when we change behaviour and not
  implementation detail, it stands to reason that we need to separate those
  concerns.

** On types of complexity
   As software engineers, we have to deal with two kinds of complexity.
   - *Accidental complexity* is the complexity we have to deal with because
     we're working with computers, stuff like networks, disks, APIs, etc.
   - *Essential complexity* is sometimes referred to as "domain logic". It's the
     particular rules and truths within your domain.
     - For example, "if an account owner withdraws more money than is available,
       they are overdrawn". This statement says nothing about computers; this
       statement was true before computers were even used in banks!

   Essential complexity should be expressible to a non-technical person, and
   it's valuable to have modelled it in our "domain" code, and in our acceptance
   tests.

** Separation of concerns
   We should have the idea of specifications. Specifications describe the
   behaviour of the system we want without being coupled with accidental
   complexity or implementation detail.

   This idea should feel reasonable to you. In production code, we frequently
   strive to separate concerns and decouple units of work. Would you not
   hesitate to introduce an interface to allow your HTTP handler to decouple it
   from non-HTTP concerns? Let's take this same line of thinking for our
   acceptance tests.

** Testing on steroids
   Decoupling how the specification is executed allows us to reuse it in
   different scenarios. We can:

   *Make our drivers configurable*
   This means you can run your acceptance tests locally, in your staging and
   (ideally) production environments.
   - Too many teams engineer their systems such that acceptance tests are
     impossible to run locally. This introduces an intolerably slow feedback
     loop. Wouldn't you rather be confident your acceptance tests will pass
     before integrating your code? If the tests start breaking, is it acceptable
     that you'd be unable to reproduce the failure locally and instead, have to
     commit changes and cross your fingers that it'll pass 20 minutes later in a
     different environment?
   - Remember, just because your tests pass in staging doesn't mean your system
     will work. Dev/Prod parity is, at best, a white lie. [[https://increment.com/testing/i-test-in-production/][I test in prod]].
   - There are always differences between the environments that can affect the
     behaviour of your system. A CDN could have some cache headers incorrectly
     set; a downstream service you depend on may behave differently; a
     configuration value may be incorrect. But wouldn't it be nice if you could
     run your specifications in prod to catch these problems quickly?

   *Plug in different drivers to test other parts of your system*
   This flexibility allows us to test behaviours at different abstraction and
   architectural layers, which allows us to have more focused tests beyond
   black-box tests.
   - For instance, you may have a web page with an API behind it. Why not use
     the same specification to test both? You can use a headless web browser for
     the web page, and HTTP calls for the API.
   - Taking this idea further, ideally, we want the
     *code to model essential complexity* (as "domain" code) so we should also
     be able to use our specifications for unit tests. This will give swift
     feedback that the essential complexity in our system is modelled and
     behaves correctly.

** Acceptance tests changing for the right reasons
   With this approach, the only reason for your specifications to change is if
   the behaviour of the system changes, which is reasonable.
   - If your HTTP API has to change, you have one obvious place to update it,
     the driver.
   - If your markup changes, again, update the specific driver.

   As your system grows, you'll find yourself reusing drivers for multiple
   tests, which again means if implementation detail changes, you only have to
   update one, usually obvious place.

   When done right, this approach gives us flexibility in our implementation
   detail and stability in our specifications. Importantly, it provides a simple
   and obvious structure for managing change, which becomes essential as a
   system and its team grows.

* Enough talk, time to code
  Unlike other chapters, you'll need [[https://www.docker.com/][Docker]] installed because we'll be running
  our applications in containers. It's assumed at this point in the book you're
  comfortable writing Go code, importing from different packages, etc.

  Create a new project with:
  ~go mod init github.com/maker2413/GoNotes/scalingAcceptance~ (you can put
  whatever you like here but if you change the path you will need to change all
  internal imports to match).

  Make a folder ~specifications~ to hold our specifications, and add a file
  [[./specifications/greet.go][greet.go]]

  My IDE (Emacs) takes care of the fuss of adding dependencies for me, but if
  you need to do it manually, you'd do:
  ~go get github.com/alecthomas/assert/v2~

  Given Farley's acceptance test design (Specification->DSL->Driver->System), we
  now have a decoupled specification from implementation. It doesn't know or
  care about /how/ we ~Greet~; it's just concerned with the essential complexity of
  our domain. Admittedly this complexity isn't much right now, but we'll expand
  upon the spec to add more functionality as we further iterate. It's always
  important to start small!

  You could view the interface as our first step of a DSL; as the project grows,
  you may find the need to abstract differently, but for now, this is fine.

  At this point, this level of ceremony to decouple our specification from
  implementation might make some people accuse us of "overly abstracting".
  *I promise you that acceptance tests that are too coupled to implementation
  become a real burden on engineering teams*. I am confident that most
  acceptance tests out in the wild are expensive to maintain due to this
  inappropriate coupling; rather than the reverse of being overly abstract.

  We can use this specification to verify any "system" that can ~Greet~.

** First system: HTTP API
   We require to provide a "greeter service" over HTTP. So we'll need to create:
   1. A *driver*. In this case, one works with an HTTP system by using an
      *HTTP client*. This code will know how to work with our API. Drivers
      translate DSLs into system-specific calls; in our case, the driver will
      implement the interface specifications define.
   2. An *HTTP server* with a greet API.
   3. A *test*, which is responsible for managing the life-cycle of spinning up
      the server and then plugging the driver into the specification to run it
      as a test.

* Misc Notes
  Here is some random quotes and notes from the scaling acceptance test chapter.
  #+BEGIN_QUOTE
  In [[https://en.wikipedia.org/wiki/Software_engineering][software engineering]], the *adapter pattern* is a [[https://en.wikipedia.org/wiki/Software_design_pattern][software design pattern]]
  (also known as [[https://en.wikipedia.org/wiki/Wrapper_function][wrapper]], an alternative naming shared with the
  [[https://en.wikipedia.org/wiki/Decorator_pattern][decorator pattern]]) that allows the [[https://en.wikipedia.org/wiki/Interface_(computing)][interface]] of an existing [[https://en.wikipedia.org/wiki/Class_(computer_programming)][class]] to be used
  as another interface.[1] It is often used to make existing classes work with
  others without modifying their [[https://en.wikipedia.org/wiki/Source_code][source code]].
  #+END_QUOTE

  Sometimes, it makes sense to do some refactoring before making a change.
  #+BEGIN_QUOTE
  First make the change easy, then make the easy change
  #+END_QUOTE
  ~Kent Beck

** GRPC
   If you're unfamiliar with gRPC, I'd start by looking at the
   [[https://grpc.io/][gRPC website]]. Still, for this chapter, it's just another kind of adapter into
   our system, a way for other systems to call (remote procedure call) our
   excellent domain code.

   The twist is you define a "service definition" using Protocol Buffers. You
   then generate server and client code from the definition. This not only works
   for Go but for most mainstream languages too. This means you can share a
   definition with other teams in your company who may not even write Go and can
   still do service-to-service communication smoothly.

   If you haven't used gRPC before, you'll need to install a
   *Protocol buffer compiler* and some *Go plugins*.
   [[https://grpc.io/docs/languages/go/quickstart/][The gRPC website has clear instructions on how to do this]].

** Separating different kinds of tests
   Acceptance tests are great in that they test the whole system works from a
   pure user-facing, behavioural POV, but they do have their downsides compared
   to unit tests:
   - Slower
   - Quality of feedback is often not as focused as a unit test
   - Doesn't help you with internal quality, or design

   [[https://martinfowler.com/articles/practical-test-pyramid.html][The Test Pyramid]] guides us on the kind of mix we want for our test suite, you
   should read Fowler's post for more detail, but the very simplistic summary
   for this post is "lots of unit tests and a few acceptance tests".

   For that reason, as a project grows you often may be in situations where the
   acceptance tests can take a few minutes to run. To offer a friendly developer
   experience for people checking out your project, you can enable developers to
   run the different kinds of tests separately.

   It's preferable that running ~go test ./...~ should be runnable with no
   further set up from an engineer, beyond say a few key dependencies such as
   the Go compiler (obviously) and perhaps Docker.

   Go provides a mechanism for engineers to run only "short" tests with the
   [[https://pkg.go.dev/testing#Short][short flag]].
   #+begin_src bash
     go test -short ./...
   #+end_src

** When should I write acceptance tests?
   The best practice is to favour having lots of fast running unit tests and a
   few acceptance tests, but how do you decide when you should write an
   acceptance test, vs unit tests?

   It's difficult to give a concrete rule, but the questions I typically ask
   myself are:
   - Is this an edge case? I'd prefer to unit test those.
   - Is this something that the non-computer people talk about a lot? I would
     prefer to have a lot of confidence the key thing "really" works, so I'd add
     an acceptance test.
   - Am I describing a user journey, rather than a specific function? Acceptance
     test.
   - Would unit tests give me enough confidence? Sometimes you're taking an
     existing journey that already has an acceptance test, but you're adding
     other functionality to deal with different scenarios due to different
     inputs. In this case, adding another acceptance test adds a cost but brings
     little value, so I'd prefer some unit tests.

* Wrapping up
  Building systems with a reasonable cost of change requires you to have ATs
  engineered to help you, not become a maintenance burden. They can be used as a
  means of guiding, or as a GOOS says, "growing" your software methodically.

  Hopefully, with this example, you can see our application's predictable,
  structured workflow for driving change and how you could use it for your work.

  You can imagine talking to a stakeholder who wants to extend the system you
  work on in some way. Capture it in a domain-centric, implementation-agnostic
  way in a specification, and use it as a north star towards your efforts. Riya
  and I describe leveraging BDD techniques like "Example Mapping"
  [[https://www.youtube.com/watch?v=ZMWJCk_0WrY][in our GopherconUK talk]] to help you understand the essential complexity more
  deeply and allow you to write more detailed and meaningful specifications.

  Separating essential and accidental complexity concerns will make your work
  less ad-hoc and more structured and deliberate; this ensures the resiliency of
  your acceptance tests and helps them become less of a maintenance burden.

  Dave Farley gives an excellent tip:
  #+BEGIN_QUOTE
  Imagine the least technical person that you can think of, who understands the
  problem-domain, reading your Acceptance Tests. The tests should make sense to
  that person.
  #+END_QUOTE

  Specifications should then double up as documentation. They should specify
  clearly how a system should behave. This idea is the principle around tools
  like [[https://cucumber.io/][Cucumber]], which offers you a DSL for capturing behaviours as code, and
  then you convert that DSL into system calls, just like we did here.

** What has been covered
   - Writing abstract specifications allows you to express the essential
     complexity of the problem you're solving and remove accidental
     complexity. This will enable you to reuse the specifications in different
     contexts.
   - How to use [[https://golang.testcontainers.org/][Testcontainers]] to manage the life-cycle of your system for
     ATs. This allows you to thoroughly test the image you intend to ship on
     your computer, giving you fast feedback and confidence.
   - A brief intro into containerising your application with Docker.
   - gRPC.
   - Rather than chasing canned folder structures, you can use your development
     approach to naturally drive out the structure of your application, based on
     your own needs.

** Further material
   - In this example, our "DSL" is not much of a DSL; we just used interfaces to
     decouple our specification from the real world and allow us to express
     domain logic cleanly. As your system grows, this level of abstraction might
     become clumsy and unclear. [[https://cucumber.io/blog/bdd/understanding-screenplay-(part-1)/][Read into the "Screenplay Pattern"]] if you want
     to find more ideas as to how to structure your specifications.
   - For emphasis, [[http://www.growing-object-oriented-software.com/][Growing Object-Oriented Software, Guided by Tests]], is a
     classic. It demonstrates applying this "London style", "top-down" approach
     to writing software. Anyone who has enjoyed Learn Go with Tests should get
     much value from reading GOOS.
   - [[https://github.com/quii/go-specs-greet][In the example code repository]], there's more code and ideas I haven't
     written about here, such as multi-stage docker build, you may wish to check
     this out.
     - In particular, /for fun/, I made a *third program*, a website with some
       HTML forms to ~Greet~ and ~Curse~. The ~Driver~ leverages the
       excellent-looking https://github.com/go-rod/rod module, which allows it
       to work with the website with a browser, just like a user would. Looking
       at the git history, you can see how I started not using any templating
       tools "just to make it work" Then, once I passed my acceptance test, I
       had the freedom to do so without fear of breaking things. -->