Skip to main content

Production Testing Checklist

The following tasks should be completed before every API deployment into production unless there is a valid reason to not complete a given task.:

This reason should be documented.

Naming conventions:

  • Endpoint URL follows the format of api/v#/controllerName;
  • Other naming conventions are upheld in query params and responses as per the playbook naming guidance;

Responses:

  • All workflows in the pipeline are passing (observe in CircleCI);
  • Response time of the API is not longer than the Lambda timeout;
  • Check for long-running queries that might time out;
  • List endpoint works with each search parameter;
  • Limit and pagination is working;
  • Data returned makes sense i.e. not loads of duplicates;
  • Responses match the Swagger doc, if they don't for good reason, update Swagger;
  • Values within responses are returned in the proper format.
    • e.g. enums return the desired constant’s name/description and not its integer value in the enumeration; date/time is in a readable format;

Configuration and security:

  • All entities are created in AWS in each environment (incl. correct subnets and any other required config);
  • Check all environment variables.
    • For example, make sure staging environment variables haven't been copied over to production without changing their values to the correct production values;
  • API keys are in place;
  • API authentication is in place – no anonymous access;

Errors:

  • Failures are logged in CloudWatch;
  • Notifications are available when API is failing;
  • Correct error responses are returned;
  • The canaries for all endpoints that should have them;
  • Check that canaries are passing;
  • The canaries have someone subscribed to it;