I submit a PR that adds a new resolver to my GraphQL schema. My teammate tells me they’ve merged and deployed it. But when I try to access the new resolver, it isn’t reachable, and I can’t figure out why. I check the infrastructure test and see that the schema has changed - but my resolver is missing. I start losing my mind.

This is what could have happened if I didn’t have my tests in place.

The Problem with Testing Infrastructure-as-Code

When developing Infrastructure-as-Code (at least on AWS - I haven’t tried other platforms), I often find that the tests I write don’t actually verify the behavior I care about.

The primary goal of testing is to validate the behavior of our system. However, some tests don’t focus on behavior at all. Instead, they capture snapshots of the source code, effectively locking it in place and only alerting us to changes. At first glance, this might seem reasonable. But to me, it feels flawed - we’re not testing what the code does, just how it looks.

A Typical Schema Test

When testing Infrastructure-as-Code, we might write the following test to verify the GraphQL schema:

describe('GraphQL schema tests', () => {
  const stack = new StackWithSchema();
  const template = Template.fromStack(stack);

  it('should have a GraphQL API schema', () => {
    template.resourceCountIs('AWS::AppSync::GraphQLSchema', 1);
    
    template.hasResourceProperties('AWS::AppSync::GraphQLSchema', {
      ApiId: { 'Fn::GetAtt': [ Match.stringLikeRegexp(`GraphQL*`), 'ApiId' ] },
      Definition: 'schema {... }', // Placeholder for the actual schema
    });
  });
});

But what exactly does this test verify? It checks the schema in the .json output of the AWS CDK toolchain. It does not validate the schema deployed on the actual GraphQL server - the behavior I truly want to test.

This is essentially a snapshot test of the code, not a behavior test. It’s better than nothing, but ideally, I want to verify the real functionality. I’ve found that integration tests often implicitly cover infrastructure behavior - for example, if a resolver isn’t available via an API key, the error is clear. If a Lambda function lacks access to a DynamoDB table, the error is less obvious and requires digging into logs.

Testing the Schema via Introspection

Fortunately, GraphQL provides a built-in way to test its schema via introspection. The schema itself has a type definition:

extend type __Schema {
    description: String
    types: [__Type!]!
    queryType: __Type!
    mutationType: __Type
    subscriptionType: __Type
    directives: [__Directive!]!
}

I’m particularly interested in resolvers and the types they reference, so I wrote the following integration test:

const introspectOperations = `{
    __schema {
        queryType {
          fields {
            name
          }
        }
        mutationType {
          fields {
            name
          }
        }
        subscriptionType {
          fields {
            name
          }
        }
    }
}`;

const introspectTypes = `{
    __schema {
        types {
          name
          fields {
            name
          }
        }
    }
}`;

describe('Schema tests', () => {
    const byName = (field: any) => field.name;

    function testSchemaFields(schemaFields: any) {
        const sortedSchemaFields = schemaFields.sort(byName);
        expect(sortedSchemaFields).toMatchSnapshot();
    }

    it.each(['query', 'mutation', 'subscription'])('should verify %s operations', async (operation) => {
        const response = await sendGraphQLRequest({ query: introspectOperations });

        const schema = response.data.__schema;
        expect(schema).toBeTruthy();

        const operationFields = schema[`${operation}Type`]?.fields || [];
        testSchemaFields(operationFields);
    });

    it('should verify types', async () => {
        const response = await sendGraphQLRequest({ query: introspectTypes });

        const schema = response.data.__schema;
        expect(schema).toBeTruthy();

        const typeFields = schema.types;
        testSchemaFields(typeFields);
    });
});

Notes on the Test Code:

  • I usually avoid any types, but since these won’t be reused elsewhere, I can live with them.
  • expect(schema).toBeTruthy(); is technically optional - it just verifies that the GraphQL server responds correctly.
  • I’ve omitted the implementation of sendGraphQLRequest(), as it depends on the infrastructure, but it’s just a basic HTTP request.
  • The sorting (schemaFields.sort(byName)) ensures that test failures aren’t caused by variations in field order.

Real-World Impact

Whenever I deploy a new schema for my GraphQL service (AWS AppSync), I run this test as part of my integration tests. It gives me immediate feedback on the behavior of the deployed schema.

Now, let’s revisit the original scenario. I added a new resolver and my teammate merged and deployed the change - or so they claimed. When I ran my schema test, the new resolver was missing. We checked the logs and discovered that while they had merged my code, they hadn’t actually deployed it.

This test gave me a clear, automated way to demonstrate that the expected behavior had not changed. It wasn’t a matter of opinion but a matter of fact, making it easy to convince my teammate of the issue. And that’s the power of good testing.