The way I am developing features for full stack applications has a pattern. I’m pretty sure that I’m not inventing anything new, this is more of an exercise for me to bring clarity to myself first, but maybe someone can find this helpful too.

When one thinks about developing a frontend and a backend the thoughts are about exchanging data. It’s all about the data flow - what does each layer of the stack sends and receives. Developing that data flow in one shot was hard for me - I had to imagine the whole chain from the beginning until the end. I was doing a lot of rework, the cost of development was high.

I wanted smaller and simpler steps. I tried an exploratory approach, where I consider the whole process to be an experiment where “it has to work somehow, but I let myself explore the how”.

For the sake of clarity let’s say we have an application that manages points on a map with the tech stack below, although keep in mind that the stack is not that important here.

  • Frontend
    • UI: React
    • State: Redux
    • Middleware: Redux-Observable
    • GraphQL client: Apollo Client
  • Backend
    • GraphQL server: AWS AppSync
    • Backend API: AWS Lambda
    • DB: AWS DynamoDB

The task is to add a feature that lets a user add a point the map. So basically the data flow starts at the top of the stack when the user adds a point, goes until the backend, where the request is processed and the point is added to the database, and then the data comes back to the top with some additional data from the backed, like the new point’s ID. The key point is to go with that flow.

I will skip the development of infrastructure, the testing techniques since this would make the article too long and I will use TypeScript for the example but will often omit types and imports.

Flow endpoints

I start at the frontend, this is where the data flow begins and ends. It begins with a button click so I add a button that does nothing for now.

export function AddPointButton() {
    return (<button>Add new point</button>);
}

I have the start of the data flow, now I need create its end a well:

export function UserAddedPoint() {
    return (
        <div className="point-container">
            <span className="point-description">
                Point id '123-456' located at lat '10', lon '20'
            </span>
        </div>
    );
}

This step lets me work on the UI without caring about the logic behind.

Current data flow: UI (no flow)

Connecting the state

Now that my flow endpoints are set (AddPointButton() to UserAddedPoint()), I have to make the data flow between them in the simplest possible way - through the Redux state. I remove the data from the receiving component:

export function UserAddedPoints() {
    const points = useSelector((state: RootState) => state.pointsState);

    const userAddedPoints = points.map((point, index) => {
        <UserAddedPoint point={point} key={index} />;
    });

    return <div className="points-container">{userAddedPoints}</div>;
}

export function UserAddedPoint(point) {
    return (
        <div className="point-container">
            <span className="point-description">
                Point {point.id} located at lat {point.lat}, lon {point.lon}
            </span>
        </div>
    );
}

And to loop the flow, I send that data from the sending component:

export function AddPointButton() {
    const dispatch = useDispatch();

    const onClick = () => {
        dispatch(addPoint({ id: '123-456', lat: 10, lon: 20 }));
    };

    return <button onClick={onClick}>Add new point</button>;
}

Note that I have hardcoded all the data including the id. This may feel bad and it should, this is exactly the exploration part I was wrote about before. Now thanks to that feeling that I got while typing this id I understood it should not be here. Let’s keep it in mind and continue for now. Following that I obviously need to create a reducer that adds the point:

export function points(state = initialState, action: PointAction) {
    switch (action.type) {
        case PointActions.ADD_POINT:
            return { ...state, points: [...state.point, action.point] };
        default:
            return state;
    }

At this step the app is capable of adding a point with a button click to the state and rendering the state - I created a very simple data flow.

Current data flow: UI <–> State

Obviously we don’t want the user to be able to add a point only at lat: 10, lon: 20, but at any point on the map, this can be done later, delegated to someone, etc. This is not the important topic right now, so I will continue the the data flow.

Including the epic

For fetching data from the backed I will use a Redux-observable epic, but I’m not rushing to connect the backend with it. I rather expand the data flow with a new epic. I remember that the button should not send the id, so I add another action - a requestToAddPoint():

export function AddPointButton() {
    const dispatch = useDispatch();

    const onClick = () => {
        dispatch(requestToAddPoint({ lat: 10, lon: 20 }));
    };

    return <button onClick={onClick}>Add new point</button>;
}

This action is to be handled by the epic, it will be the input action and the previous addPoint() action is now the output of the epic.

export function remotePoints(action$) {
    return action$.pipe(
        ofType(PointActions.REQUEST_TO_ADD_POINT),
        mergeMap((action) => {
            return of(addPoint({ id: '123-456', lat: action.lat, lon: action.lon }));
        }),
    );
};

I take the lat and lon from the action and the id is attributed in the epic. All of them are used in the already exisiting action addPoint() so I don’t need to modify the reducer - I’ve just inserted another element in the data flow chain.

Now I’m aware that the data I will be sending to the backend is { lat, lon } and the data I expect to be retrieved is { id, lat, lon }. In this simple case this might have been known from the beginning, but in more difficult, real-world cases this choice may not be trivial.

Current data flow: UI <–> State <–> Epic

Connecting the backend

Yes, it doesn’t exist yet, I know, it’s fine. I can still write code, I might write a separate article on how to do that later, this is a whole separate topic. In any case I don’t need the UI or the reducer, now I expand my data flow from the epic by adding the Apollo Client:

export function remotePoints(action$, _, { pointsClient }) {  
    return action$.pipe(  
        ofType(PointActions.REQUEST_TO_ADD_POINT),  
        mergeMap((action) => sendAddPointRequest(action, pointsClient)),  
    );  
}  
  
function sendAddPointRequest(action, pointsClient) {  
    return fromPromise(  
        pointsClient.mutate({  
            mutation: addPointMutation,  
            variables: { lat: action.lat, lon: action.lon },  
        }),  
    ).pipe(  
        mergeMap((response: AddPointResponse) => of(addPoint(response.AddPoint))),  
        catchError(reportError),  
    );  
}

This will fail miserably since the backend is not implemented, so I will finally go for backend implementation to repair that and continue the flow.

Current data flow: UI <–> State <–> Epic >–< API (no flow with API)

Implementing the backend

What is the simplest way for me to make the backend work? Just add static data in the same way I did on the frontend…

export default async function addPoint(location) {
    const newPoint = {
        id: '123-456',
        lat: location.lat,
        lon: location.lon,
    };

    return newPoint;
}

Even with this minimal implementation I can test the data flow between the frontend and the backend, do some exploratory testing of the whole feature or refactor, for example, package the lat and lon in interace Location { lat: number, lon: number } on both the backend and the frontend.

Current data flow: UI <–> State <–> Epic <–> API

Connecting the DB

There are things to do still after the last step - add an id generator and store the data in the database.

import { DocumentClient } from 'aws-sdk/clients/dynamodb';

export default async function addPoint(location) {
    const newPoint = {
        id: generateId(),
        lat: location.lat,
        lon: location.lon,
    };
    
    const itemToPut = {
        TableName: 'PointsTable,
        Item: newDevice,
    };
    
    await new DocumentClient().put(scanInput).promise();
    
    return newPoint;
}

And with this step we have completed the flow!

Current data flow: UI <–> State <–> Epic <–> API <–> DB

This method may be seen as a naive thing to do. Why just not develop the whole thing in one go? To me it feels like making a huge bet that I’m not sure to win. From the beginning I’m obliged to make a bet about how the whole thing works. Loosing such a bet means I have to rework what I already spent my time on. Which means that time was not well spent.

Instead I prefer to make lots of smaller bets that I’m pretty sure to win. Each small step can be done in a tiny timeframe, easily scheduled or, as it happens sometimes, interrupted or delayed. When I get back to this topic, I need little time to restore the context back in my head and continue the development. So this is also an efficiency tool.

I imagine if such a process would be the standard feature development process in a team, this could be streamlined to a conveyor-style work between developers, with a nice kanban view of each feature status.