Supercharged forms guide: Using JSON schema in forms

If you're a web developer, you have probably spent some time in your career developing a functionality involving forms. Depending on the form and the application on the frontend, this can mean a lot of work (particularly if you're determined to provide a solid user experience).

To see the first article in this series, check out our introduction on how Remote builds onboarding flows!

On the front end, you'll need to focus on three critical tasks:

• create UI components for inputs

• support inline error messages

• write client-side validation before it's submitted to the server

And on the backend you'll need to move through the following three steps:

• validate the received data

• do whatever needs to be done with that data

• send a response that may be reflected in the form

On top of that, you’ll need synched validations between the frontend and backend. In the long term, that can quickly get out of control. Without careful planning, you could end up with a nightmare scenario to maintain.

The Remote platform depends heavily on forms. The use case of a new country launch demonstrates this in practice.

In this instance, we need to follow a process of gathering the data requirements for Remote to support a new Employer of Record country activation. For every new country launch in the platform, we need to create around eight forms with country-specific fields. Each of these fields have different specificities.

If you think about the number of countries in the world (180+), multiplied by the number of forms with different fields each (approximately 8), you can start to understand the number of different possibilities, and all the coordination that needs to happen between frontend and backend. The complexity is immense.

A simple frontend scenario explained

But let’s dive into a simple example and see what needs to be done from a frontend perspective so you can have a better understanding of the problem with developing forms at scale.

We’ll assume we have to create a form with two required fields:

1. Work Schedule: This can take one of two values – full-time or part-time.

2. Available PTO: The time that employees can take off of work while still getting paid (which can’t be less than 20 days)

To create this form, we need to work through the following workflow:

• Create input components (a radio button, and input text) which will represent work schedule and available PTO respectively. At Remote we already have a lot of input components and all of them include labels, descriptions and/or placeholders and also accommodate inline errors.

• Create the form with the inputs and handle all the boilerplate code, like getting values in and out of form state, validating the payload, rendering error messages, and handling form submission. At Remote, we created a DynamicForm component, which is a headless form that renders the form inputs based on the fields prop. That means we need to manually create this fields list with the field name, description, options, validation schema, and some other specific properties depending on the input.

• Orchestrate client-side validation (we use Yup for this) and in our case, we need to make sure both fields are required and that available_pto can’t be less than 20.

• On the backend (which we won’t focus on in this blog post), we need to validate the payload coming from the frontend, store it in the database and return a response.

1const fields = [
2  {
3    type: "radio",
4    name: "workSchedule",
5    label: "Work schedule",
6    description: "Can be either full time or part time",
7    options: [
8      {
9        value: "full_time",
10        label: "Full Time",
11      },
12      {
13        value: "part_time",
14        label: "Part Time",
15      },
16    ],
17    schema: yup.string().required(),
18  },
19  {
20    type: "number",
21    name: "availablePto",
22    label: "Available PTO",
23    description:
24      "Time that employees can take off of work while still getting paid",
25    schema: yup.number().min(20).required(),
26  },
27];
28
29export function MyForm() {
30  const fieldValues = {
31    workSchedule: "",
32    availablePto: "",
33  };
34
35  const handleSubmit = (values) => {
36    // send to server
37  };
38
39  return (
40    <DynamicForm
41      defaultValues={fieldValues}
42      fields={fields}
43      handleOnSubmit={handleSubmit}
44    />
45  );
46}

Although it’s a simple form, it’s probably enough to understand the amount of effort needed to build it. Moreover, this strategy requires a lot of good and synchronous communication between frontend and backend engineers which also means that it’s error-prone. For example, what if the backend updates the minimum available PTO from 20 to 22 but the frontend is not updated? What if for some reason the name of the property changed on the backend from available_pto to available_pto_days?

This doesn’t scale when forms are part of your core. Causing a lot of frustration for developers, potentially being a source for bugs, and ultimately delaying your time to market.

Using JSON Schemas to supercharge our forms

We needed to take action and thought about solutions that could improve our developer experience, reduce all duplicated code and ultimately accelerate our time to market in functionalities that involved forms.

How we use JSON schemas on the backend

We decide to use JSON Schemas and delegate form composition and ergonomics, plus data validations to the backend. JSON Schema is a vocabulary that is primarily used to annotate and validate JSON data.

So going back to our example, we could represent our form with the following schema:

1{
2  "type": "object",
3  "properties": {
4    "available_pto": {
5      "minimum": 20,
6      "description": "We suggest 25 days although 20 is statutory, which are pro-rated in your first year of employment. Please note that Statutory, Bank Holidays and Public Holidays, based on employee's country of residence are excluded from the above.",
7      "presentation": {
8        "inputType": "number"
9      },
10      "title": "Available PTO/holiday days",
11      "type": "number"
12    },
13    "work_schedule": {
14      "description": "Please select if it is a full or part-time job.",
15      "oneOf": [
16        { "title": "Full-time", "const": "full_time" },
17        { "title": "Part-time", "const": "part_time" }
18      ],
19      "presentation": {
20        "inputType": "radio"
21      },
22      "title": "Work schedule",
23      "type": "string"
24    }
25  },
26  "required": ["available_pto", "work_schedule"]
27}

properties are the list of fields that feeds the DynamicForm. Each schema property includes a presentation which is a non-standard property that includes “visual” data from a field. In our example, the work_schedule has inputType set to radio which will map in the UI to a RadioField component with a list of options.

When it comes to validation, both fields are required and then each one has specific validation rules, available_pto requires a minimum value of 20 and work_schedule only accepts one of two values: full_time or part_time.

Using a JSON Schema parser on the frontend

We already have DynamicForm and input components so we only needed the last piece of the puzzle: build a JSON Schema parser, that could transform JSON into a list of fields (similar to the example above). Then validate that form and plug it into DynamicForm.

From a code standpoint this could be translated to the component below:

1import { createHeadlessForm } from "@remote-com/json-schema-form";
2
3export function MyComponent() {
4  const [data, setData] = useState();
5
6  useEffect(() => {
7    fetch("api/v1/json-schema")
8      .then((response) => response.json())
9      .then((data) => setData(data));
10  }, []);
11
12  if (!data) {
13    return null;
14  }
15
16  return <MyForm jsonSchema={data} />;
17}
18
19export function MyForm({ jsonSchema }) {
20  /**
21   * Example response: 
22     {
23      fields: [{
24        description: "Can be either full time or part time"
25        label: "Work schedule"
26        name: "workSchedule"
27        required: true,
28        options: [
29          { value: 'full_time', label: 'Full Time' },
30          { value: 'part_time', label: 'Part Time' },
31        ],
32        type: "radio"
33      },
34      {
35        description: "Time that employees can take off of work while still getting paid"
36        label: "Available PTO"
37        name: "availablePto"
38        required: true
39        type: "number"
40      }],
41      validationSchema: Yup.schema
42     }
43   */
44  const { fields, validationSchema } = createHeadlessForm(jsonSchema);
45  const handleSubmit = (values) => {
46    // send to server
47  };
48
49  return (
50    <DynamicForm
51      fields={fields}
52      validationSchema={validationSchema}
53      handleOnSubmit={handleSubmit}
54    />
55  );
56}

More complex examples using JSON schemas

For the sake of simplicity and readability of this guide we’re using simple schemas. However, many use-cases are much more complex.

A common scenario presents when field properties change based on another field value. This can be done with conditional logic in JSON Schemas and it’s handled by our parser, which means that validationSchema is recalculated when a field with dependencies changes.

In our specific example, let’s say that the minimum number for available PTO changes depending on the selected work_schedule option.

1{
2  "type": "object",
3  "additionalProperties": false,
4  "properties": {
5    "available_pto": { ... },
6    "work_schedule": { ... }
7  },
8  "required": ["work_schedule", "available_pto"],
9  "if": {
10    "properties": {
11      "work_schedule": {
12        "const": "part_time"
13      }
14    },
15    "required": ["work_schedule"]
16  },
17  "then": {
18    "properties": {
19      "available_pto": { "minimum": 0 }
20    }
21  },
22  "else": {
23    "properties": {
24      "available_pto": { "minimum": 20 }
25    }
26  }
27}

In the example, if part_time is selected, then available_pto takes a minimum of 0 — otherwise (if full_time was selected), the minimum is 20.

Since the adoption of this strategy, we already started to feel the differences when it comes to development velocity, just by reducing duplicated code, mostly related to validations, and all the required harmony between frontend and backend engineers.

This also brings an extra layer of security to the Remote Platform because all the validations live on the backend, making it impossible to brute force form submission. Moreover, in the long term, we expect to reduce our Javascript bundle size significantly as we migrate older forms to the new development pattern.

In conclusion, our engineering team is very happy with the come-up solution and we’re looking forward to a massive adoption in our platform.

Next steps for our form development

We worked hard and we are wrapping up a beta version of our JSON Parser library that we hope to open-source pretty soon.

Finally, you might be thinking that at this point we need someone to manually create these schemas on the backend. I can attest this can work, but the execution can also be a very time-consuming and prone to error.

The extra effort is worth it for our team, as we now have a single source of truth for such complex forms, and most importantly, we have the foundations to move to the next step: developing an interface to create JSON Schemas 🚀.

Stay tuned for the next update on this one!