Chapter 20 — Web Programming with FastAPI

20.0. Web Programming

More and more experiments in cognitive science are being run online, using platforms such as Prolific or Connect to perform large-scale data collection. (Alas, Amazon Mechanical Turk has fallen out of favor due to the proliferation of bots and low-quality participants on the platform.)

Indeed, much of my own lab’s work is conducted using the web, which requires a completely different set of tools and skills than the ones used in traditional lab experiments. The goal of this chapter is to equip you with the basics of online experimentation by building a tiny web-based experiment. We will build a minimal demo of a survey that runs in your browser, sends responses to a local Python server, and saves those responses in a database locally on your own computer.

When building a traditional lab experiment, everything to do with the experiment can be run on the same computer: the experiment’s logic, presentation, and data collection are all performed on the same machine. Indeed, everything we have done in the class so far has made use of this same approach, including the experiment we built in Chapter 13. Unfortunately, this approach of one-computer-does-it-all becomes impossible when dealing with online experiments. Instead, web experiment programming typically relies on a client-server architecture. In this type of architecture, the components for the web experiment are split into three main parts:

The client (in this case, running in the participant’s web browser) takes care of all the participant-facing parts of the experiment. In our case, this means showing survey questions and collecting responses. This is also called the “frontend” of the experiment.
The server (in this case, a Python server running locally on your computer, though later you could make it accessible online by deploying it to a cloud service) takes care of any logic to do with setting up the experiment, as well as data collection. This includes receiving requests from the client, validating the data, storing it, and then providing some way for you (the experimenter) to access the data. This is also called the “backend” of the experiment.
Lastly, the database (in this case, a SQLite file stored on your computer) stores the data collected from the participants. There are other kinds of databases available, such as PostgreSQL or MySQL, but SQLite makes the most sense for a small, local demo like this one. We’ll cover how to use the database in Chapter 20.5.

Below is a diagram showing how data flows through the experiment, from the client to the server and eventually to the database. (There are some details in this diagram that will make more sense as you go through the chapter, such as GET and POST requests.)

flowchart LR
  subgraph Client["Client (participant browser)"]
    UI["HTML form (survey.html)"]
  end

  subgraph Server["Server (FastAPI app)"]
    R1["GET /  → serves HTML form"]
    R2["POST /submit → receives form data"]
    V["Validation (SQLModel / Pydantic)"]
    DBW["Write row to database"]
    R3["GET /responses (experimenter)"]
    EXP["Export (CSV) / summary endpoints (implemented in HW20)"]
  end

  subgraph DB["Database (SQLite file: survey.db)"]
    T["survey_responses table"]
  end

  UI -->|HTTP request| R1
  UI -->|HTTP POST form submission| R2
  R2 --> V
  V --> DBW --> T
  R3 --> T
  EXP --> T

In practice, this means that writing web experiments involves creating a full-stack web application, with both a frontend and a backend (which encompasses the database logic as well). The frontend is usually built with the languages natively understood by web browsers, such as HTML and CSS, and (perhaps most importantly) JavaScript, which is the primary programming language that a traditional web browser can execute directly (so most frontend logic is written in JavaScript). This is often supplemented with the use of various libraries and frameworks (such as React, Vue, or Svelte) to make the frontend more user-friendly and interactive, as well as developer-friendly.

However, because this is a Python course, we are going to keep the frontend extremely simple: we will build our survey using plain HTML forms and avoid any custom CSS or JavaScript.

The backend can be built with any number of languages typically used for server-side programming, such as Python, Node.js, PHP, C#, Ruby, Go, and many others, and each of these languages also has its own set of libraries and frameworks to scaffold development.

Naturally, in line with the rest of the book, this chapter will focus on building the backend using Python. We’ll use two modern Python libraries to accomplish this:

FastAPI: a lightweight framework for building Application Programming Interfaces (APIs). (Think of an API as a set of rules dictating how the client and server can communicate with each other.) Documentation available here
SQLModel: a library built on top of SQLAlchemy and Pydantic for defining database tables and validating data. (Yes, we will need to learn how to use databases in order to store our data!) Documentation available here

The docs are your friend

When in doubt, always refer to the documentation for the library you are using. Sometimes the docs can be intimidating, but they really are comprehensive, helping you understand how to use the library and its features. In these two cases, the docs are especially helpful, serving as literal tutorials for the libraries in question.

The survey we will build will be intentionally barebones. We’ll use a couple of Likert-scale items adapted from the Ten Item Personality Inventory (TIPI), plus an optional open-ended “comments” question.

As a rule of thumb, we try to avoid collecting personally identifying information (PII) by default. This means we avoid asking about names, emails, social security numbers, or anything else that could be used to identify the participant. (Of course, standard demographic questions like age or education level are not considered PII, and are commonly asked in surveys.)

What you’ll learn in this chapter

By the end, you should be able to:

Run a FastAPI server locally
Understand what a route/endpoint is and how to define one in FastAPI
Use your server’s interactive docs to debug your own routes/endpoints
Define database tables and validation schemas with SQLModel
Write API routes/endpoints that read from and write to the database
Build a tiny frontend page using plain HTML forms that can send data to your FastAPI server

20.1. Setup with `uv`

In Chapter 0, you learned how to install Python packages and manage virtual environments using uv. In this section, we are going to use uv to set up a small web project that uses FastAPI and SQLModel.

Tip

If you already have a bcog200 project with a .venv folder and a pyproject.toml, you can do all of this in that same project. If you want to keep this chapter separate, start by making a new folder with a name you can understand, like bcog200_web.

Step 1: Make (or enter) a project folder

Pick a folder you want to work in. For example, we can navigate to our bcog200 folder on the Desktop and create a new folder called bcog200_web in there:

cd ~/Desktop/bcog200
mkdir bcog200_web
cd bcog200_web

If you have not initialized this folder as a uv project yet, you can do so with:

uv init

Step 2: Create and activate a virtual environment

Create a virtual environment for this project (we will use Python 3.13, like earlier chapters):

uv venv .venv --python=3.13

Activate it:

# On Windows:
.venv\Scripts\activate

# On Mac/Linux:
source .venv/bin/activate

Step 3: Install the web packages

Install FastAPI, SQLModel, Jinja2, Python-Multipart, and the server we will use to run FastAPI (uvicorn), all with a single command:

uv add fastapi sqlmodel jinja2 python-multipart "uvicorn[standard]"

Jinja2 is a templating library for Python, and Python-Multipart is a library for parsing HTML form submissions. We will use those later on in the chapter.

This installs all those packages and updates your pyproject.toml automatically, so your dependencies are tracked correctly.

Step 4: Verify your setup

Let’s quickly check that your environment is active and the packages can be imported by running a small Python script inline (recall when we did this in Chapter 5.1). You should be able to just copy/paste this into your terminal at this point):

python -c "import fastapi, sqlmodel; print('FastAPI:', fastapi.__version__); print('SQLModel:', sqlmodel.__version__)"

You should see version numbers printed like below (and encounter no errors):

FastAPI: 0.115.6
SQLModel: 0.0.20

Now you should be good to go!

Running Python code inline

We just ran a little Python script inline. The -c flag tells Python to execute the code passed to it as a string. Semicolons are used here to separate valid statements, rather than using new lines.

20.2. A Simple FastAPI App

In this section, we are going to write the smallest working FastAPI app possible, run it locally, and use the built-in interactive documentation to test our endpoints. We will do so with the help of the FastAPI documentation, which gives us the basis for our working code below.

Step 1: Create a file called `main.py`

In your bcog200_web folder (the one where you ran uv init), create a file called main.py with the following code:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, query: str | None = None):
    return {"item_id": item_id, "query": query}

What is a “route” (aka an endpoint)?

A route (or endpoint) is a URL path, such as / or /items/1 above, that is connected to a Python function. When the client makes a request to the server at that path, the attached function is executed.

For example, the line @app.get("/items/{item_id}") says: “when the browser (or any client) makes a GET request to /items/{item_id}, run the function right below this line.” The dictionary we return in the function gets automatically converted into JSON (so it can be sent across the internet in a standard format for ease of use).

On Slugs

You may notice that in our /items route we actually passed in a variable called item_id in curly braces. This is called a slug, and it is a type of path parameter that you can use to capture a value from the URL. For example, if the URL is /items/52, then item_id will be set to 52. This approach is useful for all kinds of situations, such as accessing different blog posts or items sold in a store. As experimenters, we might use them for stimulus IDs or participant IDs. You can learn more about slugs in the FastAPI docs.

Step 2: Run the server with `uvicorn`

Make sure your virtual environment is active (you should see (.venv) or (bcog200) or whatever the name of our project environment is at the beginning of your terminal prompt). Then execute the following command in your terminal:

uvicorn main:app --reload

Let’s break this command down step by step:

uvicorn => the command to run the server
main => the file main.py (without the .py)
app => the variable named app inside that file
--reload => the flag to tell the server to restart automatically when you save any changes to your code. This is very helpful for development!

Step 3: Visit your app in the browser

Once uvicorn has started, it will print something like:

INFO:     Will watch for changes in these directories: ['/Users/yourusername/Desktop/bcog200_web']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [86932] using WatchFiles
INFO:     Started server process [86934]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

The most important line is the one that says Uvicorn running on http://127.0.0.1:8000. This means that the server is listening for requests at the URL http://127.0.0.1:8000. http:// is the protocol (as distinct from other types of protocols like ftp or ssh), 127.0.0.1 is the local IP address of your computer (you can substitute it with localhost if you prefer), and 8000 is the port number.

Now open your browser and try these URLs:

http://127.0.0.1:8000/
http://127.0.0.1:8000/items/1

The first route will return a JSON response like:

{
  "Hello": "World"
}

The second route will return a JSON response like:

{
  "item_id": 1,
  "query": null
}

JSON and query parameters

Recall that we learned a bit about JSON in Chapter 4.3. To jog your memory, JSON is a standard format for representing data that can be easily read and written by computers and humans alike. It actually looks a lot like a Python dictionary (or list of dictionaries, depending on the data structure).

Query parameters are optional key-value pairs appended to the end of the URL using a question mark. For example, http://127.0.0.1:8000/items/1?query=test has the query parameter query=test which says that the value of query is the string test.

Now, if we add a query parameter to our URL, we should be able to see the query parameter in the response. Go to http://127.0.0.1:8000/items/1?query=test and you should see the response below:

{
  "item_id": 1,
  "query": "test"
}

Step 4: Try out the interactive docs

FastAPI generates interactive documentation for you automatically at the /docs and /redoc endpoints. (When I refer to endpoints like this, I mean you should substitute the full URL, such as http://127.0.0.1:8000/docs or http://127.0.0.1:8000/redoc.)

Open /docs and try clicking on GET /items/{item_id}, then click “Try it out”, enter values for the item_id and query parameters, and click “Execute”. This will send a GET request to the server at that endpoint, just as if you had gone to the URL http://127.0.0.1:8000/items/1?query=test in your browser.

This kind of interactive documentation can serve as a sort of “debugging dashboard”, especially as we add more endpoints.

Stop the server

While uvicorn is running, you may notice that you cannot interact with your terminal as expected. This is because your terminal is “occupied” by the server. To stop it, click anywhere on the terminal and press Ctrl + C. That’s the universal shortcut to quit any process currently running in your terminal.

Troubleshooting

If you managed to get through all the steps above, great! Feel free to move on to the next section.

If you ran into problems, this section will go through some common issues you might encounter when first running your server.

“Address already in use”

This usually means something else is already running on port 8000 (perhaps you have another instance of uvicorn running in another terminal window?). You can always choose to run your server on a different port like so:

uvicorn main:app --reload --port 8001

“Error loading ASGI app. Could not import module ‘main’”

This usually means one of these is true:

You’re not in the folder that contains main.py (e.g. your bcog200_web folder) — this means your terminal is not in the correct directory
Your file isn’t named exactly main.py — simply rename it
Your FastAPI variable isn’t named app (e.g. it might be app_web or app_main or something else) — again, simply rename it

20.3. Drafting the survey with HTML

In the previous section, we built a tiny FastAPI server and learned about routes/endpoints by returning simple JSON responses. However, we would like to have our server return something a participant can actually use.

In this section, we will build the frontend of our survey using a plain HTML form. When the participant visits our server, they will see a page with a few questions, hit Submit, and then see a simple “thank you” page.

This is just for drafting purposes, so we won’t store any data yet — that is its own kettle of fish. For now, we just want to send form data from the browser to the server, to have the participant “communicate” in some sense with our server. (We’ll handle data validation and storage in the next section.)

Step 1: Ensure Jinja2 and Python-Multipart are installed

FastAPI can serve HTML pages using templates. We’ll use Jinja2, which is the most common templating library in Python. If you haven’t already installed them, be sure to install Jinja2 and Python-Multipart.

uv add jinja2 python-multipart

Step 2: Create a `templates/` folder to store our HTML templates

In the same folder as your main.py, create a folder named templates.

Your project folder should look like this:

bcog200_web/
  main.py
  templates/

Step 3: Create HTML templates in `templates/`

Alas, in order to make a frontend, we will need to write some HTML. Though we won’t go into the details of HTML here, you can learn about it at your leisure on w3schools. For our purposes, all you need to know is that it’s a simple markup language for displaying content in a web browser. Think of it as a more complicated version of Markdown that uses opening tags (e.g., <div>, <p>, <h1>, <h2>, etc.) and closing tags (e.g., </div>, </p>, </h1>, </h2>, etc.) to define the structure of the page. These tags also have various attributes that you can define.

For now, create a file called survey.html inside your templates/ folder and add the following content to it (you can just copy/paste this directly). This is a tiny survey based on the Ten Item Personality Inventory (TIPI). For demo purpsoses, we’ll only use two of the TIPI statements (plus an optional comments question).

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>BCOG 200 Demo TIPI Survey</title>
  </head>
  <body>
    <h1>Demo TIPI Survey</h1>

    <p>
      Here are a number of personality traits that may or may not apply to you.
      Please rate on a scale of 1 to 7 to indicate the extent to which you agree
      or disagree with that statement. You should rate the extent to which the
      pair of traits applies to you, even if one characteristic applies more
      strongly than the other.
    </p>

    <form method="post" action="/submit">
      <div>
        <p><strong>I see myself as:</strong> Extraverted, enthusiastic.</p>
        <label for="tipi_extraverted">Rating:</label>
        <select id="tipi_extraverted" name="tipi_extraverted" required>
          <option value="" selected>Please select an option</option>
          <option value="1">1 (disagree strongly)</option>
          <option value="2">2</option>
          <option value="3">3</option>
          <option value="4">4 (neither agree nor disagree)</option>
          <option value="5">5</option>
          <option value="6">6</option>
          <option value="7">7 (agree strongly)</option>
        </select>
      </div>

      <div>
        <p><strong>I see myself as:</strong> Reserved, quiet.</p>
        <label for="tipi_reserved_quiet">Rating:</label>
        <select id="tipi_reserved_quiet" name="tipi_reserved_quiet" required>
          <option value="" selected>Please select an option</option>
          <option value="1">1 (disagree strongly)</option>
          <option value="2">2</option>
          <option value="3">3</option>
          <option value="4">4 (neither agree nor disagree)</option>
          <option value="5">5</option>
          <option value="6">6</option>
          <option value="7">7 (agree strongly)</option>
        </select>
      </div>

      <div>
        <label for="comments">Any additional comments? (optional)</label><br />
        <textarea id="comments" name="comments" rows="4" cols="60"></textarea>
      </div>

      <button type="submit">Submit</button>
    </form>
  </body>
</html>

This page displays a form with two Likert-style questions (1-7) and an optional open-ended comments box. We’ve made the Likert scale questions required, so the participant cannot submit the form without answering them.

A few other notes:

The action="/submit" means: “send this form to the /submit endpoint on our server”.
The method="post" means: “send it as a POST request (not a GET request)”.
The name="XXX" attributes are the ways we can refer to the input fields on the server (XXX is, of course, just a placeholder here)

If you open the file directly on your computer using your web browser, you should see a very simple page with the form. Of course, it won’t do anything yet, as we haven’t added any logic to the server to handle the form data.

While we’re at it, let’s create a template for the “Thank you” page. Create a file called thank_you.html inside your templates/ folder and add the following content:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Thank you</title>
  </head>
  <body>
    <h1>Thank you!</h1>
    <p>Your response was received.</p>
    <h2>Debug view (to be removed later)</h2>
    <ul>
      <li>tipi_extraverted: {{ tipi_extraverted }}</li>
      <li>tipi_reserved_quiet: {{ tipi_reserved_quiet }}</li>
      <li>comments: {{ comments }}</li>
    </ul>
  </body>
</html>

In addition to run-of-the-mill static HTML, this page also uses Jinja2 templating syntax to display the submitted data in a dynamic fashion. This type of templating is denoted by the use of double curly braces, like { tipi_extraverted }. This means “insert the value of tipi_extraverted from the context dictionary provided to the template”. And the context dictionary is the dictionary of variables that is passed to the template when it is rendered (we’ll see how that works in the next couple of steps).

We are going to show the submitted values only as a means of making our debugging easier for now. In a real survey, you usually would not echo the participant’s answers back to them unless you have a reason to do so, but it’s quite common to use this sort of templating to provide custom feedback or debriefing to the participant.

Step 4: Update `main.py` to serve the survey page

Now we need to ensure that our server can show this HTML page to the participant when they visit, and then save the data when they submit the form. Let’s add it to the root of our server.

We’re going to update our main.py file with a new route that serves the HTML page for now. Later, we will add another that receives the form data and shows a thank-you page. We will also need to add a couple of imports to the top of the file to handle some of this functionality.

from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from typing import Any

app = FastAPI()

templates = Jinja2Templates(directory="templates")


def cast_to_string(value: Any) -> str:
    # used when dealing with `request.form()`
    # the values are typed as `str | UploadFile` but here they will just be strings
    # we use this to avoid type errors in our editor
    return value if isinstance(value, str) else ""


@app.get("/", response_class=HTMLResponse)
def survey_form(request: Request):
    return templates.TemplateResponse(
        request=request,
        name="survey.html",
        context={"request": request},
    )

(I’ve included a cast_to_string() function to avoid the editor complaining later.)

As you can imagine, this new survey_form() function returns the rendered template survey.html to the participant when they visit the root of our server. Visit the root of your server in your browser (e.g., http://127.0.0.1:8000/) to see the form. You should even see the form has some basic input validation, but pressing the “Submit” button will take you to a page that doesn’t have a route we defined yet (yielding a “{detail: ‘Not Found’}” JSON response). We’ll fix this in the next step.

Step 5: Handle the form submission

Next, let’s add a route that receives the form data and shows the “Thank you” page with the details of the submitted data, just as a sanity check to make sure everything is working. Below our previous route, add the following:



@app.post("/submit", response_class=HTMLResponse)
async def submit_survey(request: Request):
    form = await request.form()

    # Get the form data and remove any whitespace
    # from the beginning/end of the strings
    tipi_extraverted_raw = cast_to_string(form.get("tipi_extraverted")).strip()
    tipi_reserved_quiet_raw = cast_to_string(form.get("tipi_reserved_quiet")).strip()
    comments = cast_to_string(form.get("comments")).strip() or None

    # Convert the data to the appropriate types
    tipi_extraverted = int(tipi_extraverted_raw) if tipi_extraverted_raw else None
    tipi_reserved_quiet = int(tipi_reserved_quiet_raw) if tipi_reserved_quiet_raw else None
    return templates.TemplateResponse(
        request=request,
        name="thank_you.html",
        context={
          "request": request,
          "tipi_extraverted": tipi_extraverted,
          "tipi_reserved_quiet": tipi_reserved_quiet,
          "comments": comments,
        },
    )

There are a few new things to note here. First, we are associating this route with a POST request, rather than a GET request (as shown by the @app.post("/submit", response_class=HTMLResponse) decorator). HTML requests have a handful of different methods or “verbs” that dictate how the data is sent to the server, such as GET, POST, PUT, DELETE, and PATCH. Here we’ll focus on GET and POST.

GET requests are used to retrieve data from the server (as with our survey_form() function, which returns the HTML form to the client).
POST requests, on the other hand, are used when the client wants to send data to the server (as with our submit_survey() function). Of course, in practice, POST requests often also return data themselves (like the “Thank you” page returned by the submit_survey() function) so it can be a two-way street.

We won’t cover the other HTML verbs here, as GET and POST are all we need for now. However, if you’re curious, you can learn more about the other HTML verbs in the MDN documentation.

Second, this route makes use of an async function. This type of function stands in contrast to “synchronous” functions, which are the default type of function in Python, and the only type we have written so far in this course.

So what’s the difference between these two types of functions?

In a synchronous function (def), Python runs the code step-by-step, and if it hits an operation that needs to wait (for example, waiting for data to arrive from the network), that function blocks execution until it’s done.
In an asynchronous function (async def), the function is allowed to pause at specific points using await. While it is paused, the server can go do other work (like handling another request from another browser tab). When the paused operation is done, the function resumes execution. Writing asynchronous functions is a little more complicated, but it keeps the server responsive and able to handle multiple requests at once.

In our case, the key line that requires us to use an asynchronous function is:

form = await request.form()

Reading/parsing the incoming form body is implemented as an asynchronous operation in FastAPI, so we must use the await keyword to pause and wait for that data to come in. Any time you need to use the await keyword, you must define the function with the async def keywords.

A few other important details:

The request: Request parameter in the function definition is required for templates to work properly
Everything arrives as strings, so we handle type conversion ourselves (for now).

Step 6: Run the server and test the survey

Now we’re ready to try out our survey! In your terminal, run the same command you used to start the server in the previous section:

uvicorn main:app --reload

Then visit the root of your server in your browser (e.g., http://127.0.0.1:8000/) to see the survey form.

Fill out the form and submit it. You should see the “Thank you” page with the details of the submitted data!

Troubleshooting

As you’re getting started, you may encounter issues. This section will be updated with common issues faced by the class and their solutions, so if you run into an error, first check here, then contact me/the TAs to get your issue solved and potentially added to the book.

“RuntimeError: Form data requires ‘python-multipart’”

As the error message suggests, you need to install the python-multipart package.

uv add python-multipart

“TemplateNotFound: survey.html”

Make sure you created a templates/ folder in the same place as main.py and that the HTML file is exactly named survey.html and exists within that templates/ folder.

What comes next?

We can now send participant responses from the browser to our server. Next, we’ll define our data model using SQLModel and save each submission to a real database.

20.4. Modeling Survey Data with SQLModel

In our web survey, participants will submit their responses through a form in their browser. On the server side, we need a way to do three things:

Validate incoming data (e.g., make sure a 1-7 Likert rating is actually an integer between 1 and 7, not a string or a float)
Store that data in a database, so we can analyze it later
Export that data later, so we can analyze it and share it with other researchers

Luckily, we have SQLModel to help us with all of these things.

SQLModel is built on top of:

SQLAlchemy — the most popular Object-Relational Mapping (ORM) library in Python
Pydantic — a library for data validation using Python type hints (recall that we covered type hints briefly in Chapter 6.4 Data Classes)

Pydantic classes look a lot like data classes, but they come with a number of extra features. Most importantly for our purposes, they come with automatic data validation, and they can be used to create our database tables as well.

Step 1: Create a file called `models.py`

In your root project folder (where the main.py script lives), create a file called models.py. Let’s start by adding a base class for our survey responses. This will store all the shared fields for our various types of survey response classes.

from datetime import datetime, timezone

from sqlmodel import Field, SQLModel


class SurveyResponseBase(SQLModel):
    """
    The shared fields for a survey response.
    """

    # TIPI-style items (intentionally minimal; no PII).
    # Scale: 1 = disagree strongly, 7 = agree strongly.
    tipi_extraverted: int = Field(ge=1, le=7)
    tipi_reserved_quiet: int = Field(ge=1, le=7)

    # Optional open-ended response.
    comments: str | None = Field(default=None, max_length=2000)

Next, let’s create the table that will store our survey responses. This table will inherit from our base class, and we’ll set table=True to let SQLModel know that we will create an SQL table for this class. It’s important to note that in a database, each row is a unique entry, so it needs an id field to uniquely identify each row. We’ll also add a primary_key=True flag to the id field to tell SQLModel that this is the primary key for the table. In addition, it helps to know when each response was created, so we will add a created_at field to the table, such that each response has a timestamp (in UTC time) of when it was created.

class SurveyResponse(SurveyResponseBase, table=True):
    """
    This is the database table that will store our survey responses.

    Note the use of `table=True`; this will allow SQLModel to create an SQL table.
    """

    id: int | None = Field(default=None, primary_key=True)
    created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))

Next, let’s make a helper class for sending formatted survey response data from the server to the client.


class SurveyResponseOut(SurveyResponseBase):
    """
    This is what we send back to the client (it includes server-generated fields).

    This output model is used to ensure that we don't accidentally return private information
    back to the client (in this case, the `id` field).

    In a real-world application, we might also have an "input model" for when the client
    sends data to the server as well, to validate that data before trying to save it.
    """

    created_at: datetime

Interim summary

Reusable data models

Having done all that, we’ve set up a number of useful data models, one of which is simultaneously a database table. For example:

SurveyResponseBase is not a database table. It is just a reusable set of typed fields, representing the shared fields for all survey response data models we will use.
SurveyResponse is a database table because it includes table=True in its class definition.
SurveyResponseOut is not a database table. It is used to send the formatted survey response back to the client.

This pattern keeps things clean: we can reuse the same field definitions for both “API input/output” and “database storage”, but still keep the database-only fields (like id) private and separate.

`Field(...)` lets you add database and validation rules

We made use of the Field(...) function for each of the properties of the classes. This is a special function that allows us to add database and validation rules to our data models. For example:

id: int | None = Field(default=None, primary_key=True) creates an auto-incrementing integer primary key.
tipi_extraverted: int = Field(..., ge=1, le=7) validates that a Likert rating is 1-7. ge means “greater than or equal to” and le means “less than or equal to”.
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) creates a timestamp in UTC time (to avoid confusion surrounding what timezone the participant was in). Note the use of the default_factory parameter to set the default value to the current UTC time at the time of creation! This is different from other types of default values, as it is a dynamic value (differing with each new instance of the class) rather than a static one.
comments: str | None = Field(default=None, max_length=2000) validates that the comments are an optional string with a maximum length of 2000 characters.

Of course, there are many other types we could use in our database. We could use float for computed trait scores, or bool for yes/no responses, and so on. Often, for these computed values, they need not be validated as Field(...) because they are computed after the fact. So in that case you can just declare them as you ordinarily would in a data class. E.g., extraversion: float or submitted_comments: bool.

Having gone to all this effort, later on, when the client submits data via FastAPI, invalid values will automatically produce helpful error messages.

Next steps

This section was all about setting up the architecture for our database and data models. But on their own, they’re quite useless. In the next section we will need to put them to use by:

creating a database engine
creating the database tables automatically
opening a database session and inserting rows into the database
exporting the data from the database in CSV format

20.5. Setting Up the Database

Now that we’ve defined our survey data model using SQLModel, we can actually create a real database on your computer’s hard drive and start saving participant responses to it.

We will use SQLite, which is a lightweight database that stores everything in a single file. It is perfect for small demos, local development, and many real research projects. (In production, out on the web, we would typically use a more robust database like PostgreSQL.)

Step 0: Decide where the database file should live

In this chapter, we’ll keep things simple and store the database file in the root project folder; the same folder as main.py. Let’s call it survey.db.

SQLite will create this file for us automatically when we first connect to the database.

Step 1: Create a database engine

Next, we will need a way to connect to the database. We will do this by creating a new file in your root project folder called database.py (again, next to main.py) and adding the following code:

from sqlmodel import SQLModel, create_engine

# This is a "connection string" that tells SQLModel/SQLAlchemy to use SQLite and store the database
# in a file named survey.db in the current folder.
sqlite_url = "sqlite:///survey.db"

# `echo=True` prints SQL statements to the terminal. Note that we
# don't need to do this in production, as it will slow down the server, but we do it here
# for learning and debugging purposes.
engine = create_engine(sqlite_url, echo=True)

def create_db_and_tables() -> None:
    """
    Create the database tables based on any SQLModel classes with `table=True`.
    """
    SQLModel.metadata.create_all(engine)

Step 2: Create the tables from our SQLModel classes

Remember: tables are created only from the classes where you used table=True (in our case, SurveyResponse).

To make sure SQLModel knows about that table, we need to import it before we create the tables.

Update the top of main.py to include the following imports, which will allow us to create the database tables:

from database import create_db_and_tables
from models import * # This will import all the SQLModel classes from models.py

# rest of app here...

Lastly, we will call create_db_and_tables() when the app starts. The simplest way to do this would be to import it at the top of main.py and then run it inline. However, a better (but slightly more complicated) way is to use the lifespan feature of FastAPI, like so:

# first, we import the asynccontextmanager from the contextlib module
# at the top of main.py
from contextlib import asynccontextmanager

# rest of imports here...

# then, we can define a lifespan context manager
# that will run the create_db_and_tables() function when the app starts

@asynccontextmanager
async def lifespan(app: FastAPI):
    # The code below runs on startup
    create_db_and_tables()
    yield
    # Any code below here would run on shutdown
    # but we don't have any cleanup to do here,
    # so we can just leave it empty

Finally, make sure you pass that lifespan function into your FastAPI app:

app = FastAPI(lifespan=lifespan)

Now, any time the app starts anew, the create_db_and_tables() function will be called.

After you restart the app with uvicorn, you should see SQL statements printed in the terminal and a survey.db file appear in your project folder.

Step 3: Open a database session (so we can insert rows)

Now we will need a way to open a session to the database, so we can make changes to it. Update the top import statement in your database.py file to include Session from sqlmodel, as well as the Generator type hint from typing, then create a get_session() function that returns a new Session object:

from sqlmodel import SQLModel, create_engine, Session
from typing import Generator


def get_session() -> Generator[Session, None, None]:
    """
    Returns a generator yielding a new Session object.
    This is a context manager that will automatically close the session when the block finishes.
    """
    with Session(engine) as session:
        yield session

This creates a Session, which is how we talk to the database: insert rows, query rows, etc. (Don’t worry too much about the Generator type hint here — the important thing is that it will give us a Session object. We could have written a much simpler function that just returned a Session directly, but this way we won’t have to worry about closing the session manually.)

Step 4: Save a survey response when the participant clicks Submit

Now we can update our /submit route in main.py so that it creates a SurveyResponse and saves it to the database. We will use the get_session() function to open a session to the database, and then use the session object to insert the new response into the database. However, we will do so with a helpful feature in FastAPI: dependency injection.

First, let’s update our imports at the top of main.py to ensure we have everything we need.

# These should be all the imports you need for now
from contextlib import asynccontextmanager

from fastapi import FastAPI, Request, Depends
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates

from sqlmodel import Session

from database import create_db_and_tables, get_session
from models import SurveyResponse, SurveyResponseOut
from typing import Any

Then update your /submit route to be the following:

# We're adding a new parameter to the submit_survey function — `session: Session = Depends(get_session)`
# This means that the session object will be automatically created and passed to the function.
@app.post("/submit", response_class=HTMLResponse)
async def submit_survey(request: Request, session: Session = Depends(get_session)):
    form = await request.form()

    tipi_extraverted_raw = cast_to_string(form.get("tipi_extraverted")).strip()
    tipi_reserved_quiet_raw = cast_to_string(form.get("tipi_reserved_quiet")).strip()
    comments = cast_to_string(form.get("comments")).strip() or None

    response = SurveyResponse(
        tipi_extraverted=int(tipi_extraverted_raw),
        tipi_reserved_quiet=int(tipi_reserved_quiet_raw),
        comments=comments,
    )

    # Add the response to the session, commit the changes,
    # and refresh the response object
    session.add(response)
    session.commit()
    session.refresh(response)

    return templates.TemplateResponse(
        request=request,
        name="thank_you.html",
        context={
            "request": request,
            "tipi_extraverted": response.tipi_extraverted,
            "tipi_reserved_quiet": response.tipi_reserved_quiet,
            "comments": response.comments,
        },
    )

Now each time a participant submits the form, their response is saved to survey.db. This is accomplished by using the session object to add the response to the session, commit the changes, and refresh the response object.

Why do we bother calling session.refresh(response)?

You may think our work is done after we add and commit the response. However, refreshing can be useful, as it updates the Python object with any server-generated values (like id and created_at) that the database filled in, making it easier to work with as a developer. If you didn’t call refresh(response), you would not have access to those server-generated values in the Python object itself.

Step 5: Add a route to quickly view the saved data

It’s often helpful to add an “experimenter view” route while you’re building the survey.

Here’s a simple JSON endpoint you can add to main.py to view the saved data:

# other imports above here...
from sqlmodel import select


@app.get("/responses", response_model=list[SurveyResponseOut])
def list_responses(session: Session = Depends(get_session)):
    responses = session.exec(select(SurveyResponse)).all()
    return responses

There’s one more trick being done in that function: we’re declaring the response_model to be a list of SurveyResponseOut objects. This is a helpful feature of FastAPI that will automatically convert the list of SurveyResponse objects into a list of SurveyResponseOut objects, which is what we actually want to return to the client/web browser. If you visit http://127.0.0.1:8000/responses, you should now see a JSON list of stored responses. And if you make any changes to the SurveyResponseOut class, those changes will be reflected in the responses you see. For example, we can add the id field to the SurveyResponseOut class, and the responses you see will now include the id field. Right now, the responses are sanitized to remove that information, even though the original SurveyResponse objects still have that field (thanks, FastAPI!).

Conclusion

At last, we have a complete minimal experiment pipeline! We now have:

A simple HTML form for the participant to fill out and submit
A FastAPI app that can receive the form submission
Functions that use SQLModel to validate/type the data and save it to the database
The ability to view the saved data via a JSON endpoint

There are many other features we could add down the road, including exporting the database to CSV; password-protected administrator pages; a dashboard for us to monitor the experiment’s progress; and so on. But the code we’ve written so far provides all the basic functionality we need to get started with running a survey on the web (minus deployment, for which there are many options).

20.6. Lab 20

For this lab, your goal is to get our tiny online survey from Chapter 20 working from start to finish. This is especially important as you will build on this for Homework 20! Follow the steps from Chapter 20 and make sure your survey is working. This means that all of the following conditions should be met:

You can run your server with uvicorn main:app --reload and it will start up
Visiting http://127.0.0.1:8000/ shows your TIPI demo form
Submitting the form shows your “Thank you” page along with the details of the submitted data
Submitting the form creates (or updates) a survey.db file in your project folder
Visiting http://127.0.0.1:8000/responses shows a list of saved responses as JSON

Once your survey is working, submit at least 3 distinct responses. Zip up the project folder, including the survey.db file and name it lab20.zip. You will submit this zip file as your lab assignment.

20.7. Homework 20

For Homework 20, you will turn our Chapter 20 demo (from the lab) into a more complete mini online experiment. In addition to getting the survey to work and saving responses, you will implement TIPI scoring, including reverse scoring, and store the computed trait scores in your database. In addition, you will implement new endpoints for the experimenter, as well as a a script to populate the database with simulated data.

Your goals are:

Serve the full 10-item TIPI survey as an HTML form
Accept submissions with FastAPI
Save raw responses to a SQLite database using SQLModel
Compute the Big Five trait scores (including reverse scoring) on the server when the form is submitted
Save the computed scores to the database
Provide an experimenter-friendly way to view and export the data

Project folder

Create a folder called HW20/ (or reuse your Chapter 20 project folder and rename it). Inside it you should have:

main.py
models.py
database.py
templates/
- survey.html
- thank_you.html
populate_db.py (you will create this file yourself)

Part 1: The complete TIPI survey (HTML)

Below we have the new templates/survey.html file, which will display the full 10-item TIPI scale as an HTML form.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>TIPI Survey</title>
  </head>
  <body>
    <h1>Personality Survey</h1>

    <p>
      Here are a number of personality traits that may or may not apply to you.
      Please indicate the extent to which you agree or disagree with each
      statement using a scale from 1 to 7, where 1 means "disagree strongly" and
      7 means "agree strongly". You should rate the extent to which the pair of
      traits applies to you, even if one characteristic applies more strongly
      than the other.
    </p>

    <form method="post" action="/submit">
      <ol>
        <li>
          <p><strong>I see myself as:</strong> Extraverted, enthusiastic.</p>
          <label for="tipi_1">Rating:</label>
          <select id="tipi_1" name="tipi_1" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Critical, quarrelsome.</p>
          <label for="tipi_2">Rating:</label>
          <select id="tipi_2" name="tipi_2" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Dependable, self-disciplined.</p>
          <label for="tipi_3">Rating:</label>
          <select id="tipi_3" name="tipi_3" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Anxious, easily upset.</p>
          <label for="tipi_4">Rating:</label>
          <select id="tipi_4" name="tipi_4" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p>
            <strong>I see myself as:</strong> Open to new experiences, complex.
          </p>
          <label for="tipi_5">Rating:</label>
          <select id="tipi_5" name="tipi_5" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Reserved, quiet.</p>
          <label for="tipi_6">Rating:</label>
          <select id="tipi_6" name="tipi_6" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Sympathetic, warm.</p>
          <label for="tipi_7">Rating:</label>
          <select id="tipi_7" name="tipi_7" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Disorganized, careless.</p>
          <label for="tipi_8">Rating:</label>
          <select id="tipi_8" name="tipi_8" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Calm, emotionally stable.</p>
          <label for="tipi_9">Rating:</label>
          <select id="tipi_9" name="tipi_9" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>

        <li>
          <p><strong>I see myself as:</strong> Conventional, uncreative.</p>
          <label for="tipi_10">Rating:</label>
          <select id="tipi_10" name="tipi_10" required>
            <option value="" selected>Please select an option</option>
            <option value="1">1 (disagree strongly)</option>
            <option value="2">2</option>
            <option value="3">3</option>
            <option value="4">4 (neither agree nor disagree)</option>
            <option value="5">5</option>
            <option value="6">6</option>
            <option value="7">7 (agree strongly)</option>
          </select>
        </li>
      </ol>

      <div>
        <label for="comments">Any additional comments? (optional)</label><br />
        <textarea id="comments" name="comments" rows="4" cols="60"></textarea>
      </div>

      <button type="submit">Submit</button>
    </form>
  </body>
</html>

Part 2: Updating data models

Update the models.py file so that the database stores:

Raw TIPI responses: tipi_1 through tipi_10 (integers between 1 and 7)
Optional comments string (up to 2000 characters long)
Computed Big Five scores (floats):
- extraversion
- agreeableness
- conscientiousness
- emotional_stability
- openness
  - To compute these scores, you will need to implement the scoring logic in your /submit route and add the computed scores to the entry in the database
Server-generated fields: id, created_at

Validation remains important. All raw items input by the user must validate as integers between 1 and 7 using Field(ge=1, le=7). The optional comments field should be a string with a maximum length of 2000 characters.

Part 3: Implement trait scoring of the Big Five

While half of the questions are straightforward to score, half of them are reverse-scored. E.g., if the user entered a 7 for question 1, that should be scored as 1 (a 6 would become 2, a 5 would become 3, etc.).

To compute the reverse of a score, you simply do new_score = 8 - original_score. We use 8 because it’s the maximum score (in this case, 7) plus 1.

Then, you can calculate the mean of the two items that are designed to measure the trait. For example, extraversion is the mean of item 1 and the reverse of item 6. The full scoring key is given below:

Extraversion: items 1 and 6 (reverse-scored)
Agreeableness: items 2 (reverse-scored) and 7
Conscientiousness: items 3 and 8 (reverse-scored)
Emotional Stability: items 4 (reverse-scored) and 9
Openness: items 5 and 10 (reverse-scored)

Implement a function (in the main.py file or a helper module) that:

takes the responses tipi_1 through tipi_10 as integers
performs reverse scoring where appropriate
takes the mean of the two items for each trait
returns the five trait scores as floating point numbers

Then, update your /submit route so it:

reads form inputs
converts the responses to integers
calls the function you just implemented to compute the trait scores
saves everything to the database

Part 4: Experimenter features

As experimenters, we need some way to get access to the data we’ve collected, and to monitor the progress of the experiment as it’s ongoing. To this end, add two new routes:

GET /summary

this route returns JSON summarizing the data collected so far
this summary includes:
- the number of responses collected (num_responses)
- the mean of the responses for each trait (means, which is a dictionary)
- the standard deviation of the responses for each trait (stds, which is a dictionary)

For example, the JSON for your experiment’s summary could look something like this

{
  "num_responses": 42,
  "means": {
    "extraversion": 4.9,
    "agreeableness": 5.1,
    "conscientiousness": 4.2,
    "emotional_stability": 3.8,
    "openness": 5.6
  },
  "stds": {
    "extraversion": 1.1,
    "agreeableness": 0.9,
    "conscientiousness": 1.3,
    "emotional_stability": 1.4,
    "openness": 1.0
  }
}

GET /export_to_csv
- this route returns a CSV download of all responses
  - This can be accomplished by using either the csv module or the pandas library
  - For example, you can convert a given response to a dictionary using response.model_dump() and then write the dictionary to a CSV file using csv or pandas
  - Then you can return the CSV file as a PlainTextResponse (imported from fastapi.responses)
- The CSV should include the raw item responses and computed scores for each entry

Part 5: Test script to populate the database

Create populate_db.py that inserts at least 30 automatically generated fake responses into the database so you can easily test your two new endpoints from Part 4. (If your database has responses in it that are malformed or that you don’t want in there for any reason, you can delete the survey.db file and run the script again.)

Your populate_db.py script must:

generate random integers between 1 and 7 as responses for each survey question (tipi_1 to tipi_10)
include optional comments on some (but not all!) rows
- You may “roll the dice” to decide whether to include a comment (let’s say there is a 30% chance of a given entry having a comment)
- A comment should be a random string of 10-30 characters (you can use the random module to generate a random string; see Chapter 3.5, 5.0, and 8.7 for examples)
- Make sure to compute the trait scores for each response and save them to the database (using the function you implemented in Part 3)

What to submit

Submit a zip file named lastname_firstname_hw20.zip It must include your entire HW20/ folder with all the code necessary to run the experiment. This includes your automatically populated survey.db database file

When we unzip the project, it should run as-is using the command uvicorn main:app --reload. If it does not, then it will not be graded.

20.0. Web Programming

What you’ll learn in this chapter

20.1. Setup with uv

Step 1: Make (or enter) a project folder

Step 2: Create and activate a virtual environment

Step 3: Install the web packages

Step 4: Verify your setup

20.2. A Simple FastAPI App

Step 1: Create a file called main.py

What is a “route” (aka an endpoint)?

Step 2: Run the server with uvicorn

Step 3: Visit your app in the browser

JSON and query parameters

Step 4: Try out the interactive docs

Troubleshooting

“Address already in use”

“Error loading ASGI app. Could not import module ‘main’”

20.3. Drafting the survey with HTML

Step 1: Ensure Jinja2 and Python-Multipart are installed

Step 2: Create a templates/ folder to store our HTML templates

Step 3: Create HTML templates in templates/

Step 4: Update main.py to serve the survey page

Step 5: Handle the form submission

Step 6: Run the server and test the survey

Troubleshooting

“RuntimeError: Form data requires ‘python-multipart’”

“TemplateNotFound: survey.html”

What comes next?

20.4. Modeling Survey Data with SQLModel

Step 1: Create a file called models.py

Interim summary

Reusable data models

Field(...) lets you add database and validation rules

Next steps

20.5. Setting Up the Database

Step 0: Decide where the database file should live

Step 1: Create a database engine

Step 2: Create the tables from our SQLModel classes

Step 3: Open a database session (so we can insert rows)

Step 4: Save a survey response when the participant clicks Submit

Step 5: Add a route to quickly view the saved data

Conclusion

20.6. Lab 20

20.7. Homework 20

Project folder

Part 1: The complete TIPI survey (HTML)

Part 2: Updating data models

Part 3: Implement trait scoring of the Big Five

Part 4: Experimenter features

Part 5: Test script to populate the database

What to submit

20.1. Setup with `uv`

Step 1: Create a file called `main.py`

Step 2: Run the server with `uvicorn`

Step 2: Create a `templates/` folder to store our HTML templates

Step 3: Create HTML templates in `templates/`

Step 4: Update `main.py` to serve the survey page

Step 1: Create a file called `models.py`

`Field(...)` lets you add database and validation rules