AI Storybook Example

An example project showing how to make a full-stack app that integrates elements of generative AI.

This project features the following platforms and frameworks:

• Replicate
• OpenAI
• LangChain.js
• Convex

Video

This project was coded on a live stream. The video, which introduces the concepts gradually and explains how the project was built, is found on YouTube here:

https://www.youtube.com/watch?v=4DEFIEHbC_s

Steps

The project is broken up an initial state, and then three steps/stages of development that are completed during the video. The end result of each step is a separate subdirectly in this repository.

• Step 0 (The initial state): A simple, frontend-only React app that shows a caurosel view with each cell representing a page in a children's book. The left side of the cell is the text of the page of the book, which can be edited by clicking the text and typing in the resultant textarea. The right side of the cell is reserved for an eventual AI-generated image to accompany the page contents. For now this will show a simple spinner. This basic React app will be more of less unchanged as we develop the rest of the project on the backend. The page state is driven by an in-app useState React state variable, but that will soon change.
• Step 1: A full-stack app, where we've swapped out the React state for a Convex backend. We'll replace useState with useQuery, and do the other plumbing necessary to make our app state stored in the backend.
• Step 2: Initial generative AI. We'll create a background task that fetches an image to represent the illustration for each page from a Replicate model using the LangChain.js library. The prompt for any given page will just be the page contents.
• Step 3: Richer, prompt-tuned generative AI. We'll introduce using "chaining" from LangChain.js, and we'll utilize it to both improve our prompts, and to connect ChatGPT to our app to further refine our storybook illustrations.