Running Custom Stable Diffusion Models with RunPod

Want to run powerful Large Language Models (LLMs) on your own computer with flexible scalability? This guide will help you achieve just that!

Prerequisites

• A computer (local or cloud) with:

  • Docker installed

  • Git installed

  • Relatively fast upload speeds

  • At least 100GB of free storage space

• A custom model on your computer, or a public link to obtain it

Note: You currently can't build Docker images directly on RunPod.

This guide will use an example from https://github.com/runpod-workers/worker-a1111. You can create your own custom worker, but this guide will use this example for simplicity.

Setting Up Your Environment

1. Clone the repository:

   First, navigate to the directory where you want to work and run the git clone command to clone the repository:

         git clone https://github.com/runpod-workers/worker-a1111.git
       

   This will create a directory named "worker-a1111" in your folder. Access this directory with:

         cd worker-a1111
      

2. Examine the directory structure:

   You will see a few files and folders, including:

   • Dockerfile: Instructions for building the Docker image.

   • src/rp_handler.py: Python code executed as a worker.

   • src/start.sh: Script run when the container starts. This script internally calls the handler and starts the AUTOMATIC1111 API.

Replacing the Default Model with a Custom Model

To use a custom model, you need to take a few steps:

1. Remove the old model from the Docker image.

2. Add your custom model.

3. Modify the start.sh script to point to the new model.

Using a Local Model:

This example uses a civitai safetensors model from https://civitai.com/models/4823/deliberate.

• If the model is already downloaded locally, save it in the same directory and rename it to model.safetensors.

• Edit your Dockerfile by adding the following line:

        ADD model.safetensors /
     

  This will add your model file to the Docker image.

Downloading a Model from the Internet:

If you want Docker to download the model during the build process, use the RUN wget command instead of ADD:

      RUN wget -O /model.safetensors https://civitai.com/api/download/models/4823
    

Modifying the start.sh Script

• The start.sh script needs to point to the correct model file.

• If your model is named model.safetensors and is in the same directory, no changes are needed.

• If the name or location is different, change the line --ckpt /model.safetensors in the start.sh script accordingly.

Building the Docker Image

Once all changes are complete, you can build the Docker image using the command:

      sudo DOCKER_BUILDKIT=1 docker build .
    

Pushing the Image to Docker Hub (Optional)

• You can register an account at https://hub.docker.com/ to share your Docker image.

• Login to Docker Hub in your terminal and push your image to the repository you created.

Using the Template on RunPod

Once you have a Docker image, you can create a template on RunPod and populate it with the name of the image you pushed.

RunPod Documentation

Refer to the RunPod documentation for instructions on how to deploy and use your API:

• Autoscaling: https://docs.runpod.io/serverless-ai/custom-apis/autoscaling

• Using your API: https://docs.runpod.io/serverless-ai/custom-apis/using-your-api

Conclusion

While the default AUTOMATIC1111 API offers ease of use, this approach allows you to utilize custom models and manage their scalability on RunPod. Enjoy the flexibility of running your own Stable Diffusion models with the power of RunPod!