Back to videos

Build a 3D AI Teacher w/ Next.js, ChatGPT & Azure

Overview

Are you ready to take your language learning to the next level with AI? In this tutorial, we will create an interactive 3D AI teacher chatbot that leverages the power of ChatGPT for natural language processing and Azure’s Text-to-Speech (TTS) for realistic voice interaction. We'll bring it all to life using React Three Fiber to display a 3D animated teacher in a classroom setting.

By the end of this tutorial, you'll have a working chatbot that can teach you languages in an immersive and engaging 3D environment. This tutorial will also show you how to deploy your project using Next.js, ensuring a seamless web experience for users.

Enhancing Education with a 3D AI Teacher

In many regions, students face challenges like:

• Limited access to skilled teachers and high-quality educational resources.

• Flat, text-based e-learning materials that reduce engagement and comprehension.

A 3D AI teacher can address these challenges by:

• Simulating a live classroom experience that feels immersive and interactive.

• Providing real-time answers, adapting to each student’s learning pace and style.

• Simplifying complex subjects through engaging 3D visuals and models.

• Supporting language learning with realistic conversation practice.

This approach makes learning accessible, flexible, and engaging, opening opportunities for quality education anywhere.

Insights and Skills

By the end of this project, you'll learn how to:

• Animate morph targets to create natural expressions

• Implement audio-synced animations for speech

• Enable smooth transitions between characters and environments

• Set up CI/CD with GitHub and Azure for automatic deployment

Tech Stack

• Next.js for fast, server-rendered React applications.

• React Three Fiber for 3D rendering in the browser.

• Three.js for creating and manipulating 3D graphics.

• Azure Cognitive Services for natural language processing and speech capabilities.

• OpenAI API (ChatGPT) for conversational AI responses.

• Mixamo for pre-made character animations.

• Leva for real-time control panel adjustments.

Code Links

You can access the code for the final project below:

• Final code 🔗

Resources

• OpenAI API Key

• NextJS Getting Started

• Azure API Key

• Azure Speech SDK

• Virtual Girlfriend Tutorial

• Mixamo Animations

• Fiver Use my promo code WAWASENSEI to get 10% off at the checkout process

3D Character Designers:

• Beeapoo

• Sahasub

3D Classroom Designers:

• Remigiusz

• Adityia

#threejs #webdevelopment #reactthreefiber

Need help with this tutorial? Join our Discord community!