r/Convai • • Mar 12 '23

r/Convai Lounge

2 Upvotes

A place for members of r/Convai to chat with each other


r/Convai • • 2d ago

Turning Service Manuals and SOPs into Interactive Equipment Training with AI

1 Upvotes

We put together a short demo of a workflow we're building with Convai Assembly Studio.

The problem we're addressing is that a lot of industrial training material already exists, but it's scattered across service manuals, SOPs, technical specifications, and other files. Turning all of that into a structured training experience often requires a lot of manual work.

In this example, we're using a robotic arm gripper replacement procedure.

The workflow brings together the equipment's reference documents and uses them to generate a step-by-step procedure. Each step can be linked back to its source material, and the training can include knowledge checks for things like mounting bolt counts and torque specifications.

We then show how that procedure can be delivered in an interactive 3D environment, with an AI trainer guiding the learner through the equipment and individual steps.

The aim is to reduce the work involved in creating training content from existing technical documentation, while keeping the underlying sources available for review and verification.

This is particularly relevant for manufacturing, equipment maintenance, and other technical training environments where procedures need to be accurate and consistent.

More about the platform:
https://convai.com


r/Convai • • 3d ago

Bring Context-Aware AI Agents Directly to your Websites

1 Upvotes

We've been working on website AI agents with Convai, specifically around making them capable of interacting with the actual website rather than just answering questions about its content.

The idea is that a user should be able to ask an assistant to perform something on a website without having to navigate through the interface manually.

For example, instead of explaining how to switch a subscription from monthly to annual billing, the agent can interact with the billing toggle directly.

In this demo, we show an AI assistant that can:

  • Understand the context of the current page

  • Use vision to interpret the website interface

  • Switch between pricing plans

  • Filter products based on natural-language requests

  • Compare product specifications

  • Maintain conversational context across interactions

  • Work with external systems like CRM, inventory, calendars, and order management tools

One of the things we're particularly interested in is how page awareness, vision, and tool access can work together.

Page context helps the agent understand where the user is. Vision helps it interpret what's actually being displayed. Actions allow it to interact with the interface, while external integrations can extend those interactions into business workflows.

We're exploring this for product discovery, customer support, sales assistance, and other website experiences where users need to accomplish tasks rather than just retrieve information.

Curious how others are approaching reliable UI interaction and context management for website agents.

More about Convai:
https://convai.com/


r/Convai • • 4d ago

We built an AI Platform that combines CAD models and technical knowledge to generate procedures and guide technicians

1 Upvotes

We've been working on Convai Assembly Studio, a system that combines 3D CAD assemblies with technical knowledge sources like manuals, documents, and videos to support AI-driven procedure generation and interactive equipment training.

One of the challenges we're trying to address is that understanding a complex machine requires more than retrieving information from documentation.

You also need to understand its physical structure, how components connect, which parts depend on others, and the sequence required to perform a task.

Assembly Studio uses visual spatial reasoning over CAD assemblies to help with that.

In this example, we give the AI a 3D assembly of an industrial robotic arm and ask it to figure out how to replace the gripper.

It analyzes the geometry, identifies relevant components and attachment points, reasons through the disassembly sequence, and generates a step-by-step procedure.

Beyond procedure generation, we're also building toward AI-guided training workflows where technicians can interact with equipment models, ask questions about specific components, and receive contextual guidance grounded in the CAD assembly and supporting technical knowledge.

The broader application is in high-stakes, hard-skill environments like manufacturing, aerospace, and industrial maintenance, where expertise is often concentrated among a relatively small number of experienced people.

We're interested in making that knowledge easier to access and apply during training and real-world tasks.

Curious how others working with CAD, industrial AI, or technical training are approaching the combination of spatial reasoning and procedural knowledge.

More about Convai:
https://convai.com/


r/Convai • • 22d ago

Using Claude Code, Codex, and Cursor to Build AI Characters Directly in Unreal Engine with MCP

1 Upvotes

We've been working on a Convai MCP integration that lets AI coding agents like Claude Code, Codex, and Cursor work directly with Unreal Engine projects to create and configure AI characters through natural language prompts.

The idea is to reduce the amount of manual setup required when building AI-driven experiences.

Instead of configuring every component individually, developers can describe what they want their character to do and have the coding agent handle the corresponding setup.

Some of the workflows include:

  • Creating AI characters with real-time voice conversations

  • Setting up lip sync and facial animations

  • Configuring multilingual conversations

  • Enabling Vision support for characters

  • Adding built-in Actions like following and navigation

  • Connecting character behavior to custom game logic

The coding agent works through the integration, while the developer can inspect, test, and refine the results inside Unreal Engine.

This can be useful for building interactive NPCs, AI teammates, virtual guides, training simulations, and other AI-driven 3D experiences.

We put together a short reel demonstrating the workflow.

You can also check out our existing Unreal Engine MCP workflow here:
https://youtu.be/dQgOzt8BNIk

Convai documentation:
https://docs.convai.com

Unreal Engine plugin:
https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c

Get started:
https://convai.com


r/Convai • • 23d ago

Turning Natural Language Commands into Character Actions in Unreal Engine 5 Using Convai

1 Upvotes

We've been working on making it easier to connect natural language conversations with character actions inside Unreal Engine.

With Convai Actions, developers can configure AI characters to interpret player instructions and execute corresponding actions in the game world.

For example, a player can ask a character to follow them, move to a particular location, or stop, and the character can execute the appropriate built-in action.

The system also supports Custom Blueprint Actions, so developers can connect natural language instructions to their own gameplay mechanics and interactions.

This is useful for AI teammates, interactive characters, virtual guides, and simulation environments where conversations need to trigger actual behavior inside the scene.

The reel shows how dialogue can translate into actions inside Unreal Engine.

We also have a step-by-step tutorial covering the setup of built-in Actions, navigation, and object references.

Full Actions tutorial:
https://youtu.be/lB5_KRWdg_w

Actions documentation:
https://docs.convai.com/api-docs/plugins-and-integrations/unreal-engine-plugin-beta-overview/convai-actions

Unreal Engine plugin:
https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c


r/Convai • • 25d ago

We Built an AI Tool That Automatically Analyzes and Annotates 3D Environments in Unreal Engine

1 Upvotes

We recently built Scene Tagger for Unreal Engine to make it easier to annotate large 3D environments with AI.

A typical 3D level can contain hundreds or thousands of objects, but mesh names and transforms alone don't provide enough semantic information for AI systems to understand what those objects are or how they relate to the environment.

Scene Tagger helps automate that process.

In this demo, we're using a virtual museum to show how the workflow operates.

The tool can:

  • Analyze selected actors or an entire Unreal Engine level

  • Identify significant objects and capture their views

  • Generate meaningful object names and descriptions

  • Let developers review, correct, and refine annotations

  • Apply the structured information back into the project

The idea is to reduce manual scene annotation while giving AI systems more useful context about the environments they're operating in.

This can support embodied AI workflows, such as interactive characters and virtual tour guides, as well as disembodied AI agent workflows that need structured information about a 3D scene.

We also put together a full Unreal Engine tutorial showing how to set up and use Scene Tagger.

Full tutorial: https://youtu.be/k8CAaH25lJ8

Convai Unreal Engine Plugin: https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c

Documentation: https://docs.convai.com


r/Convai • • Oct 12 '26

Building AI characters for VR and mixed reality with real-time conversation, navigation, and actions

2 Upvotes

We've been exploring how Convai-powered AI characters can be used inside VR and mixed reality environments.

The idea is to make characters more than static avatars or scripted NPCs.

With Convai, developers can connect a conversational AI system to a 3D character and give it capabilities such as real-time voice interaction, environmental context, navigation, gestures, and actions.

Some applications include VR training simulations, virtual tour guides, interactive learning environments, and game NPCs.

We put together a short reel illustrating how these capabilities can come together in immersive XR experiences.

For anyone working with Unity VR, we also have a tutorial covering the setup of a Convai AI character, connecting the XR Origin player rig, and preparing the project for Meta Quest or PCVR.

Unity VR tutorial:
https://convai.com/blog/how-to-build-convai-powered-ai-npc-vr-projects-in-unity-for-meta-quest

Convai:
https://convai.com


r/Convai • • Oct 09 '26

Giving AI characters real-time vision so they can understand their 3D environment

1 Upvotes

We put together a short showcase of Convai Vision, which lets AI characters receive visual information and use it as context during conversations.

The idea is to let characters incorporate what they see into their responses.

For example, inside a virtual environment, an AI character can receive visual information about the surrounding scene and use it when answering questions.

Vision can also work with camera input, allowing characters to receive visual information about a user's real-world surroundings.

Some potential applications include:

  • AI NPCs that can perceive their game environment

  • Virtual guides that respond with visual context

  • Training assistants that can discuss what users are looking at

  • XR and mixed reality characters

  • Interactive 3D avatars with visual perception

We've also made it possible to try vision-based video calls with 3D avatars directly through the Convai Playground.

More here: https://convai.com


r/Convai • • Oct 06 '26

Real-time facial animation and lip sync for AI characters with NeuroSync

1 Upvotes

A quick showcase of NeuroSync, our real-time facial animation system for Convai-powered AI characters.

As the character generates a voice response, NeuroSync uses that response to drive synchronized lip sync and facial animation in real time.

The goal is to keep the voice, mouth movement, and facial performance working together during live AI conversations.

It can be used across games, training simulations, virtual guides, XR experiences, digital humans, and other interactive 3D applications.

More on Convai:
https://convai.com


r/Convai • • Oct 05 '26

Quick walkthrough: connecting an AI character to external tools with MCP

1 Upvotes

We put together a short walkthrough of the MCP setup inside Convai.

The flow is straightforward: add an MCP server, configure the server details and connection settings, connect it, and make the available tools accessible to the character.

Once connected, those tools can be called when a request requires access to something outside the character’s existing context.

In the current interface, examples include live web search, GitHub exploration, documentation lookup, and Convai docs through servers like Firecrawl, DeepWiki, Context7, and Convai Gitbook.

This is one of the more practical ways to extend an AI character into an agentic workflow.

More here:
https://convai.com


r/Convai • • Oct 05 '26

Quick walkthrough: giving an AI character its own multimodal Knowledge Bank

1 Upvotes

We put together a short walkthrough of Convai Knowledge Bank.

You can upload reference material like text, PDFs, images, audio, and video through In-Context Knowledge Bank and connect that content to the character.

The goal is to give the character access to the same material your team already uses, whether that is training documentation, manuals, product information, FAQs, or other domain-specific content.

Useful for training assistants, onboarding, virtual guides, support workflows, and other AI character applications where domain knowledge matters.

More here:
https://convai.com


r/Convai • • Oct 01 '26

Quick walkthrough: connecting an AI character to external APIs and tools

2 Upvotes

We put together a short walkthrough of Convai’s External API workflow.

You can create an API method, define the inputs it expects, add the implementation, test it directly in the dashboard, and then expose it as a tool the character can call when a user request needs it.

The interesting part is the agentic layer this adds. The character is no longer limited to its existing conversation context. It can reach into live services and workflows when the task requires it.

Useful for things like backend data, support workflows, internal systems, product experiences, training, and other interactive AI applications.

More here:
https://convai.com


r/Convai • • Sep 30 '26

A quick walkthrough of the Core AI Settings behind a Convai character

1 Upvotes

We put together a short walkthrough of the Core AI Settings available for a Convai character.

From the same section, you can configure supported languages, enable Long-Term Memory, connect multimodal Knowledge Bank content, choose between LLMs like GPT, Gemini, and Claude, and select voice providers such as ElevenLabs. Convai Overview

You can also compare models based on things like latency, cost, and intelligence, then enable NeuroSync for real-time facial animation.

The goal is to give developers one place to tune the AI stack behind the character before taking it into a game, simulation, training workflow, or other interactive experience.

More here: https://convai.com


r/Convai • • Sep 29 '26

A compilation of agentic AI characters reacting to live 3D environments

2 Upvotes

We put together a short compilation showing different Convai-powered characters operating inside live 3D environments.

The common thread across the demos is the same agentic loop.

Characters can receive live game state through Dynamic Context, understand tagged objects and changes happening in the scene, and then use Actions to navigate, follow instructions, interact with objects, and respond inside the world.

The interesting part for us is how these systems work together rather than as isolated features:

environment state → context → decision → action

That makes the same underlying setup useful across games, simulations, training applications, XR, and other interactive 3D experiences.

More here:
https://convai.com


r/Convai • • Sep 24 '26

Create and Test AI Avatars in Your Browser | Convai Avatar Studio

1 Upvotes

We put together a short reel showing Convai Avatar Studio, our browser-based workflow for creating and testing 3D AI avatars.

You can define the character’s AI setup, including backstory, personality, voice, language, and knowledge, then customize the avatar and environment and start testing the experience without setting up a game engine first.

The idea is to make it quicker to prototype things like virtual guides, training assistants, onboarding experiences, product demos, kiosks, and other web-based AI character experiences.

More here:
https://convai.com


r/Convai • • Sep 22 '26

Combining live game state with AI character actions in Unreal Engine

2 Upvotes

We’ve been exploring how live environment context and character actions can work together inside game-engine experiences.

With Dynamic Context, information such as player progress, tracked object states, and environmental changes can become part of the AI character’s awareness.

Convai Actions then gives the character ways to respond inside the world, including navigation, following the player, and interaction with scene objects.

We’ve used this pattern in our Unreal Engine Stack-O-Bot workflow, where an AI teammate can understand tagged objects, track pressure plate and moving platform states, navigate the environment, and help coordinate a puzzle based on live game state.

The basic idea is:

World state → character context → decision → action

For us, this is one of the more interesting parts of embodied AI inside game engines because the character is responding to both conversation and the actual state of the environment.

More: https://convai.com


r/Convai • • Sep 22 '26

Bringing AI characters into real-world AR, MR, and XR experiences

1 Upvotes

We put together a short demo showing Convai-powered AI characters operating in real-world spatial experiences.

The idea is to bring the same AI character into AR, MR, and XR environments across phones, headsets, and spatial devices.

The character can talk with the user in real time, use visual context to understand what the user is seeing, assist with tasks, and provide information about the surrounding environment.

That makes the character useful for things like spatial guides, training assistants, task support, and other embodied AI experiences where the AI exists alongside the user rather than only inside a traditional screen-based interface.

More here:
https://convai.com


r/Convai • • Sep 18 '26

We tested a vision-based AI training assistant on a wood planer workflow

1 Upvotes

We’ve been experimenting with vision-based assistance for hands-on training.

In this demo, the user is working around a wood planer while interacting with an AI assistant. Instead of only relying on predefined text context, the assistant can use what the learner is seeing as part of the interaction.

That makes it possible to ask questions naturally while working, with the guidance staying connected to the equipment or task currently in view.

The broader idea is to make training assistance more contextual and hands-free, particularly for workflows where stopping to search through documentation breaks the flow of the task.

Convai supports vision-enabled assistants that understand what the user is seeing, as well as camera-based guidance, voice, text, avatars, simulations, and custom applications.

More here:
https://convai.com


r/Convai • • Sep 17 '26

Introducing Convai Scene Tagger for Unreal Engine: Automatically Analyze, Annotate, Describe, Review, and Structure Entire 3D Environments

2 Upvotes

Introducing Scene Tagger from Convai for Unreal Engine.

We just published a tutorial covering the complete Scene Tagger workflow in Unreal Engine.

The problem is fairly straightforward: large 3D scenes can contain hundreds or thousands of objects, and manually going through important assets to give them meaningful names, descriptions, and context can become a very tedious workflow.

Scene Tagger is designed to analyze the environment, identify significant objects, capture useful views, and generate structured names and descriptions that developers can review and refine before applying them back to the project.

For the tutorial, we use a museum scene with paintings and sculptures, but the museum itself is simply the example environment.

The tutorial covers:

  • Selecting a Convai character for scene analysis

  • Using Knowledge Bank content as additional context

  • Generating an environment description from the current camera view

  • Analyzing an entire Unreal Engine level

  • Reviewing generated object names and descriptions

  • Correcting camera views

  • Providing notes and reanalyzing objects

  • Batch reanalysis

  • Removing irrelevant objects or bad captures

  • Applying Convai Object components to the scene

  • Testing the results with an AI character

  • Debugging tagged objects and spatial relationships

Once the scene is tagged, that structured information can support other AI-driven workflows that need an understanding of the surrounding 3D environment, including games, simulations, training applications, guided experiences, and XR.

Full tutorial: https://youtu.be/k8CAaH25lJ8

Convai: https://convai.com

Scene Auto Tagger Documentation: https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/scene-auto-tagger


r/Convai • • Sep 17 '26

What actually makes an AI agent agentic? Embodied vs disembodied agents

0 Upvotes

We put together a short reel around a distinction we’ve been thinking about a lot: embodied vs disembodied AI agents.

A disembodied agent operates across software environments such as browsers, apps, APIs, and computers.

An embodied agent occupies a 3D environment, has a position and point of view, and can act within that world. Its action space can include things like navigation and object interaction in addition to software-level tools.

The useful part is that both still depend on the same broader loop:

Perception / context → understanding → decision-making → action

The specific environment determines what the agent can perceive and what actions should be available to it.

That perception-action relationship is also the basis of our Actions IO framework, which looks at the information channels reaching an agent and the affordances it can use to change its environment.

More here:
https://convai.com


r/Convai • • Sep 16 '26

Giving AI characters their own knowledge base

1 Upvotes

We made a quick reel showing how Convai Knowledge Bank works.

The idea is simple:

Upload your documents, manuals, FAQs, product information, or training material
Connect that content to your AI character
Let users ask questions naturally
The character answers from the knowledge you provide

This can be useful for training assistants, virtual guides, onboarding, product demos, support agents, games, simulations, and interactive 3D worlds.

Learn more: https://convai.com

Detailed Knowledge Bank Docs: https://docs.convai.com/api-docs/convai-playground/character-customization/knowledge-bank


r/Convai • • Sep 15 '26

Connecting AI characters to live tools and external systems

1 Upvotes

AI characters can do more when they are connected to the tools and systems around them.

With Convai, your AI characters can use MCP or External API to connect with live web search, documentation lookup, GitHub explorers, internal systems, custom APIs, and more.

A user asks a question.
The character understands what is needed.
It calls the right tool.
The result comes back into the conversation.

Build AI characters that can work with live information and real systems using Convai.

Learn more: https://convai.com

Docs: https://docs.convai.com

External API Docs: https://docs.convai.com/api-docs/convai-playground/character-customization/external-api

MCP Docs: https://docs.convai.com/api-docs/convai-playground/character-customization/mcp-servers

#Convai #AICharacters #MCP #ExternalAPI #ToolCalling #AIAgents #ConversationalAI #EnterpriseAI #AIWorkflows #GameAI


r/Convai • • Sep 11 '26

We built a Unity scene with multiple conversational AI characters — here’s how the player naturally switches between them

1 Upvotes

We recently put together a short demo showing multiple conversational AI characters running inside the same Unity scene.

The main interaction problem here is pretty simple: when several AI characters are nearby, how does the system know which one the player is actually talking to?

Convai Multi-Character Sessions supports three approaches:

Look At — based on where the player is looking.

Proximity — based on distance.

Manual — controlled directly by the developer.

For this demo, we use Look At, so the player can speak with Sophia, turn toward Ethan, and then continue with Marcus naturally within the same environment.

This is Part 1 of the setup.

Part 2 will expand this into all-to-all awareness, where players and AI characters can stay aware of nearby conversations and events, allowing characters to react or engage when something is relevant to them.

Full tutorial:
https://youtu.be/GIejlNL6tfQ

Multi-Character Sessions docs:
https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/features/multi-character-sessions


r/Convai • • Sep 10 '26

We built a Scene Tagger to automatically annotate objects across large 3D environments

1 Upvotes

Large 3D projects can contain hundreds or even thousands of objects, and giving every relevant object an accurate name, description, and context manually can become a pretty significant development task.

Convai Scene Tagger is designed to automate a large part of that workflow.

It analyzes the environment, identifies significant objects, captures useful views of them, and generates titles and descriptions. You can then review the results, correct views, add additional context, reanalyze objects where necessary, remove irrelevant results, and apply the final annotations back to the project.

For this tutorial, we're using a museum because it gives us a large environment with lots of visually distinct objects, but Scene Tagger itself isn't specific to museums or tour-guide experiences.

The resulting scene information can be used for broader AI-based development and runtime workflows that need structured context about objects in the environment.

The full tutorial (coming out soon) goes through the complete workflow from scene analysis to reviewing, refining, and applying the final annotations.

Would be interested to hear what kinds of environments you'd want to use automated scene annotation for.