Skyrim NPCs are being powered by ChatGPT

Thanks to a modder, ChaptGPT is powering conversations with NPCs in Skyrim VR, and more might even be coming.

We’ve all wondered what it might be like to be able to talk to an NPC in a video game about anything, without any guardrails or predetermined dialogue paths. Unfortunately, that’s not exactly a technically feasible ask; however, that’s changing. Thanks to chatbot ChatGPT, a modder has successfully been able to use ChatGPT to generate responses on the fly to any question you might want to be answered by an NPC in Skyrim VR.

How does it work? Well, basically, modder Art From The Machine uses ChatGPT to generate a response to a question, xVASynth to turn that text into speech, and Whisper so that what you say in VR can be turned into text that can be then understood by the mod. Essentially, it means you can have actual conversations with NPCs about anything. Though, it’s still a work in progress, so it’s not exactly a fully fleshed-out feature just yet.

Another interesting feature of the mod is that there’s a basic memory system in place, too. Currently, this means that the conversation you have with an NPC can be summarized and saved for later prompts after a conversation finishes up, but Art From The Machine says that there are much more comprehensive systems for handling memory, like Langchain, that could well be relied on in the future, too.

For example, there is still a noticeable delay as the tech at play here processes what you’ve said and generates a response, and while synthesizing voice acting is certainly cool, it doesn’t sound fully natural, and sometimes, it can be hard to understand what a particular NPC is saying. In general, this mod is something of a proof-of-concept. However, Art From The Machine is hard at work on improving the mod’s systems.

Currently, this mod isn’t available for you to download just yet, but keep an eye out in the coming months if you’re interested in trying it out for yourself.