AI21 Labs managed to build enterprise-level production models which served players at Latitude to elevate the gaming experience while at the same time improving cost.
Latitude is a pioneer in the AI gaming space, offering powerful AI-generated digital gamescapes for its players.
Latitude is most famously known for AI Dungeon, a text-based adventure game that uses artificial intelligence to generate unique and unpredictable stories, depending on the choices that a player makes. In the past, computer adventure games were limited in terms of the amount of actions and scenarios they offered. By using Large Language Models, AI Dungeon grants players the flexibility to perform virtually any action and have the game respond to that action.
Latitude got good traction with its early implementation of AI Dungeon on top of GPT-3, but when planning their future roadmap, they quickly realized that they needed an LLM partner who shared their vision of immersive AI-driven experiences. They found an eager partner in AI21 Labs, who worked collaboratively with Latitude to discover roadblocks and brainstorm solutions. This resulted in improvements to issues such as latency, flexibility, and user satisfaction, to name just a few.
So, how did they do it?
AI21 Labs managed to build enterprise-level production models which served players at Latitude to elevate the gaming experience while at the same time improving cost efficiency. Latitude found AI21 Labs to be a collaborative LLM partner, which wasn’t just a plug-and-play solution to a scaling problem, but rather a trusted partner with whom they could collaborate on AI solutions.
But what were the exact problems preventing Latitude from realizing its vision? And how did AI21 Labs overcome them? Read on for the full story.
Latitude was never envisioned as a gaming company - it actually started as a Hackathon project that translated into a Dungeons and Dragons plugin. The team decided to go to market with the plugin, and within the first week, they had roughly 100,000 customers which quickly grew to a million in just the first month.
The verdict was in: there was a creative use case for large language models for game-like experiences and writing.
Latitude wanted to offer a stronger game experience, with richer text and better AI responses. For this, they partnered with the legacy GPT-3 enterprise, OpenAI.
But with rapidly evolving technology came growing pains, and OpenAI was not aligned with Latitude's vision - thus, the visionary gaming experience became impossible. It became clear to Latitude that they needed to seek out a new tech partner who was interested in solving the use case of creating an AI-Driven adventure role play game..
Latitude booked a call with AI21 Labs when they’d just launched their Jurassic-1 Model. A Jurassic prototype was set up for Latitude, and they started experimenting with the features and the accuracy of results. The final model was perfect for Latitude’s AI Dungeon use case. Soon after that, the team declared it an alpha model for their audience.
In order to get to that stage, the teams had to overcome two massive roadblocks.
OpenAI’s policy of non-violence restricted integral elements of Latitude’s AI Dungeon adventures — quests, swords, and fights. It was impossible to have a role playing expedition without weapons.
“We were interested in a partner who understands the nuances of the creative use case.” says Ryan Seamons, VP of Product at Latitude.
Latitude prioritized finding a partner who was not only aligned philosophically but willing to become a true partner and collaborator in building a quality AI-driven role-play experience.
“We need to offer AI-assisted role-play at scale to thousands of daily players. We need models that are fast, reliable, and creative.” says Seamons.
AI21 Labs’ team of developers, creators, and stakeholders were analytical problem-solvers who created reliable production models for Latitude.
A precise production model is critical for a gaming company like Latitude. It defines the customer experience and determines whether users will come back. So, it needs to be hyper-accurate and reliable. To achieve this, there has to be excellent communication during prototyping.
Latitude and AI21 Labs had a system for rapid back-and-forth between teams and numerous collaborative checkpoints. As a result, they got to the production stage quickly.
Even after launching the model, the AI21 Labs team worked tirelessly to ship updates, create new features, and build on feedback.
“We’ve experimented with other tools, but they have been demanding and daunting with direct orders and no problem-solving. But not AI21 (Labs). They’re enthusiastic, collaborative, and innovative — a true gaming visionary.” says Seamons.
Latitude offers AI-assisted role-play at scale to thousands of daily players.
Not only did Latitude want to offer existing models to new players, they also wanted to create more immersive and realistic worlds.
They wanted an LLM partner who offered customizable pricing because gaming models are extremely expensive to run at scale.
“AI costs are one of our most significant ongoing expenses, and so finding ways to reduce our AI costs help us to operate sustainably as a business.”
AI21 Labs put in hours of skilled talent to make the large language model efficient enough to be cost-effective. They helped Latitude transition from Jurassic-1 Jumbo to Jurassic-1 Grande to scale performance at the right cost.
Jurassic-1 Jumbo, AI21’s biggest model, processes 178 billion parameters to generate an output. But after running multiple experiments, their team concluded that Latitude’s game did not need this many parameters. They switched from Jumbo to Grande (which uses 17B parameters), and the result was the perfect sweet spot for a compelling experience at a sustainable price with faster response time. The cost savings are great, but the faster response times are also something players can benefit from. The lower costs also let Latitude experiment with additional ways to improve output quality, such as Hydra mode which generates multiple responses and selects the best one for a player.
Being able to experiment with new data sets more quickly and affordably also yielded positive results and will be a major part of Latitude’s AI strategy this year.
“The biggest benefit of working with AI21 (Labs) is for agile companies who want more hands-on attention in service. What I’ve been most impressed with is the proactiveness in optimizing costs for multiple use cases. It’s been a breath of fresh air compared to competitors.” says Seamons.
Latitude discovered that the Grande model struck the right balance of cost vs. performance, all while generating higher quality results than small models such as GPT-J or GPT-3 Curie.
By using an ensemble model approach called Hydra, Latitude was able to achieve quality similar to that of larger models.
At every stage, the team at AI21 Labs devised innovative solutions to roadblocks and proactively and continuously improved cost efficiency.
Whether through large-scale prototyping or improving production after release, Latitude found AI21 Labs to be a creative, efficient, and reliable partner on every level.
