The other day, I appeared on the Product Hacker podcast, where I was asked, “Why aren’t there more products like Luzia?” So I’ve decided to organize my thoughts and distill six months of hustle launching a genAI solution into this post.
A decade ago, if someone had told me we’d form meaningful relationships with our smartphones, I would’ve advised them to start writing sci-fi. We lived in a world where there was an ever-present promise of smart assistants that were actually useful, a promise that Alexa, Siri and Google did not live up to. Enter generative AI (genAI). With the unprecedented success of chatGPT, amassing 100 million users (1) faster than any other consumer app, genAI is not just a buzzword — it’s an essential part of modern tech toolkits. What three years ago seemed impossible, it’s now part of the day-to-day of millions of people worldwide.
Amid this hype, critics have started to label this trend as a bubble, questioning why we haven’t seen more AI marvels like Luzia. I aim to debunk this notion in this post. Far from being a bubble, genAI is already revolutionizing non-consumer sectors from academia (2) to consulting (3), passing through industrial applications ([reference]). However, when it comes to consumer-focused applications like Luzia, several obstacles stand in the way. Let’s go over them.
In this article, I’ll outline the four key barriers hindering the market entry of genAI products, especially in the B2C sector: processing capacity, software readiness, specialized talent, and escalating variable costs.
A few hurdles to overcome
The Big Bottleneck: Capacity
So you’ve got a killer feature and the budget to match for genAI. Fantastic! Time to deploy. You call Azure — sorry, no GPU capacity. AWS? Same deal. A promising startup? Can’t scale. Welcome to 2023 (5), where computational power is like gold, hoarded by a few industry giants. Even well-funded startups and big corporations languish on waiting lists for months.
Having the ideal model or a must-have feature doesn’t guarantee you a spot in this resource-starved ecosystem. We know the struggle at Luzia; apologies to our users for the bumps back in May and June. But for bigger, established companies, the bar is even higher. We at Luzia managed to boost our capacity through some intense optimization and made it grow as we ramped up our user base. But if you’re Spotify and are kicking off with 200 million daily active users, delivering flawless service from the get-go is a fantasy.
Those long waiting lists, you see? They’re not just marketing ploys; more often than not, they’re the fallout of this capacity bottleneck.
Twitter Demos vs. Scale Reality
We all love those flashy Twitter demos — AI generating stories (6), composing tunes (7), and even automating marketing teams (8; side note: I tried this one, almost killing my DB!). But let’s not get it twisted. Those demos are the tip of the iceberg, coming after countless failures and not showcasing the hurdles of catering to millions of users. Remember the initial hype about projects like AutoGPT (9)? They looked great on paper, and don’t get me wrong, they are extremely promising starts for a long-distance run, but scaling and bringing that to industrial/consumer applications is a different beast altogether — think hundreds of millions in investment and endless compute hours.
The good news is some players are stepping up their game. Langchain with Langsmith (10) and Huggingface with Text Generation Inference (11) are making strides in scalability. Plus, a new wave of companies is breaking ground to simplify the infrastructure side of things.
A more technical note: LLMs are stochastic models, meaning the output for a given input isn’t guaranteed. In layman’s terms, let’s say you prompt the model for a JSON representation; 90% of the time it’ll nail it, but there’s that pesky 10% where it won’t. Parsing that misfire is a challenge. GPT-4–0613 (12) has improved this with function calling, but it also ups the consumption.
Talent Pool: More Like a Puddle
Here’s the elephant in the room: talent. Sure, there are people who can make a demo where AI looks like it’s ready for Broadway. Absolutely, AI tech is evolving at warp speed. And yes, GPT-4 can code like a pro. But here’s the kicker: only a handful of applications have scaled to millions of DAUs, which means an even smaller pool of experts knows how to manage these systems at that level. Clasical chicken-and-the-egg problem. Startups like Luzia have an edge; we’ve scaled our expertise in tandem with our user base, turning that experience into a valuable commodity.
The Unseen Cost of inference at scale
An eye-opening Economist article put it best (13): running an inference with GPT-4 at almost full context (32k tokens or 40ish pages) will set you back a staggering $2.4 per query. Do the math: 100 queries a day totals $240, or $7.2K a month — far more than the average employee salary.
Bottom line: inference costs skyrocket, and they do it quickly. Beyond Twitter demos, scaling requires a razor-sharp focus on cost management. And that’s not even diving into agent-based systems where the LLM becomes the core engine, multiplying inference calls manifold.
At Luzia, we deploy a range of strategies to slash these costs. For instance, one of our newest, still-under-wraps features has been tested with a small but notable user group at just 3% of its initial cost back in April.
Conclusion: The Future’s Looking Bright
Navigating the genAI space is no cakewalk, but let’s not mistake caution for stagnation. This isn’t a bubble — it’s a carefully inflating balloon. Companies will rise and fall, but the wise navigators will be the real disruptors. With the industry’s best and brightest on it, we’re at the threshold of a revolution loaded with productivity and societal impact.
Luzia’s Blueprint
How does Luzia stand out in this dynamic ecosystem? Simple. We had a plan, and we’re sticking to it (14): build a rock-solid foundation, scale up, and only then start adding the frills. We’re playing the long game.
Luzia isn’t just another player in the genAI arena; we’re setting up to rule it. Agile and adaptable, we tackle challenges head-on, using them as catalysts for innovation. Our goal isn’t just to ride this wave but to direct it. While we may not have built the foundations of genAI, we’re all about making it user-friendly and elegantly simple. So keep watching; we’re only getting warmed up.
