Here’s something nobody warns you about when you build AI into your product: you pay for the work every single time a user touches it. If you’re a well-funded team, that’s just another line item. But for a bootstrapped founder, it can sink the business.
Arjun Singh Jamwal sees both sides of that. He’s a Senior PM at Verizon, where he’s worked on Verizon Cloud and now Verizon Home. On the side, he’s bootstrapping Goodmuncher, an app that helps people cut back on ultra-processed foods, and where every token he spends comes straight out of his own pocket.
His fix is something he calls “giving the model a layup.” Rather than having AI analyze every ingredient from scratch whenever someone scans a label, Goodmuncher scores ingredients and products ahead of time and keeps them in a vector database. Users get the same answer, and Arjun pays a fraction of what it would cost to run the analysis live.
This one’s a little different from our usual episodes: it’s a live demo. Arjun shares his screen and runs Goodmuncher’s content pipeline right there on the call. One command, 30 seconds, and out comes a finished 10-slide social carousel. Then he walks us through how it’s all built.
Along the way, we get into:
Why you should think of tokens as OpEx, and how a little upfront prep keeps your AI costs predictable
How giving the model a “layup” lets cheaper models punch above their weight, and makes it easier to swap models later
Why he hard-codes the rules he can’t break instead of trusting the model to remember them
His step-by-step process for evaluating agents, from golden datasets to LLM-as-judge and observability tracing
1. Why Arjun treats every AI token like an operating expense
Plenty of companies are spending on AI simply because leadership told them to. Arjun doesn’t think that will last. Because those are real dollars, and sooner or later, that cost is going to add up.
Arjun thinks about it the way a finance person would: Every token you spend at runtime is a variable cost, and if you just throw raw data at a model and let it sort everything out, that cost gets hard to predict:
“Think of tokens as a variable cost. This is like your OpEx. When you just throw everything at it to crunch the numbers, you’re actually working with an unpredictable OpEx.”
The other option is to do some of the work up front, one time. Arjun thinks of that as CapEx.
And at some point, the math just stops making sense:
“There’s no point in spending $200,000 on tokens when I could just hire a great full-stack engineer who could do that for me.”
Product takeaway: Treat AI costs like any other cost on your P&L. If a feature uses tokens every time someone clicks, taps, or asks a question, your bill grows right alongside your user base. Before you launch, estimate what that feature would cost if usage grew 10x, so you aren’t caught off guard when the invoice comes.
2. How Arjun gives the model a layup by doing the hard work ahead of time
Goodmuncher has to come up with a processing score for every food item a user looks up. The obvious approach is to have users snap a photo of the ingredient label, send it to a multimodal model, and let it work everything out from scratch. That’s fine with a handful of users. With 10,000, it gets expensive fast.
So, Arjun does the heavy lifting ahead of time. He pulls ingredient data from open sources, scores it in advance, and stores it as embeddings in a vector database.
“What I’d rather do is pull in those ingredients, crunch that data, and already pre-attribute a processing score to it, so that when a user actually looks up the pesto sauce, for example, that score’s already ready.”
Now the model isn’t reasoning through a long chain of steps. It’s just finding an answer that already exists.
Those savings add up. Arjun puts it at at least a 30X difference at scale, and says it’s let him cut what he plans to charge users by more than half.
Product takeaway: Before you reach for a bigger, newer model, map out your workflow and ask which steps really have to happen in real time. Anything you can pre-process, cache, or look up ahead of time is work the model doesn’t have to do, and that’s money you can save yourself AND your customers.
3. Not every job needs a frontier model
You don’t need the newest, most powerful model for everything. Frontier models shine on complex, multi-step reasoning. You can use them for simple tasks too, but you’re paying for horsepower you don’t need.
Keeping your workflows lean pays off in another way, too. When a better model comes out, you can swap it in without re-testing a long, tangled chain of steps:
“When you have to swap out that model, the fewer steps you have in the workflow, the fewer steps of evaluations you need to run. So your evals will have a better time, your change management team will have a better time, your devs will have a better time.”
Product takeaway: Picking a model is a product decision, not just an engineering one, and the only way to know which one fits is to try a few. Get in the habit of testing the smaller, cheaper option first, and only move up when it can’t keep up.
4. How Arjun hard-codes the rules his AI should never break
Goodmuncher is all about helping people eat less ultra-processed food, but it can’t claim to cure diseases or lower anyone’s medical bills. The FDA would have strong words for Arjun if it did.
During his live demo, Arjun shows how his Goodmuncher content pipeline checks every output against a “tripwire” file, a running list of banned phrases like “prevent dementia” or “save medical bills.” That’s not a call he leaves up to the model.
“It’s important to make this deterministic in nature because we don’t want to compromise on this. These are the red lines that you cannot cross.”
He also made a point of keeping that check separate from the step that writes the content, instead of baking it into the prompt:
“I could have easily put my tripwire check into the content generation file, but I didn’t because I want it to be a different process, and I know this will build out over time.”
Product takeaway: Models follow the rules most of the time. But “most of the time” doesn’t cut it for compliance, safety, or anything that could hurt a user. Figure out what’s non-negotiable for your product and enforce it in your code, as its own step, so you can test it and add to it over time.
5. Arjun’s step-by-step process for evaluating agents
Arjun mentors startups through the Cornell Tech community, and he keeps running into the same blind spot:
“It’s so easy just to throw an agent at any problem. But what we’re not being mindful of is the eval aspect of it.”
For Goodmuncher, this isn’t abstract. If a recipe recommendation ignores someone’s nut allergy, a real person could get hurt. Here’s how Arjun approaches evals:
Run an exploratory data analysis (EDA) on your production data to understand its categories and shape
Build a golden dataset of 10–50 examples you know are 100% correct
Reverse-engineer a rubric from that dataset (e.g., “Do the ingredients respect the user’s allergies?”)
Create a staging database that represents production, with PII anonymized and de-identified
Run evals against the rubric, using a mix of human evals and LLM-as-judge, with 5–10 examples per category
Add observability tracing to pinpoint exactly where a workflow breaks
Build a dashboard of those traces, if you can
Product takeaway: Building a golden dataset and rubric takes some work up front, but like everything else Arjun builds, that investment pays for itself every time you swap a model or chase down a bug.
Chapters
00:00 Introduction
02:03 Arjun’s product background
04:18 Treating tokens as unpredictable OpEx vs. a One-time CapEx
08:34 Pre-processing data so smaller (and cheaper) models can do the job
13:28 Arjun's Demo: Turning one command into a 10-slide social carousel
19:05 Inside the pipeline architecture and its agents
26:22 Arjun's framework for evaluating AI agents
32:07 Conclusion
Links
What does LogRocket do?
LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.