Last month, a client asked me something that made me realize how broken our assumptions about AI visibility are. They'd been paying a SaaS tool $49/month to tell them whether ChatGPT mentions their product, and the dashboard was giving them contradictory signals every week. One day they're position 1, the next day position 5. They asked me to debug it. I couldn't, because the tool wasn't buggy. The AI models themselves just aren't deterministic.
That conversation stuck with me. So when I stumbled across a developer's approach to solving this problem with a weekend of work and zero dependencies, I had to write about it. Because here's what bothered me: we've let marketing vendors convince us that AI visibility is some complex, proprietary thing worth hundreds of dollars a month. It's not. It's three measurements. That's it.
What We're Actually Trying to Measure
The original article nails something I've been thinking about for months. When someone asks "does an AI model mention my brand," they're asking exactly three questions, whether they know it or not:
First: does the model mention you at all in a category query? That's a yes/no. Second: where does that mention appear? Position matters, position 1 in a headline carries weight that position 7 in a nested list doesn't. Third: who else gets mentioned, and how often? Context is everything.
Everything else sold to you as an "AI visibility metric" is noise or inference on top of these three facts. Citation graphs, trend dashboards, heatmaps showing "engagement velocity"-I've seen all of it. It's usually someone's attempt to justify the price tag.
The 300-Line Solution That Made Me Rethink Premium Tools
The developer built this in Python using only the free Gemini API tier and urllib. No ML libraries. No infrastructure. No dashboard. The script asks category questions, parses the responses for your brand name and competitors, detects whether you show up in headings versus nested lists, and generates a weekly HTML report.
I kept reading and thinking: "This is what my client needed. Not the $49/month dashboard. This."
Here's the core of it, the function that actually matters:
def find_brand(text, brand, competitors):
text_l = text.lower()
mentions = {brand: text_l.count(brand.lower())}
for c in competitors:
mentions[c] = text_l.count(c.lower())
# Only count top-level mentions, not nested ones
position = None
for i, line in enumerate(headings_and_top_level_lists(text)):
if brand.lower() in line.lower():
position = i + 1
break
return {"mentions": mentions, "position": position}
The genius part? The headings_and_top_level_lists filtering. The developer caught a real problem: if you count every bullet point, you end up with false positives. A competitor mentioned as "the leader" in a heading will show up before you even if you're mentioned in a sub-bullet. That's a bug in the methodology, not the code.
My Take on the Bootstrap-vs-Premium Question
I respect what the developer did here, and I'm also being realistic about where this tool sits in the market. For a solo founder or a bootstrapped startup? This is perfect. It costs zero dollars, runs once a week on GitHub Actions, and gives you enough signal to decide "should I rewrite this landing page or ship a product feature?"
But I'm not going to pretend this replaces Otterly or Peec for a serious marketing team that needs daily tracking across multiple models. That's just honest. The tradeoff is real.
What changed my thinking, though: most companies don't need daily tracking across four AI models. They need to know if they're invisible, and a weekly signal on Gemini is often enough to answer that. They're overpaying because vendors convinced them that more data equals better decisions.
The other honest admission in the original post landed hard for me too, the non-determinism problem. LLMs return different wording each time. So position 1 this week might be position 3 next week, even if nothing on your site changed. The solution isn't a fancier dashboard. It's looking at a 4-week rolling median instead of obsessing over single-week changes.
What I'd Do Different
If I built this for a production service, I'd add fallback error handling for API failures and some basic rate limiting. I'd also probably add Claude and ChatGPT support, relying only on Gemini limits the insight. But that's scope creep. The developer stayed focused on the minimum viable thing.
Your Move
If you're spending money on AI visibility tools and you're not a team of 20+ marketing people, fork that repo. Spend an hour customizing it. Run it for a month. See if the signal matters to your actual business decisions. Chances are, you'll find that $49/month was solving a problem you didn't have.
What gaps am I missing? Is there something about tracking brand mentions across AI models that actually requires premium tooling, or have we just gotten used to overpaying for dashboards?
Source: This post was inspired by "How to check if ChatGPT, Gemini, Claude and Perplexity mention your brand (free Python script)" by Dev.to. Read the original article
