How we went from helping Google find words to helping AI understand what we mean — and why SEO needs to catch up.
I remember when keywords were exciting.
It was around 2006 or 2007, and I was working for Governor Arnold Schwarzenegger in California, helping lead the state’s digital transformation. One of our biggest challenges was surprisingly simple: How do we help citizens find the government services they need?
We had websites full of valuable information, but that information wasn’t particularly useful if people couldn’t find it. Government has its own language, and it isn’t necessarily the language citizens use when they’re looking for help. We might call something by its formal program name, while someone searching for that service would use an entirely different set of words.
Then along came the opportunity to make our websites more discoverable through Google.
We learned that if we included the words people were likely to use when searching for a service in our website content, Google was much more likely to surface the appropriate pages. It sounds incredibly basic now, but at the time it was exciting. We could actually help people find what they needed simply by being more thoughtful about the language we used.
Google’s mission was to organize the world’s information and make it universally accessible and useful. From our perspective, it was delivering on that promise.
Of course, the underlying technology was considerably more sophisticated than a simple lookup table. Google was already using indexing, link analysis, and other signals to determine which pages were relevant and authoritative. But the practical experience for those of us managing websites felt remarkably like a giant lookup system. Put the right words in the right places, and Google could connect the search with the information, and it worked.
When Words Became an Industry
As Google’s popularity grew, something else was happening. Businesses realized that being found in search results had enormous commercial value.
In those early days, Google advertising looked quite different from what we see today. We became accustomed to the familiar blue organic links, with sponsored advertisements appearing alongside them, including in a column on the right side of the page.
Advertisers could bid on words and phrases associated with what people were searching for. Suddenly, choosing the right words wasn’t simply about making information accessible. It was about making money. An entire industry grew around that opportunity.
Search engine optimization, or SEO, had already begun emerging in the 1990s, but Google’s growth helped turn it into a major commercial discipline. Keyword research became a specialty. Companies hired experts to determine which words people were searching for, how often they searched for them, and how those words should appear on websites. And, as often happens when people discover how a technology works, some began looking for ways to exploit it.
I remember the tricks. Repeating keywords over and over in website content. Stuffing them into titles and metadata. Creating pages designed primarily for search engines rather than people. Some even hid keywords in the background of pages so search crawlers could read them while human visitors couldn’t see them.
The thinking was straightforward. If Google was looking for words, give it more words. For a while, some of those techniques worked too. Still, there was a fundamental problem with this approach. Finding the same words on a website doesn’t necessarily mean the website contains the best answer. We thought we were so smart, but Google knew it all along.
The Difference Between Reading Words and Understanding Meaning
This is where the story gets really interesting to me. Way behind the scenes, things were already starting to change.
Years earlier, while I was at Harvard, I had been involved in work exploring something the research team called cultural resonance analysis. We were interested in how language carries meaning beyond the literal definitions of words.
Consider the word cool.
If I tell you it’s cool outside, you probably understand that I’m talking about temperature. But if I tell you a new technology is cool, you know I’m not suggesting that you put on a sweater.
The word is identical. The meaning is completely different.
We were interested in how culture, context, and human experience influence the way language is interpreted. It was fascinating research because it gets at something fundamental about human communication. We don’t communicate through isolated words. We communicate through meaning. Meaning depends on context.
For years, search engines had been getting progressively better at recognizing that the use of langurage. Google introduced improvements to semantic understanding, including its Knowledge Graph in 2012 and Hummingbird in 2013. RankBrain followed in 2015, helping Google interpret queries in ways that went beyond simple keyword matching.
Then came BERT.
The B in BERT Changed Everything
Google introduced BERT into Search in 2019. BERT stands for Bidirectional Encoder Representations from Transformers, and while that may sound like something only a computer scientist could love, the concept behind it is surprisingly easy to understand.
I remember speaking about BERT at a Digiday conference shortly after its release and telling the audience that if they remembered nothing else, they should remember the B.
B stands for bidirectional.
Why was that important?
It is important because language doesn’t work one word at a time, independently of everything around it. The meaning of a word depends on what comes before it and what comes after it.
BERT represented a major advancement in a machine’s ability to interpret those relationships. Instead of relying primarily on individual terms or limited sequences, its language representations could consider context from both directions.
Google offered examples showing how small words could dramatically change the meaning of a search. In one example, understanding the word to in a query about a traveler going from Brazil to the United States made a significant difference in identifying the information the person actually needed.
Think about that for a moment. A word that a traditional keyword strategy might have considered relatively unimportant could completely change the meaning of a question. The keywords were Brazil and United States. It could be culture, food, anything to do with either of those countries. Adding the small contextual words of “from” and “to”, it is clear this is about how to get from one place to another, not the anything to do with places themselves.
For me, this was particularly exciting because I could see a connection to the questions we had been exploring years earlier at Harvard. Words were no longer simply strings of characters to be matched. Machines were becoming much better at interpreting how words relate to one another.
Natural language processing wasn’t new, but the advancements were extraordinary. Today, with generative AI and large language models, we experience those capabilities in ways that would have seemed almost unimaginable twenty years ago.
We can ask a question conversationally without regard to keywords. We can describe a problem without knowing the technical terminology. We can ask follow-up questions, clarify what we mean, and expect the technology to maintain context.
That is a far cry from typing a few carefully chosen words into a search box and hoping the right website appears.
So Why Are We Still So Obsessed With Keywords?
This is the part I find puzzling. For years, Google has been telling website owners to focus on creating useful, reliable, people-first content rather than content designed primarily to manipulate search rankings. Its search systems have become progressively more capable of understanding meaning, relationships, context, and intent.
Google’s guidance around E-E-A-T — Experience, Expertise, Authoritativeness, and Trustworthiness — reflects its emphasis on the qualities that help make information credible and useful. E-E-A-T is not a single ranking score, but it is a helpful way to think about the kind of content Google wants its systems to recognize.
Yet, I still see SEO recommendations that sound remarkably similar to conversations we were having twenty years ago.
How many times should a keyword appear on the page? Should we repeat the exact phrase in every heading? How many variations of the same keyword can we work into the content? Can we create another page targeting a slightly different version of the same search?
I understand why these practices persist. Keywords are measurable. Search volume is measurable. Rankings are measurable. They give marketers something tangible to manage and report.
To be clear, keywords still matter.
They help us understand the vocabulary people use, identify topics they care about, and recognize how they describe their problems. They remain important in paid search, where advertisers use keywords as one of several ways to indicate which searches they want to reach.
But there is an enormous difference between using keywords to understand what people are looking for and using keywords as the primary strategy for creating content.
One is research. The other can become an exercise in trying to satisfy a machine that has already moved well beyond the original rules of the game.
Search Is No Longer Just About Finding a Page
I know. We all want the world to love our website and visit our pages. That was the point of search, right? The emergence of generative AI has taken this evolution even further. For most of the internet’s history, search engines served primarily as navigational tools. You asked a question, received a list of possible sources, and clicked through to find your answer.
Today, AI-powered search experiences can interpret a question, gather information from multiple sources, synthesize an answer, and present it directly to the user.
Sometimes the person still clicks through to a website. Sometimes they don’t.
That changes the relationship between content creators, search platforms, and the people looking for information. Whether you understand it or not, if you own a website, you are a content creator.
The challenge is no longer getting a page to appear for a particular keyword. It is creating information that search and AI systems can understand, trust, retrieve, and use when responding to a question, even if there is not click to the page. Perhaps even more importantly, our challenge is creating information that is genuinely useful to the person asking.
This doesn’t mean that traditional search has disappeared or that SEO is dead. Google still uses multiple retrieval and ranking systems, and exact terminology can still matter enormously, particularly for product names, technical subjects, local services, and specific questions.
The trajectory is unmistakable. We have moved from matching words toward interpreting meaning. And we are increasingly moving from retrieving documents toward generating answers grounded in information. That requires a different approach to content.
What Should Businesses Be Doing Now?
I would begin by changing the question.
Instead of asking, “What keywords do we want to rank for?” ask, “What does someone need to understand before they would consider doing business with us?”
Think about someone considering a major home renovation, choosing a tax preparation service, evaluating a healthcare provider, or researching a fitness program. They may have dozens of questions before they ever decide which company to contact.
Some questions are practical. Others involve cost, risk, trust, alternatives, timing, or uncertainty. Many won’t be expressed using the industry’s preferred terminology. The businesses that can answer those questions clearly, credibly, and comprehensively are creating something far more valuable than keyword-optimized pages. They are creating useful information.
Start with the questions, not the keywords.
Talk to your sales teams, customer service representatives, and customers. Find out what people actually ask, what confuses them, what concerns them, and what they need to know before making a decision. Keyword research can help validate and expand that understanding, but it shouldn’t replace it.
Write for meaning and clarity.
Explain concepts in natural language. Use the terminology your customers understand, while introducing technical language when it helps. Organize information logically, explain relationships, and provide enough context for a reader to understand not only what something is, but why it matters.
Bring something original to the conversation.
This is becoming increasingly important in a world flooded with AI-generated content. If everyone is publishing variations of the same information, what reason does a person — or an AI system looking for useful sources — have to value yours? Firsthand experience, original research, practical examples, distinctive insights, and demonstrated expertise give content substance that generic rewriting cannot reproduce.
Make your expertise visible and verifiable.
Identify who is responsible for the information, explain relevant experience, support factual claims, and keep content accurate and current. Use clear page structures, descriptive headings, accessible text, and appropriate structured data so both people and machines can interpret what you’ve published. None of these techniques guarantees inclusion in an AI-generated answer, but they create a stronger foundation for discoverability and credibility.
Measure more than keyword rankings.
Rankings and organic traffic still have value, but they tell only part of the story. Look at whether people are finding answers, engaging with your content, recognizing your brand, returning directly, and ultimately choosing your business. As AI changes the way information is discovered, the relationship between visibility and website clicks will continue to evolve.
And finally, stop creating content simply because a keyword research tool says there is search volume. Search volume can tell you what people type. It cannot, by itself, tell you what they need to understand, what they believe, what they fear, or what might change their minds.
Those are questions about human behavior. Human behavior has always been the more interesting part of this story.
The Technology Changed. The Human Need Didn’t.
When I think back to those early days in California government, what strikes me is how much the technology has changed and how little the underlying objective has. We wanted to help people find the information they needed. Keywords helped us do that.
Today, AI gives us the ability to go much further. We can create information that is easier to understand, more responsive to the questions people actually ask, and more useful throughout the process of making a decision.
The hard part is we have to be willing to let go of some of the assumptions we developed when search engines worked differently. I don’t think the future belongs to businesses that figure out how to insert the greatest number of keywords into their websites. Nor do I think it belongs to those who simply produce the greatest volume of AI-generated content.
I think it belongs to businesses that understand their customers well enough to create information worth finding, understanding, and trusting. After all, the goal was never really to rank for a word.
The goal was to help someone find an answer, gain their trust and convert them into a customer.
And twenty years later, that is still the point.
By Carole Lawson
Co-Founder & Chief Data Officer, MarketStorm