The notion of artificial intelligence has been discussed quite a bit since 2022. I say that just in case you’ve been living under a rock and didn’t notice. As a front-developer, an important consideration for me is how the use of Large Language and Multimodal Models like ChatGPT and Gemini impacts search behavior. If I’m creating a product for a business that is unlikely to be served up to their customers, then, for obvious reasons, they’re unlikely to hire me in the first place. Understanding how AI impacts Search Engine Optimization (SEO) is, therefore, important to anyone creating websites or other products which have traditionally relied on search engines such as Google and Bing. This article will take a look at how the rise of AI has impacted SEO. And just to save you the suspense – it doesn’t for the most part.
Below is a list of topics I’ll be addressing in this article. If you need assistance with a project, then click the button below to contact us online or by telephone.
Table of Contents:
- Why developers should care about SEO (jump to section)
- Why AI hasn’t changed SEO (jump to section)
- SEO fundamentals to understand (jump to section)
- Myths about AI and SEO (jump to section)
Why developers should care about SEO
Search engine optimization is important to a developer’s potential clients
The reason why front-end developers should care about SEO should be obvious – quite a few of your potential clients are going to care about it. I regularly build websites and other consumer-facing applications for clients. These clients will need the product to have a chance of doing well in search engines so they can connect with their potential customers or end users. This is why I’m often asked, “how will you get my website on Google?” when someone calls me for the first time. If you’re not building a product that will have a chance at appearing in the results for various search queries then you’re doing your potential clients an injustice and hurting your own business in the process.
Developers must understand what SEO is and isn’t
A key part of “caring about search engine optimization” is to understand what the term means in the first place. After all, it’s hard to focus on something without clearly defining it first. In short, real “SEO” consists of two aspects. The first aspect is ensuring that content on a page, website, or within an application, is relevant to the end user of that product. Also, this content (whether it be verbiage, photos, or information) must consist of unique material that is not simply copied or regurgitated. Second, the product must provide a proper user experience. This second aspect is not as subjective as it sounds. The search engines provide objective criteria as to what they consider to be a “good user experience.” Such criteria is provided by the search engines themselves. See Google’s webmaster guidelines as a result. While some may act as if what search engines are looking for is a mystery, the truth is that they publish it and make it readily available.
The above should be taken to mean that search engines want you to create a product which provides useful information to a user in a way that is easy to use. The reason for this is simple. Suppose you own Google or Bing (congratulations – you’re rich). If I go to a search engine and query “how do I ‘xyz’”, and you return me a list of results that aren’t relevant or are frustrating to use, then I’m going to quit using your search engine. If I quit using your search engine, then I’m never going to come back and click on ads. All a search engine cares about, therefore, is providing me with useful and usable information, so I keep coming back. This is why content and user experience are the cornerstone of “optimization.” Later in this article, we’ll dive into some more specifics on how to provide a good user experience.
AI has not changed SEO
If you’re wondering how AI has changed SEO, the answer is “it hasn’t.” I say this due to the fact that, as I just discussed above, the goal of a search engine is to provide a user with the most useful and usable information. If someone is throwing a query at an AI agent then they are also looking for the most useful and usable information. The goals of a search engine and an AI agent are, therefore, the same. Google has explicitly stated this by saying:
The best practices for SEO continue to be relevant because our generative AI features on Google Search are rooted in our core Search ranking and quality systems.[1]
In other words, when Google is deciding whether to include a website and its information in the “AI” search results, it is simply relying on the same algorithms that it uses for search. Optimizing for AI, therefore, is the same as optimizing for search. Google has even gone as far as to explicitly dispel quite a few myths that are popular in regard to how one should “optimize for AI.” This, again, is due to the fact that search and AI results function in the same way. That’s about all there is to that.
Fundamentals for SEO (and AI) that developers should understand
There are some core concepts that developers should understand when it comes to optimizing sites for eligibility in search results and AI agent responses. While this list is far, far, far, far (far) from being comprehensive, these are some of the big areas which should be focused on. These areas include:
- Creating “non-commodity” content
- Providing a proper user experience
- Ensuring the website or application can be indexed
- Submitting an XML sitemap to search engines
- Including proper meta data
- Including structured data
There’s also a lot of advice thrown around online which is just flat-out wrong. Such advice includes:
- The need to include an LLMS.txt file
- The need to “chunk” your website’s content
- The need to manufacture social media engagement
Let’s take a look at each of these in this and the next section of this article.
Creating non-commodity content
“Commodity content” is that which is based on generally agreed-upon information and is not unique. An example of this would be a webpage that is nothing but a list of “ingredients for bread.” This list is going to be the same no matter who is writing it. If, on the other hand, the verbiage on the page addressed “why I only use organic, stone-ground flour when making bread,” then a less-generic point of view is being provided that goes beyond common and recyclable knowledge.
Why do search engines frown on “commodity content?” The answer is simple. The entire goal of a search engine is to return the most relevant information to a query. If Website One and Website Two both contain the exact information (i.e. a list of ingredients for bread), then there is nothing making Website One more relevant than Website Two and vice versa. Google, therefore, would have no reason to return one above the other. The only way Google is going to care about a page is if it provides information that is not readily available across the internet or through common knowledge.
Providing a proper user experience
The idea of providing a “good user experience” is not subjective as far as search engines are concerned. The search engines actually provide quite a bit of guidance on what they consider a proper UI and UX. For a deep dive into what’s good and what’s not, I’d suggest familiarizing yourself with Google’s Search Essentials documentation[2] (also mentioned above). This documentation is kept up to date and should be understood by anyone holding themselves out as a front-end developer.
What constitutes a good UI and UX can be a rather lengthy separate course on its own. For the purposes of this article, I’ll dive into three of the big mistakes that developers make.
- Neglecting site speed – one of the big factors search engines look at is how quickly a website or application loads and how long it takes for it to become interactive. This only makes sense. After all, when’s the last time you said, “I wish this was taking longer?” Free tools from Google for checking site speed, which also provide feedback on what needs correction, can be found at Google’s Page Speed Insights tester[3].
- Avoiding intrusive popups – Ever been on a web page and you can’t interact with it until you fill out a pop-up? That’s a “no-no” as far as search engines are concerned. Google’s unintrusive dialog guidance[4] should be followed for any forms or captures placed in an application or website.
- Avoiding excessive advertising or distractions – Search engines are based on the idea that people are clicking a link to gain information or access to a service. When a page is filled with advertising or other distractions, to the point that it is difficult to use, then the search engines are less likely to consider it a quality experience.
Providing a good user experience is also important for AI agents for one obvious reason – if an agent recommends a website, and someone clicks through to check the source, then it’s problematic for the agent if the website provides a poor experience. Page experience, therefore, matters for “AI search.”
Ensuring that a webpage or application can be indexed
While this may sound like a “duh” statement, there are a few situations in which developers often err and make information unavailable to search engines. The first is when pages or an entire website are blocked through a robots.txt file. This can occur when a developer has the file in place for staging purposes and simply forgets to remove it after deployment. This can also happen when a “developer” simply doesn’t understand how such files work. A second, more common reason why search engines may not be able to access content is when it is blocked behind a paywall or login requirement. There are many reasons why one may not wish to give away information for free, but still needs the website or application to appear in search results. Google’s guidance for paywalled content[5] can be followed to ensure that the pages are protected from non-paying users, but accessible to search engines.
Submit an XML Sitemap to search engines
Google cannot index a page unless it knows it exists. Submitting an XML sitemap to the search engine, through Google Search Console[6], provides the search engine with a complete list of your pages that stays up to date. The search engine will then begin the process of crawling and indexing the pages in the sitemap. Bing and other search engines have their own equivalents of Google Search Console.
Interestingly, Open AI does not have a method for submitting an XML sitemap. So, while Gemini can find pages through a sitemap that has been submitted in Google Search Console, ChatGPT is reliant on finding data solely by crawling the internet.
Including proper meta data
Metadata helps search engines, and AI agents, to understand the meaning of a page as well as its objectives. This can include a description of what the page conveys as well as the title of the document which appears in search results. If you don’t include this data (and many pages don’t), you are leaving search engines and AI agents to guess when it comes to meaning. Google keeps an up to date list as to its supported metadata, which can be found in its Search Essentials documentation.
Include structured data
It’s important to understand that search engines and AI agents can’t read. They can look at a block of text and interpret patterns, but they can’t understand a broader context in the way we do as human beings. Including structured data in a page or application helps search engines and AI agents to understand the meaning of a page as well as various values (i.e., that a four-star review is on a scale of five stars).
Structured data to include on a page, in a website as a whole, for products, and elsewhere, requires the assigning of values. These values can be found at schema.org[7] and are regularly changing and being updated. To test the validity of structured data, Google has provided a Schema markup testing tool[8].
Ignore common myths about how AI impacts SEO
There are several myths that have quickly spread around the new-fangled internet about how to “do well” in AI results. Google and others have directly addressed these. Let’s look at a few myths you should avoid.
Including an LLMS.txt file
A common piece of advice you’ll see is to include what’s known as an LLMS.txt file in the public root of your website’s directory. This is a machine-readable file which some attempt to claim will help Large Language Models with finding the content on your website. Google, however, has explicitly stated the following:
“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.”[9]
These files also are not accessed by services such as ChatGPT. I regularly view our server logs and have yet to see a request from AI crawlers attempting to access them. Unfortunately, it is common for developers to say these files are necessary in order to make it sound as if they are “wise when it comes to AI.” There is no need for an LLMS.txt file. Including one will, in no way, hurt your website. It won’t help either.
Chunking web content
It’s commonly stated that content on a web page should be “chunked.” This would mean breaking content into smaller bits so AI can “better understand it.” Words and content will be crawled regardless of whether they are “chunked.” Google has repeatedly stated to provide content which will engage your audience and not to break up content in order to target an AI agent.
Fake social media engagement
One of the more laughable “AI” strategies that people mention is to create additional or fake Reddit profiles to mention a website or business. Google has a name for this kind of conduct – “Spam.” Attempting to gain what Google refers to as “inauthentic mentions” does not help.
Conclusion
The use of AI has not changed the fundamentals of search engine optimization. This is due to the fact that the goal of an AI agent is the same as the goal of a search engine – to provide useful information. Google has explicitly stated that the two are one and the same. If you are a front-end developer, then it is important to understand this fact, as well as the basics of SEO, when it comes to helping your clients.
Cheers!
References
[1]Optimizing your website for generative AI features on Google Search, Google Search Central. Accessed at: https://developers.google.com/search/docs/fundamentals/ai-optimization-guide on September 10, 2026
[2] Google Search Essentials, Google Search Central. Accessed at: https://developers.google.com/search/docs/essentials on September 10, 2026.
[3]https://pagespeed.web.dev/
[4] Avoid intrusive interstitials and dialogs, Google Search Central. Accessed at: https://developers.google.com/search/docs/appearance/avoid-intrusive-interstitials on September 10, 2026.
[5] Structured data for subscription and paywalled content, Google Search Central. Accessed at: https://developers.google.com/search/docs/appearance/structured-data/paywalled-content on September 10, 2026
[6] https://search.google.com/search-console/about
[7] https://schema.org/
[8] https://developers.google.com/search/docs/appearance/structured-data
[9] https://developers.google.com/search/docs/fundamentals/ai-optimization-guide