What business owners need to check before worrying about AI visibility
Your website is live. You can open it on your laptop. Your customers can visit it. It appears when you type the business name into Google.
Surely that means ChatGPT, Gemini and Claude can find it too?
Possibly.
But a website being publicly available does not automatically mean every search engine and AI platform can access every important page, understand the information or retrieve it when answering a question.
A platform might be able to find your homepage but miss a service page.
Google might index a page while ChatGPT cannot use it in search.
An AI platform might access the page but struggle to extract important information because it is hidden inside an image, animation or awkward website feature.
And even when everything is technically accessible, your business may still not be selected as a recommendation.
There are several steps between “my website exists” and “an AI platform can use it as a helpful source”.
Let’s work through them.
The short answer
For AI platforms to find and use your website, your important pages generally need to be:
Publicly accessible
Available to the relevant crawler
Indexable where search indexing applies
Linked clearly within your website
Written in readable text
Clear about the subject, business and service
Supported by accurate technical signals
Useful enough to retrieve for a relevant question
Passing these checks makes your website eligible to be found and used. It does not guarantee a citation, mention or recommendation.
That part is worth repeating.
Technical accessibility opens the door. It does not force anybody to invite you onto the shortlist.
What does “find your website” actually mean?
The phrase can describe several different things.
They are related, but they are not interchangeable.
The platform can access the page
A crawler or user-directed retrieval system can request the URL and receive the page content.
It is not blocked by your website, hosting provider, security service or robots.txt instructions.
The page can be indexed
A search engine can process the page and consider storing it within its search systems.
A page being accessible does not guarantee it will be indexed. Search engines can choose not to index pages they consider duplicated, low-value, poorly connected or otherwise unsuitable.
The content can be understood
The platform can identify what the page is about, which business it belongs to and how it relates to the question being asked.
A technically accessible page can still be confusing.
The page can be retrieved for a relevant question
The platform considers the page useful enough to surface when answering a particular query.
This depends on relevance, quality, context and the other information available.
The business can be recommended
The platform goes beyond using the page as information and includes the business as a suitable option.
This usually requires more than access. It may involve clarity, relevance, supporting evidence and confidence in the business information.
When business owners say, “ChatGPT cannot find me,” the actual problem could sit at any of these stages.
AI search access and AI model training are not the same thing
This distinction causes plenty of confusion.
Allowing a platform to access your website for search is not necessarily the same as allowing it to use your content for model training.
OpenAI uses different web crawlers for different purposes.
OAI-SearchBot
OAI-SearchBot is used to surface websites in ChatGPT search results.
OpenAI states that websites opting out of OAI-SearchBot will not be shown in ChatGPT search answers, although they may still appear as navigational links.
If your goal is to make Orca articles eligible to appear as sources in ChatGPT search, this is the important OpenAI crawler to allow.
GPTBot
GPTBot is used to crawl content that may contribute to the development of OpenAI’s generative AI foundation models.
A website owner can disallow GPTBot while allowing OAI-SearchBot.
The two controls are separate.
ChatGPT-User
ChatGPT-User may access a webpage when somebody asks ChatGPT or a Custom GPT to visit it.
This is a user-directed action rather than automatic web crawling. OpenAI states that ChatGPT-User is not used to determine whether content can appear in ChatGPT search.
Anthropic also separates its crawlers by purpose.
Claude-SearchBot
Claude-SearchBot navigates the web to improve the quality and relevance of Claude’s search results.
Blocking it may reduce a website’s visibility and accuracy within those results.
Claude-User
Claude-User may retrieve content when a person asks Claude to access a website.
Blocking it can prevent Claude from retrieving your content in response to a user’s request.
ClaudeBot
ClaudeBot collects public web content that may contribute to model development.
Once again, search visibility and model training are treated separately.
This gives website owners more control, but it also means a broad instruction blocking every AI-related crawler may do more than intended.
Some businesses attempt to prevent model training and accidentally reduce their eligibility for AI-powered search at the same time.
What about Google’s AI search features?
Google handles AI Overviews and AI Mode as part of Google Search.
Google states that a page must be indexed and eligible to appear in conventional search with a snippet before it can appear as a supporting link within these AI features.
There is no separate special crawler that businesses need to allow specifically for Google AI Overviews.
Googlebot access and normal search eligibility remain central.
Google also says there are no additional technical requirements or special schema markup needed specifically for AI Overviews or AI Mode.
The familiar SEO foundations still matter:
Allow crawling
Make important pages indexable
Use internal links
Provide helpful content
Keep important information in textual form
Ensure structured data matches the visible page
Maintain current Business Profile information
You do not need to sprinkle “AI optimised” code around the website and hope something magical happens.
Check 1: Can the public open your website normally?
Begin with the unglamorous basics.
Open your website in a private or incognito browser window.
Check:
The homepage loads
Important service pages load
No password is required
No “coming soon” page appears
The correct domain opens
HTTPS works
Mobile pages load properly
Essential content appears without logging in
There are no security warnings
Pages do not redirect in a loop
This catches more issues than you might expect.
A website owner may see the correct version because they are logged into the website platform, while public visitors encounter a password page or old homepage.
Test the website as a stranger, not as its owner.
Check 2: Can Google index the important pages?
Google Search Console is the best starting point.
Use the URL Inspection tool to check each important URL.
Look for confirmation that the page:
Is known to Google
Is eligible for indexing
Uses the intended canonical URL
Is not blocked by robots.txt
Does not contain a noindex instruction
Can be crawled successfully
Has been indexed, where appropriate
Check the pages that matter commercially:
Homepage
Primary service pages
About page
Contact page
Important location pages
Major articles
Case studies
Key conversion pages
Do not assume the homepage represents the condition of the whole website.
A homepage can be indexed while several service pages are missing.
You can also search Google using:
This may give you a rough view of pages Google knows about, but it is not a complete indexing report. Use Search Console for a proper diagnosis.
Check 3: Is your robots.txt file blocking access?
A robots.txt file tells compliant crawlers which parts of a website they may access.
You can usually view yours by adding /robots.txt to the domain:
https://www.yourdomain.com.au/robots.txtLook for instructions involving:
Googlebot
OAI-SearchBot
GPTBot
Claude-SearchBot
Claude-User
ClaudeBot
Broad wildcard rules applying to every crawler
A complete block may look like:
User-agent: OAI-SearchBot
Disallow: /The forward slash tells that crawler not to access the website.
An empty Disallow instruction generally means access is allowed:
User-agent: OAI-SearchBot
Disallow:Do not edit robots.txt casually.
A misplaced instruction can affect large parts of the website. Website platforms and security services may also manage these settings differently.
If you are unsure, have the file checked before changing it.
Check 4: Is the page hidden from search?
A page can be publicly accessible while carrying a noindex instruction.
This tells search engines not to include it in their index.
Website platforms often provide a setting called something like:
Hide page from search results
Noindex this page
Discourage search engines
Search visibility
SEO visibility
These settings can be helpful for thank-you pages, private resources and internal content.
They are less helpful when accidentally enabled on a major service page.
Check the SEO settings for every important page.
Also check whether the website itself has a site-wide privacy, password or search-hiding setting enabled.
Check 5: Are you using the correct canonical URL?
A canonical tag tells search engines which version of a page should be treated as the primary version.
This is useful when similar or duplicated URLs exist.
Problems occur when an important page points to a different URL as its canonical version.
For example, you inspect:
https://www.yourdomain.com.au/seo-servicesBut the canonical tag points to:
https://www.yourdomain.com.au/servicesSearch engines may treat the second URL as the preferred version.
This can happen after:
Rebuilding a page
Changing URLs
Duplicating a page
Migrating platforms
Leaving an old version live
Applying an incorrect SEO setting
Search Console can show which canonical URL Google selected.
If it differs from the page you expected, investigate why.
Check 6: Can AI platforms access a specific page?
You can run a simple manual check.
Open a new conversation and provide the exact URL.
Ask:
Can you access and summarise the information on this webpage? Please tell me whether you retrieved the live page, which business it belongs to and the main subject it covers: [insert URL]
Then ask:
Which specific details were you able to retrieve from the page, and was any important content unavailable or unclear?
Test the same page in more than one platform.
Remember that failure to retrieve a page does not always mean the website permanently blocks the platform.
Temporary errors, platform limitations, login requirements, security controls and retrieval failures can all interfere.
Record the result and test again before drawing a firm conclusion.
Check 7: Is the important information available as text?
A beautiful image can communicate a great deal to a human visitor.
A crawler usually needs important information to exist in readable text too.
Avoid placing essential details only inside:
Graphic tiles
Infographics
Hero images
Menus uploaded as images
PDF brochures
Social media posts
Videos without supporting text
Decorative animations
Image-based testimonials
For example, if your service areas appear only on a map graphic, the information may be less accessible than a clear written list.
If a restaurant uploads its current menu as an image, platforms may have difficulty reliably extracting dish names, dietary information and prices.
If an event date appears only in an Instagram tile, your website provides no clear textual version for a crawler to retrieve.
Images and video still have value. Google specifically encourages businesses to support useful written content with high-quality visual material.
The important point is to provide the facts in text as well.
Check 8: Can someone reach the page through your website?
Important pages should not sit alone like tiny digital islands.
Internal links help visitors and search systems discover content and understand how pages relate to one another.
Every major service page should be linked from relevant places, such as:
Main navigation
Homepage
Related service pages
Articles
Footer, where appropriate
Case studies
Calls to action
Articles should also link to other useful resources.
For example, an article about ChatGPT accuracy can link to:
The AI Visibility Test
AI Visibility Services
A related article about recommendation gaps
The About page when author expertise is relevant
Do not add links simply to achieve a predetermined number.
Add them where they help the reader continue exploring the subject.
If an important page has no internal links pointing to it, crawlers may take longer to find it and have less context for understanding its role.
Check 9: Does the page explain itself clearly?
A crawler may access a page perfectly and still struggle to understand its purpose.
Check whether the page clearly communicates:
The business name
Main topic
Service
Intended audience
Location or service area
Author or responsible expert
Relevant experience
Relationship to the rest of the website
Use descriptive page titles and headings.
A heading such as:
Solutions designed around you
sounds polished but explains very little.
A heading such as:
SEO Audits for Established Australian Businesses
provides a much clearer subject.
You can still write with personality. Clarity does not require your website to sound like it was assembled by a committee of instruction manuals.
Check 10: Is JavaScript hiding important content?
Many modern websites use JavaScript for interactive features.
Search engines have become better at processing JavaScript, but complex implementation can still create problems.
Important content may be:
Loaded only after a user clicks
Inserted after the initial page load
Hidden inside an app
Pulled from an external system
Unavailable when scripts fail
Rendered differently for crawlers
Missing from the underlying page source
You do not need to become a JavaScript developer to recognise a possible issue.
Warning signs include:
Content appears after a noticeable delay
Important text disappears when scripts are blocked
Search Console cannot see sections visible in your browser
Interactive tools contain all the useful information
AI platforms retrieve only the page heading and nothing beneath it
If you suspect this problem, a technical review can compare the page seen by a browser with the version available to crawlers.
Check 11: Are security tools blocking legitimate crawlers?
Web application firewalls, content delivery networks, bot-protection services and hosting security tools can block automated access.
This may happen even when robots.txt allows crawling.
Security systems might challenge or reject requests based on:
User agent
IP address
Request frequency
Geographic location
Bot-detection rules
Firewall settings
CAPTCHA requirements
OpenAI publishes IP ranges for its crawlers. Anthropic also provides information to help website owners verify its bots.
If your robots.txt file looks correct but platforms repeatedly cannot access the website, hosting or security controls may need investigation.
Do not disable security broadly to solve this.
The goal is to allow verified legitimate access without inviting every badly behaved bot on the internet in for tea.
Check 12: Is your sitemap working?
A sitemap gives search engines a list of important URLs available on the website.
Most major website platforms generate one automatically.
It is commonly found at:
https://www.yourdomain.com.au/sitemap.xmlCheck whether:
The sitemap loads
Important pages are included
Removed pages are no longer listed
URLs use the preferred domain
Old or temporary pages are absent
The sitemap has been submitted in Google Search Console
A sitemap helps discovery, but it does not guarantee indexing.
It is a useful guide, not an instruction Google must obey.
Check 13: Have redirects been handled correctly?
When a page URL changes, the old URL should usually redirect to the most relevant new location.
Poor redirect handling can cause:
Broken links
Lost authority
Confusing search results
Old pages remaining visible
Visitors reaching irrelevant content
Crawlers following unnecessary chains
Avoid redirecting every removed page to the homepage.
If an old AI visibility article has moved, redirect it to the new version of that article or the closest relevant resource.
Keep redirects direct wherever possible.
Old URL → Current URL
Not:
Old URL → Older replacement → Temporary page → Current URL
Check 14: Are you expecting schema to solve an access problem?
Structured data can help search engines interpret certain information.
It cannot make a blocked page crawlable.
It cannot force a page into an index.
It cannot replace visible content.
It cannot manufacture authority or guarantee an AI recommendation.
Schema should accurately reflect information people can see on the page.
Use it as supporting structure, not a substitute for a functioning website.
Do you need an llms.txt file?
You may encounter recommendations to add an llms.txt file to your website.
At this stage, it should not be treated as a universal requirement for AI visibility.
Google explicitly says businesses do not need new machine-readable AI files or special AI markup to appear in its AI search features.
The major priorities remain:
Crawl access
Indexability
Readable content
Internal linking
Clear business information
Useful resources
Accurate structured data where appropriate
Strong supporting evidence
Do not allow a fashionable technical extra to distract you from a blocked service page or a website that never clearly states what the business does.
How can you tell whether the problem is technical or informational?
Ask the platform to retrieve a specific page.
Then ask it to explain the business without providing the page.
Use these prompts:
Can you access this exact page and summarise its main content? Please identify any sections you could not retrieve: [insert URL]
What can you tell me about [business name] in [location]? Please include its services, service areas, key people and current sources.
Which pages from [domain] did you use, and what information did each page support?
The results can suggest different problems.
The platform cannot access the URL
Investigate crawl restrictions, security settings, page status and technical availability.
The platform accesses the page but misunderstands it
The problem may involve unclear headings, vague content, hidden information or poor page structure.
The platform understands the page but does not mention the business elsewhere
The page may be accessible but weakly connected to the broader business entity.
The platform finds the website but does not recommend the business
The issue may sit beyond technical access.
Look at relevance, authority, supporting evidence, consistency and competitor strength.
What should you fix first?
Work in this order.
1. Public accessibility
Confirm the correct website and important pages load for ordinary visitors.
2. Search eligibility
Check indexing, noindex settings, canonical URLs and Search Console reports.
3. Relevant crawler access
Review robots.txt and security controls for Googlebot, OAI-SearchBot, Claude-SearchBot and other relevant systems.
4. Text availability
Make sure essential information appears clearly in readable text.
5. Website structure
Connect important pages through useful internal links and navigation.
6. Content clarity
Explain what the business does, who it helps, where it operates and why each page matters.
7. Supporting evidence
Build case studies, reviews, expert information and credible third-party references.
8. Repeat testing
Retest the same URLs and prompts after changes have been processed.
This order prevents you from investing heavily in authority or content before correcting an access problem.
Can you make ChatGPT crawl your website immediately?
There is no universal submission button guaranteeing that a page will appear in ChatGPT search.
For Google, you can submit a sitemap through Search Console and request indexing for important URLs.
For AI platforms, make sure relevant crawlers are allowed and that the website is technically accessible.
Then allow time for the systems to revisit and process the content.
OpenAI says changes to robots.txt may take approximately 24 hours to be reflected by its systems. Changes to the content itself may take longer to be discovered and used.
Repeatedly asking ChatGPT to visit the page does not guarantee that it will be added to search systems.
Can you guarantee that accessible content will be cited?
No.
Crawl access makes a page eligible to be considered. It does not guarantee that the platform will retrieve, cite or recommend it.
The page must still be relevant to the question.
It also competes with other available sources.
A technically perfect page containing vague, generic or unsupported content may remain absent from useful answers.
Likewise, a strong page can be accessible and indexed without appearing in every relevant AI response.
This is why AI visibility cannot be reduced to a crawler checklist alone.
A final checklist for business owners
Before concluding that AI platforms are ignoring your business, check:
The live website is publicly accessible
The correct homepage is active
Important pages are not password-protected
Google can crawl and index key pages
Noindex settings are not enabled accidentally
Canonical URLs point to the intended pages
Robots.txt does not block relevant search crawlers
Hosting security is not rejecting verified crawlers
Important information exists as text
Service pages are linked internally
Titles and headings explain each page clearly
The sitemap contains the correct URLs
Old URLs redirect appropriately
Structured data matches visible content
Business profiles contain current information
AI platforms can retrieve specific URLs when tested
If those foundations are sound, move beyond the technical layer.
Ask whether your content is useful, your services are clear and your claims are supported.
Being accessible is the beginning.
Being worth retrieving is the next part.
Make your website findable before asking it to be chosen
It is easy to jump straight to the exciting question:
How do I get ChatGPT to recommend my business?
But first, establish whether ChatGPT and other platforms can access the right information at all.
A blocked crawler, hidden page or image-only service description can limit visibility before questions of authority and recommendation even begin.
Once the technical foundations are working, you can focus on the wider picture:
Does the platform understand the business accurately?
Does it connect the business with relevant customer needs?
Is there credible evidence supporting its claims?
Does it appear consistently across different prompts?
Which competitors are selected instead?
You can use the Orca AI Visibility Test to assess more than one isolated response.
You can also read What Does ChatGPT Know About Your Business? to check how accurately your business is being described.
For help investigating technical search foundations, explore Orca’s SEO Services.
For a broader examination of AI search and recommendation visibility, explore AI Visibility Services.
CHECK YOUR TECHNICAL VISIBILITY
Not sure whether AI platforms can access the right information?
Orca can examine your website’s technical accessibility, content clarity and wider digital presence to identify what may be limiting its visibility.
You will receive practical priorities based on the problem, not a generic checklist of AI tricks.
DISCUSS YOUR SEARCH VISIBILITYABOUT THE AUTHOR
Sally Townsend
Sally is the founder of Orca Creative and an SEO and AI visibility strategist helping Australian businesses strengthen how they are found, understood and chosen online. Her work combines practical SEO, content strategy, digital authority and AI visibility without the smoke, mirrors or mysterious promises of a guaranteed ChatGPT ranking.
MORE ABOUT SALLY