Noindex
A meta robots or X-Robots-Tag directive telling search engines not to store a page in their index, even though they may still crawl it.
In depth
What it really means
Noindex is how you actually remove a page from search. This is worth stating clearly because the intuitive alternative, blocking it in robots.txt, does the opposite: it prevents crawling, so the noindex tag is never read and the URL can linger in results.
The correct pattern for a page you want gone but whose links you still want followed is noindex, follow, with the page left crawlable.
How it works
- A meta robots tag or X-Robots-Tag HTTP header specifies noindex.
- The crawler must be able to fetch the page to read it.
- The page is dropped from the index at the next crawl.
- With follow, links on the page continue passing equity.
Pros & cons
Pros
- The only reliable way to remove a page from the index.
- Works on non-HTML files through the X-Robots-Tag header.
- Preserves link flow when combined with follow.
Cons
- Requires the page to stay crawlable, which uses crawl budget.
- Removal takes effect only at the next crawl.
- Applied wrongly at template level it can deindex whole sections.
Common mistakes
- Combining noindex with a robots.txt Disallow, which stops the tag from ever being read.
- Applying it to a template and deindexing an entire section by accident.
- Using noindex where a 301 redirect or canonical would be more appropriate.
- Forgetting to remove it after a staging site goes live, the classic launch disaster.
Best practices
- Keep noindexed pages crawlable so the directive can be read.
- Use noindex, follow when the page’s outbound links still matter.
- Use the X-Robots-Tag header for PDFs and other non-HTML files.
- Audit for stray noindex tags immediately after any launch or migration.
FAQs
What is noindex?
A directive telling search engines not to store a page in their index. It can be a meta robots tag or an X-Robots-Tag HTTP header.
Noindex or robots.txt?
Noindex to remove a page from search. Robots.txt blocks crawling, which prevents the noindex from being read, so a blocked page can stay indexed indefinitely.
Does a noindexed page still pass link equity?
With noindex, follow, yes. Links on the page continue to be followed and pass equity even though the page itself is not indexed.
Keep reading
Related on LymLyt
Beyond LymLyt
Further reading
Want this working on your site?
We build the content behind the term, ranked in search and cited by AI.
Book a 30-min call →