.Google.com's John Mueller addressed a concern concerning why Google.com marks pages that are actually disallowed from creeping through robots.txt and also why the it is actually safe to disregard the relevant Browse Console documents concerning those creeps.Robot Website Traffic To Concern Specification URLs.The person talking to the question recorded that robots were actually making links to non-existent query criterion Links (? q= xyz) to pages with noindex meta tags that are likewise blocked out in robots.txt. What motivated the question is that Google.com is actually crawling the web links to those webpages, obtaining blocked out through robots.txt (without noticing a noindex robots meta tag) after that getting reported in Google Browse Console as "Indexed, though blocked through robots.txt.".The individual talked to the complying with question:." Yet right here is actually the major question: why will Google mark pages when they can not also see the information? What is actually the advantage during that?".Google's John Mueller confirmed that if they can not crawl the webpage they can not view the noindex meta tag. He also makes a fascinating reference of the web site: hunt operator, urging to disregard the outcomes given that the "typical" individuals won't view those outcomes.He wrote:." Yes, you are actually appropriate: if our team can not crawl the webpage, our company can't find the noindex. That stated, if our experts can't creep the webpages, at that point there's not a whole lot for our team to index. So while you might find several of those web pages with a targeted website:- concern, the typical customer won't view them, so I wouldn't fuss over it. Noindex is additionally great (without robots.txt disallow), it merely suggests the URLs will find yourself being crept (and end up in the Search Console file for crawled/not listed-- neither of these conditions cause problems to the remainder of the web site). The important part is that you do not produce all of them crawlable + indexable.".Takeaways:.1. Mueller's answer affirms the constraints in operation the Web site: hunt accelerated search operator for diagnostic reasons. Among those reasons is given that it is actually not linked to the normal hunt mark, it is actually a distinct thing entirely.Google.com's John Mueller discussed the internet site hunt driver in 2021:." The brief answer is that a website: inquiry is not implied to be total, neither utilized for diagnostics objectives.A web site question is actually a details kind of search that confines the end results to a particular web site. It is actually primarily just words website, a colon, and afterwards the website's domain name.This inquiry limits the outcomes to a certain internet site. It's not indicated to be a detailed selection of all the pages from that internet site.".2. Noindex tag without using a robots.txt is actually fine for these type of scenarios where a crawler is actually connecting to non-existent webpages that are actually getting found by Googlebot.3. URLs with the noindex tag will definitely generate a "crawled/not listed" entry in Look Console and also those won't have a damaging impact on the rest of the web site.Review the concern and respond to on LinkedIn:.Why would Google mark webpages when they can not also see the content?Included Picture by Shutterstock/Krakenimages. com.