robots.txt and noindex rule

Step 1 : One misconception that we see often is the use of robots.txt for noindexing of content Step 2 : Disallowing search engines to crawl a specific directory on your site does not mean search engines will not index it. The best practice for this case is to use the meta robot tags Step...