#Pinterestbot setup User-agent: Pinterestbot Crawl-delay: 1 # Crawlers Setup User-agent: * # Directories Disallow: /app/ Disallow: /bin/ Disallow: /dev/ Disallow: /lib/ Disallow: /phpserver/ Disallow: /pkginfo/ Disallow: /report/ Disallow: /setup/ Disallow: /update/ Disallow: /var/ Disallow: /vendor/ Disallow: /customer/account/login/referer/ # Paths (clean URLs) Disallow: /*.php$ Disallow: */review/ Disallow: */search/ Disallow: */catalogsearch/ Disallow: */catalog/product_compare/ Disallow: */catalog/category/view/ Disallow: */catalog/product/view/ Disallow: */customer/ Disallow: */sendfriend/ Disallow: *?___store=default Disallow: *?customFilters # Paths (not clean URLs) Disallow: /*SID= # Parameters to prevent server overload Disallow: /*?customFilters Disallow: /*?priceMin Disallow: /*?priceMax Disallow: /*?sortKey Disallow: /*?sortDirection # Website Sitemap Sitemap: https://www.halosleep.com/media/sitemap_halous.xml Sitemap: https://www.halosleep.com/media/blog_sitemap_halous.xml