# Allow only major search spiders User-agent: * Sitemap: https://www.musixmatch.com/community_sitemap/sitemap.xml Crawl-delay: 60 User-agent: Mediapartners-Google Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Googlebot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: MSNBot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: bingbot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Slurp Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Yahoo! Slurp Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: DuckDuckBot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Baiduspider Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Yandex Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: grapeshot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin User-agent: Twitterbot Disallow: Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin # Block all other spiders User-agent: * Disallow: / # Block Directories for all spiders User-agent: * Disallow: /search Disallow: /*/search Disallow: /profile Disallow: /*/profile Disallow: /contribute Disallow: /*/contribute Disallow: /*/annotation Disallow: /verify-user Disallow: /crowdin