User-agent: Mediapartners-Google Allow: / User-agent: * Disallow: /*.php User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: /