User-agent: * Disallow: /api/ Disallow: /backend/ Disallow: /customer/ Disallow: /frontend/