How would you generate a sitemap and robots file to help search engines properly crawl a large Next.js website?

You create special files named sitemap.js and robots.js in your app folder, exporting functions that return the list of URLs to include in your sitemap and the crawling rules for search engines, and Next.js automatically builds these into the proper sitemap.xml and robots.txt files.

// app/sitemap.js
export default async function sitemap() {
  const posts = await getAllPosts();
  return posts.map(post => ({
    url: `https://example.com/blog/${post.slug}`,
    lastModified: post.updatedAt
  }));
}