How does the Distributed Map state in Step Functions support processing extremely large datasets that would exceed the limits of a standard Map state?

The Distributed Map state is designed to process massive datasets, potentially containing millions of items sourced directly from an S3 bucket, by automatically spreading the workload across many concurrent child workflow executions, far beyond what a standard Map state's iteration limits can support, making it well suited for large scale data processing tasks like transforming every file within a massive S3 bucket without needing to build a custom distributed processing system yourself.

{
  "Type": "Map",
  "ItemReader": {"Resource": "arn:aws:states:::s3:listObjectsV2", "Parameters": {"Bucket": "my-bucket"}},
  "MaxConcurrency": 1000
}