Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters
During a traffic peak, a request can sit in a queue on a saturated HyperPod cluster while GPUs on another cluster, sometimes in another AWS Region, sit idle. You have already paid for that capacity, but your requests can’t reach it. In Introducing Amazon SageMaker HyperPod Inference Gateway, we showed how the Inference Gateway gives … Read more