Introducing container caching in Amazon SageMaker AI for faster model scaling

Today, we’re excited to announce container image caching for Amazon SageMaker AI inference, the next major advancement in our faster scaling optimization journey. This speeds up end-to-end latency by up to 2x for generative AI models during scale-out events. Over the years, Amazon SageMaker AI has continued to reduce latency across these scaling stages: detecting … Read more

All the latest news on Android 17, Wear OS 7, and Android XR

Google’s Android 17 update includes highlights like new floating “Bubble” app windows for easier multitasking, a Screen Reaction recording mode, and a 50/50 split gaming mode for foldable phones. Meanwhile, Wear OS 7 brings Live Updates, better battery life for smart watches, and prepares connections for new Android XR smart glasses that will launch this … Read more

Android 17 arrives on Pixel phones today

Floating Bubble windows are among the biggest changes in the OS update. | Image: Google Following its official debut last month, Google is now rolling out Android 17 to compatible Pixel phones, alongside additional exclusive features as part of the June Pixel Drop. Not every feature announced alongside the OS at the pre-I/O Android Show … Read more

Parallelize speculative decoding with P-EAGLE on Amazon SageMaker AI

As large language models (LLMs) grow in size and complexity, maximizing inference throughput while minimizing latency remains a critical challenge for enterprise production deployments. Speculative decoding is one effective strategy to address this, utilizing a lightweight draft model to guess future tokens which are then verified by the target LLM in a single forward pass. While state-of-the-art frameworks like Extrapolation Algorithm for Greater … Read more