How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

Thanks to Amit Gupta, Senior Director- AI/ML and Mark Wang, Software Engineering PMTS from Salesforce for your contributions in the vision and partnership of cross-Region inference routing on SageMaker AI. Salesforce’s Agentforce Foundation team serves a wide variety of models for products across Salesforce, including Agentforce and the Salesforce Trust Layer, in 30+ global Regions. … Read more

How Social Department built culture-aware video discovery

This post is co-written by Mariah Miller and Sirajus Salekin from AWS and Nicholas Chase from Social Department. Entertainment marketing teams manage tens of thousands of video assets across studios, networks, and streaming applications. Most campaigns begin with the same video discovery problem: finding the one moment in the footage that will resonate with an … Read more

Connect Amazon Bedrock AgentCore Gateway to private APIs in a hub-and-spoke network with centralized inspection

In many enterprises, a central network team owns every virtual private cloud (VPC) and subnet, and every flow between VPCs must pass a central firewall in both directions. Agents that call internal REST APIs and Model Context Protocol (MCP) servers through Amazon Bedrock AgentCore Gateway, a capability of Amazon Bedrock AgentCore, must meet the same … Read more

How Stretto built an AI voice agent with Amazon Nova 2 Sonic

This post is co-written with Danya Gordin from Stretto and Marcelo Wzorek from CI&T. Before Stretto built an AI voice agent, consumers considering bankruptcy on its Bankruptcy.com site filled out a web form with a few fields: debt amount, attorney status, and contact information. These form-generated leads carried thin margins. The gap between a web … Read more

Managed NVIDIA Dynamo on Amazon SageMaker HyperPod Inference

Serving large language models (LLMs) efficiently requires sufficient GPU memory and careful coordination of prompt processing and token generation. GLM-5.3’s FP8 weights, for example, exceed the 640 GB of GPU memory on a single ml.p5.48xlarge instance, before accounting for KV cache and runtime overhead. Long prompts can also slow token generation when both share the … Read more

Reducing token costs and latency in Amazon Bedrock AgentCore JSON workloads

As enterprises build production agents with dozens of tools and large structured inputs, token consumption and latency become the primary blockers to a good user experience. When these agents process large JSON data, every additional tool invocation injects the full payload into the large language model (LLM) context window, causing token usage and response latency … Read more

Amazon SageMaker AI real-time endpoints now support KV cache aware routing

Today, Amazon SageMaker AI introduces KV cache-aware routing (KV-aware routing), a new routing strategy for real-time endpoints. It uses the cache state that the model container reports. It sends each request to an instance that already holds a long cached match for its prompt and isn’t too busy. It’s for teams that serve large language … Read more

Govern and discover records with common metadata in AWS Agent Registry

As organizations build more agents, Model Context Protocol (MCP) servers and skills, they need a consistent way to describe, govern, and find them. AWS Agent Registry provides a catalog for registering, approving, and discovering these resources and custom records. Each record carries a name, a description, a protocol descriptor, and an approval status. However, every … Read more

Secure OAuth patterns for Snowflake Cortex AI through Amazon Bedrock AgentCore

Enterprises today need a clear way to extend their authentication and authorization controls into the agentic layer, often requiring either broad service account permissions or custom-built auth integrations that add operational overhead. When an AI agent running on Amazon Bedrock AgentCore (a platform to build, connect, and optimize agents at scale, with any framework or … Read more

Announcing AI data preparation capabilities in Amazon SageMaker: Build high-quality datasets for model development workflows

Teams that ship generative AI to production know that a model is only as good as its data. That is why preparing high-quality training and evaluation data comes first. Fine-tuning an open-weight model, running reinforcement learning, or building an evaluation suite all depend on data that is accurate, well-labeled, and representative of your task. For … Read more

Getting AI out of the demo and into businesses globally

AWS Forward Deployed Engineering is now available in 15 countries including U.S., Mexico, Brazil, Canada, Australia, Singapore, India, South Korea, Japan, UK, Germany, Netherlands, France, Sweden, and Saudi Arabia for commercial and public sector customers; and powered by dozens of built in production-ready accelerators. Every enterprise I talk to has built something with AI that … Read more

Reason over time-series data with OpenTSLM on Amazon SageMaker JumpStart

Across industries, decision-making starts with understanding what is changing and why. Industrial organizations collect data that informs decisions across their operations. This data helps prevent production stoppages, diagnose vehicle faults, explain drops in data-center performance, and adjust pricing as supply and demand shift. However, interpreting that data still requires specialist time, and collecting more data … Read more

Responsible AI for agentic AI: A phased roadmap for platform and application teams

Companies build agentic AI platforms to govern the applications, tools, skills, models, and storage that agents use. The nondeterministic nature of generative AI models introduces a complexity that previous software development practices didn’t anticipate. Consider a real estate app that generates property listings from retrieved data: does it accurately reflect the property, or hallucinate features … Read more

How ArisGlobal scaled generative AI inference on Amazon Bedrock

ArisGlobal, a provider of life sciences technology spanning pharmacovigilance, regulatory, medical affairs, clinical, and quality functions, developed NavaX, an AI engine purpose-built for life sciences on Amazon Bedrock, to automate adverse-event case processing. ArisGlobal chose Amazon Bedrock for managed access to foundation models, EU geographic cross-Region inference, and managed prompt caching. As enterprise customers added … Read more

Build memory-aware AI agents with Agentic Retrieve on Amazon Bedrock Managed Knowledge Base

Users of a knowledge-grounded application expect it to remember them, not only to search well. They ask follow-up questions that make sense only in the context of the previous turn. They expect an answer about a policy or a procedure to reflect what they told the application about themselves last month. And they still expect … Read more

How Duco transforms financial data automation with Amazon Bedrock AgentCore

Financial institutions face a critical challenge: processing massive volumes of data with speed, accuracy, and compliance while operating under some of the world’s strictest regulatory frameworks. Duco is the agentic Operations platform for financial services, serving tier-one banks and other financial institutions. It processes 20 billion transactions monthly for 200+ clients, including 7 of the … Read more

Taxonomy of multi-agent design patterns, anti-patterns, and lessons from manufacturing

Enterprise teams are building multi-agent systems, and when early implementations fall short, the cause is rarely the model. Multi-agent coordination is a distributed systems problem, and many teams taking it on haven’t yet built distributed systems instincts. The numbers are consistent across sources: ~88% of AI pilots never reach production. Stanford HAI demonstrated that two … Read more

Modernizing mainframe Assembler to Java using Amazon Bedrock and AWS Transform

Imagine a batch job that has run every night for decades without failing, and the only person who understands its code is about to retire. We have seen this situation again and again while working with banking, insurance, and airline customers. Their Assembler programs still run critical business operations, but fewer people can read them … Read more

Novo Nordisk modernizes manufacturing process documents with Amazon Bedrock AgentCore

Novo Nordisk (Novo) is one of the world’s leading global pharmaceutical companies, manufacturing medicines for chronic diseases such as diabetes and obesity, and delivering them to patients across many countries. Behind every product lies a manufacturing network that adheres to strict quality requirements. Every quality-relevant process step must be described, controlled, and validated through formal … Read more

Get answers from tables with tabular search in Amazon Bedrock Managed Knowledge Base

Amazon Bedrock Managed Knowledge Base now supports tabular search, so you can get answers from tables through agentic retrieval. When a question needs data from CSV, XLSX, or XLS files, tabular search looks up values and runs calculations across every relevant row in one or more tables or sheets. It then combines the results with … Read more

Customize answers per tenant in Amazon Bedrock Managed Knowledge Base

With retrieval instructions in Amazon Bedrock Managed Knowledge Base, you can customize answers per tenant. The same product and policy documents often serve several audiences, such as customers in different regions or internal teams, and each audience needs its own answer. You set the voice, format, retrieval guidance, and answer policy that agentic retrieval applies … Read more

Use prompt caching with open-weight models on Amazon Bedrock

Many generative AI applications send the same long context with every request, and prompt caching avoids processing it again each time. That context can be a system prompt, reference documents, tool definitions, or the conversation so far. Open-weight models on Amazon Bedrock can reuse it across requests, which helps reduce input costs and response latency. … Read more

Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1

GS Caltex uses Amazon SageMaker Canvas and Amazon SageMaker Pipelines to connect process engineers’ models to production workflows. The South Korean energy and chemical company divides model development and operations between engineers with complementary skills. Process engineers build and validate models in SageMaker Canvas without writing code, drawing on their process knowledge. Machine learning (ML) … Read more