AWSML

Eliminate GPU waste. Reduce first-token latency by up to 82%. Install one Kubernetes-native addon with zero application changes. The problem: Naive routing wastes your most expensive resource Running large language models (LLMs) at scale on GPU clusters is expensive. The default Kubernetes load balancers are making it worse. Round-robin and least-connections algorithms have no visibility … Read more

Hiring at scale in industries such as retail, logistics, hospitality, and others has its fair share of challenges. Recruiting teams are expected to fill hundreds of roles within tight timelines, often with limited capacity and with tools that weren’t designed to seamlessly work together. As a result of this, applications pile up, phone screens get … Read more

When building a Retrieval Augmented Generation (RAG) solution with Amazon Bedrock Knowledge Bases, selecting the right vector store impacts performance and cost. Amazon Bedrock Knowledge Bases offers a fully managed option and a customer-managed option where you choose your own vector store. This post focuses on the customer-managed path, comparing the three supported backends: Amazon … Read more

Git activity is one of the richest signals engineering teams produce that can provide continuous observability into development analytics. The challenge is extracting these Git metrics at scale, which has traditionally required hand-rolled extract, transform, and load (ETL) jobs, dedicated infrastructure, and ongoing maintenance. Further, with modern development tools becoming more prevalent, teams need a … Read more

Building a working agentic prototype takes an afternoon. Getting it to production is where the work explodes. The moment an agent has to serve more than one user, a new layer of engineering appears and it’s critical to tell whether the agent is doing the right thing on real traffic. Concurrency, session isolation, identity, persistent … Read more

Basic AI chat isn’t enough for financial services organizations that need secure, self-service AI agents. In financial services, employees need AI that can work with internal systems and sensitive client data, stay inside a governed environment, and remain auditable and cost-transparent. All of this must happen without every team standing up its own tools. This … Read more

Each Model Context Protocol (MCP) tool invocation on Amazon Quick is an access event that can require defense-in-depth authorization at the tool and parameter level. This applies in addition to a valid token. Without granular controls, a single misconfigured permission can bypass the access requirements that organizations might need to fulfill for compliance purposes. In … Read more

AI agents built on foundation models (FMs) often misapply healthcare and life sciences (HCLS) decision frameworks, even when they’ve seen the guidelines in training and in the system prompt. Ask an agent to classify a TP53 missense variant using ACMG/AMP criteria. It will cite the correct framework but misapply evidence categories, skip population frequency thresholds, … Read more

Large-scale distributed training jobs run for hours or days across dozens of nodes. At that scale and duration, interruptions are statistically inevitable: network partitions, memory errors, software exceptions, or infrastructure events will eventually disrupt at least one worker. A single GPU fault triggers a cascade: NVIDIA Collective Communication Library (NCCL) timeouts propagate to healthy workers, … Read more

In a previous launch post, we introduced AgentCore optimization, a capability of Amazon Bedrock AgentCore that can help you improve the quality of your agents. Improving a low-scoring agent has traditionally been a manual process. You review long traces to find where the agent goes wrong, tune individual components such as prompts, tool descriptions, and … Read more

Organizations that process thousands of scanned documents daily, including medical forms, insurance claims, and financial records, face a recurring compliance need: personally identifiable information (PII) redaction before documents are shared with third parties or processed downstream. Manual redaction doesn’t scale: It consumes staff hours, introduces human error, and creates compliance exposure. Redaction is also a … Read more

Prompt caching in Amazon Bedrock can reduce your input token costs by up to 90 percent when you repeatedly send the same context to foundation models, based on Amazon Bedrock prompt caching pricing. Without caching, a 10,000-token contract sent alongside 50 user questions means 500,000 input tokens billed at full price for content the model … Read more

Retail catalogs rarely arrive as clean, structured attributes. Product names, descriptions, and category paths come from many sources and change continuously. Search, recommendations, and catalog navigation depend on consistent tags, but manually applying those tags across thousands of stock keeping units (SKUs) is slow and difficult to keep consistent. A general-purpose frontier model can generate … Read more

Getting access to the right GPUs when you need them is one of the biggest challenges in training or customizing AI models at scale. During peak demand periods, your preferred GPU may not be immediately available – and when your job is tied to one specific GPU configuration, the only option is to wait or … Read more

AI agents now run in production at a scale of billions of operations a day, and a recurring architectural pattern has surfaced: agents need a compute scratch pad. Not only for coding tasks, but for data aggregation, analysis, verification, and any workflow where semantic reasoning alone isn’t enough. Abnormal AI, a behavioral security service that … Read more

AI agents often need to access services such as GitHub and Slack on a user’s behalf. Before an agent can act, the user must authenticate with the provider and explicitly approve the requested access. The application must then securely associate the resulting OAuth grant with the user who authorized it. This process is called session … Read more

Financial institutions participating in open finance (the network where banks share customer-authorized financial data with third-party applications through standardized APIs) face a persistent integration challenge. Every bank exposes APIs with its own field names, formatting conventions, and gaps relative to the Financial Data Exchange (FDX) standard. Validating those APIs, mapping fields, and scoring readiness for … Read more

This post shows you how to pick the right generative AI customization approach for your workload without over-engineering or under-investing. AWS provides access to foundation models from Anthropic, Meta, Mistral, and Amazon through Amazon Bedrock, along with the infrastructure to build everything from chatbots and code assistants to document processors and autonomous agents. The models … Read more

AWS Machine Learning Blog: technical walkthrough. Commands and expected outputs are from a working deployment. Substitute your own account values throughout. Replenishment automation starts with a demand forecast, and in retail that forecast has a short shelf life. By the time a planner exports it, checks it against supplier availability, and works down tens of … Read more

Multi-agent systems in production experience issues in ways that traditional monitoring misses. For example, the agent can’t invoke its foundation model (FM) and returns an empty response. This could be because of a missing AWS Identity and Access Management (IAM) permission on an agent’s execution role that doesn’t throw a 500 error. A supervisor agent … Read more

Organizations building generative AI applications usually compare models the same way: dollars per million tokens. It’s the number on every pricing page, so it becomes the number in every spreadsheet. But production workloads don’t buy tokens. They buy outcomes: a resolved support ticket, a completed research brief, a correct financial summary. Between the pricing page … Read more

As customers shift to interacting with digital services through AI hosts like ChatGPT and Claude, organizations need a way to make their services accessible across these applications with rich UI, not only plain text. They also need to do this without coupling to a single host. MCP Apps and Amazon Bedrock AgentCore provide exactly that. … Read more

When you build an application on top of a large language model (LLM), the prompt you send to the model typically has two parts. There’s a fixed part that sets up context (instructions, reference documents, conversation history) and a variable part that contains the actual user input. Take a customer service bot as an example. … Read more

When you deploy a large language model (LLM) for inference on Amazon SageMaker HyperPod, there’s a gap between when you request a pod and when it’s ready to serve traffic. This gap is dominated by two sequential downloads: the inference server container image from Amazon Elastic Container Registry (Amazon ECR), and the model weights from … Read more

Today we’re announcing the general availability of TwelveLabs Marengo Embed 3.0 as an embedding model in Amazon Bedrock Knowledge Bases. Video and media assets remain largely unsearchable by meaning. Teams in media, sports analytics, education, security, and retail need to find specific moments in hours of footage using natural language. An example query is “show … Read more

Your teams get an AI assistant that handles real work while your data stays in your environment and your conversations stay private Today, the Amazon Quick desktop application is generally available on macOS and Windows. We’re also adding a new activity feed to the mobile experience on iOS and Android that consolidates email, calendar, CRM, … Read more

Build an end-to-end Request for Information (RFI) questionnaire workflow using Amazon Quick Automate to solve a challenge organizations face at every scale. A typical enterprise might handle hundreds of RFI questionnaires each year, and each arrives as a complex, multi-tab workbook packed with hierarchical question sets, category metadata, and varied response types. The volume, variety, … Read more

A configurable, instruction-driven detector that runs on any large language model (LLM) managed on Amazon Bedrock, evaluated on five public PII corpora across nine LLM-based detectors, including the OpenAI PrivacyFilter. Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and … Read more

Multi-turn agents fail in ways that single-turn evaluation misses: one early mistake quietly corrupts every later turn. This post introduces the Agent Evaluation Metric (AEM), a decomposable, turn-level way to measure agent quality. We apply it to its first dimension, correctness. We show how AEM pinpoints the one turn that caused a failure and separates … Read more

This post is co-written with Petra Lafond, Product Manager, and Maarten Cardinaels, Tech Lead at AvioBook, a Thales Group Company. Airlines run on tight, cascading schedules, and accessing operational data holds the key to protecting them. During turnarounds, a few minutes of delay at one gate can ripple through an entire day of connections. It … Read more