Transitive Compatibility Types in Schema Registry for Apache Kafka

Apache Kafka provides schema management capabilities through the Schema Registry, including various levels of compatibility. In this article, we will discuss three types of transitive compatibility: BACKWARDTRANSITIVE, FORWARDTRANSITIVE, and FULL_TRANSITIVE. Additionally, we will analyze their pros, cons, and provide code examples.

BACKWARD_TRANSITIVE

Description

Backward transitive compatibility means that a new version of the schema must be compatible with all previous versions. This is useful in systems with a long data lifecycle.

Code Example

curl -X PUT 
  -H "Content-Type: application/vnd.schemaregistry.v1+json" 
  --data '{"compatibility": "BACKWARD_TRANSITIVE"}' 
  http://localhost:8081/config/<subject_name>

Java Code Example

import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class BackwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "BACKWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
}

Pros

• ✅ Ensures compatibility with historical data.
• ✅ Useful for long-term data storage.

Cons

• ❌ May restrict adding new fields or changing the structure.

FORWARD_TRANSITIVE

Description

Forward transitive compatibility requires that a new version of the schema be compatible with all future versions. This is useful when consumers are updated asynchronously with producers.

Code Example

curl -X PUT 
  -H "Content-Type: application/vnd.schemaregistry.v1+json" 
  --data '{"compatibility": "FORWARD_TRANSITIVE"}' 
  http://localhost:8081/config/<subject_name>

Java Code Example

import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class ForwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FORWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
}

Pros

• ✅ Ensures stability for old consumers with new data.

• ✅ Convenient for systems with asynchronous updates.

Cons

• ❌ Does not guarantee compatibility with historical data.

FULL_TRANSITIVE

Description

Full transitive compatibility combines the requirements of backward and forward compatibility. A new schema version must be compatible with both previous and future versions.

Code Example

curl -X PUT 
  -H "Content-Type: application/vnd.schemaregistry.v1+json" 
  --data '{"compatibility": "FULL_TRANSITIVE"}' 
  http://localhost:8081/config/<subject_name>

Java Code Example

import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class FullTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FULL_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
}

Pros

• ✅ Maximum flexibility and resilience.
• ✅ Guarantees compatibility with all schema versions.

Cons

• ❌ The strictest compatibility type.
• ❌ Requires careful planning for schema changes.

Comparison Table

| Compatibility Type | Compatibility with Past Versions | Compatibility with Future Versions | Ease of Use | Applicability |
|—-|—-|—-|—-|—-|
| BACKWARDTRANSITIVE | ✅ | ❌ | Medium | Long-term data storage |
| FORWARDTRANSITIVE | ❌ | ✅ | Medium | Asynchronous updates |
| FULL_TRANSITIVE | ✅ | ✅ | Low | Maximum resilience |

These compatibility types allow you to configure the system to meet your stability and flexibility requirements. The correct choice depends on your system architecture and use cases.