NOTE
Managed Kafka
What managed Kafka services operate for you, what semantics remain your responsibility, and how to evaluate compatibility, networking, scaling, and cost.
This is a historical learning note and may contain outdated or incomplete understanding.
1. What Managed Kafka Changes
A managed service can operate brokers/controllers, patching, replacement, monitoring, and storage for you.
Some services run Apache Kafka closely; others provide Kafka-protocol-compatible engines with different storage/control-plane internals.
2. What the Application Still Owns
- topic/partition design;
- key and ordering strategy;
- producer acknowledgement/idempotence;
- consumer idempotency and offset handling;
- schema compatibility;
- retention requirements;
- lag and business-SLO monitoring.
3. Evaluate Compatibility
Check:
- supported Kafka protocol/client versions;
- transactions/idempotence;
- consumer groups;
- quotas;
- Connect/Schema Registry integrations;
- security/authentication;
- networking/private connectivity;
- partition/storage limits.
4. Cost Model
Managed Kafka cost often includes provisioned/elastic compute, storage, cross-zone/region network traffic, and retained data.
A technically valid design with excessive partitions or cross-region replication can become unnecessarily expensive.
5. Availability Semantics
A “multi-AZ” label does not by itself define producer durability. Application guarantees still depend on acknowledgement, replication, ISR, and provider-specific failure behavior.