Skip to main content

Senior Confluent Kafka & Flink Streaming Architect

apolisColumbus, OHManager L7
Job DescriptionJob DescriptionJob Title: Senior Confluent Kafka & Flink Streaming Architect Onsite in Columbus, OH Experience: 12 years overall (8 years for Senior Engineer level); 5 years in event streaming About the Role: We are looking for a Senior Streaming Architect / Engineer to design, build, and run our enterprise real-time data platform on Confluent Cloud and Apache Flink. You will lead the move from legacy batch and ETL workloads to a cloud-native, event-driven architecture on AWS and GCP. You will set technical direction, mentor engineers, and work with business and executive stakeholders.
Key Responsibilities: " Architect multi-region, highly available Confluent Cloud environments with DR strategies (Cluster Linking, cross-region replication) supporting 200K events/sec. " Design and build real-time pipelines in Apache Flink (DataStream, Table, and SQL APIs): event-time processing, watermarks, late-event handling, windowed aggregations (tumbling, hopping, session), and stream joins. " Implement stateful, fault-tolerant processing with checkpointing, savepoints, and exactly-once semantics; tune parallelism, state backends, and operator chaining. " Build and manage Kafka Connect source/sink connectors (CDC, RDBMS, S3, analytics platforms) with SMTs and DLQ/retry patterns. " Own schema governance through Schema Registry (Avro/JSON/Protobuf) and compatibility rules. " Develop event-driven microservices and Kafka Streams / ksqlDB applications using Java and Spring Boot. " Lead migrations of on-prem middleware and Confluent Platform workloads to Confluent Cloud and AWS (EKS) with zero-downtime cutover. " Automate infrastructure with Terraform (Confluent, AWS, GCP providers) and GitOps CI/CD (Harness, Jenkins). " Implement secure connectivity (Private Service Connect, VPC, PrivateLink), RBAC, ACLs, SSL/SASL, and audit logging. " Set up observability (Prometheus, Grafana, CloudWatch, Splunk); lead incident response, RCA, and performance tuning. " Optimize partitioning, retention, and capacity to reduce infrastructure cost. " Lead and mentor Kafka/Flink engineers, run training sessions, and communicate architecture trade-offs to leadership.
Required Qualifications: " 12 years in software, integration, or data engineering, with 5 years hands-on with Apache Kafka / Confluent in production. " Strong hands-on experience with Apache Flink (DataStream API and Flink SQL), including state management and checkpoint tuning. " Deep Kafka knowledge: partitioning, replication/ISR, retention, compaction, Kafka Connect, Kafka Streams, ksqlDB. " Experience running Confluent Cloud in an enterprise setting on AWS and/or GCP. " Advanced Java (11 ) and Spring Boot; proficiency in Python and SQL. " Experience with Terraform, Docker, and Kubernetes (EKS / GKE / OpenShift). " Experience with CI/CD tools (Jenkins, Harness) and monitoring stacks (Prometheus, Grafana, Splunk). " Proven cloud migration experience, from assessment and planning through cutover. " Strong communication skills and experience with stakeholders in regulated industries.
Preferred Qualifications: " Confluent Certified Developer / Administrator for Apache Kafka; Confluent Certified Data Streaming Engineer (Flink). " AWS Solutions Architect certification. " Experience with Amazon MSK, Managed Service for Apache Flink, Kinesis, EMR, Glue, and MWAA. " PySpark experience for batch and streaming ETL. " Familiarity with Snowflake, BigQuery, PostgreSQL, Oracle, and DB2 as sources or sinks. " Experience with CDC patterns and legacy integration platforms (IBM MQ, IIB, mainframe). " Domain background in healthcare, insurance, banking/finance, or retail.
Tools & Technologies: Confluent Cloud, Apache Kafka, Apache Flink, ksqlDB, Kafka Streams, Kafka Connect, Schema Registry, AWS (EKS, MSK, S3, IAM, VPC), GCP (GKE, Pub/Sub, PSC), Terraform, Kubernetes, Docker, Jenkins, Harness, Java / Spring Boot, Python, PySpark, Prometheus, Grafana, Splunk. "
Must-have: Confluent Kafka production experience and hands-on Apache Flink (not just Kafka Streams). " Ask about: Flink checkpointing/savepoints, watermarks and late events, exactly-once guarantees, Cluster Linking DR design. " Strong signal: Confluent Cloud migration from on-prem, Terraform-based provisioning, and Confluent certifications. " Profile titles to search: Kafka Architect, Confluent Architect, Streaming Data Engineer, Flink Engineer, Event Streaming Solution Architect.
APPLY NOW (opens in a new tab)

Matching similar jobs

Level

Manager L7

Location

Columbus, OH

Occupation

Computer Systems Engineers/Architects

Industry

Computer Systems Design Services

Posted

yesterday

APPLY NOW (opens in a new tab)