Role Purpose
We are seeking a Senior Solution Architect with deep experience designing and delivering enterprise-scale big data, streaming, and real-time data integration solutions.
The role is focused on solution architecture across Cloudera-based big data platforms, Kafka-based streaming applications, and real-time data integration / analytics solutions, including cloud-hosted data platforms such as Cloudera on AWS or equivalent AWS-native services.
This is not a pure data modelling, reporting, analytics, or governance role. The successful candidate must have a strong architecture background and be comfortable operating across both solution design and hands-on technical delivery.
Key Responsibilities
- Own the end-to-end solution architecture for large-scale distributed data, streaming, and data integration solutions.
- Design solutions across Cloudera CDH/CDP, data lakes, operational data stores, event streaming platforms, and real-time data pipelines.
- Architect and document solutions using technologies such as Hive, Impala, HBase, Spark, NiFi, Flink, Kafka, and equivalent AWS services.
- Design Kafka-based streaming applications, including event ingestion, topic design, consumer patterns, error handling, replay, scalability, and resilience.
- Define solution patterns for Spring Kafka consumers, ingestion into HBase, and consumption through Spring Boot / API-based services.
- Provide architecture leadership across data ingestion, streaming integration, transformation, storage, APIs, and downstream consumption.
- Work with engineering teams to ensure solutions are deliverable, scalable, secure, supportable, and aligned to enterprise architecture principles.
- Review technical designs produced by engineering teams and ensure alignment with the approved solution architecture.
- Identify technical risks, dependencies, assumptions, non-functional requirements, and design trade-offs.
- Support delivery planning by helping break solutions into iterative delivery increments and sprint-level outcomes.
- Engage with business stakeholders, programme teams, enterprise architects, engineering leads, and governance forums.
- Build, represent and own solution designs(Architecture Summaries and High level designs) to architecture review boards and senior technical stakeholders where required.
- Develop and promote architectural patterns and best practices and engineering standards across big data, streaming, integration, APIs, and cloud-based data platforms.
- Essential Experience
- Candidates must demonstrate the following:10+ years’ overall IT experience, with significant experience in data platform, integration, or distributed systems delivery.5+ years’ experience in solution architecture, technical architecture, or lead design roles.2+ years’ experience architecting streaming or real-time data applications, preferably using Kafka or equivalent event streaming technologies.
- Proven experience designing solutions on Cloudera CDH/CDP or equivalent big data platforms.
- Strong demonstrable experience with some of the distributed data technologies (if not all) such as:HBase
- Spark
- NiFiKafka
- Flink
- Hive
- Impala
- Experience designing and delivering large-scale data ingestion, transformation, storage, and consumption patterns.
- Experience with Kafka consumer architecture, including consumer groups, offsets, retries, dead-letter handling, ordering, partitioning, and scalability.
- Experience with API-based data consumption, ideally using Spring Boot / Java-based services.
- Strong understanding of batch, micro-batch, and real-time streaming architectures.
- Experience working in complex enterprise environments with multiple systems, vendors, platforms, and governance processes.
- Strong understanding of non-functional requirements including performance, scalability, resilience, security, observability, and operational support.
- Required
Technical Skills
- The successful candidate should have hands-on exposure to several of the following:
- Cloudera CDH/CDP, ideally on AWS or hybrid cloud environments
- Apache Kafka or AWS equivalents such as MSK, Kinesis, Event
- Bridge, or similar
- Spark / Spark Streaming
- HBase / Cassandra / NoSQL data stores
- Hive / Impala
- Apache NiFi / ingestion frameworks
- Apache Airflow Orchestrations
- Apache Flink or equivalent stream processing technologies
- Java / Spring Boot / Spring Kafka
- Python and PySpark
- REST APIs and service-based data consumption
- Change Data Capture and event-driven integration patterns
- Data lake, ODS, and distributed storage design
- Cloud data platform design, preferably on AWSDesirable Experience
- Financial services or banking experience.
- Experience with regulated, secure, enterprise-scale data platforms.
- Experience presenting to architecture governance boards.
- Exposure to enterprise data governance, lineage, metadata, and data quality tooling.
- Experience with DevOps, CI/CD, automated testing, and modern engineering delivery practices.
- Experience with cloud-native data services on AWS.
