Senior Data Engineer

Job Purpose

The Senior Data Engineer will own the structural elements of data, including data storage, data piping and interfacing with analytics platforms and manage all data aspects related to use cases that are developed within the DCoE. The Senior Data Engineer will drive data requirements, modelling and testing for the use cases responsible for and define the flow of data in a project from input through to storage, including interfaces with analytics tools or end user software.

Duties & Responsibilities

• Collaborate across teams on the design and maintenance of our Operations data mart (ETL, data modeling, metric design, reporting/dashboarding) to assure a stable reporting infrastructure
• Lead investigations streams of content usage and A/B testing
• Be exemplary in promoting Data Engineering best practices within the team
• Introduce new skills and knowledge into the team adding to the already dynamic environment our Data Engineering team has createdhare knowledge to compliment team-wide expertise
• Develop high quality software at a senior level
• Assist with the elicitation and documentation of system requirements
• Understand business and technical requirements and constraints to design effective software solutions
• Create conceptual architectures and detailed designs for software solutions
• Providing technical guidance related to data architecture and data models to Digital Architect, senior IT and Business leaders.
• Define and implement data flows through and around digital products.
• Define and implement measures to ensure data availability, quality, and completeness for use cases you are part of.
• Participate in Data modeling and testing.
• Extract relevant data to solve analytical problems; ensure your use case team has the required data.
• Working in close cooperation with data scientist, software engineers, and IT architects, as well as data management function to ensure state-of-the-art data quality for use cases.
• Qualifications & Experience:5+ years of experience equivalent work in a data engineering, business analytics, business intelligence or comparable data engineering role, including data warehousing and business intelligence tools, techniques and technology.3+ years of experience with Stored Procedures, Common Table Expressions, Window Functions, Partitioning and other advanced RDBMS tasks.
• Solid understanding of ETL concepts and workflows.
• Willing to work in new areas and across multiple platforms such as MSSQL, Postgresql, Hadoop, Linux.
• Experience in Big data integration, Hive
• QL, Hadoop, Map
• Reduce, REST API Denodo or Datameer.
• Deep understanding of data lake or lakehouse architectures.
• Expertise in data warehousing concepts, dimensional modeling, and advanced schema optimization.
• Advanced SQL capabilities, including complex query optimization, partitioning strategies, and experience with columnar or MPP databases.
• Experience with data governance frameworks (lineage, cataloging, metadata management) and enterprise-level security (encryption at rest/in transit, RBAC, regulatory compliance).
• Experience with advanced data management systems (e.g., Teradata, Hadoop, Oracle etc.).
• Experience in developing applications in high volume data staging/ETL environments and is proficient in advanced SQL