Data Engineer

• BH
• Data Engineer
• Permanent

On behalf of our innovative fin-tech clients, Brainhunter Systems is looking for a skilled Data Engineer to join their growing data and AI team. In this role, you’ll be responsible for working with product and delivery teams to help them in determining their data needs and finding innovative ways to build elegant and scalable data products that support different types of analytics within their environment. You will

continuously develop your expertise in data management. You’ll be challenged to set your own goals, priorities and pace of delivery while contributing to the overall success of the team.

Summary of key responsibilities

• Build massive reservoirs for big data.
• Develop, construct, test and maintain architectures such as databases and large-scale data processing systems.
• Create and maintain optimal data pipeline architecture.
• Develop data set processes for data modeling, mining and production.
• Define data requirements, test data needs for product/delivery teams.
• Create custom software components (e.g. specialized UDFs) and analytics applications.
• Implement and maintain DataOps processes and data processing best practices
• Implement and maintain CI/CD pipelines for continuous integration and continuous delivery of data pipelines and data products.
• Recommend ways to improve data reliability, efficiency and quality.
• Employ a variety of languages and tools (e.g. scripting languages) to marry systems together.
• Collaborate with data architects, data scientists and product team members on project goals.
• Research and implement new tools and techniques for team use.
• Improve the process for team members to access and understand customer data.

Qualifications

• Minimum a BSc in computer science or engineering, with 4+ years experience as a data engineer
• Demonstrated proficiency in at least one major language (e.g. Python, Ruby, etc).
• Bonus points if you also have experience with PySpark, Java, R and MATLAB
• Experience with building & maintaining ETL/ELT pipelines in Data Warehouses & Data Lakes
• Experience with integration of real-time and streaming data
• Experience with Database systems (both SQL & NoSQL), such as MySQL, PostgreSQL, Cassandra and MongoDB
• Experience with designing & building CI/CD pipelines
• Experience with containerization & Kubernetes technologies (Preferably on GCP)
• Experience with working in an Agile/Scrum environment

Nice-to-haves

• GCP Professional Data Engineering Certification, with experience in building Data APIs
• Experience with distributed systems and distributed processing of large data sets using
• Apache Hadoop and Apache Spark is desirable
• Bonus points if you’ve built a startup, can demonstrate your unique contributions to open source projects and/or have experience with human interface/human UX

How to Apply

• Online: Apply online now at www.Brainhunter.com
• Email: If you would prefer to email a copy of your resume for later review, we would be pleased to consider your resume for future upcoming opportunities, please send resume, availability and compensation expectation to: megan.degagne@brainhunter.com

Please note: While we appreciate and welcome all applications, only those who satisfy employment eligibility in Canada will be considered.