We are looking for a Senior Data Engineer who will report directly to our Manager of Data Engineering. This position is responsible for developing data solutions critical to our company’s descriptive and predictive analytics. The successful candidate will perform a broad range of tasks, including mapping business problems to data solutions, developing ETL/ELT processes, designing logical and physical data models, and promoting data quality. Additionally, they will collaborate with senior management and partner with project and resource managers to develop project scope, dependencies, estimates, plans, schedules, status reports, and manage issues/risks.
Job Responsibilities
- Lead the design and development of data engineering solutions using Databricks, SQL, DBT, Python, C#, and Terraform for critical and complex projects.
- Collaborate with team members across the organization to foster a data-driven culture that enhances revenue and customer growth.
- Gain a comprehensive understanding of HQY data and work closely with the business to map pressing problems to data and analytic solutions.
- Engage in problem-solving across the data platform architecture and specific solutions.
- Play a central role in designing the data warehouse.
- Advocate for automation by designing and implementing automated deployment pipelines.
- Promote data governance and quality by automating data quality checks.
- Enable the data science team to build ML models by providing high-quality data.
- Address issues in unfamiliar technologies or areas of expertise.
- Coach and mentor junior team members.
Requirements
- BS degree in Computer Science, Engineering, Mathematics, or a related field with 8 years of work experience.
- Passion for delivering ETL/ELT development projects efficiently using Agile methodologies.
- Advanced problem-solving and troubleshooting skills.
- Proven ability to innovate, drive projects to successful completion, and create enduring solutions.
- Extensive experience with cloud services (e.g., Azure, AWS, GCP).
- Advanced knowledge of massive parallel processing datastores (e.g., DataBricks, Delta Lake, Apache Spark, Snowflake, Redshift).
- Proficient in SQL and Relational Database Management Systems (e.g., Azure SQL, SQL Server, Oracle, MySQL) and NoSQL databases (e.g., MongoDB).
- Advanced knowledge of ETL processing tools (e.g., DBT, Databricks, Fivetran, SSIS).
- Strong understanding of data modeling techniques (e.g., 3NF, Dimensional) and their application.
- Proficient in Terraform and able to interpret execution plans.
- Proficiency in programming languages relevant to Data and Analytics (e.g., Python, C#).
- Commitment to data security and privacy.
Brazil
