Staff Data Engineer (Databricks)

Affinity stitches together billions of data points from massive datasets to create a powerful, accurate representation of the world's professional relationship graph. Based on this data, we offer our users the insights and visibility they need to nurture and tap into their team's network of opportunities.

Reporting to the Director of Data and AI, you'll support creating the magic that underlies Affinity's industry-leading relationship intelligence model as the key technical leader of Affinity’s Data Platform team.

In this role, you’ll leverage your past experiences and deep understanding of data warehousing and data lake concepts to help shape and execute Affinity's roadmap. You’ll champion engineering best practices, delivery velocity, and act as a technical mentor for other engineers on the team. You’ll play a significant role in defining the future of how businesses around the world use their relationships.

What you’ll be doing

• Design scalable and reliable data pipelines to consume, integrate and analyze large volumes of complex data from different sources to support the growing needs of our business.
• Help define our data roadmap. You'll collaborate with our team of data engineers, machine learning engineers, product, and business leaders to help to answer these questions and more.
• Build frameworks for measuring and monitoring data quality and integrity.
• Establish CI/CD processes, test frameworks, and infrastructure-as-code tooling.
• Implement, and build data solutions using Spark, Python, Databricks, and the AWS ecosystem (S3, Redshift, EMR, Athena, Glue).
• Mentor, coach, and inspire the engineers on the team.
• Identify and fill gaps in the team, and create the processes necessary for the teams’ success.

Qualifications

Don’t meet every single requirement? Studies have shown that women and people of color are less likely to apply to jobs unless they meet every qualification. At Affinity, we are dedicated to building a diverse, inclusive, and authentic workplace, so if you’re excited about this role, but your past experience doesn’t perfectly align with the qualifications above, we encourage you to apply anyways. You may be just the right candidate for this or other roles.

Required:

• You have 8+ years of experience working in data engineering, with at least 3+ years of acting as a senior team lead or staff engineer, leading complex, sometimes ambiguous engineering projects across team boundaries.
• You have extensive hands-on experience in building scalable data platforms and reliable data pipelines using technologies such as Spark, Hadoop, Databricks, AWS SQS, AWS Kinesis, and/or Kafka.
• You have experience working with large, multi-terabyte datasets and are comfortable with high-scale data ingestion, transformation, and distributed processing tools such as Apache Spark (Scala or Python).
• Experience with AWS, DBX or related cloud technologies.
• You're comfortable with the building blocks of modern back-end systems, such as horizontally scalable data infrastructure, event-driven architecture, and beyond and can clearly articulate the pros/cons of different approaches, while also providing a recommended solution based on the current context.
• You have familiarity with databases and analytics technologies in the industry, including Data Warehousing, Data Lakes, ETL and Relational Databases.
• You have experience mentoring and helping the engineers around you grow.
• ​​You have experience partnering with product and machine learning teams on large, strategic data projects and routine partner work.
• You take pride in delivering exceptionally high quality work in terms of data accuracy, performance, and reliability.

Nice to have:

• Experience leveraging machine learning to improve the quality of ingested data.
• You have worked with multiple third party data vendors and have experience in conflict resolution approaches.

How we work:

Our culture is a key part of how we operate as well as our hiring process:

• We iterate quickly. As such, you must be comfortable embracing ambiguity, be able to cut through it, and deliver incremental value to our customers each sprint.
• We are candid, transparent, and speak our minds while simultaneously caring personally with each person we interact with.
• We make data driven decisions and make the best decision for the moment based on the information available.

Join us in enabling every professional on the planet to succeed by harnessing the power of their relationships.

If you’d want to learn more about our values click here.

What you’ll enjoy at Affinity:

• We live our values as playmakers, obsessed with learning, care personally about our colleagues and clients, are radically open-minded, and take pride in everything we do.
• Health Care coverage and flexible personal sick days. We want our team to be happy and healthy :)
• We provide an annual budget for you to spend on education and offer a comprehensive LD program – after all, one of our core values is that we’re #obsessedwithlearning!
• We support our employee’s overall health and well-being and reimburse monthly for things such as; Transportation, Home Internet, Meals, and Wellness memberships/equipment.
• Virtual team building and socials. Keeping people connected is essential.

Please note that the role compensation details below reflect the base salary only and do not include any equity or benefits. This represents the salary range that Affinity believes, in good faith, at the time of this posting, that it will pay for the posted job.

A reasonable estimate of the current range is $193,500 to $255,500 CAD. Within the range, individual pay depends on various factors including geographical location and review of experience, knowledge, skills, abilities of the applicant.