Senior Assurance Engineer, Machine Learning

About Trail of Bits

Trail of Bits helps secure the world's most targeted organizations and products. We combine high-end security research with a real-world attacker mentality to reduce risk and fortify code.

As a cybersecurity research and consulting firm, our team has worked on some of the most innovative software projects millions of people use worldwide. We help the best companies in the world — Epic Games, Google, Microsoft, Zoom, and others — with their most complex security challenges by designing and building new technology, researching new techniques, and reviewing the security of the latest available products before they hit the market.

Our team considers developing and sharing our knowledge as a natural part of doing business. We push to publish every tool or project so the public — not just our clients — benefit from our expertise and innovations. It's a practice that's earned us industry accolades and helped contribute to our double-digit bottom-line growth.

Role

Our assurance team is responsible for reviewing high-assurance software in the tech, finance, and defense industries. Our evaluations allow our clients to make informed decisions about risk to their systems, including how to minimize or mitigate safety and security issues. In addition to dedicated teams focusing on application security, cryptography, blockchain security, and emerging platforms security, Trail of Bits has a machine learning (ML) practice that creates tools and techniques for the exploration of new attack surfaces and failures that can lead to the degradation of model performance, exploitation of ML system assets, and manipulation or lack of robustness of resulting ML outputs. Trail of Bits has also created and maintains more than 200 free and open-source tools (available in our GitHub repositories) and offers research and engineering services for the public and private sectors.

As a Senior Security Engineer, Machine Learning you will be a part of a new machine learning assurance team that aims to identify and taxonomize classes of failure modes which directly impact ML model performance, as well as novel hazards and threats that can compromise the ML operations pipeline for our clients. You will provide value to our clients through your expertise and experience within the ML development lifecycle as well performing risk assessments and threat modeling.

Responsibilities

• Research and development of machine learning assurance frameworks and contributing to the shape of the new field of ML auditing and assurance.
• Carrying out risk assessments and threat modeling on machine learning model capabilities and their failure modes.
• Analysis of machine learning model degradation and failure from systematic defects.
• Work directly with leading industry teams to review system code and architecture, and help assure their products through system analysis and modeling.
• Develop and contribute to policy and regulatory frameworks to enable the assurance and auditing of machine learning and artificial intelligence to be deployed within mission critical applications.

Requirements

• 3+ years of experience with the machine learning data pipeline or machine learning development lifecycle.
• 3+ years of experience with systems, safety, or security engineering principles for mission critical applications.
• 3+ years of experience performing or evaluating analysis methods to assess machine learning performance, safety, or security from system-level requirements (risk assessments, system hazard analysis, STPA, etc. included).
• Knowledge of ISO 38507, ISO 25012 or similar AI standards such as EU Commission Ethics Guidelines for Trustworthy AI.
• Excellent verbal written communication skills.
• Proven success working in a geographically dispersed and remote work environment.
• Can adapt and drive innovation and performance in a rapidly changing field.
• Technical background to be able to look at systems and analyze their behaviors and realize their failures.

The base salary for this full-time position ranges from $175,000 to $225,000, excluding benefits and potential bonuses. Various factors influence our salary ranges, including the specific role, level of seniority, geographic location, and the nature of the employment contract. An individual's specific work location, unique skills, experience, and relevant educational background will determine the final offer within this range. The presented salary range encompasses the starting salaries for all U.S. locations. For a precise salary estimate tailored to your preferred location, please discuss it with your recruiter during the hiring process.

Trail of Bits, Inc. participates in E-Verify, the US federal electronic employment eligibility verification program. Learn more.

Company Perks

• Before, during and after COVID-19, our workforce works flexibly. Many employees choose to work from home around the globe. As long as you deliver against your goals, we encourage you to harness your personal working style to let you work best.
• Liberal expense policy for acquiring the equipment and software that help you do your job. If we need hardware to work effectively, we buy it.
• We offer exceptional and tailored technical, leadership and organizational training for our team members. Everyone is encouraged to identify additional opportunities for personal professional growth with working at Trail of Bits.
• We routinely highlight the amazing work our employees do via our blog, product offerings, and conference talks. We celebrate you!
• We're at the forefront of a number of markets and have the internal expertise and the ambition to capitalize on those opportunities. Our employees see their work in use and valued by many others.

Highlighting some of our US Canadian Full-Time Employee Benefits:

• Multiple generous health, vision, and dental insurance plans.
• Ancillary benefits including life and disability insurance.
• Retirement plan with 5% company matching.
• 4 months paid parental leave.
• Moving expenses to NY: $5,000 one-time.
• Charitable donations matching up to $2,000.
• One time $1,000 at home office expense stipend.
• $500/year personal learning development budget.
• Executive coaching for managers above.
• Bonuses for recruiting, public speaking, tool releases, blog posts, academic posters, proposals, and whitepapers, and end-of-year bonuses based on company, team, and personal performance.
• 20 days of PTO per year for vacation, sick, and personal time off.
• 15 company holidays per year.
• Carbon offsets for your personal and corporate carbon emissions through Project Wren.

Dedication to diversity, equity inclusion

Trail of Bits is committed to creating and maintaining a diverse and inclusive workplace where our employees can thrive and be themselves! We welcome all persons into our community. We embrace the diversity of gender, gender identity or expression, race, color, religious creed, national origin, ancestry, age, physical and mental disabilities, medical condition, genetic characteristic, sexual orientation, marital status, family care or medical leave status, military or veteran status, or perceived membership in any of these groups.