Position Overview:
We are seeking a dedicated and detail-oriented LLM Enablement and Quality Manager to oversee and improve the quality and engagement of contributors across various AI-related projects. The ideal candidate will have a strong background in managing contributor quality through training, feedback loops, and audit processes. This role will be pivotal in ensuring that client expectations are met by driving improvements in prompt generation, review processes, and overall project quality metrics.
Key Responsibilities:
- Training amp; Webinars:Establish and lead frequent training webinars designed to maintain contributor quality, update prompt approaches, and enhance competencies. Training sessions will be tailored to specific projects or layers within projects and will align with updated client feedback and requirements..
- Feedback Collection; Problem-Solving:brGather and process feedback from contributors related to tasking challenges, such as instruction clarity, expectations, and technical terminology. Provide targeted feedback to project management and clients, identifying viable solutions that maintain engagement without compromising quality.
- 1-on-1 Contributor Support:brHost office hours and 1-on-1 training sessions, including “last chance” sessions for contributors flagged for quality issues. Ensure consistency and performance improvements across the contributor base..
- Documentation; Rubric UpdatesMaintain and regularly update project instruction documents and rubrics for contributors, reviewers, and QA staff. Ensure these materials support clear and consistent execution across all layers of the project.
- Project Metrics Reporting:Analyze and condense project-wide quality metrics, such as prompt rejection rates and response rating accuracy, into actionable insights. Use these insights to create story points for clients and project management to visualize trends and identify training focus areas.
- Quality Audits Spot Checks: Conduct random spot audits on contributors to measure prompt quality, detect potential spammers, and flag contributors for time theft or noncompliance with project requirements.. Additional Opportunities: Take on responsibilities that emphasize technical requirements, specialized terminology, and evolving project needs. Contribute to broader project development and the definition of KPIs to refine contributor quality management processes.
Qualifications:
- Proven experience managing contributor quality and engagement in AI or data-focused projects.
- Strong ability to create and lead effective training sessions, both group-based and individual.
- Experience in quality auditing, feedback processing, and reporting key metrics.
- Excellent communication skills, with the ability to work across diverse teams.
- Ability to adapt and manage multiple projects or project layers concurrently.
- Familiarity with prompt generation, rating systems, and AI model training is a plus.
United States
