Job Description

Job Responsibilities:

• Apply linguistic expertise in syntax, semantics, pragmatics, and sociolinguistics to support LLM and generative AI systems
• Collaborate with linguists, data operations teams, and ML engineers on data collection, curation, annotation, and localization efforts for model training and fine-tuning
• Contribute to the development and maintenance of annotation schemas and guidelines for LLM training data (e.g., instruction-tuning, preference labeling, RLHF)
• Evaluate and quality-check datasets used for pre-training, fine-tuning, and alignment of language models
• Support the development of programmatic methods for generating synthetic annotated data at scale
• Assist in model evaluation efforts including prompt-based testing, red-teaming, and linguistic error analysis
• Contribute to AI safety and responsible AI practices through linguistic review of model outputs (e.g., hallucination detection, bias identification, tone and pragmatic appropriateness)
• Participate in experiments to assess data quality, annotation consistency, and downstream model performance.

Skills:

• Native or near-native fluency in English and at least one additional language
• Knowledge of syntax, semantics, pragmatics, sociolinguistics, corpus linguistics, and other areas of linguistics
• Familiarity with Large Language Models (LLMs), their applications, and data practices (training data, evaluation, prompting, fine-tuning)
• Exposure to LLM evaluation methodologies (human evaluation, automated metrics, adversarial testing
• Experience working with semantic ontologies, taxonomies, or intent/slot frameworks
• Proficiency using AI Agents/Chatbots
• Experience with database queries and data analysis processes (SQL, Python, spreadsheets, R, Unix, or others)
• Experience working with speech and text data in multiple languages
• Comfortable working in a fast-paced, highly collaborative environment with evolving priorities
• Master's degree in Linguistics, Computational Linguistics, Language Technologies, or a related field
• Familiarity with machine learning frameworks, NLP libraries, and tools (e.g., Hugging Face, spaCy, NLTK, PyTorch)
• Exposure to statistical language modeling or training data pipelines
• Strong organizational skills and attention to detail

Education/Experience:

• Bachelor's degree in Linguistics, Computational Linguistics, Computer Science, Speech Science, or related field
• Master's degree in Linguistics, Computational Linguistics, Language Technologies, or a related field

Job Tags

Similar Jobs