وصف الوظيفة
Role Overview
LLM Research & Validation Specialist at ADIB, Abu Dhabi, UAE. The role leads frontier research, quantitative evaluation and independent validation of large language models, multimodal models, retrieval-augmented generation systems and agentic AI used or proposed by ADIB, translating mathematical and scientific methods into reproducible validation tests, challenger analyses, runtime controls and decision-useful evidence for model governance.
Role Purpose
Combine deep technical research with second-line effective challenge to build validation toolkits and evaluation harnesses, independently assess conceptual soundness and production behaviour, and communicate material limitations clearly to technical teams, senior management and governance forums. The role does not own model development or production approval.
Key Responsibilities
Validation and Assessment
- Lead independent validation of LLM, multimodal, RAG and agentic AI use cases across design, implementation, deployment and ongoing monitoring.
- Assess transformer architecture, tokenisation, embeddings, attention, context-window behaviour, decoding, fine-tuning, alignment, quantisation and inference configuration.
- Evaluate task performance, hallucination and factuality, calibration, robustness, stability, long-context behaviour, retrieval quality, grounding, citation faithfulness and uncertainty.
- Perform deep testing of prompt injection, indirect injection, data leakage, tool-use safety, excessive agency, multi-step failure propagation, kill-switches and human oversight.
- Review data provenance, representativeness, contamination, benchmark validity, leakage, drift and limitations of synthetic or LLM-generated evaluation data.
Evaluation Framework and Tooling
- Design reproducible evaluation harnesses, golden datasets, adversarial suites, counterfactual tests, canary sets and statistically defensible acceptance criteria.
- Build and maintain reusable Python-based validation tooling, automated test pipelines, experiment tracking, results repositories and technical documentation.
Research and Analysis
- Apply probability, statistics, optimisation, information theory, numerical methods and experimental design to develop challenger tests and quantify uncertainty.
- Conduct structured research on emerging model architectures, interpretability, mechanistic analysis, scalable oversight, model evaluation and AI safety methods.
Governance and Communication
- Independently challenge model owners, vendors and developers, document findings, propose risk-based restrictions and track remediation without assuming first-line ownership.
- Prepare validation reports, research notes, standards, committee papers and senior-management briefings that clearly distinguish evidence, judgement and residual uncertainty.
Team Development
- Mentor junior validators, improve team methodology and support knowledge transfer across Model Risk.
Qualifications & Experience
- Master's degree in Theoretical Physics, Applied Physics, Mathematics, Applied Mathematics or a closely related quantitative discipline is required. PhD or research-intensive master's is strongly preferred.
- Typically one to three years of relevant experience in AI research, machine learning, quantitative modelling, model validation, scientific computing or a closely related field. Exceptional research profiles may be considered based on demonstrated capability.
- Deep understanding of probability, statistics, linear algebra, optimisation, numerical computation, experimental design and uncertainty quantification.
- Strong understanding of transformers, LLM training and inference, embeddings, RAG, fine-tuning, alignment, evaluation, agentic systems and AI safety failure modes.
- Ability to read research papers critically, reproduce methods, design controlled experiments and convert findings into bank-grade validation evidence.
- Excellent technical writing and communication, including the ability to explain mathematical concepts, assumptions and limitations to non-specialist stakeholders.
- Banking experience is advantageous but not mandatory.
Skills & Competencies
- Advanced Python proficiency and experience with scientific and ML libraries.
- Exposure to PyTorch, Hugging Face, evaluation frameworks, experiment tracking, SQL, Git and cloud AI platforms is expected.
- Experience with red teaming, adversarial testing, interpretability, calibration, robustness, privacy, security or model risk management is strongly advantageous.
- Willingness to develop knowledge of financial services, Islamic banking, CBUAE expectations and ADIB governance.
Additional Information
Validation conclusions must be reproducible, evidence-based and proportionate to use-case risk. Reusable evaluation assets and automation should measurably improve validation coverage, consistency and efficiency. Material LLM and agentic risks must be identified early, clearly communicated and translated into actionable controls or use restrictions. Research outputs strengthen ADIB validation methodology and remain traceable to tested evidence rather than unsupported claims. Stakeholders receive constructive, independent challenges while second-line ownership and decision rights remain clear.