📢 جديد: تابع عروض اليوم على قناتنا في واتساب
Jobiglo

لا توجد نتائج.

Senior AI Evaluation Engineer - LLMs & Agentic AI

Management Solutions International (MSI) · Doha

جديد
Senior 🇬🇧 English
RAGAS DeepEval TruLens OpenAI Evals LangSmith Arize Phoenix A/B testing Statistical analysis

وصف الوظيفة

About the role

A company in Qatar is looking for an experienced Senior AI Evaluation Engineer to lead the evaluation, testing, validation and continuous improvement of advanced AI and Large Language Model (LLM) systems. This is not a conventional machine learning or generative AI developer role focused on building and deploying applications; instead, the engineer determines whether AI systems perform correctly, reliably and consistently in real-world production environments.

The role covers designing evaluation frameworks, benchmark datasets, automated testing pipelines and statistical validation methods to measure quality, accuracy, factual reliability and performance of LLM-powered and agentic AI systems.

Key responsibilities

  • Design and implement evaluation frameworks for LLMs, generative AI and RAG-based applications, measuring factual accuracy, relevance, hallucinations, contextual understanding, consistency and response quality.
  • Implement and validate LLM-as-a-Judge approaches for automated evaluation and scoring, and benchmark different models, prompts and configurations.
  • Build and maintain automated evaluation pipelines using tools such as RAGAS, DeepEval, TruLens, OpenAI Evals, LangSmith or Arize Phoenix, and create datasets, test cases, ground-truth benchmarks and scoring rubrics.
  • Develop repeatable testing processes and set evaluation metrics and quality thresholds for production readiness.
  • Apply statistical methods to assess model performance, run controlled experiments, A/B tests and comparative evaluations, and perform hypothesis testing, significance analysis and confidence interval calculations.
  • Analyse results to identify performance gaps, model weaknesses and improvement opportunities.

Required profile

  • 7+ years of experience.
  • Master's degree or higher in Computer Science, Software Engineering or a related engineering discipline.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec Management Solutions International (MSI).
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

لماذا تبلغ عن هذا العرض؟

شكراً لإبلاغك. سنراجع هذا العرض.

اكتشف المزيد

الرواتب والأدلة وعمليات البحث في قطر.

قدم طلبك في 30 ثانية

أدخل بريدك الإلكتروني للتقديم. سيتم إنشاء حساب تلقائياً.

قدم الآن →

بالمتابعة، أنت توافق على شروط الاستخدام.

لديك حساب بالفعل؟ تسجيل الدخول

لديك سؤال حول هذا العرض؟

اطرحه هنا: ستصلك تفاصيل العرض كاملة عبر البريد الإلكتروني، فوراً.

منشور منذ 56 دقيقة

ينتهي شهر من الآن

6 مشاهدات · 0 مهتم

عزز فرصك

حمّل سيرتك الذاتية وسنقترح عليك الوظائف التي تناسب ملفك.

جاري تحليل سيرتك الذاتية...

Management Solutions International (MSI)

Doha