AI Evaluation Engineer – Business & Operations Domain

About the role

Gramian Consultancy seeks an analytical professional to design and evaluate AI benchmark tasks that reflect real‑world business and operational workflows. You will create multi‑step scenarios that test AI systems on process optimization, decision‑making, and reporting challenges across diverse business environments.

Key responsibilities

• Design realistic business and operational workflow scenarios for AI evaluation.
• Create structured tasks involving analytics, reporting, operational reasoning, and process optimization.
• Develop clear specifications, expected outcomes, and validation logic for each task.
• Identify edge cases, bottlenecks, and failure scenarios within workflows.
• Assess AI‑generated outputs for reasoning quality, usefulness, and accuracy.
• Collaborate with reviewers and researchers to refine benchmark scenarios.

Required profile

• 3–10 years of experience in operations, analytics, consulting, or related business systems.
• Strong analytical thinking and problem‑solving skills in operational contexts.
• Hands‑on experience with operational workflows, reporting systems, CRM tools, or business analytics.
• Proficiency in documenting tasks, specifications, and evaluation criteria.

Required skills

• Spreadsheets
• Dashboards
• Operational reporting
• Workflow automation
• CRM tools
• Exposure to AI systems or automation platforms