Agent Architect

About the role

Pencil is building an agentic operating system for marketing, turning generative AI into a professional, brand‑safe, and scalable creative engine. As an Agent Architect you will design the core "brain" that powers multi‑agent workflows, bridging creative intent with reliable machine execution.

Key responsibilities

• Design and implement multi‑agent workflows, including task decomposition, state management, and tool‑use such as Retrieval‑Augmented Generation and API integration.
• Develop systematic evaluation frameworks (e.g., LLM‑as‑a‑judge, promptfoo, DSPy) to benchmark and improve agent performance at scale.
• Create reusable agents and prompt libraries that enable the platform to serve 1,000+ brands with distinct voices simultaneously.
• Collaborate with AI Engineering and Product teams to decide when to use prompting, RAG, or fine‑tuning for specific behaviors.
• Lead technical delivery for complex client deployments, translating high‑level creative briefs into deterministic AI workflows.

Required profile

• 3+ years of hands‑on experience with generative AI models (LLMs and multimodal systems).
• Strong systems‑thinking mindset, understanding latent spaces, context windows, and inter‑agent data flow.
• Obsessive focus on measurement; comfortable building benchmarks, A/B tests, and evaluation pipelines.
• Ability to translate creative direction into concrete prompt engineering parameters.

Required skills

• Python programming
• JSON data handling
• API documentation and integration
• Experience with orchestration frameworks such as LangChain, CrewAI, or AutoGen
• Familiarity with LLMs (GPT‑4, Claude, Gemini) and multimodal models (Stable Diffusion, Midjourney, video generation)
• Knowledge of evaluation tools like promptfoo and DSPy