Senior Applied AI Engineer - Health Tech AI Startup

Role purpose: The Senior Applied AI Engineer designs, builds, and operates the AI-driven services that provide core functionality to the platform. You will shape the technical architecture of the intelligent processing and machine learning systems, with a focus on LLM-powered document understanding, structured extraction, retrieval pipelines, and agentic workflows. You will operate with genuine autonomy, treating technical decisions as product decisions, and taking ownership from ambiguous problem statements to measured production outcomes. You will collaborate closely with Product Management, System Architects, DevOps, and Customer Success, serving as a technical lead who mentors less experienced engineers and engages directly with customers on complex solution design or escalations.

Team: Engineering

• Reports to: VP of Engineering
• Direct reports: None / individual contributor

What you will do: AI & LLM systems engineering Design, develop, and maintain the LLM-powered services and pipelines at the core of the platform — document understanding, structured data extraction, classification, and summarization across highly variable real-world clinical and administrative documents Build agentic and multi-step workflows where they demonstrably outperform simpler approaches — routing, validation, exception handling, and human-in-the-loop review — with guardrails and fallbacks appropriate to a clinical setting Select the right approach for each problem — hosted frontier models, fine-tuned open-weight models, or classical ML — based on quality, latency, and cost, and implement fine-tuning where it demonstrably beats prompting and retrieval on those measures Create and maintain the high-quality datasets that model development, fine-tuning, retrieval, and evaluation depend on, including labeling strategy for documents where ground truth is genuinely ambiguous Partner with Product Management and Quality Assurance so AI solutions meet real customer needs and performance requirements, following the company's software development life cycle and its reliability, scalability, and reproducibility standards Evaluation, observability & code quality Build and maintain evaluation suites for LLM-powered features — curated golden datasets, automated scoring calibrated against human review, and regression detection — so that model, prompt, and pipeline changes ship with evidence rather than intuition Establish observability for production AI services, including request tracing, token and cost accounting, and monitoring for quality drift and failure modes such as malformed or unfounded model output. In healthcare, a confidently wrong extraction is the failure mode that matters most, and the systems you build should surface it rather than pass it downstream Lead code reviews with particular attention to ML- and LLM-specific concerns such as data processing, prompt and model versioning, experiment tracking, evaluation coverage, and production readiness Problem solving & optimization Provide problem-solving support for ML and AI systems, including performance optimization, debugging model and retrieval behavior, and improving pipeline efficiency Optimize inference for production environments — latency, throughput, token and compute cost, batching, caching, model routing, and model right-sizing — treating inference cost as a product constraint, not an afterthought Optimize ML pipelines for scalability, latency, and resource utilization at real production document volumes Customer engagement & escalation support Serve as a technical point of contact for escalated customer issues involving ML and AI services, partnering with Customer Success to own each issue end to end — reproduce, root-cause, fix, and close the loop with affected stakeholders Participate directly in customer-facing engagements when required — solution design sessions, technical discussions, and integration support — representing the team's capabilities credibly to technical and non-technical audiences, and translating customer needs into engineering direction Feed lessons from customer engagements back into product, dataset, and system improvements Technical thought leadership & mentoring Drive complex projects from ambiguous problem statements through design, implementation, and measured rollout, actively identifying and pursuing high-impact opportunities that align with company strategy Influence the architecture of ML/LLM systems and mentor engineers by sharing best practices, guiding implementation efforts, and providing constructive feedback that raises the team's overall technical standard What we look for Substantial production experience building LLM-powered systems that real users depend on — not prototypes, demos, or notebooks that never met a difficult document Depth in document understanding and structured extraction, and a realistic view of what OCR, layout analysis, and scanned or faxed source material actually do to a pipeline An evidence-driven practice: you build the evaluation before you tune, and you can show how a change moved a measured outcome Judgment about approach — knowing when a fine-tune, a retrieval pipeline, or plain classical ML beats reaching for the largest available model Comfort owning production systems, including debugging under pressure and treating cost and latency as things you are accountable for The ability to handle regulated data responsibly. Experience working under HIPAA with PHI is a strong plus; a willingness to work carefully within those constraints is a requirement Enough customer-facing ability to be trusted in a solution design session with a customer's technical and clinical stakeholders Key relationships Internal: Customer Success, DevOps, Product Management, Quality Assurance, System Architects External: Customer technical and operational stakeholders — engaged as needed for escalations, solution design, and technical integration discussions, typically in partnership with Customer Success and implementation teams About the engineering department The Engineering department designs, develops, and maintains the company's software products, covering development, infrastructure, operations, quality assurance, and product management. Its objective is to build and sustain products that satisfy customer needs and align with the long-term business strategy, to research and integrate new technologies that improve internal products and operations, and to ensure everything it ships meets the company's IT, information security, and compliance requirements. Location Remote-first — open to candidates across the United States. Customer-facing scope This role has the ability to be up to 25% customer-facing. The work is primarily engineering, but the successful candidate should be comfortable stepping into customer-facing solution design, technical discussions, and escalation support as needed.