We are specifically looking for candidates who have assessed or audited actual AI applications, not just candidates with traditional IT audit, policy governance, or general GRC experience. Candidates should be able to explain how they would audit an enterprise chatbot or autonomous AI agent, identify its risks, evaluate existing controls, and recommend improvements to ensure secure, responsible, and compliant operation. This would be a Hybrid opportunity - 3 days a week.
Job Description: AI Governance & Audit Specialist
Location: Middlesex County, New Jersey
Role: AI Governance & Audit Specialist – Generative AI / LLM Applications
Experience:
6+ years in IT Audit, Technology Risk, AI Governance, or related fields, including hands-on experience with AI applications.
Job Summary:- We are seeking an experienced AI Governance & Audit Specialist to assess, audit, and establish governance controls for enterprise AI applications, including Generative AI solutions, AI-powered chatbots, conversational assistants, autonomous AI agents, and LLM-based applications.
- The ideal candidate will have practical experience evaluating AI systems for security, regulatory compliance, data privacy, accuracy, transparency, reliability, and operational risks.
- This role requires someone who understands how AI applications operate and can independently assess their risks, identify control gaps, and recommend appropriate governance and audit measures.
- Key ResponsibilitiesAI Application Auditing
- Conduct risk-based audits of AI-powered applications, including chatbots, virtual assistants, AI agents, and Generative AI solutions.
- Evaluate AI application design, implementation, data handling, user interactions, and operational controls.
- Review LLM-based solutions, Retrieval-Augmented Generation (RAG) applications, and agentic AI workflows.
- Assess risks related to hallucinations, inaccurate responses, bias, data leakage, prompt injection, and unauthorized actions.
- Evaluate AI agent permissions, tool access, autonomy, and human approval mechanisms.
- Review application logs, audit trails, testing evidence, and model performance monitoring.
AI Governance & Risk Management:
Develop, implement, and assess AI governance policies, standards, controls, and procedures.
Evaluate AI applications against established governance frameworks such as NIST AI Risk Management Framework (AI RMF) and ISO/IEC 42001.Assess AI lifecycle governance, including development, validation, deployment, monitoring, change management, and retirement.
Review third-party AI platforms, external LLM providers, and associated vendor risks.
Evaluate data privacy, security, responsible AI practices, and regulatory compliance requirements.
Identify governance gaps and recommend remediation measures.
Controls, Compliance & Reporting:
Design and evaluate AI-specific risk and control matrices.
Conduct control testing and document audit findings, risk ratings, and remediation recommendations.
Assess explainability, accountability, human oversight, and responsible AI safeguards.
Work with AI engineering, cybersecurity, compliance, risk management, and internal audit teams.
Prepare audit reports, governance dashboards, and executive-level risk assessments.
Track remediation plans and validate corrective actions.
Required Qualifications:
6+ years of experience in IT Audit, Technology Risk, Model Risk Management, AI Governance, Cybersecurity Risk, or related disciplines.
Demonstrated hands-on experience auditing, assessing, or governing AI applications, particularly Generative AI, LLM-based chatbots, or AI agents.
Strong understanding of AI application architecture, LLM integrations, APIs, RAG, and agentic AI workflows.
Practical knowledge of AI risks, including hallucinations, prompt injection, sensitive data exposure, bias, and autonomous agent behavior.
Experience developing or testing technology controls, conducting risk assessments, and documenting audit findings.
Knowledge of frameworks such as NIST AI RMF, ISO/IEC 42001, and OWASP Top 10 for LLM Applications.
Understanding of data governance, privacy, access management, security controls, and AI application monitoring.
Strong communication, documentation, and stakeholder management skills.
Ability to work onsite in Middlesex County, NJ.
Preferred Qualifications:
Experience auditing enterprise AI/GenAI applications in production environments.
Experience with AI platforms such as Azure OpenAI, AWS Bedrock, Google Vertex AI, or similar technologies.
Familiarity with agent orchestration frameworks such as LangChain, LangGraph, or equivalent platforms.
Experience with AI red-teaming, adversarial testing, model evaluation, or AI assurance tools.
Certifications such as IAPP AIGP, CISA, CRISC, CISSP, or ISO/IEC 42001-related qualifications.
Prior experience in regulated enterprise environments.
