AI Quality Strategy

We are building an AI agent for verifying user credentials.

Users login into a website, and the AI agent looks at the page and identifies required information on that page. If the required information is on a different page, the agent identifies that and prompts the user to navigate to the said page.

For example, if a user wants to prove that they study at Stanford - we’d ask them to login into stanford.edu ; Once they login, the AI Agent will identify their name, field of study and graduation year from their profile page.

We support over 50k websites from where a user can verify their credentials - university portals, employment portals, HR/Payroll systems, Govt ID portals, ecommerce websites, fintech apps etc.

This is a hard technical AI problem. But, that is something that we have built expertise in. However, the real deal is when the rubber hits the road! That is, does the AI perform as it is expected to for the long tail of websites we support? To build confidence, we need to keep testing and improving our AI. That means we need real users to test our products by logging into the websites we support and try generating a proof - and report if there is any issue encountered. We want to do this proactively, so that the issues are identified before a user experiences these problems in prod.

For that reason, we’re starting a new team to annotate the performance of our AI. It is an ongoing, ever-changing role. The requirements of the type of annotations and demography to get the annotations from will keep changing. Today, we might need thousands students to login and test our university page AI Agents. Tomorrow, we might need PhD candidates to review our AI’s intermediate reasoning steps.

This role is to make sure the AI we’re building is continuously benchmarked on real-use, and help improve the agent.

What you will do

• Understand requirements from business and engineering functions, and ensure delivery
  • Example - Annotate AI performance for 1000 universities by onboarding students from 1000 universities and asking them to login into their respective university portal
• Launch targeted acquisition campaigns globally
  • Example - University Students, Professionals with licenses, Gig workers
• Design and manage scalable referral/affiliate programs to source the required annotators
• Build and lead a global volunteer ambassador network
• Partner with student clubs, online forums, Reddit mods, Discord servers, and HR/payroll communities
• Write and publish content (social posts, DMs, landing pages, newsletters, etc.)
• Track KPIs and continuously optimize acquisition funnels

Who you are

• Are self driven, creative and decisive, capable of operating without getting blocked on “superiors”. A.k.a, you are a leader.
• Have 2–4 years experience in growth, community, or product marketing
• Have launched referral or affiliate programs (even small-scale or experimental)
• Are a scrappy operator: cold DMs, subreddits, Telegram raids, student org outreach
• Know your way around low-code tools like Notion, Zapier, Typeform, Airtable
• Enjoy solving messy growth problems and building early-stage systems
• Bonus: Experience in Web3, decentralized identity, HR tech, or edtech