Bilingual Korean AI Evaluation Specialist Job at AuraOne Human Data, Remote

  • AuraOne Human Data
  • Remote

Job Description

Bilingual Korean AI Evaluation Specialist is a remote Korean specialist track for evaluating korean evaluation outputs against native-speaker standards. Reviewers spot fluency, register, and cultural-context errors that automated checks miss, and write structured rationale that the modeling team can act on in the next training cycle.

Why this role matters

Multilingual model quality depends on reviewers who can read Korean the way a native speaker does. AuraOne uses these reviewers to set the bar for tone, formality, and cultural register before a model ever ships to Korean users.

Responsibilities

  • Evaluate Korean model responses for fluency, accuracy, register, and cultural fit for Bilingual Korean AI Evaluation Specialist assignments.
  • Compare paired korean evaluation outputs and pick the stronger response with a written rationale.
  • Flag hallucinations, code-switching errors, and formality mismatches with task-specific severity tags.
  • Capture native-speaker edits that demonstrate the correct phrasing alongside each issue.
  • Calibrate against the AuraOne Korean rubric in weekly reviewer-quality cycles.
  • Surface ambiguous prompts back to the program team so the rubric can be sharpened.
  • Maintain reviewer notes that document tricky idioms, regionalisms, and emerging usage.

Qualifications

  • Native or near-native Korean fluency with strong written-English ability for Bilingual Korean AI Evaluation Specialist work.
  • Demonstrable experience editing, translating, or evaluating Korean content.
  • Comfort applying multi-page rubrics consistently across long evaluation batches.
  • Clear written reasoning that names the issue and the standard being applied.
  • Reliable async availability for at least 10 hours per week.
  • Prior model-evaluation, annotation, or human-rater experience is a plus.

Example tasks

  • Score a pair of Korean responses to a customer-support prompt and pick the better one with a 1-2 sentence rationale.
  • Flag formality mismatches in korean evaluation outputs (T/V usage, honorifics, regional register).
  • Rewrite a model's Korean response so it reads naturally and tag the original error category.
  • Audit a 50-row evaluation batch for rubric consistency and surface drift back to the program lead.

Nice to have

  • Background in Korean linguistics, journalism, or translation studies.
  • Experience with TMS tools, MQM error taxonomies, or other structured QA frameworks.
  • Familiarity with LLM evaluation rubrics and inter-rater agreement workflows.

Skills

  • Multilingual evaluation
  • Native-speaker review
  • Rubric application
  • LLM output evaluation
  • Korean evaluation

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

$25–$30 / hr

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Job Tags

Remote job, For contractors, 10 hours per week

Similar Jobs

Sibanye Stillwater

Electrical Planner Job at Sibanye Stillwater

 ...and Skills:Working knowledge of mining and maintenance including safety (MSHA Rules & Regulations), maintenance, equipment and environmental regulations.Knowledge in applying modern-day technology to improve reliability.Knowledge in executing safety and quality control... 

Widenet Consulting

COMPETITIVE INTELLIGENCE ANALYST Job at Widenet Consulting

Job Description: Work location: Remote We are seeking a Competitive Intelligence Analyst who will Support the company-wide win loss programs operational needs and per quarter read-out content needs, including qualitative and quantitative analysis, Create, update, and maintain...

Framework

Virtual Elementary & Middle School Tutors Job at Framework

 ...Framework is now hiring Certified Elementary and Middle SchoolTutors! Framework is seeking certified, experienced, and creative educators to provide engaging, high-quality virtual tutoring to a diverse group of elementary, middle, or high school students covering... 

Mussett Nicholas & Associates

MEP BIM/CAD Technician Job at Mussett Nicholas & Associates

 ...Technology, or related field. Previous experience with BIM and/or CAD software essential Previous experience using AutoCAD and Revit preferred Previous experience in an architectural environment preferred Understanding of structural drafting and design.... 

Health Advocates Network

EMT/Emergency Medical Technician Job at Health Advocates Network

Benefits We Offer: Comprehensive health, prescription, dental, vision, life, and disability plans Competitive pay rates Referral opportunities ? Refer a friend & Cash in! Travel reimbursement and per diem allowances Employee discounts Educational opportunities...