about company

I am currently working with a company focusing on AI, LLM and Computer Vision.

Salary structure: Base + bonus! Hybrid working arrangement- 2 days office, 3 days WFH. Office location at CBD. 4 rounds of interview to offer stage.

about job

• Design and implement robust frameworks to evaluate the performance of generative AI systems, including text and multi-modal models for Large Language Models (LLMs), including but not limited to GPT-based models, BERT, T5, and other state-of-the-art architectures
• Perform technical AI evaluations, benchmarking and "red-team" tests on LLM model, including assessing them for robustness in performance, embedded biases, vulnerability to jailbreaks and prompt injection attacks
• Work with stakeholders to design strong LLM models, custom evaluation approaches and a suite of technical and analytical AI evaluation frameworks and tools
• Define and refine metrics for evaluating model performance, such as perplexity, BLEU, ROUGE, accuracy, coherence, factual consistency, and bias detection
• Lead efforts in curating and managing large, high-quality datasets for evaluating LLMs

skills and requirements

• Min 5 years of total working experience with min 2 years in building and deploying LLMs
• Strong experience in evaluating LLMs using metrics such as perplexity, BLEU, ROUGE, and human-centered evaluation techniques
• Proven track record of managing and analyzing large, complex language datasets, including text preprocessing and tokenization
• Solid programming skills in Python and experience building automated pipelines for continuous model evaluation
• To apply online please use the 'apply' function, alternatively you may contact Stella at 96554170 (EA: 94C3609 /R1875382)