Re-advertisement – Consultancy: Azure Artificial Intelligence/Large Language Models (AI/LLM) Expert, ICTD Digital Headquarters (remote)

Job title:

Re-advertisement – Consultancy: Azure Artificial Intelligence/Large Language Models (AI/LLM) Expert, ICTD Digital Headquarters (remote)

Company

Unicef

Job description

Those who applied to the previous advertisement do not need to re-apply, their application is still being considered.UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.And we never give up.For every child, a connected worldThe overarching strategic goal of UNICEF’s Information and Communication Technology Division (ICTD) is to transform and build partnerships with our stakeholders to successfully implement UNICEF programmes globally through innovative technology-enabled solutions.How can you make a difference?UNICEF ICTD requires an AI technical expert to develop a GenAI solution using the Microsoft Azure AI platform. The GenAI solution will enable UNICEF staff members to upload a large volume of UNICEF-internal structured and unstructured documents that are domain-specific, submit prompts and get responses as quickly and accurately as possible. UNICEF will grant access to the necessary infrastructure, regulations, and guidance to support the development and operationalization of the GenAI solution.UNICEF developed UniBot, a GenAI/LLM that is designed to interact with specialized internal knowledge from vetted internal resources as foundational GenAI, in addition to the ability to ingest individual users data, which is only available to that specific user to complement the foundational GenAI knowledge to produce high-quality responses. Every UNICEF staff can ask UniBot questions, however only a select group of specialists is allowed to upload additional knowledge. This is done to maintain the quality of the knowledge base and responses.The UniBot Architecture has four high level components, an ingestion pipeline, a LLM answer flow, a front-end application, and a reporting system. Documents to be included in the knowledge base land in the Azure Storage Container, are transformed / processed by the AI Search Indexer, and placed in the AI Search Index. When a user comes on to the platform to ask a question, that question is routed through the Prompt Flow endpoint, and a response is provided back to the user. Additionally, to support transparency and maintainability, a reporting database and PowerBI reports are built off the application data. Finally, users access all the available features through the frontend application. Refer to Annex 1: UniBot diagram for additional details.Scope of Work:Under the supervision of the UNICEF ICT Division (ICTD) lead, the consultant will continue the development of UniBot and perform the following activities:1. Application Architecture Development:

• Develop and add features to GenAI solution that leverages Microsoft Azure AI services such as Azure AI Studio, Azure Cognitive Search, Prompt Flow, etc. integrated with OpenAI’s LLMs such as GPT4 and the data sources mentioned above. Similar experience with Google Vertex or AWS Sage a plus.

• Suggest opensource options if any.

2.Data Management:

• Develop and maintain data pipelines for handling a high volume of structured and unstructured documents uploaded by a user, such as:

o Upload files and folderso SharePoint integration and content transformationo Web scraper

• Ensure data security and compliance with relevant UNICEF data protection regulations.

3.User Administration

• Implement roles and users’ groups using Microsoft Entra ID

Create basic UI for users’ administration.4. User Interface Development:

• Build a simple, user-friendly UI/UX for UNICEF staff members to upload documents and interact with the GenAI.

• Implement features to automate document upload and chat-based querying.

5.Cost Tracking

• Implement a feature to track and display the cost associated with chat interactions.

• Implement a feature to track backend transactions.

• Implement basic reports with options for duration, frontend and/or backend transactions.

6.Logs, Usage Reports and History

• Log timestamped training events, algorithm change events, prompts, responses, user assessment (thumbs up/down, hallucination, bias, accuracy)

• Implement Provide method(s) to manage log size, backups and archival.

• Create basic usage reports and guidance for PowerBI users.

• Provide users with access to chat history for reference and review.

• Provide the ability to reset the session/start new session.

7.Prompt Engineering and Model Fine tuning:

• Document efficient prompt examples that help the LLMs generate accurate and relevant responses.

• Document the best ways to train and fine-tune LLMs for domain-specific knowledge and applications.

• Employ RAG, Prompt or other techniques to anchor the GenAI responses to the provided data sources and establish techniques to mitigate hallucinations and increase accuracy.

8.Security and Compliance:

• Ensure compliance with UNICEF security, ethics, and privacy standards.

9.Testing and Deployment:

• Conduct thorough testing of the chatbot for functionality, accuracy, and security.

• Deploy the solution on UNICEF’s Azure.

• Develop a quality plan template for testing validating and revalidating the solution.

10.Documentation and handoff

• Document the work and application in a simple and detailed manner.

• Provide documentation and basic training for end-users and administrators

Work Assignments OverviewDeliverables/OutputsDelivery deadline

• Understand UniBot and UNICEF’s requirements and present application architecture and needs from the team to implement the work

• High level Application Architecture document
• Technical roadmap divided into two major milestone for deliverables.

20%Deadline: 2 weeks from contract start dateMilestone I

• Functional, data, security, performance, quality enhancements.

• First set of data ingestion enhancements as per roadmap in phase I
• Security enhancements
• First set of functionality enhancements as per roadmap in phase I
• Enhance prompt flow logic for additional use cases.
• Validation/Test Plan

30%Deadline: 10 weeks from contract start dateMilestone II

• Functional, data, security, performance, quality enhancements.

• First set of data ingestion enhancements as per roadmap in phase II
• Security enhancements
• First set of functionality enhancements as per roadmap in phase II
• Enhance prompt flow logic for additional use cases
• Validation/Test Plan
• Handoff through presentation of the system
• Handoff documents

50%Deadline: 20 weeks from contract start dateFinancial Proposal:Please indicate your ability, availability and your overall rate – or rate per deliverable (in US$) to undertake the terms of reference above. Applications submitted without a rate will not be considered.To qualify as an advocate for every child you will have…Education Qualification

• An advanced university degree (Master’s or higher) in Computer Science, Data Science, Mathematics, AI and/or other quantitative field.

Certifications in Azure AI, AWS Sage Maker, or Google Vertex AI. * Certification in data science.Knowledge/Expertise/Skills required

• A minimum of 5 years of relevant professional experience in building AI solutions.
• Proven experience in Retrieval Augment Generation (RAG) in a cloud computing environment
• 5 years’ experience preferably with Microsoft Azure AI infrastructure, or Google Vertex AI, and AWS Sage Maker.
• Proficient in creating data pipelines and managing large datasets.
• Experience in prompt engineering and implementing AI security measures.
• Knowledge of basic UI/UX design principles. Knowledge in accessibility a plus.
• Excellent problem-solving and communication skills.

• Fluency in English is required. Knowledge of another official UN language (Arabic, Chinese, French, Russian or Spanish) is an asset.

For every Child, you demonstrate…UNICEF’s values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS).To view our competency framework, please visit .UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. , irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.UNICEF offers for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment.UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.Remarks:Only shortlisted candidates will be contacted and advance to the next stage of the selection process.Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.Advertised: 06 Aug 2024 GMT Daylight Time
Deadline: 13 Aug 2024 GMT Daylight Time

Expected salary

Location

New York City, NY

Job date

Thu, 08 Aug 2024 00:12:26 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (un-jobs.net) you saw this job posting.