Fondazione Icaro ETS – Independent AI Evaluation Activities
Impartiality and Conflict of Interest Policy
Fondazione Icaro ETS (“Icaro”) is currently conducting independent third-party evaluations of artificial intelligence models and systems for frontier model providers. Icaro is committed to ensuring that its evaluation activities are conducted impartially, independently, and free from commercial, financial, organisational, or personal influence that could compromise professional judgment.
Independence of evaluation activities
Icaro retains independent professional judgment over the design, execution, analysis, and reporting of its third-party evaluations.
Clients and evaluated organisations may define the general objectives and scope of an engagement and may provide relevant technical information and comments. They may not determine or alter evaluation findings, scores, conclusions, or the professional judgment of Icaro’s evaluation team.
Where a client disagrees with a finding, its position may be recorded separately without modifying Icaro’s independent conclusion.
Financial independence
Compensation for evaluation services is not contingent upon:
- achieving a particular evaluation result or score;
- identifying or not identifying particular risks or vulnerabilities;
- the successful release or deployment of a model;
- any other outcome that could create an incentive to modify evaluation findings.
Commercial considerations shall not influence the methodology, interpretation, or reporting of evaluation results.
Identification and management of conflicts of interest
Before assigning personnel to an independent evaluation, Icaro assesses potential conflicts of interest that could reasonably affect, or appear to affect, the impartiality of the evaluation.
Relevant conflicts may include:
- financial interests in the organisation being evaluated;
- employment with the evaluated organisation or relevant competitors;
- significant personal or organisational financial relationships capable of influencing professional judgment;
- any other circumstance that could reasonably call the evaluator’s independence into question.
Potential conflicts are documented, assessed, and appropriately managed by the Managing Director.
Recusal
Individuals with a material conflict of interest shall not participate in decisions concerning the methodology, execution, review, approval, or reporting of the affected evaluation.
Organisational independence
Icaro shall not permit an evaluated organisation to exercise organisational control over its evaluation activities, evaluation personnel, governance, or findings.
Personnel conducting an independent evaluation operate under the direction and responsibility of Icaro.
Methodological and editorial independence
Icaro retains professional autonomy in determining the evaluation methods appropriate to the agreed scope, including, where applicable:
- evaluation metrics and sampling strategies;
- elicitation and adversarial-testing techniques;
- agent scaffolding and tool configurations;
- scoring criteria and rubrics;
- interpretation of evidence;
- limitations and uncertainty;
- conclusions and recommendations.
Icaro retains editorial control over its evaluation reports and conclusions.
Confidentiality, responsible-disclosure, security, and information-hazard requirements may restrict the dissemination of particular information, but shall not be used to alter evaluation findings or conceal material adverse results.
Disclosure
Icaro discloses relevant actual or potential conflicts of interest to the appropriate recipients of an evaluation where such conflicts could reasonably affect the interpretation or perceived independence of the work.
Each independent third-party evaluation includes a specific declaration describing the evaluator’s independence and any relevant conflicts or safeguards applicable to that engagement.
Confidentiality and impartiality
Confidentiality obligations towards evaluated organisations do not override Icaro’s obligation to conduct evaluations impartially or to report findings accurately.
Where contractual, legal, security, or responsible-disclosure requirements restrict external publication, such restrictions shall not permit the evaluated organisation to modify Icaro’s independent technical findings.
Oversight
Icaro periodically reviews its independence arrangements and conflict-of-interest procedures and updates this policy where necessary to reflect changes in applicable regulation, standards, professional practices, and independent AI evaluation methodologies.