Guidelines for the ethical conduct of Paul Ramsay Foundation commissioned measurement, evaluation, research and learning
Last updated November 2024
The Paul Ramsay Foundation (PRF) is committed to working alongside our partners so that we learn understand the impact of our work together. We believe that ethical evaluation and research is essential to this learning process. These guidelines are designed to provide PRF, our partners, and evaluators/researchers with clear principles to conduct evaluation and research that is not only rigorous but also respectful, fair, and culturally sensitive. Many of these principles are informed by recognised codes of ethics and cultural safety frameworks.1
Historically, evaluation and research have failed to include the perspectives of disadvantaged or marginalised groups; often instead being used to reinforce their exclusion from decisions that affect their lives. As a result, they mistrust of evaluation and research work and are a skeptical whether evaluation and research will benefit them rather than be used to reinforce their exclusion. The application of these principles will help overcome this situation; by involving these people in research in evaluation activities in a way that empowers them and gives them agency over decision that affect their lives.
- Cultural Safety
Principle
Evaluation and research activities must respect the cultural values, beliefs, and practices of all participants in the research or evaluation, particularly when working with populations that are vulnerable. We take a broad and inclusive view of evidence, recognising that valuable knowledge extends beyond mainstream technical definitions to encompass diverse cultural perspectives and lived experiences. This approach fosters a rich understanding of social impact and helps shift perceptions of what constitutes valid knowledge within the broader community.
Guidelines
a. When selecting evaluation, research or learning partners, prioritise those with demonstrated cultural capability or lived experience relevant to the population that is the focus of the work.
b. Engage with community leaders and culturally competent researchers during the evaluation or research design and implementation stages to build trust, ensure relevance, and address ethical concerns. This may include consultations to obtain community-based ethics approvals.
c. Ensure that methods and outcomes are culturally appropriate, and that findings are shared in a meaningful way with the community.
d. Include people with lived experience or their representatives in the governance and oversight of research or evaluation.
e. Build flexibility into the design of evaluative activity to develop trust with communities and to recompense any associated costs.
- Free and Prior Informed Consent
Principle
Evaluation and research partners will obtain free informed consent from all participants before they take part in any research or evaluative activity.2
Guidelines
a. Ensure that participants fully understand the purpose, procedures, potential risks, and benefits of the evaluation/research.
b. Consent should be obtained voluntarily without any coercion, and participants should be informed of their right to withdraw from the evaluation or research process at any time without penalty.
c. Participants should be explicitly asked to reaffirm consent at appropriate points during the evaluation or research, especially when covering sensitive topics.
d. Obtain appropriate consent when using secondary data as inputs to the evaluation or research.
- Equity and Fairness
Principle
Evaluations and research will be conducted in a manner that respects the rights of all participants.
Guidelines
a. Ensure that comparison groups are created through transparent and equitable processes.
b. Pay special attention to ensure that vulnerable populations are not disproportionately burdened or excluded.
c. Consider methods (e.g. waitlists or phased rollouts) in contexts of scarce resources to ensure fairness.
d. Compensate community members and people with lived experience for their contributions (e.g. through direct payment or offering co-authorship in reports) while ensuring these forms of recognition do not influence their responses.
- Confidentiality and Privacy
Principle
Personal or private information of participants will be protected throughout the evaluation or research process.
Guidelines
a. Act in a manner that is consistent with the Australian Privacy Principles.
b. Implement robust data protection measures to ensure that all personal data are kept confidential and used only for the predefined evaluation or research purposes, unless consent for other use has been explicitly given.
c. Anonymise data and ensure secure data storage, such as using local servers and access controls.
d. Do not use data for purposes other than those for which consent has been given.
e. Refrain from collecting, recording, or using culturally sensitive information unless absolutely necessary.
f. Obtain consent before publishing any material that may affect the privacy of those involved in the evaluation/research.
- Data Sovereignty and Ownership
Principle
We will respect the sovereignty and ownership of data used or collected in an evaluation or research context and ensure that these are not transferred without consent.
Guidelines
a. Follow all legal, regulatory, and professional standards regarding data sovereignty and ownership.
b. Ensure that all contracts and agreements explicitly protect the sovereignty and ownership of data.
c. Do not share data with others without explicit consent from those who have sovereignty over it
d. PRF staff will stay informed of all relevant legal, regulatory, and professional standards regarding data sovereignty.
- Minimising Harm and Managing Distress
Principle
Evaluations and research will actively work to avoid or mitigate any distress and minimise potential harm to participants.
Guidelines
a. Compensate participants for costs they have incurred for their participation.
b. Assess the risks associated with the evaluation or research and implement strategies to mitigate them.
c. Ensure that the benefits of the evaluation or research justify any potential risks.
d. Ensure that evaluation or research staff are not placed in situations that could affect their safety or wellbeing.
e. Conduct a thorough risk assessment to identify potential sources of distress for participants prior to the evaluation and implement strategies to minimise them.
f. Provide participants with information about support services available to them, including counselling or helplines, particularly if sensitive topics are being addressed.
g. Train evaluation staff to recognise signs of distress in participants and to respond appropriately, including pausing the evaluation if necessary.
h. Establish clear protocols for participants to express their discomfort or distress, and ensure they know how to access support at any time during the evaluation.
i. After the evaluation, follow up with participants to assess their well-being and address any lingering concerns resulting from their participation.
- Transparency and Accountability
Principle
Evaluation and research processes and results will be transparent and accountable.
Guidelines
a. Be open about the goals, methods, and potential conflicts of interest of the evaluation
b. Develop a clear protocol outlining the study’s aims, methodology, participant recruitment, consent process, and data management plan.
c. Publish study protocols in advance of the evaluation or research.
d. Share all findings, whether positive or negative, with participants, partners, and relevant stakeholders.
e. Publish results to contribute to the broader knowledge base and ensure that lessons learned are accessible.
f. Ensure that all analyses are amenable to independent scrutiny and verification.
- Collaboration and Learning
Principle
Evaluations and research will be collaborative and foster mutual learning.
Guidelines
a. Work closely with partners throughout the evaluation or research process, ensuring that their insights and knowledge inform the design and interpretation of results.
b. Include people with lived experience or their representatives in the sense-making process that translates evaluation or research evidence into insights.
c. Use evaluation and research as a tool for collective learning, with a focus on improving future practice and outcomes.
- Justice and Benefit to others
Principle
The evaluation and research activity will contribute positively to the well-being of participants and their communities.
Guidelines
a. Ensure that the evaluation or research does not exploit participants, especially those from vulnerable or marginalised groups.
b. Plan how the findings will be used to improve services and outcomes for those who participate.
c. Consider using adaptive trial designs that allow for modifications based on interim results, to address ethical concerns in real-time.
d. Consider ways to grow community capacity around evaluation, research and data collection, by working with appropriate organisations.
- Post-Evaluative activity Follow-Up
Principle
Participants’ contributions will be acknowledged and valued, and they will have access to initiatives or support following the conclusion of evaluations or research, particularly if the program is found to be beneficial.
Guidelines
a. Report back to participants the findings of the evaluation or research and what actions will flow from it.
b. If evaluation and/or research activity demonstrates that a particular program is effective, ensure that participants, including those in control groups, have access to the benefits of that program.
c. If evaluation and/or research demonstrates that a particular program is not as effective as intended, ensure that appropriate steps are taken to avoid harm, such as ending the program or changing its approach.
d. Consider how findings could be applied or scaled to maximize positive impact.
- Ethical Review
Principle
All evaluations and research activity will undergo formal ethical review.
Guidelines
a. Prior to implementation, evaluations will be reviewed by an independent ethics committee or an evaluator’s internal review board to ensure they meet ethical standards.
b. Evaluations and research will conform with appropriate professional standards, including those for the Australian Evaluation Society, and the National Health and Medical Research Council Ethical Considerations in Quality Assurance and Evaluation Activities.
c. Researchers and field staff will have training in ethical conduct, data collection procedures, and how to handle sensitive situations.
d. Regular monitoring and adjustments should be made as needed to address any ethical issues that arise during the evaluation or research process.
- Informed Decision Sharing
Principle
Participants should be kept informed about how their contributions will be used and the outcomes of the evaluation.
Guidelines
a. Clearly communicate to participants that the evaluation report will be shared with them and outline how and when this will occur.
b. Provide participants with a summary of findings in accessible language that highlights key points relevant to them.
c. Ensure that participants understand their rights in relation to the information shared, including the option to provide feedback or ask questions about the report.
d. Encourage an open dialogue with participants about their experiences during the evaluation process, fostering a collaborative environment where their insights are valued.
e. Address any potential concerns participants may have regarding the sharing of evaluation results and ensure that their confidentiality is maintained
Appendix 1: Guidelines for working with First Nations peoples and communities
In addition to the principles mentioned above, evaluators/researchers working with First Nations communities and people should be guided by the following principles. They should also consult the following frameworks that provided more detailed guidance:
- The AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research
- NSW Treasury, The First Nations Investment Framework: A Guide to design, economic appraisal and evaluation of First Nations initiatives
- NSW Department of Education, Re-imagining Evaluation: A Culturally Responsive Evaluation Framework
- The Lowitja Institute, Protocols and Principles
- Cultural Safety
Principle
When working with First Nations people and communities, evaluations and research must be conducted with the utmost respect for their cultural values, beliefs, and practices. Evaluation and research processes must integrate two-way learning, where First Nations knowledge systems and Western evaluation and research methods are co-created and adapted to reflect the values and aspirations of the communities. Evaluative activity should be designed to foster reciprocal learning, building trust and collaboration.
Guidelines
a. For initiatives that involve First Nations places and communities, prioritise evaluation and research partners who are First Nations owned and controlled. If their expertise aligns with the needs of the evaluation and/or research, these partners should be given preference to ensure that the evaluation is conducted in aculturally sensitive and respectful manner.
b. Ensure that First Nations people are involved in the selection of evaluators and/or researchers.
c. Engage with community leaders and culturally competent researchers to build trust, ensure relevance, and address ethical concerns.
d. Work with First Nations communities to create or use formal governance structures, such as Community advisory boards, that will oversee all aspects of evaluation and research design, implementation, and reporting to ensure alignment with Community protocols, priorities and ethical standards.
e. Ensure that evaluation and research methods and outcomes are culturally appropriate, and that findings are shared in a way that is meaningful to the community.
f. Continue to build PRF’s understanding of First Nations owned and controlled research and evaluation providers within Australia to support these activities.
- Data Sovereignty and Ownership
Principle
We will follow relevant First Nations principles on Indigenous Data Sovereignty and ownership
Guidelines
a. When working with data on First Nations peoples and communities, follow the Maiam nayri Wingara Key Principles of Indigenous Data Sovereignty and Indigenous Data Governance. In particular, develop Indigenous Data Governance mechanisms with advice from Indigenous Data Governance experts.
b. Data collected as part of any evaluation or research will be governed by Indigenous Data Governance mechanisms, and communities will have full ownership and control over their data and their use.
c. Establish mechanisms to prevent Indigenous cultural intellectual property (ICIP)from being collected, where it is not relevant to program evaluation or research. Data management plans should identify any data collected that is part of ICIP to be excluded from all data holdings.
- Justice and Benefit to others
Principle
The evaluation and/or research will aim to contribute positively to the well-being of participants and their communities.
Guidelines
a. Where appropriate, invest in the training, capability-building, and infrastructure development required to embed measurement, evaluation, and learning practices directly within First Nations communities and programs. This can be done by working with ACCOs and/or First Nations researchers, if appropriate and possible.
b. When engaging external evaluators/researchers, their role should focus on coaching, mentoring and capacity-building for First Nations evaluators, researchers and communities, ensuring that First Nations-led teams can conduct evaluations and research independently in the future.
- Ethical Review
Principle
All evaluations and research will undergo formal ethical review to ensure adherence to these guidelines.
Guidelines
a. Prior to implementation, evaluations and research should be reviewed by an independent ethics committee or an evaluator’s internal review board to ensure they meet ethical standards. Work with Aboriginal ethics boards (AHRECs), if available.
b. Evaluations and research will conform with appropriate professional standards, including the AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research.
Appendix 2: List of Ethics Review Boards
These committees vary in their scope, but they all play a crucial role in ensuring that research involving vulnerable populations is conducted ethically, with appropriate safeguards in place. It’s important to choose the right HREC based on the specific population, location, and type of research being conducted.
Human Research Ethics Committees (HRECs)
- National Health and Medical Research Council (NHMRC)-registered HRECs: These committees review human research projects to ensure they are ethically acceptable and in accordance with relevant standards and guidelines. Many universities, hospitals, and research institutions have their own HREC.
- Aboriginal Health and Medical Research Council (AHMRC) Ethics Committee: Specifically reviews research that impacts Aboriginal and Torres Strait Islander peoples in New South Wales.
- Central Australian Human Research Ethics Committee (CAHREC): Covers research in the Northern Territory and South Australia, focusing on Indigenous populations.
- Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) Research Ethics Committee: Specializes in reviewing research projects involving Indigenous peoples across Australia.
State and Territory Health Department Ethics Committees
- Victorian Department of Health and Human Services (DHHS) Human Research Ethics Committee: Reviews research involving vulnerable populations within the state of Victoria.
- Queensland Health Human Research Ethics Committees: Includes several committees that review research conducted within Queensland, particularly involving public health services.
University-based HRECs
- University of Melbourne Human Research Ethics Committee: Reviews a wide range of research projects, including those involving vulnerable populations.
- UNSW Sydney Human Research Ethics Committee: Provides oversight for research conducted by the university and associated entities.
- Australian Catholic University Human Research Ethics Committee: Reviews research conducted by or affiliated with ACU, which often includes work in the social impact sector.
Hospital-based HRECs
- Sydney Local Health District (SLHD) Human Research Ethics Committee: Reviews research within hospitals and other health services in the Sydney area.
- Royal Children’s Hospital Melbourne Human Research Ethics Committee: Specializes in pediatric research and other studies involving vulnerable children.
Specialised Committees
- Social Research Ethics Committees: Some universities and research institutions have ethics committees specifically for social research, which often involves vulnerable populations.
- National Disability Insurance Scheme (NDIS) Quality and Safeguards Commission Ethics Committee: Focuses on research involving people with disabilities, particularly within the NDIS framework.
Other Indigenous-focused Ethics Bodies
- Western Australian Aboriginal Health Ethics Committee (WAAHEC): Reviews research proposals involving Indigenous populations in Western Australia.
- Menzies School of Health Research HREC: Specialises in research in Indigenous health, particularly in the Northern Territory and Northern Australia.
Other
- Bellberry: a private not-for-profit that provides scientific and ethical review of human research projects across all phases of clinical and device trials, and social and behavioral studies.
1 See especially the Australian Evaluation Society, the AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research, the National Health and Medical Research Council Ethical Considerations in Quality Assurance and Evaluation Activities.
2 See also the UNHCR, Free Prior and Informed Consent – An Indigenous Peoples’ right and a good practice for local communities