What is the Be You Programs Directory?
The Be You Programs Directory is a searchable database of mental health and wellbeing programs offered by a wide range of providers to schools and early learning services. It is a tool designed to help educators find and make informed decisions about programs to implement in their learning community.
The Programs Directory includes general information about the program. For example, who offers the program, how many sessions in the program, cost, location – as well as ratings for the program’s evidence base and how it’s implemented.
Why did Be You redevelop the Programs Directory?
The Be You Programs Directory was redeveloped based on educator feedback. The Programs Directory now includes the following:
- A ratings system, which reflects the degree to which a program has been evaluated and found to be effective.
- Minimum entry requirements – to be listed in the directory, all programs must have completed at least one evaluation or research study that demonstrates a positive impact on a child or young person’s mental health and wellbeing, or their educational outcomes.
- Guidance for program providers to encourage them to evaluate and continue to assess the research underpinning their program – and by doing so improve their evidence rating in the Programs Directory.
- Updated with a new look and functions that make it easier to use, including a search filter allowing users to search for programs based on their ratings, the ability to compare two programs side by side, and an online application form available on accessible from the Be You website.
The initial Be You Programs Directory was launched in November 2018 as an evolution of the programs guides from the former KidsMatter and MindMatters initiatives.
Why is evidence important for programs?
Evidence is used to demonstrate the effectiveness and quality of a program, that is whether it has a positive impact on child mental health and wellbeing or educational outcomes. Evidence is a way of demonstrating that a program does what it says.
- There are many thousands of mental health, wellbeing and social and emotional learning (SEL) programs in Australia. It is useful for learning services, schools and educators to know how the evidence for a program has been established – whether this information comes from the program provider or whether it’s been independently established. Evidence supports learning communities to make informed decisions about which programs may be best suited to their needs.
- There are many program providers taking the time and effort to evaluate and gather evidence to demonstrate the impact of their program. This is leading practice, and helps to ensure we are making a difference to child mental health and wellbeing. Be You, the national mental health in education initiative, lists evidence and implementation ratings in the Be You Programs Directory to recognise and acknowledge the work programs are doing to provide high-quality evidence.
- Many programs have an associated cost. Educators want to know that the program they choose (and pay for) has been shown to be effective and has positive impacts on child mental health and wellbeing. It is a way of ensuring that schools and services receive value for money.
- Be You understands that establishing evidence takes time. We are focused on building the capacity of program providers to establish and develop program evidence. Our ratings system is designed to encourage program providers to continue to evaluate and gather evidence with the aim of improving the evidence base for the program.
What does the ratings system mean?
All programs listed in the Programs Directory have been through a rigorous assessment process designed in collaboration with the Murdoch Children’s Research Institute (MCRI) and the University of Melbourne.
MCRI assesses all applications to be on the Programs Directory against a set of criteria. At a minimum, all programs must have completed at least one evaluation or research study that demonstrates a positive impact on mental health outcomes.
As part of this assessment, programs are awarded two ratings: one for evidence and one for implementation.
- The evidence rating provides an assessment of the research underpinning the program, including impact and quality.
- The implementation rating provides an assessment of the program’s training, support and acceptability by participants and instructors.
Each rating is on a scale from 1 to 4. Four is the most robust score, while a score of 1 indicates there is scope for a stronger evidence base to be developed.
The Programs Directory also includes information about the context for the evidence. For example, the locations, year levels and groups where its impact was demonstrated.
How are the program ratings calculated?
Individual program ratings are calculated based on information collected from program providers in the application form. The information includes the following:
- General program information, such as details of the program’s implementation, and its cost, audience, location, overall aims and delivery.
- Program evaluation or research, which includes information about any evaluations or research studies conducted, with a focus on the research participants, design, and quality. Program providers are required to upload relevant documents (e.g. reports, publications, analysis) to support their answers.
- Outcome data, which includes details of the measures or outcomes used in each evaluation study.
What are the 4 evidence ratings?
- A rating of 1 (‘emerging’) means the program meets the minimum entry standard for research design and rigour and have at least one evaluation or research study demonstrating their effectiveness in improving child health outcomes.
- A rating of 2 (‘developing’) means the program has demonstrated research design of increasing rigour and a positive impact on mental health outcomes for children and young people. Studies generally have a pre-post design with no comparison group.
- A rating of 3 (‘promising’) is for programs with moderate evidence of a long-term positive impact through multiple rigorous evaluations. Research design generally reflects non-randomised controlled comparison.
- A rating of 4 (‘established’) is given to programs with extensive and strong evidence through rigorous research design (such as a randomised controlled trial) and a positive impact through multiple rigorous evaluations.
What are the 4 implementation ratings?
- A rating of 1 (‘emerging’) means the program meets the minimum criteria for providing training and support, and for acceptability by participants and instructors.
- A rating of 2 (‘developing’) means the program meets some of the criteria for providing training and support, and for acceptability by participants and instructors.
- A rating of 3 (‘promising’) is for programs with moderate implementation support meeting most of the criteria for providing training and support, and for acceptability by participants and instructors.
- A rating of 4 (‘established’) is given to programs with well-established program implementation support meeting all the criteria for providing training and support, and acceptability (including multiple delivery options, training for instructors, ongoing support offered to instructors and participants and accompanying resources provided to participants).
How does the program assessment process work?
- Minimum entry requirements: If a program meets the minimum entry requirements, the provider can apply via the online application form.
- Program provider completes online application form: The provider submits information about the program, its implementation and evidence.
- Assessment of application: Be You research partner Murdoch Children’s Research Institute (MCRI) assesses the provider’s responses against a set of criteria relating to the quality and rigour of the program’s evidence and effectiveness. If the program is awarded a rating, it will be for both evidence and implementation.
- Confirming the rating: The provider will be sent a report detailing the program’s ratings. Providers are offered the opportunity to ask to be reassessed if they do not think the ratings are a true representation of the evidence provided. If this occurs, an expert panel will review the application for quality assurance and determine whether a different rating should be awarded.
- Program is published: After this engagement between Be You and the program provider, the program is added to the Programs Directory and published on the Be You website.
A program we’re running, or want to run, has a low score. Is that bad?All programs awarded ratings have demonstrated a positive impact on mental health outcomes for children or young people. A program with a low rating may be in the early stages of development, and still building their evidence base. Over time, as more evaluation studies are completed, the rating may go up.
When can program providers apply to be listed?
Program providers can submit an application at any time. Once the application is verified as meeting the minimum entry requirements, it will be assessed within 7 weeks.
Be You offers guidance for program providers to help them evaluate and continue to assess the effectiveness of their program, and improve their ratings in the Programs Directory.
What are the minimum entry requirements to be included in the Programs Directory?
Programs must meet the following minimum entry requirements to be considered the Programs Directory. A program must:
- align with one or more of the five Be You professional learning domains
- align to the Australian Curriculum or National Quality Framework
- be available in Australia
- be supported by a training/delivery/implementation manual or guide
- be offered as more than a one-off session (i.e. offer multiple sequential sessions, either as a set series of sessions or on an as-needs basis)
- target one of the following audiences as the intended beneficiary: children, young people, parents, carers or families, early childhood educators, teachers, School Age Care educators
- have at least one research or evaluation study that demonstrates:
- a positive impact on mental health outcomes for children or young people
- a minimum of 20 participants in the study who received the program.
Where have all the old programs gone?
All programs listed in Be You’s superseded Programs Directory were invited to reapply to be listed in the current Programs Directory.
A program can apply at any time to be listed in the Programs Directory via the online application form on the website.
Will there be more programs added to the Programs Directory?
Yes. The process of assessing programs and adding them to the Programs Directory is ongoing.
Program providers are invited to check if they meet the minimum entry requirements and submit an application at any time via the online application form.
To ensure a broad representation of programs in the directory, Be You is working with other mental health and education-focused organisations to encourage more program providers to apply.
What about programs for diverse audiences?
We are actively working with the mental health and education sector to ensure more programs are developed and included in the Be You Programs Directory for diverse audiences. These include programs for Aboriginal and Torres Strait Islander communities, programs for culturally and linguistically diverse communities, and programs for children and young people with diverse needs.
I can’t find the program I’m looking for. Does this mean we shouldn’t be implementing it?
There are many programs offered to schools and early learning services in Australia that do not appear in the Programs Directory. The directory is not exhaustive as it is up to program providers to apply to be listed.
The Be You team are working with the mental health and education sectors to ensure we continue to list as many high-quality programs as possible.
Does Be You endorse the programs listed in its Directory?
A listing on the Be You website is not an endorsement, validation, advertisement or recommendation of that program for your learning community. The Programs Directory is a list of programs that have been evaluated and rated to help you choose an appropriate program, based on your context and needs, for your learning community.
I am a program provider. How can I promote my program’s listing in the Be You Programs Directory?
Program providers are welcome to refer to your listing in the Be You Programs Directory on your website. Please see these guidelines for promoting your listings and how to obtain the Be You logo.
My program was previously listed in the Programs Directory but has been removed. Can I apply again?
Yes. If your program meets the minimum entry requirements you can re-submit your program via the online application form.
Be You offers guidance to help program providers evaluate and continue to assess the effectiveness of their program – and by doing so improve their rating in the Programs Directory.
My program is listed, but we have just completed another evaluation. Do I have to apply again?
If your program is already listed in the Programs Directory, please get in touch with the Be You team for advice about re-applying with new evidence.
What if I do not agree with my program’s ratings?
All program providers have the opportunity to review their program’s ratings prior to being listed in the Programs Directory. If you do not agree with the ratings, you may ask for a reassessment. You will need to provide a written submission, and your application will be reassessed by an expert independent panel.
Programs Directory FAQs
Learning communities and program providers can browse these FAQs to find out more about the Programs Directory – from what it does to why evidence is important for programs, what the ratings mean and, if you’re a provider, how to apply to list your program.