ABSTRACT

The recently-completed EU-funded project "The Development of Indicators and Assessment Tools for CSO Projects Promoting Values-based Education for Sustainable Development" or ESDinds for short (http://www.esdinds.eu) has succeeded in developing indicators to measure the impact of values-based education for sustainable and responsible behaviour. While the methodologies are new and require further development, case studies have shown their effectiveness in contexts as wide as indigenous school children in Mexico, former child soldiers in Sierra Leone, and businesses in Europe. While the indicators are effective in a wide variety of situations, their interpretation and the vocabularies used for the relevant values depend on the specific organizational and cultural contexts. This does not distract from their internal consistency. A web site for the community of interest in values-based indicators has been established at http://www.wevalue.org. The PERL network should find in this approach a powerful tool to measure the effectiveness and demonstrate the utility of education for responsible living.

INTRODUCTION

Living responsibly or being a good consumer citizen are concepts rooted in values, since values define behaviour that benefits society. An individual operates on a spectrum from egotistical to altruistic, infantile to mature, base impulses to cooperative. In society this is expressed as power-hungry, seeking status and social dominance, versus conscientious, egalitarian, communitarian (Shetty, 2009). The latter qualities generally contribute to greater social good and higher integration.

Many Civil Society Organizations (CSOs) are values-driven or work at the level of values, but they have seldom been able to demonstrate the usefulness of this work to others in any concrete way. How can such organizations measure what they are trying to do? Are there indicators that can measure the changes they are trying to bring to their target audience? Can values-based change be made more tangible? These are the questions the researchers set out to answer.

The European Union, through its FP7 research programme, funded a two year project (January 2009-April 2011) on the Development of Indicators and Assessment Tools for CSO Projects Promoting Values-based Education for Sustainable Development, or ESDinds for short (Podger et al., 2010) (http://www.esdinds.eu/). The partners in the project were the University of Brighton (UK) and Charles University (Prague) as academic partners, and Civil Society Organizations including the Alliance of Religions and Conservation (ARC, UK), the Earth Charter Initiative (Sweden/Costa Rica), the European Bahá'í Business Forum (EBBF), and the People's Theatre (Germany). The International Federation of Red Cross and Red Crescent Societies did not formally join the project, but contributed some case studies. Professor Arthur Dahl was a partner and senior advisor to the project. The research team at Brighton was led by Professor Marie Harder, and at the Charles University Environment Centre by Dr. Tomas Hak. *

METHODS

The project was unique in EU-funded research projects in that is was driven by the CSOs, not the researchers. The CSOs defined what values were important to them and what they wanted to measure, i.e. implementing values or spiritual principles. This often meant clarifying their values in a way they had not done before. They needed to be clear about what they were trying to do in a way that might make it measurable. Often the organizations discovered values that they had not realized were important. This crystallization of their values was itself an important project outcome, as it added a new and valuable dimension to their work.

The role of the researchers was to help to define assessment methodologies and indicators relevant to the identified values. They used various research approaches to compile the explicit values in each civil society organization. Then they looked for implicit values by interviewing staff and participants. The values found were verified with the CSOs and compared to the research literature on values. While hundreds of terms for values were compiled, these were often found to be context-specific, and could not be used consistently across projects and organizations.

From all the values identified, the CSO partners selected six for the initial trials:
- Unity in diversity
- Trust/Trustworthiness
- Justice
- Empowerment
- Integrity
- Respect for the community of life (the environment)

The researchers then developed long lists of indicators pertinent to each value, drawing on literature from across the social sciences. Through the field projects, where some indicators were refined and others dropped, a final list of 166 indicators was produced which seemed to be broadly relevant across all the case studies tested, often measuring more than one value. While the vocabularies for values differed greatly between cultural and institutional contexts, the behaviours described by the indicators seem to be more universal, although certainly still far from comprehensive.

SAMPLE OF WE VALUE INDICATORS

1. Everyone has their place in the team
8. Work environment is supportive of people being able to fulfil their responsibilities in their families or personal relationships
17. Decision-making takes into account the social, economic and environmental needs of future generations
20. People participate actively in developing the entity's code of ethics
38. Women feel that they are given equal opportunities to participate in decision-making processes
45. People share their skills and abilities freely with one another, regardless of nationality, ethnic origin, skin colour, gender, sexual orientation, creed or religion
70. People are taking the opportunity to develop their own visions and goals for projects, and/or for the whole entity
75. Entity's activities or events connect participants emotionally to the community of life
83. Mistakes are understood as opportunities to learn and improve
94. People do not backbite about others within the entity
96. People feel they can participate in the vision and activities of the entity or project without compromising their personal beliefs or values

CASE STUDIES

The researchers then visited CSO field projects, where they jointly identified and adapted relevant indicators with local project leaders and CSO personnel, and then trialled the selected indicators in the projects, followed by a joint evaluation and sharing of experience. The indicators that were developed and tested in the first phase were shared more widely with other activities both inside and outside the project, and further tested in some additional case studies.

The following descriptions of some of the initial case studies illustrate this approach, and show the diversity of situations selected to demonstrate that values can be measured.

Echeri Consultores, Mexico (Earth Charter)

Echeri Consultores is a small non-governmental organization affiliated with the Earth Charter Initiative, working to increase environmental awareness and an understanding of Earth Charter values in rural indigenous communities in Mexico.

One of its projects is a programme working with 9-13 year olds in 15 schools in the Purepecha indigenous communities. It includes arts workshops on environmental conservation and values; guided reflection on local ecosystems; and tree planting workshops, enabling the children to establish tree nurseries in the school grounds and to conduct reforestation activities in the wider community.

For this project with school children, it decided to focus on two values: collaboration in diversity, and care and respect for the community of life. This led to the choice of 22 indicators, such as:
- We feel girls are valued;
- Different points of view are listened to;
- Emotional connection to community of life;
- Quality in outputs (training in tree planting).

Many different types of assessment tools were used, such as:
- Use of a spiral of coloured scarves on which pupils stood (spatial/corporal method);
- Hand painting (paint how you felt, when we finished the last project) and word elicitation (what words go with these pictures that you painted?);
- Focus groups;
- Theatrical comprehension (can you "act out", like in a play, how you plant a seedling?) (Test of knowledge);
- Key informants.

Before the project, they used to measure the number of trees planted and the number of children involved as their indicators of success. Now they can measure as well the emotional connection to nature, gender equality, equality of the indigenous members, and empowerment. They now realize that these things that are important to them are also of interest to their funders, since their funding was extended at a time when many projects were cut.

A second Echeri Consultores project in Mexico was with a multi-cultural group of around 19 youth aged 12-21, called Juatarhu ("Forest" in Purepecha), meeting every week. The activities of Jutarhu are similar to those of the schools programme, but with greater scope and depth, incorporating large reforestation campaigns and municipal arts festivals.

Lush Cosmetics, Italy (EBBF)

Lush is a multinational company producing and distributing handmade cosmetics and toiletries from fresh organic ingredients. The company has a strong ethical policy that precludes the purchase of any ingredient from any supplier that tests any of its materials on animals, as well specific environmental education projects and numerous policies relating to waste, energy, aviation and the environmental impact of ingredient sourcing. The central office of Lush Italy is based in Milan and has a small number of employees (less than 20 in total) working in management, communication, marketing, accounting and retail. There are also employees preparing certain cosmetic products and distributing imported products. Four shops are located in the Milan area. The indicators were successfully tested in this corporate context.

People's Theater, Germany

People's Theater is a non-profit programme based in Offenbach, Germany, and staffed mainly by youth volunteers aged 18-25, that uses drama workshops to help school children to explore social responsibility and non-violent conflict resolution. Its philosophy is based on a positive image of humanity, and the view that individuals have a duty to build their own character, develop positive social values in themselves and serve the wider community. The indicators were used to measure the effectiveness of the training of the youth volunteers and their commitment to the values of the project, and have become a regular part of project evaluation. They are now exploring how they can be used to measure their impacts on the school children they work with.

University of Guanajuato, Mexico (Earth Charter)

The Environmental Institutional Programme of Guanajuato University (PIMAUG) is a cross-faculty initiative structured around 6 strategic areas:
a) Assisting students to develop a holistic vision of the environment;
b) Promoting sustainable resource use and waste management;
c) Diffusion of a culture of environmental awareness, through a variety of media;
d) Interdisciplinary research;
e) Training in environmental issues through diplomas and Masters programmes; and
f) Social participation and inter-institutional partnership.

The programme decided to engage in the indicators project because the Earth Charter is about transforming values into action, which is the "heart" of the University mission. The University already has good environmental measures, but there was no way to know rigorously the deeper dimension of the Earth Charter vision, and the degree to which those values were present and transformative. The values-based indicators provided a way. The indicators articulated deeply-held aspirations and priorities which had not previously received systematic attention. The process of reflection and selection of the indicators, even before measurement, had a significant cultural impact on the PIMAUG unit and enthused participants, becoming a process of transformational learning.

Among the key benefits was the change of culture experienced in PIMAUG. The Earth Charter workshop leaders reported a greater sense of effectiveness as a result of a clearer and more precise focus on values in their workshop delivery. The personal impact of the indicators affected how a manager dealt with conflict, and generated a much more participatory approach in her work with volunteers. The unit has a greater unity of vision, and participants in the focus group discussions have reported having reconnected or been re-inspired in their work. Integrating the indicators into regular evaluation has increased group insight into their own application of values and led to understanding success in terms of values in a practical way.

Youth as Agents of Behavioural Change, Sierra Leone (Red Cross)

The Principles and Values Department of the International Federation of the Red Cross and Red Crescent Societies (IFRC) has initiated and conducted a worldwide programme called "Youth as Agents of Behavioural Change" (YABC). YABC seeks to empower youth to take up a leadership role in positively influencing mindsets, attitudes and behaviours in their local communities towards a culture of peace, respect for diversity, equality and social inclusion.

As part of this programme, the Sierra Leone Red Cross Society has established an agricultural project composed of four teams of 30 members each. It brings together members of different tribes and chiefdoms, even those who fought on opposite sides during the civil war, which ended ten years ago. These youth live and work together on agricultural sites and participate in YABC workshops relating to non-discrimination and respect for diversity, intercultural dialogue, social inclusion, gender, and building a culture of non-violence and peace. The indicators were trialled successfully during a weekend workshop, and provided the organizers with new insights on the effectiveness of their work and on some problems still to be addressed.

The IFRC was sufficiently pleased with the results that they asked the researchers to participate in a regional conference in Jordan with youth leaders from 45 countries of Asia and the Middle East, to share the methodology and encourage its widespread application.

WE VALUE

With the end of the EU-funded project, some of the participants have joined in a new partnership to create an on-line platform, WeValue.org, to access and use the indicators developed and to encourage a community of practice for values-based indicators by taking the work forward and sharing experience.

The We Value system can help with three things:
- Measuring, by finding personalized ways to assess what is important;
- Crystallizing what is actually important to an entity or project; and
- Communicating to funders and to the public, building a new, shared, values-based vocabulary.

For rigorous measurement of the presence of a particular value, the indicators considered valid for it need to be defined clearly, and more than one indicator and measurement method should be used. If it is not necessary to be rigorous every time, a simple measurement is sufficient.

The project confirmed in various ways that it was clearly values that were being measured, including as a control a case study of a financial services company in Luxembourg. Ultimately it is for each organization or user to decide what the presence of a value looks like in any particular context. It is this internal consistency that validates and makes the tool useful. The master list of 166 indicators has been derived from the experience of many CSOs and has demonstrated its usefulness, but each user needs to decide what would be considered a good measurement with that indicator.

Despite the wide range of case studies, the methodology will still require further development. It was not possible in the short duration of the project to extend it to Asian cultural contexts or to test the methodologies with a wide variety of religious and cultural traditions, which have their own norms and vocabularies. However the project methodology of bottom-up identification of values and determination of behaviours and indicators that measure those values should be applicable in almost any context.

RESULTS

The project established as a proof of concept that values can be measured with indicators. It showed that measuring behaviours or feelings linked to values is possible with reasonable scientific validity in context-specific situations. By agreeing to a common values interpretation within a project or organization, the measurements have internal consistency and validity. Such indicators can show state or change over time. While the methodologies are quite clear, they require a certain understanding of standard social science techniques, so some human facilitation may be necessary to adapt the approach in organizations without adequate internal capacity.

The project showed that previously invisible dimensions of an activity like values could be made visible using indicators. When something can be measured, it becomes important. Values can then be consciously encouraged or cultivated, and the organization or project becomes more values-driven. Strong values are linked to more effective outcomes.

Using indicators as tools, values can be embedded more widely in many kinds of human activity that can benefit from stronger values. The measurement methodologies are sufficiently flexible to adapt to most situations and can incorporate almost any values framework. Measuring desirable behaviours and values becomes positively reinforcing.

The project concluded with an international conference "Making the Invisible Visible: An Emerging Community of Practice in Indicators, Sustainability and Values", which took place on 16-18 December 2010 at the University of Brighton, England. The project results were presented to a wide range of organizations including educators, businesses, civil society organizations and social enterprises, with discussions focused on measuring human values at the project level. Conference reports including presentations and videos of the main speakers are available at the project web site: http://www.brighton.ac.uk/sdecu/research/esdinds/conference/index.html and at the International Environment Forum web site: https://iefworld.org/conf14.html. Scientific papers on the results of the project are in preparation.

The PERL network should find in this approach a powerful tool to measure the effectiveness and demonstrate the utility of education for responsible living.

* In addition to Marie Harder, Tomas Hak and Arthur Dahl, researchers involved in the project included Gemma Burford, Elona Hoover, Svatava Janouskova, Georgia Piggot, Dimity Podger, Ismael Velasco, and Martin Zahradnik.

REFERENCES

Podger, Dimity, Georgia Piggot, Martin Zahradnik, Svatava Janouskova, Ismael Velasco, Tomas Hak, Arthur Dahl, Alicia Jimenez and Marie Harder. 2010. The Earth Charter and the ESDinds Initiative: Developing Indicators and Assessment Tools for Civil Society Organizations to Examine the Values Dimension of Sustainability Projects. Sustainability Indicators 4(2): 297-305.

Shetty, Priya. 2009. Novels help to uphold social order. New Scientist, 17 January 2009, p. 10