Critical Evaluation of a Program to Foster Reflexive Antiracism

This paper documents the evaluation of a three-day program entitled “Race, Culture, Indigeneity and the Politics of Disadvantage,” which was delivered in 2010 in Melbourne, Australia with the aim of promoting Reflexive Antiracism (RA), a novel diversity training approach. To assess the impact of the program on its participants, the Reflexive Antiracism ScaleIndigenous (RAS-I) was devised and administered before and after the program both to participants and a matched control group. The program increased Reflexive Antiracism among participants through an enhanced understanding of whiteness, racialisation and White Racial Identity. Future studies are required to advance both the concept of Reflexive Antiracism and its measurement.


Introduction
Reflexive Antiracism (RA) aims to foster a sustainable and effective approach to antiracism that avoids both the dangers of essentialism and 'backlash' effects in terms of emotional responses. It has been found that motivated antiracists, i.e., those who have an internal motivation to respond without prejudice (Plant & Devine, 1998), have a tendency to experience discomfort, distress, guilt, fear, anxiety, anger, inaction and withdrawal (Nile & Straton, 2003;O'Brien, 2009;Tatum, 1992). These emotions can be caused, in part, by a disconnection between an antiracist worldview and 'unconscious' racist feelings, thoughts and behaviours that reflect prevailing social norms (Sommers & Norton, 2006).
A certain level of such negative emotions may be associated with achieving antiracist (or non-prejudiced) behaviour in the short-term (Fehr & Sassenberg, 2010;Monteith, Mark, & Ashburn-Nardo, 2010;Sommers & Norton, 2006). However, in the long-term such emotions may result in 'burnout' (Slocum, 2009;Smith & Redington, 2010), defensiveness/resistance (Ancis & Szymanski, 2001;Kernahan & Davis, 2007), reduced support for affirmative action; and/or heightened prejudice (Case, 2007). It is also possible for these emotional reactions to result in paternalism or passivity during contact with members of minority groups (Ridley, 1995). Some scholars have noted the potential for diversity training to both essentialise minority and white identities as well as create 'backlash' effects, with some studies indicating that up to one fifth of participants have increased levels of racial prejudice after attending diversity training (Fredericks, 2008;Paradies et al., 2009;Trenerry, Franklin, & Paradies, 2010).
As 'a strategy for situating knowledges' (Rose, 1997: 306), reflexivity fosters consideration of a researcher's history and 'cultural baggage', including their relationship with research participants (de Jong, 2009). Whilst the notion of reflexivity has been employed extensively in academia (e.g. Bourdieu, 2004;Etherington, 2004), it is gaining increasingly recognition as a tool to critique cross-cultural practices within health care and social work (de Jong, 2009;Kondrat, 1999;Murray-Garcia, Harrell, Garcia, Gizzi, & Simms-Mackey, 2005). Reflexivity has been found to enhance learning in diversity training and lead to more flexible behaviours (Chick, Karis, & Kernahan, 2009;Lillis & Hayes, 2007). In relation to prejudice, greater awareness of the process of one's own thinking has been associated with minimising bias (Murray-Garcia et al., 2005). Reflexive individuals should be more willing to 'take risks, to view mistakes as part of the learning process and to grow and change' (Johnston, 2009: 649), while also engaging 'with issues such as racism and colonialism' (Pon, 2009 ).
To be effective antiracists, particular aspects of reflexivity are required. White people need to avoid a blanket view of minorities as 'good' as well as the converse understanding of white people as 'bad'. They also require an ability to recognise how white people benefit from unearned privilege without becoming 'paralysed' by guilt and anxiety, while at the same time accepting that such privilege is unavoidable in most contemporary societies.
A central component through which RA offers a mechanism for understanding these concerns is the concept of 'racialisation'. Racialisation has been defined as "injecting racial elements International Journal of Social Science Research ISSN 2327-5510 2014 into a situation or the process of creating a race" (Delgado and Stefancic (2001 p. 154) or a "racialized discourse… [with] racism as (one of) its expressive objects" (Goldberg, 1993, p. 41). Our definition also encompasses racism and antiracism under the more general rubric of racialisation. Racialisation occurs when individuals are differentiated by race, with power unequally dispersed (or created) contingent on racial identification. Racialisation can manifest through attitudes, beliefs, behaviours, laws, norms, and practices that either reinforce or counteract power asymmetries (Paradies, 2005).
Within a racialised societal system, actions can either enhance or reduce power asymmetries between the two (or more) unequal racial groups, with antiracism serving to decrease power differentials via advantage accruing to subordinate groups or creating disadvantage for dominant groups. Racism can be defined as the opposite to this (i.e., action leading to an increase in power differentials between two racialised groups) (Paradies, 2005, 3). This usually occurs through advantaging dominant racial groups and/or disadvantaging subordinate racial groups.
This definition allows that everyone has the potential to both engage in racism and/or anti-racism, regardless of their ethnic/racial background (Sawrikar & Katz, 2010). While 'obvious' in many instances, the line between racism and anti-racism can be unclear in some cases. 1 For example, paternalistic behaviour may have an anti-racist intention but fail to reduce power differentials (Jensen, 2006). We contend that it may not always be possible to definitively conclude whether power differentials are increasing or decreasing.
Another key element of racialisation is the way that minority and dominant identities are constituted. Whiteness studies seeks to understand how 'white' people (i.e. dominant identities) are provided unearned privileges which contribute to minority group oppression (Ancis & Szymanski, 2001;Kivel, 1995;McIntosh, 1990;B.D. Tatum, 1997). Importantly, white is not considered a 'natural' category based on skin colour. Rather, it is the structure (i.e., socially constructed) through which cultural dominance is naturalised and, thus, reproduced and maintained (Frankenberg, 1993). As such, those who are encompassed within the unmarked category of whiteness can vary over time. For example, both Jewish (Brodkin, 2000) and Irish people (Ignatiev, 1996) 'became' white during the 20 th century. Notably, 'white' antiracists include those who benefit from advantages such as wealth and education without these individual necessarily having white skin.
In many progressive discourses, it is assumed that it is unavoidable for whiteness to be a cause of oppression for minorities (Kowal & Downing, 2011;Miller & Harris, 2005). Therefore, conflict and consequent negative emotions (e.g., guilt and anxiety) can often ensure between antiracist and white identities (Swim & Miller, 1999). A RA approach seeks to foster reflection and acceptance of this conflict (Helms, 1994;B.D Tatum, 1992). RA bears some resemblance to 'reflexive race cognizance' (O'Brien (2001), the ambiguity and tension between white privilege and antiracism noted within 'White dialectics' (Todd & Abrams, International Journal of Social Science Research ISSN 2327-5510 2014 2011) and the need for 'critical reflection on one's own position' (Hollinsworth, 2012). More detail on Reflective Antiracism is provided elsewhere .
In addition to highlighting the interplay between whiteness and racialisation, RA considers the impact on antiracist practice. An example drawing on Australian discourses of disadvantage for Indigenous people has been explored by Kowal and Paradies (2005). Rather than acknowledging the possible contribution of Indigenous people's own agency in Indigenous disadvantage (which could be construed as racist), antiracists generally focus on societal factors that limit choices for Indigenous people (Kowal & Paradies, 2005). Since both agency and structure contribute to Indigenous disadvantage, the eliding of individual choice has profound implications for understanding the nature of social problems and their consequent solutions (Kowal & Paradies, 2005).
RA encourages insight into the effects of racialised discourses and identities on the way that antiracists think about oppression and construct potential solutions. RA is a novel approach to diversity training designed for white people who already identify as antiracist. It aims to avoid the potential for inaction and withdrawal that may arise from a disjunction between awareness of one's own racialised feelings, thoughts and behaviours and strongly internalised antiracist ideals .
This article details the evaluation of a program aimed at developing RA. Developed and facilitated by the second and third authors, 2 "Race, Culture, Indigeneity and the Politics of Disadvantage" (RCIPD) aims to assist professionals and practitioners to develop RA in relation to their work in Indigenous affairs in Australia. Focusing on identity formation, knowledge production and cultural recognition, 3 the program includes a range of practical exercises exploring Indigenous disadvantage and its implications.
Indigenous Australians suffer from significant health and social disparities (AIHW, 2011), caused primarily by colonisation and its ongoing manifestations (Anderson, Baum, & Bentley, 2008;Carson, Dunbar, Chenhall, & Baillie, 2007). For example, the health system is one structure in which the effects of colonisation continue to be experienced. Some Indigenous peoples avoid Western health care systems and others face significant barriers when accessing Western health care systems (National Aboriginal Health Organisation, 2006). Some Indigenous people seek more from government health systems (Australian Medical Association, 2002;Ring & Brown, 2002), while others consider such systems to be inherently paternalistic and/or neo-colonialism (Pearson, 2000). Thus, professionals working in Indigenous affairs in Australia must contend with a complex and highly contested political climate (Kowal & Paradies, 2005). 2 In terms of racial identity, the second author identifies as Aboriginal-Asian-Anglo Australian man while the third author identifies as Jewish-Polish-Australian woman. Research conducted within the U.S. government suggests that pairs of diversity trainers that differ in terms of race/ethnicity or gender produced significantly more learning among participants than homogeneous trainer pairs (Hayles, 1996) with more recent scholarship suggesting that race/ethnicity rather than gender is of particular importance in a diversity training context (Liberman, Block, & Koch, 2011). ISSN 2327-5510 2014

Computer exercise
Undertake the 10-15 minute Implicit Associations Test (IAT) which assesses strength of associations between ethnic/racial groups and positive/negative concepts The 2010 three-day RCIPD program entailed ten sessions, each lead by the second and third authors, involving small group discussion and activities. The first session briefly overviewed discourse, construction and critique as key concepts. Key themes of antiracism, racism and whiteness/White Racial Identity (WRI) were interrogated in subsequent sessions. The course also investigated a range of approaches to Australian Indigenous health and social policy (see Table 1). The program also considered agency and structure as key concepts relating to Indigenous ill-health. While agency is the ability to freely choose one's actions, structure determines and hence constrains choices (Kowal & Paradies, 2005).
Since 2003, RCIPD has been offered once or twice a year for up to 26 professionals in research, policy or service delivery roles within Indigenous affairs. The evaluation detailed in this paper occurred as part of two courses conducted in February and April 2010. Attendance at the course is generally through self-selection, with employers covering costs for participants. The majority of participants attending the course identify as 'antiracist' in that they support Indigenous rights, have careers focused on addressing Indigenous disadvantage and are opposed to racism.

Design
Participants consisted of 46 professionals (32 female and 14 male) who work in Indigenous affairs in Australia. A pre/post-test with control (i.e., quasi-experimental) design was employed to evaluate the RCIPD program. This project had ethics approval from the Human Research Ethics Committee of the University of Melbourne (#0932659).
The survey instrument, RAS-I (see below as well as  was administered to participants a few days prior to the program, and again one month following their completion of the program. An immediate post-program survey was not administered due to concerns relating to participant load and the possibility that familiarity with the questions could lead to social desirability or consistency bias among participants. That is, participants may have felt a need to either maintain consistency with their previous responses (which they would have recalled well) or a need to have evidenced change as a result of the program. Furthermore, it was considered more important to assess changes that were relatively sustained (i.e. still evident after a month) than to measure transitory changes that existed immediately after program completion but did not persist over the medium or long-term. It was also noted that a one month delay was the average time frame for post-evaluations of diversity training programs (Kulik & Roberson, 2008).
The intervention group (n=22) was recruited from participants in the February 2010 program. The control group (n=24) was recruited from two sources. First, individuals who had expressed interest in participating in the April 2010 program were approached to participate in the study as controls. It was presumed that people who wished to attend RCIPD would have comparable characteristics regardless of whether they attended the February or April program. Both the February and April 2010 programs were advertised at the same time and received the same public exposure. The response rate among February program participants International Journal of Social Science Research ISSN 2327-5510 2014 was high (96% at baseline with 88% completing the follow-up survey). However, as only 14 out of the eventual 26 participants in the April program had registered by February, not enough controls could be recruited from this source.
In order to boost the number of controls, individuals from the February program who had consented to participate in the evaluation were asked to nominate two people who they considered to be similar to themselves (in terms of age, occupation, gender, ethnicity and general attitudes to Aboriginal and Torres Strait Islander people). It was left up to the discretion of the participants to decide the compatibility of the individuals they nominated. Twenty-one people were nominated as potential matched controls, with 14 eventually participating.
All three categories of potential participants (February program participants, April program participants, and nominated matched controls) were contacted via email and sent an information sheet and a consent form. Those in the intervention and control groups who consented to participate completed the survey in the three days prior to the February program commencement. A total of 24 intervention participants completed the baseline survey while 28 control participants completed the baseline survey. As two participants in the intervention group and four in the control group did not complete the post-survey, the final sample size was 22 in the intervention group and 24 in the control group.

The Scale
The scale comprised three different sections. The first section assessed participant socio-demographic characteristics. These questions assessed participant age, gender, highest level of education, racial/ethnic background, Indigeneity and white identity. Furthermore, participants outlined the focus and duration of any diversity/cultural training programs they had undertaken within the past year. The remaining questions ascertained how many years participants have worked in Indigenous affairs and which Australian state they most recently worked in (see Appendix A).
The post-survey required participants to restate their racial/ethnic background, Indigeneity and white identity. These items were included a second time to gauge whether participants had an altered view of their racial identity following the program. Program participants were also asked in the post-survey the extent to which they had read the non-compulsory reading material provided to program participants some weeks before the program began, to assess whether this had any impact on survey results (see Appendix B).

Statement Items
We identified five broad areas with which to evaluate RA from existing antiracism and diversity training scholarship. These were: racism and whiteness; culture, structure and agency; White Racial Identity (WRI); antiracism; and emotional responses. The scale included 18 items in the form of statements that assessed these key themes. It should be noted that some of the items in the scale generate a strong response from some participants as they involve sensitive topics that are central to white identities and antiracist identities. ISSN 2327-5510 2014 However, as discussed above, sustained critiques of both antiracism and diversity training in the literature indicate that a novel approach to antiracism is justified.

Racism and Whiteness
While antiracists generally understand race and ethnicity as well as the way in which racism creates differentials in power, opportunity, benefit or resources (Paradies et al., 2009), a more nuanced understanding of racism among those high in RA includes an awareness of unconscious, subtle and covert racism. This facilitates acceptance and constructive management of individual racialised thoughts, feelings and behaviours .
As such, agreement with the statement: "Everyone has been racist at some point in their lives" (item 5) was thought to indicate high levels of RA. The statement: "White people get unearned privileges in Australian society" (item 3) was included to evaluate awareness of white privilege (McIntosh, 1990), with agreement indicating high RA. Yamato and Anzaldua (1990) assert that white people who want to be allies to people of colour should do so without the expectation of gratitude. A characteristic of reflexive antiracists is the recognition of their own racism and the need to address it for their own sake. In order to measure views on whiteness, an item was adapted from Yamato & Anzaldua (1990): "white people should address racism for their own sake rather than just to help Indigenous people" (item 9).
Those high in RA should recognise that moral positions change over time and that current antiracist thinking may be criticised in the future. This was assessed with one item: "Some policies in Indigenous affairs that are now thought to be fair and just will be considered racist by future generations" (item 11).

Culture, Structure and Agency
Recognising that cultures are fluid and in constant flux is indicative of high RA . As such, we hypothesised that those low in RA would agree that: "To improve Indigenous health, we must encourage the restoration of Indigenous cultures" (item 2). A second item focused on the concept of culture asked if: "Indigenous health and social outcomes can be improved without Indigenous people changing their culture" (item 12). White antiracists who seek to differentiate themselves from assimilationists are often uncomfortable with the fact that Indigenous cultures and traditions change over time (Kowal, 2008;Kowal, Lea, & Cowlishaw, 2006). Participants low in RA would therefore be likely to disagree that Indigenous cultural change is a required consequence of improvements in health and social outcomes. Conversely, those high in RA understand cultural change (including some aspects of assimilation) as inevitable and not necessarily something to be avoided. As such, those high in RA should disagree with, or have a ambivalent response, to item 12.
Antiracists who have low reflexivity characteristically show an eagerness to preserve not only Indigenous cultures in their "traditional" form but also the "moral integrity" of Indigenous people (Kowal, 2008;Kowal et al., 2006). This manifests in a reluctance to attribute any aspect of Indigenous disadvantage to the actions and choices of Indigenous people International Journal of Social Science Research ISSN 2327-5510 2014 themselves, hence avoiding the well-recognised problem of 'victim-blaming' (Cowlishaw, 2004). On the other hand, as discussed above, a more reflexive understanding of antiracism entails a recognition that placing greater emphasis on the structural rather than agential forces can down-play or ignore the agency of Indigenous people in either improving or compromising their own ill-health (Kowal & Paradies, 2005). One item focused on whether Indigenous people may or may not legitimately care less about their health than non-Indigenous people due to cultural difference and/or agency; "It is racist to suggest that Indigenous people don't care about their health" (item 17). While such a statement can be utilised as a form of victim blaming (i.e., attributing ill-health to Indigenous agency through a lack of care), ambivalence or even disagreement may be indicative of RA if such a orientation towards health is perceived as 'legitimate'.
A second item expressed this idea in another way, also assessing endorsement of agential explanations as a cause of Indigenous ill-health: "An important contributing factor to the poor health of Indigenous people is that they have different priorities to non-Indigenous people, such as family obligations or strong ties to their ancestral land" (item 10). We predicted that less reflexive antiracists would disagree with this statement, attributing Indigenous ill-health to external factors alone rather than granting some role to Indigenous agency, while more reflexive antiracists may express a greater degree of agreement.
A related item was: "Indigenous people want the same as other Australians-a nice house and car, healthy kids, regular income and leisure time" (item 15). Some antiracists would reject the idea that Indigenous peoples want the "same" as other Australians as this would be seen as endorsing the assimilation of Indigenous peoples into the dominant Australian culture. However, those high in RA may recognise the possibility of such similarity among Indigenous and non-Indigenous Australians, and may be reflexive regarding the discomfort that antiracists feel at the idea that some Indigenous people may seek to be similar to non-indigenous people. As a result, they may be more likely to express ambivalence about or endorse this item.
An modified item from the Attitudes Towards Indigenous Australians (ATIA) scale (Pedersen, Bevan, Walker, & Griffiths, 2004) was used to examine the interplay between structure and agency; "The 'bad' Indigenous people in our society are a product of discrimination and marginalisation" (item 4). Some antiracists would agree with this statement, attributing disadvantage to structural factors in order to uphold the 'moral integrity' of Indigenous people as a racialised group. On the other hand, reflexive antiracists may recognise that, as a result of both agencial and structural influences, 'bad' people can be found in any societal group, and may be reluctant to attribute Indigenous disadvantage exclusively to structural causes, hence disagreeing with this item.

White Racial Identity (WRI)
The White Racial Identity Attitude Scale (WRIAS), developed by Helms and Carter (1990) was designed to measure six "ego statuses" posited by Helms (1990). Each status has an information processing strategy that reflects certain attitudes, behaviours and feelings that a person utilises to deal with race-related information and incidents (Daniels, 2001). As White International Journal of Social Science Research ISSN 2327-5510 2014 people move through the stages of the model they have an increasingly complex understanding of race and racism. As self-selected antiracists, participants of the program would be situated within the last three ego statuses which constitute the second stage of the model ("Defining a Nonracist White Identity") (Helms, 1994).
We included three items from the WRIAS. "Being part of a multicultural community is a must for me" (Helms, 1990) (item 1) was included to assess the importance that individuals place on multiculturalism and diversity. We predicted that less reflexive antiracists would agree with this statement, which is also associated with a higher ego status of White Racial Identity.
The statement: "White people need to embrace 'whiteness' as an important part of who they are" (Helms, 1990) (item 16) assessed whether whiteness was considered an identity that can be accepted rather one to be denied or ashamed of, with agreement indicative of high RA as well as a more advanced White Racial Identity. The item: "In most situations, I express my honest opinion when an Indigenous person is present without worrying if I will appear racist" (item 6) was also adapted from the WRIAS (Helms, 1990). Individuals higher in WRI are more likely to agree with this item. We anticipated that reflexive antiracists would also be more likely to agree with this statement because they are more aware of, and thus less sensitive about, their antiracist identities. In contrast, less reflexive antiracists may be uncomfortable expressing their honest opinion when this could threaten their antiracist identity. On the other hand, it is possible that reflexive antiracists may instead be more aware of their tendency to self-censor in the presence of an Indigenous person than other antiracists. As such, it is not clear whether reflexive antiracists differ from other antiracist in only their awareness or also in their behaviour, or how the interplay between awareness and behaviour would affect responses to this item.

Antiracism
We anticipated that a change in levels of RA could result in two possible broad effects on item responses. First, as antiracists become more reflexive they may become ambivalent about issues they previously felt strongly about. Second, increased reflexivity can bring greater awareness of thoughts, feelings and behaviours that are in conflict with their antiracist aspirations. Both of these effects may produce changes in responses that, in some cases, could mimic a shift towards more racist attitudes.
To distinguish between RA and racism, we included three items in the RAS-I that indicative of racist attitudes and, therefore, with which antiracists should consistently disagree. These were: "there is no racism against Indigenous people in Australia today" (item 7); "the government should not provide special programs for Indigenous people" (item 13); and "I would be concerned if a close friend or relative married an Indigenous person" (item 18). The first item assessed awareness of racism, the second taps opposition to affirmative action and the last item is a measure of social distance adapted from the Challenging Racism survey (Forrest & Dunn, 2007).

Emotional Responses
The emotional responses of white people to their own group membership is an important consideration during diversity training, including the psychosocial costs of racism to whites . Three items adapted from the Attitudes Towards Indigenous Australians scale ATIA (Pedersen et al., 2004) were utilised. The first was: "I feel angry about Indigenous disadvantage" (item 8). Whilst anger can result from feeling defensive about an attack to a positive white identity (Kivel, 2002), it may also result from moral outrage directed at a lack of government action on Indigenous disadvantage. Thus, item 8 was included to explore feelings of anger in relation to reflexivity. The second statement to assess emotions was: "I feel anxious about interacting with Indigenous people" (item 14). It was anticipated that social desirability bias would lead less reflexive antiracists to disagree with this statement to a greater extent than reflexive antiracists. A more nuanced understanding of the disjuncture between antiracist intention and outcomes as well as an awareness of the interplay between structural forces and Indigenous agency may lead reflexive antiracists to disagree with this statement more so than other antiracists. Alternatively, those high in RA may be more aware of guilty feelings they may harbor.
A numerical value was assigned to each possible statement item response as follows: 1 = Strongly Agree, 2= Agree, 3= Neither Agree nor Disagree, 4= Disagree and 5= Strongly Disagree.

Analytical Approach
The study assessed change in responses between (a) intervention and control groups, at both time 1 and time 2, and (b) time 1 and time 2 for the intervention and control groups. The statistical differences for (a) were assessed using The Wilcoxon U test (Mann & Whitney, 1947), a non-parametric test for assessing differences between the values of two independent samples. An alternate test, the Wilcoxon signed-rank test (Chou, 1960), a non-parametric test for assessing differences between repeated measurements on a single sample, was used to assess (b).

Results
The demographic characteristics of both the intervention and control groups are detailed in Table 2. As assessed by a chi-squared test and the Fisher's exact test, there were no statistically significant differences in characteristics between the intervention and control groups. The two groups are highly comparable in relation to sex, age and education. A majority of participants were female (73% of intervention group participants and 67% of control participants) while the highest proportion of participants was aged between 30 and 44 (41% of intervention participants and 42% of control participants). Among the intervention group, 45% had a Bachelors degree while this figure was 54% in the control group. A majority of people in both groups identified as white only. Of those who did not identify as white, two intervention group participants and four control participants identified as Indigenous. Most people in both groups worked in Victoria, the state where the training took place while only a minority had completed diversity training within the past year. ISSN 2327-5510 2014 ISSN 2327-5510 2014 Although the Wilcoxon tests utilise raw data for each item, the values included in Table 3 are the mean responses for both groups at time 1 and time 2 to allow easier comparison across groups and time periods. Statistically significant differences between the intervention and control groups, and between time 1 and time 2 for each group at the two-tailed p<0.05 and p<0.01 levels are noted.

Antiracism
The responses of both intervention and control group participants to the antiracism items (7, 13 and 18) were as expected given strong disagreement with these items both pre and post, among both groups.

Emotional Responses
Items 8 and 14, which assessed changes in participants' affective reactions as a result of the RCIPD program, showed no significant change from pre to post. Both groups reported anger about Indigenous disadvantage and low levels of anxiety about interacting with Indigenous people.

Racism and Whiteness
Two racism and whiteness items showed significant change in the intervention group (3 and 11). For item 3, intervention group participants indicated stronger agreement that white people experience more privileges in society, whilst the response of the control group to this item did not change significantly. For the intervention group, the response to item 11 changed significantly from agreement that policies in Indigenous affairs that are now thought to be fair and just may be considered racist by future generations at time 1 to a more ambivalent response at time 2. The control group also had a significantly different response at time 2 to this item, but in the opposite direction, moving from ambivalence to agreement. This was the only item for which the control group had a significant change over time.
The remaining item relating to racism and whiteness (item 5) did not show significant change in either group. Both groups agreed that everyone has been racist at some point in their life.

Culture, Structure and Agency
Items 2 and 12 showed significant change in the intervention group, whilst the response of the control group to these items did not change significantly. The intervention group were less likely to agree that Indigenous cultures must be restored in order to improve Indigenous health and that Indigenous health and social outcomes can be improved without Indigenous people changing their culture. Three other items (4, 10 and 17) did not show significant change in either group. The results of both groups indicated ambivalence that the "bad" Indigenous people are a product of discrimination and marginalisation, disagreement that a significant contributing factor to Indigenous poor health is due to Indigenous people having different priorities to non-Indigenous people and agreement that it is racist to suggest that Indigenous people do not care about their health.

White Racial Identity
The responses of the intervention group to all White Racial Identity items (1, 6, 9 and 16) changed significantly from pre to post while control group responses did not. The intervention group was less likely to agree that being part of a multicultural community is a must for them and that they can express their honest opinion when an Indigenous person is present without worrying if they will appear racist. They agreed more strongly that white people should address racism for their own sake and that white people need to embrace their "whiteness".

Discussion
The above results suggest that the RCIPD program achieved a number of intended outcomes. Importantly, responses to the three 'antiracism' items remained stable indicating that antiracist ideals held by participants before the program were not altered as a result of the program. Although significant changes over time were recorded for eight items, a further seven showed no change. Interpretations of these findings by key topics are outlined below.

Emotional Responses
Items 8 and 14 showed no significant change from time 1 to time 2. For item 8 ("I feel angry about Indigenous disadvantage"), respondents in both the intervention and control groups reporting feeling angry about Indigenous disadvantage both pre and post. Anger resulting from awareness of white privilege can have both negative and positive outcomes in relation to antiracism. On the one hand, anger can stem from feeling defensive about white identity (Kivel, 2002). Thus, anger is a potentially counterproductive reaction. Alternatively, anger may be related to sincere reactions towards injustice that may serve as a motivating response to address societal racism (Krehbiel & Cropanzano, 2000;Leach, Iyer, & Pedersen., 2006). Self-focused anger is associated with self-correction and wanting to "fix" a situation (Leach et al., 2006;Tangney, Wagner, Hill-Barlow, Marschall, & Gramzow, 1996). Therefore, the lack of change in response at time 2 could indicate that participants did not react negatively to the program by experiencing a heightened anger in the form of defensiveness. Although not an explicit aim of the program, it could be argued that the program failed to encourage participants to feel intensified anger at injustice.
For item 14 ("I feel anxious about interacting with Indigenous people") respondents expressed a lack of anxiety both pre and post. Fear of appearing racist may have precluded participants from reporting feelings of anxiety. From this point of view, it could be argued that the program did not encourage a reflexive view of inter-racial relations. However, the results of White Racial Identity items don't support this interpretation. Although, as detailed in Table 2, length of professional experience in Indigenous affairs varied considerably among participants, it is possible that, for at least some participants, low levels of anxiety stem from regular interaction with Indigenous people in their professional roles.

Racism and Whiteness
There was significantly more agreement with item 3 ("White people get unearned privileges International Journal of Social Science Research ISSN 2327-5510 2014 in Australian society") among the intervention group than among the control group at time 2. This result aligns with the aim of the program to encourage an understanding that white people accrue certain privileges through racist social systems and not through merit alone. Although we expected this item to be more strongly endorsed as a result of the program, responses to item 5 ("Everyone has been racist at some point in their lives") did not change significantly in the intervention group. It seems that a high degree of reflexivity already existed in relation to this issue.
While program participants agreed with item 9 ("White people should address racism for their own sake rather than just to help Indigenous people") at time 1, their agreement was significantly stronger at time 2. Greater agreement with this statement is also consistent with a positive White Racial Identity. For whites to develop positive, non-racist definitions of whiteness, Helms (1993) argues they must become consciously aware of the ways in which racism works to their advantage so that they may make a deliberate effort to work against white privilege.
Compared to time 1, intervention group were more likely to disagree with item 11 at time 2 ("Some policies in Indigenous affairs that are now thought to be fair and just will be considered racist by future generations"). The finding was unexpected in that the program aimed to demonstrate how concepts of race and racism are subject to continual contestation and reinterpretation, and that, for this reason, past policies and practices are now often considered to be racist. Even more perplexing was the finding that control group participants agreed with this statement significantly more at time 2 than at time 1. Further research is required to determine the reason for this unexpected finding.

Culture, Structure and Agency
Item 2 assessed respondents' attitudes towards the role of culture, structure and agency in explaining Indigenous ill-health and social disadvantage. Among the intervention group, there was less agreement with item 2 ("To improve Indigenous health we must encourage the restoration of Indigenous cultures") after the program, with the average response changing from somewhat in agreement to ambivalent. This indicates an altered view of culture; from perceiving culture as something that can be 'restored' to viewing it as something that is in a continual process of change. Following the program, participants may have considered it simplistic to view Indigenous health improvement as synonymous with 'restoring' Indigenous culture.
For item 12 ("Indigenous health and social outcomes can be improved without Indigenous people changing their culture"), respondents moved from an initially ambivalent average response to disagreement. By disagreeing more readily with the statement at time 2, it appears that respondents were more likely to endorse the view that cultural change is inevitable. Prior to the program, respondents may have been uncomfortable with the idea that cultural change in Indigenous communities is required to improve health and social outcomes. The observed shift in attitude may reflect a more reflexive approach to cultural change. ISSN 2327-5510 2014 Items 4 (The 'bad' Indigenous people in our society are a product of discrimination and marginalization), 10 ("An important contributing factor to the poor health of Indigenous people is that they have different priorities to non-Indigenous people, such as family obligations or strong ties to their ancestral land"), 15 ("Indigenous people want the same as other Australians-a nice house and car, healthy kids, regular income and leisure time") and 17 ("It is racist to suggest that Indigenous people don't care about their health") did not show significant change over time. The average response to item 10 was disagreement, ambivalent for items 4 and 15, and agreement for item 17. The responses to items 10 and 17 are consistent with reduced RA, while the ambivalent responses to items 4 and 15 are consistent with a relatively more reflexive understanding.

International Journal of Social Science Research
In contrast to items 2 and 12 which ask participants to comment on Indigenous health and social outcomes (and where a change in the intervention group was observed), a commonality between items 4, 10, 15, and 17 is that they asked participants to make a judgment about Indigenous people. It may be that participants are more likely to report change on items about inequality and structural influences as opposed to those that require a judgment about Indigenous people and their agency. As aforementioned, Kowal and Paradies (2005) argue that as white antiracists are concerned with avoiding 'victim-blaming,' they are generally more comfortable with attributing Indigenous disadvantage to structural influences as opposed to the cultural practices of Indigenous people. Although the RCIPD program encourages antiracists to be reflexive about formulations that may be considered 'racist' and thus dissonant with their antiracist identity, this was not evident in responses to these items.

White Racial Identity
Item 16 changed in a way that was consistent with White Racial Identity (WRI) theory and indicates that participants further developed their WRI as a result of the program. Compared to time 1, the intervention group agreed more strongly to item 16 at time 2 ("White people need to embrace 'whiteness' as an important part of who they are"). A positive response to this item may demonstrate a developing ability to accept a white racial identity despite the discomfort that the acknowledgment of white privilege produces for white antiracists.
Items 1 and 6 also changed significantly as a result of the program. However, they changed in the opposite direction to that expected under WRI theory. Item 1 ("Being part of a multicultural community is a must for me") was taken from the WRI scale, and agreement with this item is thought to be an attitude characteristic of the WRI autonomy stage (Helms, 1990). However, the intervention group moved from stronger to weaker agreement as a result of the program. It is possible that including the word 'must' created considerable ambivalence as participants' struggled to reconcile their desire to be part of a multicultural community with the reality that, due to socio-economic disparities, they live in communities that are relatively homogenous in term of race/ethnicity. Seeking an active engagement with individuals from different races in a multicultural community (especially when expressed henas a 'must') could be perceived as a tokenistic gesture related to maintaining white antiracist identities, hence further reducing agreement with this statement. Thus, participants International Journal of Social Science Research ISSN 2327-5510 2014 may have gained an enhanced reflexivity about how their lived experiences contrast with their antiracist ideals.
Item 6 ("In most situations, I express my honest opinion when an Indigenous person is present without worrying if I will appear racist") was taken from the WRI scale and is considered a behaviour associated with the WRI autonomy stage (Helms, 1990). However, similar to item 1, respondents did not agree as strongly with this statement at time 2. Particularly as the item was expressed in the present tense rather than as a behavioural intention, participants may have agreed that they should express their honest opinion but increased reflexivity allowed them to recognise that they don't do so in practice.
This interpretation of the response to item 6 can only be tentative at this stage, as it is not possible to tell whether program participants were reporting on what they would ideally do or what they actually do. In future research, it would useful to differentiate between ideal, actual and intended behaviours. As utilised in other research (Monteith & Voils, 1998) it would be useful to include an item that specified whether or not respondents would express their opinion in an ideal situation, in conjunction with a separate item that asked respondents to indicate their most likely response in an everyday situation.

Conclusion
This evaluation aimed to determine the effectiveness of the RCIPD program in encouraging RA, a novel approach to antiracism. Study findings suggest that, in relation to its aims, the RCIPD program positively influenced the program. Furthermore, the development and implementation of the RAS-I scale was partially effective as a quantitative measure. As predicted, participants strongly disagreed with the antiracism items at both times. These results supported the validity of the other study findings. The responses of eight statement items (three WRI items, three racism and whiteness items and two culture, structure and agency items) changed significantly at time 2 for the intervention group, with no changes among control participants. It is thus highly likely that these changes resulted from the RCIPD program.
Conversely, four whiteness and racism items did not show change. Furthermore, responses to items assessing affective reactions suggest that these reactions were not altered by the program. Further research is required to determine if the lack of change across these items is replicated in other cohorts of course participants. The results also indicate that White Racial Identity and RA are related but distinct concepts. Future research could aim to discern the differences between RA and WRI as well as distinguishing between ideal and actual behaviours. It will also be important to examine unexpected findings in further research, especially changes in items (e.g., item 11) that were in an unanticipated direction.
Whilst the RCIPD program appears to have been useful as a strategy to encourage RA, a number of limitations were evident in this evaluation. It is not clear which practices employed by the RCIPD program resulted in change. Further research could establish the effectiveness of the discrete sessions within the training program by comparing the effect of delivering various session combinations. Although the results detailed above were evident a International Journal of Social Science Research ISSN 2327-5510 2014 month after the program, a survey administered 12 months after the program could be used to assess long-term change (Kernahan & Davis, 2010). The evaluation of future RCIPD programs will also help to assess the validity and reliability of the findings from this initial study.
Nevertheless, the evaluation of the RCIPD program contributes to the limited literature on reflexivity in antiracism. By establishing and investigating RA as an alternative conceptual framework, it is hoped that this research will prompt further exploration of the concept beyond the context of Indigenous affairs and in other national contexts. Both quantitative and qualitative research is required to further refine RA and explore its potential as a paradigm for conducting and assessing antiracist training and practice.