To inform the framework, USIP conducted a literature review on each of the target locations to get a thorough understanding of each area, relationships within and between communities, past experiences with conflict, and the dynamics in the post-ISIS period. The literature review highlighted the following main issues impacting target location dynamics: land disputes; lack of effective governance and service delivery; displacement, returns, and demographic shifts; security concerns; and lack of intra- and inter-community trust. Starting in Wave 7, USIP expanded data collection to Mosul district and incorporated impact of climate change on conflict dynamics.

USIP then developed indicators for each of the thematic issues based on metrics from the Measuring Progress in Conflict Environments framework. USIP worked with Iraq experts and its research contractors, Social Inquiry and RM Team, to develop survey questions linked to these indicators and adapted to the context of the Iraqi target locations. Over successive waves of data collection, USIP also infuses context from its work on the ground—including feedback from local stakeholders such as government and community decisionmakers—to inform the data framework.

• The CSMF indicators are measured using quantitative population surveys. Despite the quantitative nature of the survey, questionnaires are designed to engender detailed responses.
• The scope of the population survey aimed to cover and statistically represent the most demographically represented ethno-religious groups in northern and western Nineveh.
• Given that a significant proportion of some population groups were still in displacement at the time of the CSMF’s establishment, in addition to returnees, the survey targeted IDPs originally from these areas but displaced elsewhere.
• The data collection design consists of on average between 90-100 interviews with the different ethno-religious groups living in the target sub-districts. These figures guarantee a sufficient statistical significance of the data. See the Methodology Notes section for more about the sample’s confidence interval and margin of error.

Waves

Data collection occurred through household surveys in five districts—Hamdaniya, Sinjar, Tal Afar, Tal Keif and Mosul—over seven waves. The breakdown of sample sizes for each round is below.

• Wave 7 – March 2023 – sample size 3,021 – gender balance approximately 71% men to 29% women, with 2,137 men and 884 women
  • Sinjar (including IDPs): 600
  • Tal Afar: 700
  • Hamdaniya: 701
  • Mosul: 1020
• Wave 6 – November-December 2021 – sample size 2,002 – gender balance approximately 67% men to 33% women, with 1,338 men and 664 women
  • Sinjar (including IDPs): 605
  • Tal Afar: 792
  • Hamdaniya: 605
• Wave 5 – May-June 2021 – sample size 2,009 – gender balance approximately 64% men to 36% women, with 1,277 men and 732 women
  • Sinjar (including IDPs): 600
  • Tal Afar: 809
  • Hamdaniya: 600
• Wave 4 – October-November 2020 – sample size 2,002 – gender balance approximately 60% men to 40% women, with 1,225 men and 777 women
  • Sinjar (including IDPs): 591
  • Tal Afar: 810
  • Hamdaniya: 601
• Wave 3 – May 2019 – sample size 1,851
  • Sinjar (including IDPs): 542
  • Tal Afar (Zummar and Ayadhiya sub-districts): 437
  • Tal Keif: 179
  • Hamdaniya (including IDPs): 693
• Wave 2 – October 2018 – sample size 1,505
  • Sinjar (including IDPs): 475
  • Tal Afar: 468
  • Tal Keif: 268
  • Hamdaniya (including IDPs): 294
• Wave 1 – February 2018 – sample size 1,100
  • Sinjar (including IDPs): 273
  • Tal Afar (Rabia and Zummar sub-districts): 390
  • Tal Keif: 180
  • Hamdaniya (including IDPs): 257

Differences across Waves

• Based on lessons from and feedback on Waves 1-3, USIP and Social Inquiry adjusted the CSMF framework starting Wave 4 to include new indicators and questions on governance structures and public participation; reconciliation, peace agreements, and prospects for accountability; functioning of civilian administration and law enforcement; and community perspectives on returns of those still displaced, with a specific focus on attitudes towards those families with perceived ISIS affiliation added starting in Wave 5. Waves 1-3 were organized according to 48 indicators, whereas starting in Wave 4, waves are organized according to 92 indicators. With Wave 7, the theme of Climate Change was incorporated and new indicators around governance and security were added, bringing the total number of indicators to 103.
• USIP and Social Inquiry also adjusted the sampling strategy starting Wave 4 to allow statistical analysis at the sub-district level, whereas in Waves 1-3 the sampling breakdown was at the district level. The adjusted sampling method will allow longitudinal comparison of dynamics over time disaggregated by gender, age, urban-rural classification, socio-economic status and ethno-sectarian identity.
• The sample sizes increased across waves. Starting Wave 4, the sample size increased substantially and included all sub-districts of the three districts covered; however, Tal Keif was not covered starting Wave 4. Sampling also increased across Waves 1-3 because data collection expanded to areas that were previously inaccessible due to security dynamics. In particular, Sinjar Center (Sinjar district) became accessible in Wave 2, and Ayadhiya sub-district (Tal Afar district) became accessible in Wave 3. To add Ayadhiya, another location had to be dropped, so Rabia sub-district (Tal Afar district) was not covered in Wave 3.
• In Wave 3, the Hamdaniya sample size was expanded to focus on dynamics between Hamdaniya residents to inform a local dialogue process implemented by USIP and its partners. The sample size increased from 294 in wave two to 693 in Wave 3.

The following section covers methodology starting Wave 4 and future rounds. For methodology notes on Waves 1-3, see PDF on the CSMF Nineveh Waves 1-3 Findings page.

The statistical representation of Waves 4, 5, 6 and 7 samples is:

1. At district level, 5% margin of error within a 95% confidence interval.
2. At the sub-district level, between a 5% and 10% margin of error within a 95% confidence interval.
3. At the ethno-religious group level per sub-district, 10% margin of error within a 95% confidence interval (only groups categorized as major and minor).

Weighting the Sample

Starting in Wave 4, sampling was stratified by sub-district and ethno-religious group to ensure large-enough sample sizes for each of these units of analysis.

However, so that survey results are representative of the actual population size of each ethno-religious group, sub-district and district, a three-step weighting methodology was applied:

• Weighting the population size for each district and sub-district adjusts the sample size to the actual population for each administrative unit. As no official or reliable population data exists, district and sub-district population size was estimated based on 2012 World Bank population estimates and the population return figures reported by IOM Iraq’s Displacement Tracking Matrix (DTM). The World Bank data is more accurate given coordination with Iraq’s Central Statistics Office, but is outdated. In particular, it does not account for significant population changes due to conflict-related displacement and other demographic movements over the last decade. By contrast, DTM data estimates these recent changes based on an aggregation of reporting from key informants. Still, the DTM data is based off of returnee movements and thus excludes any families that remained in their locations during and after the ISIS conflict. The CSMF estimates therefore take  an average from both the World Bank and DTM figures per sub-district.
• Weighting for size of each ethno-religious group within a sub-district, based off of the total estimated population in a sub-district, adjusts for the proportion of ethno-religious groups who reside there. Little data on ethno-religious breakdown exists, and figures that purport to be accurate are highly contested. Therefore the CSMF takes a qualitative approach to approximate the size of the relevant groups. The initial ethno-religious composition estimates were based on DTM returns data. These proportions were then adjusted further through discussions with the enumerator teams who reside in the target areas and belong to the different ethno-religious groups themselves. The ethno-religious groups were assigned one of three categories: major (close to half of the sub-district population); minor (smaller proportion but still greater than 10% of the sub-district population); and non-negligible (roughly 10% of the sub-district population) as shown in the table below. The major category is weighted as double the minor category, which is weighted as double the non-negligible category.
• Weighting for gender balance adjusts to represent the equal presence of both genders within the district or sub-district. 

To be statistically representative, sampling included 90-100 respondents from each ethno-religious group categorized as major or minor as reflected in the table below. Groups categorized as non-negligible were also sampled to give indicative results (not statistically representative). Percentages in the table below are as of Wave 7.

Qualitative Methods

Findings were supplemented by additional qualitative work. This included key informant interviews with government (e.g. mayors or mayor office representatives) and local civil society organizations. It also included validation sessions with community members.

Enumerator teams were from the respective district and sub-district locations, comprised members of each of the predominant ethno-religious groups sampled, and interviewed residents from their own group to engender trust. Teams were also gender-balanced. Waves 4, 5 and 6 occurred during the Covid-19 pandemic, all enumerators were provided with personal protective equipment (masks and hand sanitizer) and where possible conducted interviews outdoors or in well-ventilated spaces (e.g., respondents’ yards, rooms with open windows, etc.).

Because the CSMF directly samples Iraqi respondents from the conflict-affected target communities, data collection is challenged by constantly evolving security and conflict dynamics. Many of the themes are also highly subjective, so that while indicators were designed to be as clear and specific as possible, in some instances the CSMF uses vocabulary that is subject to interpretation. USIP welcomes feedback, including questions and recommendations, to improve the CSMF.