19TH SESSION OF THE ASSEMBLY OF STATES PARTIES
10 December 2020
Name of the Side Event: Human Rights Documentation by Civil Society: A Pathway Toward Accountability
Report by: Sanne Keijer & Zselyke Szantho, Junior Research Associates PILPG-NL
Highlights:
Technological tool developers and legal experts should work together and a dialogue between them needs to be encouraged and continued.
Professor Margaret deGuzman stressed the importance of a clear purpose for a documentation mission, especially when interacting with victims and witnesses. This allows victims to make informed decisions.
Bethany Houghton highlighted that recent documentation tools need to be analyzed from a legal perspective. Once the legal community pronounces whether or not these tools meet evidentiary standards of international tribunals, civil society and tool developers can make more informed decisions about the tools they adopt.
Summary of the Event:
Hosted by the Public International Law & Policy Group (PILPG), this event concerning Human Rights Documentation by Civil Society offered excellent insights into the current international efforts and challenges to improve human rights documentation. Since 2018, PILPG has published two reports on documentation: Documenting Atrocity Crimes Committed Against The Rohingya in Myanmar's Rakhine State (2018), and Human Rights Documentation Solutions: Human Rights Documentation by Civil Society – Technological Needs, Challenges, and Workflows; Perspectives from Documenters, Transitional Justice Experts, and Tool Developers (2020). During this ASP side event, the panel, consisting of Milena Sterio, Margaret deGuzman, Adrienne L. Frick, and Bethany Houghton, focused on the particular difficulties civil society documenters face when documenting atrocity crimes for the purposes of accountability and shared their experiences of documentation in Bangladesh/Myanmar.
Dr. Gregory Noone welcomed everyone and started the conversation by asking an important question. How can civil society establish a method of documentation which can help achieve accountability goals? This underscored the importance of strong methodology and workflow which will assist documentation. There are common struggles human rights documenters face, which highlights the need for more discussion and knowledge sharing.
Dr. Gregory Noone continued with a question on the main findings of the 2018 Rohingya report directed at Adrienne Fricke. She responded that reasonable grounds exist to believe crimes against humanity, war crimes, and genocide have been committed in Myanmar’s Rakhine State. The report is based on a large-scale investigation in which Adrienne Fricke participated as an investigator.
Dr. Gregory Noone then turned to Bethany Houghton, asking her about the Human Rights Documentation Solution project. She argued that with respect to technology, the report found that there was not one need; rather a whole host of different challenges that human rights documenters face. The report contributes to methodologies, the interoperability of combining different tools, and other wide-ranging needs. In the next phase, PILPG will co-design a tool to assist with information analysis and management.
Dr. Gregory Noone then steered the discussion back to the Rohingya investigation, asking Adrienne Fricke about methodology. Adrienne Fricke expanded our knowledge of the methodology used during the Rohingya investigation. She explained that it was unique, as they used not only criminal investigative measures, but also an epidemiological sampling-based survey design.
A total of 1024 people were interviewed in the setting of a refugee camp, which had grown to host 800,000 people. Both the choice of household and the choice of interviewee within the household were randomized. Within the refugee camp, the researchers made sure to interview each quadrant adequately in order to assure that the population is descriptive.
The methodology the report used required a fair amount of training and discipline. All individuals who participated had to be trained. This included the investigators, interpreters, and local guides. Dr. Gregory Noone returned to Adrienne Fricke’s point on the importance of methodology, by arguing that where people are failing these endeavors, is where they walk into a camp and simply interview the first 50 people who speak English. He stresses that such methods gravitate to the same people NGOs and media find. Using only these sources would skew the database
Adrienne Fricke responded to Dr. Gregory Noone’s question about which documentation tools were used. She explained that the essential tool used was a questionnaire developed by the state department. The questionnaire was based on the one used in Darfur, but an improved version of it, amended in light of all lessons learned. The investigators also have had the opportunity to review the document and other foundational documents for good subject knowledge.
Adrienne Fricke further noted that they have had group discussions on the challenges faced by all participants. This included a focus on the physical and mental wellbeing of the team and discussions on self-care in the field. Over time, this proved to be one of the most important, foundational features of the background work, as conditions were challenging.
Dr. Gregory Noone pointed out the physical difficulties investigators faced. The program required walking long distances in the heat. These challenging conditions explain why the importance of self-care and safety was highlighted repeatedly. He also added that another important aspect was the composition of the smaller teams, which included both females and males, in case one needed to have a same-sex conversation. He then asked Milena Sterio for a further account on methodology.
Milena Sterio described that one essential point about the methodology has been the importance of controlling the sample. This ensured that the legal analysis was based on accurate information. This included, as previously mentioned, not only talking to the first 10 people the researchers encountered. It also involved excluding minors or people who have not had first-hand accounts of the atrocities. Additionally, the report only included people who fled the region since October 2016. Dr. Gregory Noone then asked Milena a question on which part of the methodology has been most scrutinized. Milena responded and argued that methodology is so crucial since if the methodology is wrong, the entire report is based on flawed information. If the report concludes that there is reasonable ground to believe that genocide or crimes against humanity were committed, and the methodology is flawed, that means the entire report is based on skewed information and its findings are also inadequate. For example, a non-randomized sample, including respondents who do not have first-hand information, is an instance of flawed methodology.
Dr. Gregory Noone confirmed that appropriate collection and coding of the information is essential to its analysis. He highlighted that in the process of coding, consistency has been key. As there was a big pool of information, researchers needed to make sure that they code the same way so the results were comparable. This required continued training.
Dr. Gregory Noone then referred to another interesting issue, asking Margaret deGuzman to describe some of the pitfalls she sees in human rights documentation. In her response, she focused on the general level of the type of challenges that can be addressed for any investigation, which in turn can lead to better outcomes. The most important question to ask is what the ultimate purpose of collecting the information is. That can often be challenging, as researchers might not always know going into the investigation what the purpose is. Still, keeping the purpose in mind is crucial in several aspects.
For instance, responding to questions can be traumatizing for victims, especially if they are not given an idea of what is going to happen with the information. Knowing what the process aims for can be a factor in deciding whether or not they want to engage in the interviews. Knowing the aim of the research is also crucial from the point of view of collecting the information. Here, knowing what the data is going to be used for is important in deciding some features of data collection. Collecting evidence in a court of law is different from a truth commission, engaging in a different level of detail.
The overarching question is whether the research is primarily for the population that has suffered and/ or has a more global purpose. This determines what is being taken into account. Research with global purpose might also have different goals than the one with local objectives; these goals might clash with each other. Global society organizations have to reflect on this in order to know how to deal with strategizing the collection of information.
Dr. Gregory Noone asked Bethany to respond with some ideas on different understandings of chain of custody, from the perspective of the newly published report. In response to Dr. Gregory Noone’s question on the different understandings of chain of custody, Bethany provided an interesting insight. She explained how tool developers conceptualized chain of custody, and how lawyers were conceptualizing chain of custody differs greatly. Although there is a common goal of accountability and justice, when it comes down to finer details, there is tension in how you go about doing it.
She added that we need to be critical of encryption, as encrypted data may always be in the wild. It should not be seen as a fully safe solution. Here, Bethany also highlighted multiple times that we need more opportunities for dialogue between the technological and the legal side.
A question was then raised from the audience, on how the local guides were selected. In response, it was mentioned that the selection process of local guides was not that important for the investigation. Interpreters were more important. Local guides were not involved in the collection of data themselves.
Adrienne Fricke highlighted the key role of the interpreters, who served as intermediaries between the respondents and the investigators. Their work was highly demanding from several aspects. The emotional toll of listening to and translating the respondents’ recollection of trauma can be significant. Additionally, interpreters had to have a good working relationship with their investigators, as the teams oftentimes had to do iterative work. In the recounting of stories, vocabulary and verbs matters, the description of colors and patterns can prove to be essential when describing an insignia on a uniform. Interpreters were selected on the basis of their understanding of dialect, keeping in mind the need for both female and male interpreters. Additionally, their understanding of culture and their proximity to it was essential in collecting data of high quality.
In turn, Dr. Gregory Noone confirmed the relevance of common understanding, especially in the case of weaponry and geographical locations. Having an accurate description of the weapon or the location is essential, yet, highly challenging. For instance, if respondents speak of distance in walking hours, investigators might have trouble with translating that into actual distances and location on maps. Lastly, Dr. Gregory Noone explained how traumatic some of the issues they witnessed were. Therefore, a psychologist was included in the team to help them deal with the severity of the things they were witnessing and hearing.
Dr. Gregory Noone asked Milena Sterio how we should navigate these different sources, to hold people accountable. In response to Dr. Gregory Noone, Milena Sterio highlighted that if accountability is not the overall goal, you run into trouble. A problem for PILPG, but in particular for local organizations, is that they may be unaware of relevant evidentiary standards. What can be helpful here, are particular technological tools that are designed to collect evidence that meets evidentiary standards. She pointed towards a knowledge gap between the legal community and civil society. Margaret deGuzman pointed out that we can acknowledge that the field has come a long way and that civil society has become a big part of that. Margaret deGuzman pointed out that she hopes the rise of technological tools can make it easier to meet evidentiary standards. These tools and the work PILPG is doing in this regard is very important to make the process easier, especially where evidentiary standards are so complex.
Bethany then explained that winds of change are pushing for multiple tools being developed to help data collecting. These tools, such as EyeWitness and Proofmode, often automatically add metadata. However, what is missing, is that from a legal perspective we need to look at these tools and evaluate whether or not they meet legal evidentiary standards. From the side of the tool developers, we need to help them by confirming. This will result in more Civil Society members using these tools, as they will be ensured of their validity, and can hereby make more informed decisions.
To sum up the event, Dr. Gregory Noone asked all speakers to provide us with their last thoughts, or key insights tool developers and people in the field should take away from this conversation. Adrienne Fricke responded and said that what ends up being the most useful from the perspective of documentation, is what seems more boring in the moment. How did you know, which sensory details did you experience, what was the time of day? When listening to the difficult narratives of atrocities, it is easy to become overwhelmed by the actions, Adrienne Fricke added. We need to remember what will matter to the investigator in twenty years, as those are the details that may not feel important in the moment.
Milena Sterio added on with her final thoughts, highlighting that we need to know what the evidentiary standards are. Tool developers need to work together with lawyers, and civil society should be working together closely with the international community if accountability is the goal. To sum up, Milena Sterio argued that there is no one size fits all solution, and we need to be creative to overcome these challenges.
As a final thought, Margaret deGuzman pointed out that she would like to see more overarching coordination happening at the international level. Historically it was run out of the United Nations. Recently, the UN has not been able to play as active a role for various reasons including a lack of funding and a lacking level of technological development, but would be great if either UN or other organization could serve as a HUB for this civil society documentation work. This may be something to think about in the future. Margaret ended by pointing out the tension between universality and pluralism.
As a final thought, Bethany Houghton reinforced one of the key insights, which was the importance of starting to think where the end will be, accountability. What really is the end goal of documentation, having that goal in mind is critical at the beginning of any documentation effort.
Dr. Gregory Noone concluded the event by thanking the panelists and participants, and added that he admires and applauds all people involved in the field.
Read the reports here:
https://www.publicinternationallawandpolicygroup.org/rohingya-report https://www.publicinternationallawandpolicygroup.org/hrds-phase-i-report-launch
A recording of the event can be found here:
https://www.publicinternationallawandpolicygroup.org/hrdocs-icc-asp