by Amina Abdu, Palak Jain, Priyanka Nanayakkara, Ivoline Ngong, Jayshree Sarathy
The Differential Privacy Beyond Algorithms Workshop took place at the OpenDP Community Meeting at Harvard University on August 23, 2024. The workshop was organized by Rachel Cummings, Gabriel Kaptchuk, Priyanka Nanayakkara, Elissa M. Redmiles, and Jayshree Sarathy. The goal of the workshop was to bring together participants who are thinking about aspects of differential privacy (DP) deployments beyond algorithms and technical tools. In particular, the aims were to (1) foster community around these research efforts, (2) develop community principles around doing interdisciplinary DP research, and (3) generate future research directions around sociotechnical aspects of deploying DP.
The workshop began with a discussion of shared community principles, which are summarized below. Next, participants heard engaging short talks by Joe Near on challenges around usability and setting standards for DP, Ryan Steed on adoption of privacy-preserving analytics on the ground, and Aiping Xiong on communicating DP concepts to data subjects. Recordings of these talks are available here.
Afterwards, participants engaged in a series of breakout sessions—(1) implementing DP in practice, (2) evaluating DP deployments on-the-ground, (3) facilitating engagement around DP, and (4) crafting law and policy for responsible DP deployments. These discussions were led by Amina Abdu, Palak Jain, Priyanka Nanayakkara, Ivoline Ngong, Jayshree Sarathy and are summarized below. These discussions outline future directions for research and practice.
Community principles
One major goal of this workshop was to come together to build principles for this nascent community. The workshop organizers came up with the following guidelines as a starting point.
1. Engage in and encourage rigorous, effective interdisciplinary work
– Engage deeply with other disciplines. Bring on collaborators who have expertise in other fields as well as non-traditional experts who can share real-world experiences.
2. Be aware of how our personal perspectives can impact and enrich our work
3. Be thoughtful and precise with how we represent differential privacy
4. Welcome and support junior members of the community
The organizers then led a discussion with participants about the guidelines, with the goal of building shared principles as a community. The discussion was positive and affirming, highlighting the excitement around the opportunity to build and shape a new research community.
Participants were pleased to see a focus on interdisciplinary work as part of the principles. One participant added that there is a tendency to view technical disciplines as “above” social science or humanities, and interdisciplinary collaborations sometimes devolve into the technical contributors “teaching” others. To caution against such implicit hierarchies, the participant suggested that all disciplines and parties should be viewed as having valuable contributions to this growing field.
A second participant added that it is critical that this community includes people from all parts of the DP deployment pipeline. In addition to computer scientists and privacy engineers, we should reach out to lawyers, policymakers, data users, and data subjects and engage in collaborative research.
Along the same lines, multiple participants emphasized that this community should be supportive of work that does not typically gain merit in academic spaces, such as practical playbooks and policy work. Having impact requires going beyond traditional modes of academic research and engaging in translational work that can shepherd DP into practice.
Finally, one participant cautioned that in the DP community, we often feel that we have the right definition of privacy. This participant encouraged the community to be open to engaging with other understandings of privacy. Such open-mindedness is critical for advancing tools that speak to the needs and perspectives of diverse communities.
Facilitating engagement around DP among data subjects, data users, policymakers, and other relevant parties
The breakout group on facilitating engagement around DP discussed participatory DP deployments and how to create effective transparency.
Deploying DP in a participatory way refers to involving relevant parties, like data subjects, in decision-making about specifics of a DP deployment. This includes, but is not limited to, choices around parameters and deployment model. The group discussed whether all responsible DP deployments must be participatory. To make this question concrete, the group raised two scenarios: in the first, data are already used (e.g., to train a model on users’ data) with no privacy protections in place and in the second, data are collected and privacy protections are offered to data subjects as assurance. The group felt that in the first scenario, applying DP may represent a net improvement over prior practices and thus, engagement may be less necessary. On the other hand, the group saw a pressing need for participatory deployments in the second case, where data are being freshly collected as the protections might influence data subjects’ willingness to be included in the dataset.
In addition to discussing the need for engagement, the group identified challenges to participatory deployments. First, if there are many parties, it may not be feasible to directly involve everyone. In such a case, it could be helpful to consider a system where representatives are elected or otherwise chosen to advocate for the needs of different groups. Second, receiving input from a wide range of parties requires abstracting some technical implementation details for some, raising questions around how to do so effectively.
Along these lines, the group discussed challenges and opportunities for effective transparency around DP deployments. Transparency for security professionals might involve providing enough information to audit a DP system in order to verify that privacy guarantees are not violated. Other audiences will likely require different abstractions of such information. In particular, the group felt that transparency, or explanations, for data subjects, should empower them to determine whether the privacy protections they desire align with what DP offers. To best learn what different parties need, the group also suggested not assuming needs, but rather directly learning from members of different groups what kinds of information they require to assess privacy protections.
Implementing DP in practice, technical and sociotechnical challenges
The breakout group on implementing DP in practice centered around the complexities of deploying DP in real-world environments, particularly technical and sociotechnical challenges. One key theme was the question of when DP should and should not be used. While DP offers strong privacy guarantees, its application may not be viable for small or marginalized groups, where the noise introduced could obscure meaningful insights. A related technical challenge occurs when organizations remove user identifiers early in their data pipelines – this makes it impossible to later track and properly implement DP’s privacy protections, as the system can no longer distinguish between contributions from different users.
Cost was a recurring topic throughout the discussion. Organizations face financial pressures when considering DP, as applying it can increase storage and processing requirements, slowing down data pipelines. The group drew parallels to encryption, where high costs were eventually accepted as necessary for best practices.
Similarly, the group discussed the need for concrete evidence to demonstrate that the benefits of DP—such as protection against privacy attacks—outweigh concrete costs. Simulating both the monetary impact and risks of privacy attacks was seen as crucial for convincing stakeholders, especially those less familiar with DP’s theoretical foundations, about the viability of incorporating DP into their systems.
Another important point raised by the group was the need for transparency and careful evaluation of DP’s impact, especially in sensitive areas like healthcare, where outliers in data are critical for drawing insights. In some cases, DP might limit the ability to make accurate inferences, leading to a loss of trust among data users. The group also highlighted the importance of integrating DP with other privacy-enhancing technologies and developing better threat models that take into account the variety of adversarial scenarios and auxiliary data available to potential attackers.
In summary, while DP offers the potential to release more data than traditional methods, this breakout session underscored the importance of careful planning, evaluation, and transparency in deployment. The challenge lies not just in applying DP, but in ensuring that its implementation aligns with the needs of various parties without compromising utility or trust.
Evaluating DP deployments on-the-ground
The breakout session on evaluating DP deployments on-the-ground explored challenges in assessing DP’s effectiveness in real-world applications. Key themes included communicating DP’s role in an ecosystem, empowering the general public to form opinions on the technical and legal systems that are meant to protect them, and defining the role of experts in critiquing and encouraging deployments.
The group acknowledged the complexity of communicating the scenarios in which DP is relevant, noting the need to bridge perspectives between companies, policymakers, and computer scientists in order to determine what constitutes a “correct” or “reasonable” deployment. They discussed the creation of a catalog of potential attacks, noting that such a library would need to consider a wide range of potential vulnerabilities. They noted that the gap between realized and optimal attacks could mean that the use of such a library would result in the underestimation of privacy risks. They also discussed the importance of balancing the usefulness of structured tools for communicating privacy risks with their usefulness for misrepresenting or overclaiming protections of systems.
When discussing on-the-ground deployments, the group felt that empowering the general public to evaluate their own privacy expectations—and whether they’re being met—is a key challenge. Drawing an analogy to sunscreen (SPF) ratings, the group proposed that expert-created yardsticks can be extremely useful when carefully developed and effectively communicated. The group suggested developing similar yardsticks for privacy protections. Relatedly, the group explored the idea of translating formal DP definitions into more tangible concepts like attack success rates and legal implications. This translation was seen as crucial for bridging the gap between technical implementations and real-world privacy expectations.
Lastly, the group debated approaches to critiquing DP deployments, noting the importance of constructive feedback while recognizing the delicate balance between maintaining rigorous standards and discouraging ongoing efforts. They suggested that DP experts develop norms around the level of criticism they provide based on where a deployment is in its life cycle in order to create some consensus and consistency in their communication. They also discussed the importance of approaching public criticism with care, noting that experts should consider how their words might be used to support arguments they do not endorse.
In summary, the group recognized the need for multifaceted approaches to evaluating DP deployments, discussing how one might go about involving various parties, considering different perspectives on privacy, and developing more comprehensive and accessible evaluation methods.
Crafting law and policy for responsible DP deployments
The breakout group on law and policy discussed the role that policy should play in guiding responsible data privacy practices.
While DP has been discussed in recent policy proposals, significant uncertainty remains around how and when DP should be deployed to meet privacy goals. Importantly, technical implementation decisions like model and parameter choices are complex and value-laden. While policy could help to address these uncertainties, the group agreed that specific technical details should not be written into regulation. Because of the importance of context, the need for expert judgment, and the potential for technical advances, a one-size-fits all approach to privacy standards was seen as infeasible. Rather than bright lines, the group saw promise in natural language principles that might outline key desiderata and help frame privacy goals. For example, policy might dictate whether data privacy ought to be seen as a minimum requirement or a social choice problem between privacy and accuracy. However, to address the challenges of interpreting these natural language regulations, the group identified the need for technical guidance to help connect high-level goals to specific designs.
Moreover, the group agreed that not all DP implementations are created equal in terms of ensuring privacy. To ensure that policy goals are met, the group discussed the need for robust auditing infrastructure. However, the details of such an auditing infrastructure remain unclear. The group discussed whether the need for oversight might be addressed by private auditing companies, by a government regulator, or by explicit transparency requirements.
Overall, the group agreed that DP holds promise for helping policymakers meet privacy goals. However, this task requires striking a balance between ensuring strong privacy protections and enabling flexibility across contexts and in the face of technical advances. To strike the right balance, the group saw promise in learning from both successes and failures in cybersecurity, environmental protection, and other domains where effective policy ultimately relies on technical decisions and expertise.
Interested in joining the community?
If you are doing work on or have an interest in sociotechnical aspects of deploying DP, please join the community! Joining entails including your profile on the website (SIGNUP HERE) so that others (researchers, journalists, etc) can find you and learn more about your work.