# PathCheck: Privacy-Preserving Crowdsourcing for Citizen Engagement

![](/files/-MYy5lbJVZu8UnRzaRjQ)

**Crowdsourced citizen data** can complement and connect incomplete and fragmented data that cripples pandemic responses. But, large-scale participation requires citizens’ trust and engagement. We solve this with a unique **“NoPeek” privacy** and **personalization** delivered with a free, open-source toolkit for governments and large institutions.

![](/files/-MYyRUo2S2AImYF_M7IJ)

{% embed url="<https://youtu.be/yRDc1pxctN8>" %}

{% embed url="<http://pathcheck.org/>" %}

{% content-ref url="/pages/-MZJukvnYmBpAAdznSmM" %}
[Frequently Asked Questions (FAQs)](/frequently-asked-questions-faqs)
{% endcontent-ref %}


# Background

![](/files/-M_uczGTu6-u_TO0Zuor)

For hurricanes, we use satellites to sense the progress of the hurricane and then give instructions to help citizens avoid the storm.  10 years ago, during the 2009 flu pandemic, the best we could do was to sense the extent of contagion and overall prevalence using surveillance technologies and then create and enforce regulations to address these challenges.

Today smartphones are our new sensors. We can gather data from smartphone applications to create personalized alerts for everyone in the system. &#x20;

![](/files/-MZK1RIRCJ2-gdbwownQ)

Imagine John, a refugee in Sao Paulo, Brazil. He's at a social event and is exposed to a virus. After some time he has an onset of symptoms and gets tested. Later he is treated and eventually vaccinated. At present, most of the information about John’s user journey is hidden or disjointed from public health.

As public health uses test-trace-and-vaccinate programs to predict virus spread and alert citizens, significant gaps emerge in the data. Some countries resorted to draconian measures to harness citizen data for filling the gaps. With a surveillance state, they managed to tame the pandemic within three months.\ <br>


# Problem

Our goal is to fill the **gaps in public health knowledge** by crowdsourcing data from citizens to improve pandemic responses.&#x20;

![](/files/-MYy5lbJVZu8UnRzaRjQ)

In a pandemic, data is one of the most important tools for policymakers, public health officials, and health system agents. Citizens need to have easy access to exposure alerts, symptoms, testing, treatment, and vaccination, but much of this journey is not shared with public health and remains invisible to the health system.

These gaps in data gathering, especially about activities, contacts, and status of citizens are one of the key challenges for an effective response. **The data-scarce planning leads to inefficiencies, lives lost, and socio-economic costs**.&#x20;

Technological app-based solutions can be used to fill in gaps, but they currently suffer from two key problems that make citizens uncooperative and disengaged:&#x20;

1. A **lack of incentives** for users to adopt and engage with the app
2. Citizen’s **privacy concerns** and the fear of the government becoming a surveillance state.

How can we achieve crowdsourcing, privacy, and personalized engagement in a pandemic? And how can we simultaneously provide planning tools for public health by making citizen data available in real-time?

<br>


# Stakeholders

We have three major stakeholders:&#x20;

1. **Public health agency (PHA) planners at both the country and regional level**
2. **Organizations with large campuses**
3. **Citizens (including vulnerable populations)**

**1. Public health agency (PHA) planners at both the country and regional level**

For PHAs, we create a data-rich stream of citizen activity to fill the gaps and complement other data-gathering efforts to understand case rates, the severity of symptoms, equitable deployment, and effectiveness for vulnerable populations.  &#x20;

**2. Organizations with large campuses**

For organizations with campuses (i.e. universities or large employers), the decision-makers usually take the lead in adopting innovation and expensive solutions to ensure safety, encouraging adherence to procedure (e.g. frequent testing), maintaining continued operations, and managing incentive programs (e.g. automatic sick leave). A significant frustration amongst these leaders is waiting for city or state solutions.

**3. Citizens (including vulnerable populations)**

For citizens, early exposure alerts, personalized risk scores for activities based on their health conditions, and visualizing local pandemic spread are critical. For vulnerable populations, our solution works for citizens without a smartphone. We developed a [paper-based QR code](https://vaccine-docs.pathcheck.org/) with cryptographically tamper-evident digital signatures for citizens to engage in the test-trace-treat-vaccinate ecosystem without disclosing any personally identifiable information.

Further, for public planners, we have already worked with **dozens of U.S. states and regions** to understand their covid response needs. **Seven** have deployed PathCheck open-source Exposure Notification software that is now used by millions. We use these relationships as well as the fact that we are a **trusted non-profit spinout of MIT** to understand their future needs about anticipating and responding to the next pandemic and any future COVID-19 spikes.

We actively seek **feedback** on ideas, designs, and prototypes from planners and residents to prioritize our feature development, reduce deployment risk and maximize health impact. For residents, we especially work on **willingness-to-use** (incentives, UI/UX, etc.).

We have deployed our exposure notification app in **seven** places - <https://www.pathcheck.org/en/covid-19-exposure-notification-app>

{% embed url="<http://pathcheck.org/>" %}

<br>


# Solution

**Our solution** - A novel open-source computational privacy software from MIT and PathCheck that captures crowdsourced health information, analyzes it for public and precision health, and engages users via personalized recommendations. We provide computational techniques to handle the five steps of the pandemic progress - **exposure, symptoms onset, test, treatment, and vaccination.**

![](/files/-MYy6GXxzXywRyEvEPdx)

Our toolkit provides services and computational methods at each step

1. **Exposure** - We provide a privacy-preserved solution for exposure notification and contact tracing.&#x20;
2. **Symptom onset -** The users can upload their symptoms and keep track of how the disease is progressing. Furthermore, they have access to download the symptom trajectory of other users in the same demographic and comorbidities group.&#x20;
3. **Test** - We provide a solution to find the nearest testing sites, schedule testings, and also create verifiable credentials to share testing results. Further, if the user agrees, they can share the data (in a privacy-preserving way) with the health officials who can learn more about the population level trends of the disease.
4. **Treatment -** The users can upload their ongoing treatment and what their recovery trajectory looks like. In addition, similar to the symptoms curve, the users can also access the treatment progression curves of the relevant group (based on demographics, comorbidities, etc).
5. **Vaccination** - We have the solution for vaccine scheduling, second dose reminder, adverse event reporting, etc along with verifiable and tamper-proof credentials. Moreover, we offer this with the help of low-tech solutions - paper credentials as well.

Each of these is explained in detail in the Technologies section. Furthermore, in each of the above 5 stages, our technology will be able to help governments and as well as public health agencies with privacy-preserved aggregated data from the crowdsourced toolkit.&#x20;

All the above 5 stages have 3 fundamental building blocks in them, which we explain as the **"Three step process - Capture, Analyze and Engage ".**

{% content-ref url="/pages/-MZJukvnYmBpAAdznSmM" %}
[Frequently Asked Questions (FAQs)](/frequently-asked-questions-faqs)
{% endcontent-ref %}


# The Three Step Process

Our privacy-first data-driven approach can be broadly segregated into three main stages - capture, analyze, and act.

![](/files/-MYPw0BOCsrNGcoCYy4L)

Our technology stack uses the “Capture, Analyze, and Engage” loop.

**Capture:** (i) background (age, health history, ...) (ii) status (symptoms/tested/vaccinated, non-user data: mobility data, previous disease data, vaccination trends, social media data) and (iii) Activity: (sensor trail e.g. GPS/BlueTooth trail, ..)  All using NoPeek protocols to preserve privacy.

**Analyze:** This module uses federated learning modules without looking at raw, individual citizen data.  It uses differential privacy, multi-party computation, and split learning for analysis and prediction. It also utilizes novel methodologies such as GPU accelerated Agent-Based Models and representation learning for multimodal data.

**Engage:** This module delivers personalized Information (risk score, cases nearby, precautions, ...), test or vaccine credentials, creates nudges, supports gamification, incentivizes safe actions, and makes recommendations specific to each citizen. Receiver privacy is maintained. With NoPeek, the server never discovers what message was shown to any citizen. <br>


# How do we solve the grand challenge?

Our solution **directly addresses this grand challenge** in public health as follows:

1. Respond to **emerging unusual symptoms** (symptom reporting and heat maps) and identify approximate heatmaps of the location of sick individuals.
2. Respond to **spread** (encourage citizens to test after exposure, automated alerts to close contacts after positive tests and monitoring of quarantine) to break the chain of infections.
3. Respond to **lack of early knowledge** about treatment plans (we gather treatment and outcome info directly from citizens).
4. Respond to **vaccination needs** (convey eligibility, track and remind the second dose, support upload of authentic adverse side effects, and create private vaccine credentials).
5. Respond to the **needs of vulnerable populations** with paper credentials (monitor equitable deployments, communicate with citizens who are reluctant to engage with overzealous law enforcement).
6. Respond to **socio-economic pressures** by making it highly affordable for government and campuses: In a pandemic, it is unclear who should pay for the pandemic response effort. Our free app for citizens is cost-effective.


# Unique parts of our solution

Our strategy is similar to what some Asian countries used to combat the pandemic with invasive data-rich, decision making, and enforcement. Although, our solution is completely privacy preserved and ***does not let any data be shared/stored in the raw form***. While most public health solutions try to create tools for PHAs and a top-down ‘command and control’ solution, we take the bottom-up approach. We solve the problems in **technology, trust,** and **open innovation**. We are inspired by Mozilla, one of the few large-scale consumer-facing open-source software projects focused on privacy.

&#x20;Our effort is innovative and unique for three reasons.&#x20;

1. **Tech:** Unlike other organizations that use consent and anonymity as a poor substitute for privacy, we use the NoPeek approach developed at MIT. The set of NoPeek algorithms are computationally hard to breach and are being rigorously reviewed with mathematical proofs for cryptographic guarantees. We believe consent and anonymity-based approaches are prone to data security breaches. Governments are unwilling to let private for-profit or non-profit players deploy a solution for their citizens with such security risks.
2. **Trust:** Unlike commercial solutions with vendor lock, sensitive health data and guidance require transparent, unbiased, open-source solutions.&#x20;
3. **Open Innovation:** Similar to Mozilla foundation, our open-source software is an open and transparent innovation platform drawing talent and partnerships for growth.


# Core Technical Innovation

Our core technical uniqueness is innovation with NoPeek privacy, partially developed at MIT.  It has 6 notable capabilities.&#x20;

* **Social proximity:** We were the first to publish a decentralized contract tracing protocol and the first to launch a US app for privacy-preserving contact tracing (mid-March 2020). &#x20;
* **Uploading by citizens:** [NoPeek computational privacy](https://prep.pathcheck.org/technologies/privacy-preserved-ai-for-healthcare) goes beyond consent or anonymization-based data protection. Our NoPeek module is able to create latent representations that share the knowledge to solve the problem at hand, without revealing the actual data. This is achieved by extracting non-sensitive knowledge from the raw data and securely aggregating it at a population scale.  Furthermore, we make use of differential privacy and Secure Multi-Party Computation (SMPC) for user data and representation learning models. This unique technology provides an extra layer of protection based on cryptographic guarantees, which protects user data even in the event of device or central server compromise.  Citizens can rest assured that sharing information from their mobile phones will not result in any of their raw data being released to the world.
* **Downloading by citizens:** Conversely, our app can deliver personalized healthcare information to a user’s phone without the server knowing which message was delivered to which phone. In most cases, the phone downloads a large dataset partition from the server database and downselects a tiny portion relevant to citizens. In other cases, the server sends the algorithm to the phone and the phone runs this algorithm to find the answer locally.
* **Avoiding data silos:** Concerns about data privacy amongst both users and organizations lead to the formation of “data silos'' – where data becomes compartmentalized and restricted to specific groups. An inability to collect this data for analysis leads to massive inefficiencies, which in turn can lead to poor response or execution in a crisis. This not only increases costs but also sub-optimizes outcomes for stakeholders. Our approach is radically unique by breaking this chain of privacy →  data silos →   inefficiency using powerful, new computational privacy methods.
* **Beyond the limits of consent or anonymization:** Today’s data protection schemes fall short due to data breaches and illegal data harvesting that lead to issues for citizens (identity theft, financial fraud, job discrimination, targeting (e.g. for advertising, robocalls) as well as organizations (inferred trade secrets, employee poaching). It is well known that consent or anonymization-based schemes are too weak when rogue employees or state actors are involved in data breaches. To avoid data breaches, the best practice is to prevent raw data from ever reaching the server; i.e. “no peeking allowed”.&#x20;

The best of both worlds, privacy, and crowdsourcing: Traditional NoPeek solutions are simple on-device calculations with no uploads of raw data. However, they limit the possibilities of crowdsourced collective knowledge. Our open-source software based on SMPC, federated or split-learning for un/supervised machine learning, and differential privacy - enhances “NoPeek” to allow for safe crowdsourcing.  We explain more in the Privacy Preserved AI for Healthcare section.

{% content-ref url="/pages/-MXlJM8y\_haqQJ1hq6iK" %}
[Privacy Preserved AI for HealthCare](/privacy-preserved-ai-for-healthcare)
{% endcontent-ref %}

{% content-ref url="/pages/-MZJukvnYmBpAAdznSmM" %}
[Frequently Asked Questions (FAQs)](/frequently-asked-questions-faqs)
{% endcontent-ref %}


# Testimonials

Our clients and stakeholders are governments and organizations with campuses. Our product is free, open-source software with **three components:  a privacy-preserving, NoPeek consumer app; server-side components for analytics, prediction, and correlation; along with an engagement platform allowing us to send highly personalized messages**. Our business plan, then, is to charge governments and large campuses for customization, professional services, and maintenance.  This is the same business plan we have been using over the past year at Pastor Rick Foundation as well, mainly for exposure notification and vaccination solutions.

For the next three years, in year one we would like to build the NoPeek crowdsourcing app and other components. In year two, we want to run AB trials in four pilot sites on four continents.  And finally, in year three, we would like to deploy solutions on various campuses with government global partnerships.&#x20;

We are delighted that we have already confirmed planned pilots in Sao Paulo, Brazil; Alabama in the US, in the country of Nigeria, and in Mumbai, India.

In some scenarios, our supporters are looking for a bottom-up solution because their top-down solutions are not scaling. In Mumbai, they would like to complement current public health efforts.  In Alabama, we are interested in doing A/B trials with a research organization and on a major university campus, given they already have our contract risk solution.&#x20;

Moving forward we will engage with many critical partners in the pandemic ecosystem, such as Pfizer, which remains at the forefront of developing m-RNAvaccine.  They were interested in studying with us post-vaccination data coming directly from these crowdsourcing apps. Patients-Like-Me, which is typically used for the terminally ill, would be our partner and we'll be using that platform.

Please find the quotes from different stakeholders in the subsequent sections.<br>


# Academicians

PathCheck Foundation, in collaboration with The University of Alabama at Birmingham (UAB) and the Alabama Department of Public Health, deployed the official Alabama statewide exposure notification app. We are aware that PathCheck Foundation is building longer term pandemic response solutions. UAB would like to support such innovation. Once such solutions are ready, we will conduct important due diligence before conducting a pilot.&#x20;

***Prof. Sue Feldman RN, MEd, PhD***&#x20;

***University of Alabama***

"Brazil has been severely affected by the current pandemic and top-down solutions have limitations. PathCheck foundation's bottom-up approach by incentivizing citizens in tracking symptoms, contacts and vaccination is unique. Engagement with citizens is necessary.

After receiving permission from authorities, at  Unifesp Federal University of São Paulo, we are eager to run a pilot with PathCheck foundation for engaging citizens for coordination in an outbreak"

***Dr. Paulo Schor, MD, Director of Innovation,***

***Federal University of São Paulo***


# Policymakers/States/Cities

The City of Boston is using various tools to help residents become aware of vaccination plans for the city… PathCheck is building an app to help residents receive notifications of vaccine availability and keep track of their doses in a private vaccine diary. Once the application is ready in the App Store and Play Store, we will conduct in-house diligence and consider further use upon review of the application. This letter is to request Apple and Google to enable the PathCheck Foundation to build and release the application.\
\- Rita Nieves, RN, MPH, LICSW (Boston Public&#x20;

Health Commission, Interim Directo)

"MIT and PathCheck Foundation's work is empowering citizens during a pandemic for contact tracing or vaccination. It is open-source and free. We will be delighted to conduct a pilot project in Nigeria for a pandemic preparedness solution" &#x20;

\- Emmanuel Benyeogor, Nigeria CDC


# Public

**In Bahasa Melayu** "Bagus juga kalau saya boleh kekal berniaga. Dua minggu itu masa yang lama, banyak duit yang saya rugi, jadi kalau kita dapat kekal berniaga walaupun ada kes COVID-19 lebih baik, asalkan kita dapat jamin kesihatan pelanggan dan mereka yang disahkan positif untuk COVID-19 dikurung"

**Translation**: "It would be great if we could stay in business. 2 weeks is a very long time, I lose a lot of money, so if we can stay in business even if there are cases of COVID-19, that would be great as long as we can somehow ensure the health of other customers and ensure that customers who test positive for COVID-19 are quarantined"

\- Johan, Uncle Bob's Fried Chicken vendor at Kota Damansara, Petaling Jaya night market in Selangor, Malaysia


# Privacy Preserved AI for HealthCare

![](/files/-MYy6GXxzXywRyEvEPdx)

Remember our example of John the refugee? As we think about how the app supports the five stages of his journey, we also need to think about how it supports public health analysis.  I introduce Jane, who is a public health planner in San Paulo, Brazil.  Jane wants to know where sick people are.   So as John is engaging with his app and indicating his symptoms, Jane can start seeing the map of where the sick people are.

As John starts informing his close contacts through the exposure and notification app, Jane should be able to see how the contagion is spread, the people who contact John, and how they are traveling in some other parts of San Paulo.  As John takes medications, he indicates through the app, which medications are working or not, and his current symptoms. Jane can take the aggregate data and realize which medications and which interventions are most effective. What’s more, as John gets medication on certain days and in certain parts of the city, Jane and her partners can see if this implementation conforms to equity and other ethical concerns.&#x20;

Our system architecture has three components: Capture, analyze, and engage. For capture, we are building no-peek computational software. For analysis, we use privacy-preserved machine learning and NLP.  For engagement, our platform extends to the vulnerable population. As we'll explain later, we also support paper credentials where individuals can simply collect QR codes along the way that are cryptographically secure as well as an SMS-based chatbot platform.\
\
John shares his medical history, his test status, what kind of tests he took, what kind of virus variant was recorded along with this test, and his vaccination status on the app. &#x20;

We can capture John's symptom diary over several days, including how he recovered along with his treatments and medications. If John has tested positive, we also want to capture the Bluetooth exposure keys. In addition, significant sensor data from John's phone is also being collected, of course, all with no-peek privacy.

On the other hand, Jane is very concerned about the four goals we discussed earlier, which are understanding the prevalence and spread of illness, the effectiveness of treatment, and analyzing equitable distribution.

Our engagement platform allows John to see what kind of exposure alert he should receive. He understands what symptom trajectories look like over time. He can also see what the regional information is, and he can understand his own personal risk score based on activities he is planning.

**The four main technologies that we use in no-peek primarily are minimum upload exposure notification, secure multi-party computation, differential privacy, and federated or split learning.**<br>


# Exposure Notification for Automated Contact Tracing (Download only solution)

![](/files/-MYPuoQ9RrAnR2xjW2Wy)

For an on-device solution in exposure notification, an infected phone and healthy phones would exchange Bluetooth keys.  Only the infected phone uploads the Bluetooth keys to the server, and then all the other healthy phones download this global key file locally to their phone and see if there's an overlap between the keys that they have and the keys downloaded from the server. As you can see, because it's an on-device calculation and download only, the server does not know anything about John or his contacts. At the same time, if somebody did receive an exposure alert, the server does not know who exactly got the exposure alert.

We have an official exposure notification app deployed in 7 places, details can be found in the link below.

{% embed url="<https://www.pathcheck.org/en/covid-19-exposure-notification-app>" %}

![](/files/-Ma-ztMCv_XQJGKiuYUY)

{% embed url="<https://youtu.be/GXr7TxD7QKA>" %}

{% embed url="<https://youtu.be/eoLE9OXvxiU>" %}

**References**

* [**Apps Gone Rogue: Maintaining Personal Privacy in an Epidemic**](https://arxiv.org/abs/2003.08567)
* [**Comparing manual contact tracing and digital contact advice**](https://arxiv.org/abs/2008.07325)
* [**Proximity Sensing: Modeling and Understanding Noisy RSSI-BLE Signals and Other Mobile Sensor Data for Digital Contact Tracing**](https://arxiv.org/abs/2009.04991)
* [**Proximity Interference with Wifi-Colocation during the COVID-19 Pandemic**](https://arxiv.org/abs/2009.12699)
* [**Spatial K-anonymity: A Privacy-preserving Method for COVID-19 Related Geo-spatial Technologies**](https://arxiv.org/abs/2101.02556)
* [**COVID-19 Contact-Tracing Mobile Apps: Evaluation and Assessment for Decision Makers**](https://arxiv.org/abs/2006.05812)
* [**Contact Tracing: Holistic Solution beyond Bluetooth**](http://sites.computer.org/debull/A20june/p67.pdf)
* [**Contact Tracing to Manage COVID19 Spread – Balancing Personal Privacy and Public Health**](https://www.mayoclinicproceedings.org/article/S0025-6196\(20\)30424-9/fulltext)
* [**The Architecture of Trust in Contact Tracing**](https://f.hubspotusercontent40.net/hubfs/8097148/documents/evaluating-contact-tracing-apps.pdf)
* [**Adding Location and Global Context to the Google/Apple Exposure Notification Bluetooth API**](https://arxiv.org/abs/2007.02317)


# Risk Calculator

![](/files/-MYob_LGJNDH2X8FzZR2)

The next example for on-device calculation is a risk calculator. John contributes background status and activity with no-peek, and then alerts and risk scores are delivered to John also using no-peek methods. The calculator can have inputs including John’s age, comorbidities, test results, data from phone sensors, as well as external data such as mobility, the caseload in the region, and so on. Further, the calculator can also be told the activity John expects to do:  Is it very vigorous, like going to a gym and for how long, and is the gym crowded? Is it indoors or outdoors? And is John going to spend time with other people who are vaccinated?  By taking this input, the risk calculator uses machine learning to calculate John’s risk.

The server sends these risk calculations in no peek form to John's phone so that he can conduct operations locally without the server knowing the end results. This maintains John’s privacy.<br>

![](/files/-MYobyLhR9aM11O38n7q)

**References**

* [**COVID-driven Risk Profile**](https://www.researchgate.net/publication/345823059_COVID-driven_Risk_Profile)
* [**COVID-19 Outbreak Prediction and Analysis using Self Reported Symptoms**](https://arxiv.org/abs/2101.10266)
* [**Can Self Reported Symptoms Predict Daily COVID-19 Cases?**](https://arxiv.org/abs/2105.08321)
* [**https://www.symptomchallenge.org/**](https://www.symptomchallenge.org/)

{% embed url="<https://github.com/PrivateKit/CovidSymptomChallenge>" %}


# Secure Multi-Party Computation

![](/files/-MZ4H3VOZTm01tD6OhYs)

Let's look at the third method, which is secure MPC. Imagine Jane wants to know how many phones are reporting fever. John sending “one”  means he has a fever and “zero” means he does not. To do this, John’s phone sends “70” to server P on the left and the number “minus 69'' to server Q on the right. Of course, 70 minus 69 is one. The second phone sends 38 and -37. Third sends 42 and -42. The Left server adds up to 150, and the right server to -148. And that leads to the answer 2 people with fever which Jane sees without knowing about John or his fever status.

The goal behind using secure MPC in our toolkit is to **facilitate anonymous symptoms reporting** by the public, without compromising their individual privacy.

{% embed url="<https://en.wikipedia.org/wiki/Secure_multi-party_computation>" %}

{% embed url="<https://crypto.stanford.edu/prio/>" %}


# Differential Privacy

![](/files/-MYPuUjsPeAAm6-jMBYx)

The fourth method is differential privacy which works by adding statistical noise that gets canceled out when aggregating over multiple entries. So if John wants to upload his curve of symptoms -- it could be fatigue for example -- then instead of sending the green curve, which is the authentic curve, the app adds some noise to the curve and uploads a slightly noisy curve to the server. As you can see on the top left multiple such red curves appear at the server. The server does not know what the authentic symptom curves are for each of these phones. When the server adds up all these red curves, the server gets the blue curve, and that is the final estimate for the server. Jane can look at this final estimate and realize that people like John and others who are infected have these particular types of symptom trajectory. At the same time, Jane cannot know who contributed to which symptom curve but gets a precise final estimate because this has been averaged for dozens or even hundreds of people. \[1] \[2]

{% embed url="<https://en.wikipedia.org/wiki/Differential_privacy>" %}

**References**&#x20;

\[1] [**Differentially Private Supervised Manifold Learning with Applications like Private Image Retrieval**](https://arxiv.org/pdf/2102.10802.pdf)

\[2] [**DAMS: Meta-estimation of private sketch data structures for differentially private COVID-19 contact tracing, PPML-NeurIPS 2020**](https://github.com/PrivateKit/PrivacyDocuments/blob/master/DAMS_Meta-estimation_of_private_sketch_data_structures_for_differentially_private_contact_tracing.pdf)


# Federated and Split Learning

![](/files/-MYPubB8PvUxU8Ou-gky)

The final technique is federated and split learning. Jane wants to understand which medical treatment for given comorbidity leads to a successful recovery.  John’s phone trains a local AI to see how his health status and medical treatment lead to some kind of recovery or hospitalization. Although this local AI will be noisy because it only has data from John. If every user starts calculating such local AI and transmits the local AI parameters to the server. The server can then create the master AI by averaging these multiple local AI parameters.&#x20;

As you can see, John did not upload his private health data or whether he recovered.

{% embed url="<http://splitlearning.mit.edu>" %}

{% embed url="<https://en.wikipedia.org/wiki/Federated_learning>" %}

**References**<br>

\[1] Vepakomma, Praneeth, et al. “NoPeek: Information Leakage Reduction to Share Activations in Distributed Deep Learning.” ArXiv:2008.09161 \[Cs, Stat], Aug. 2020. arXiv.org, <http://arxiv.org/abs/2008.09161>.


# Solution Videos

**PathCheck: Privacy-Preserving Crowdsourcing for Citizen Engagement**

{% embed url="<https://youtu.be/yRDc1pxctN8>" %}

**Exposure Notification**

{% embed url="<https://youtu.be/VmzcXNZmft0>" %}

**Case Management with Path Check: Beyond Exposure Alert**

{% embed url="<https://youtu.be/DGTpJTwQlxk>" %}

**Vaccination Credintials**

{% embed url="<https://youtu.be/8yfGrnjcW5g>" %}

**Digital Pandemic Response Demo**

{% embed url="<https://youtu.be/U9FAMw3eUVg>" %}

**GPS Demo**

{% embed url="<https://youtu.be/r3MkAMy2b58>" %}

**Vaccination App**&#x20;

{% embed url="<https://www.dropbox.com/s/kc5acl7xetelonf/90sec_vaccineapp.mp4?dl=0>" %}

{% content-ref url="/pages/-MZJukvnYmBpAAdznSmM" %}
[Frequently Asked Questions (FAQs)](/frequently-asked-questions-faqs)
{% endcontent-ref %}


# The PathCheck Foundation

![](/files/-MXiTPiZlnMv09hbsWeR)

We are a global nonprofit of around 2700 dedicated to creating healthy and resilient communities by protecting public health, containing pandemics, and strengthening economies while preserving individual privacy through privacy-preserving innovation and research at the intersection of technology and public health.&#x20;

### Leadership

Ramesh Rasker, Founder and Chief Scientist; Associate Professor, MIT Media Lab

Thomas Kingsley, Chief Epidemiology and Health Officer; Mayo Clinic

Lee Sanders, Chief of Health Equity and Analytics; Stanford University

Brooke Struck, Chief Behavioural Scientist

Antigoni O Polychroniadou, Chief of Cryptography

Vitor Pamplona, Director Vaccination Program

{% embed url="<http://pathcheck.org/>" %}


# Team Members

![](/files/-MYdUdypATY2qqzx3No7)

[Abhishek Singh](https://www.linkedin.com/in/tremblerz/)\
[Albert Johnson](https://www.linkedin.com/in/albert-johnson-6258864/) \
[Aniket Vashishtha](https://www.linkedin.com/in/aniket-vashishtha-476413198/)\
[Anshuman Sharma](https://www.linkedin.com/in/sharmaanshuman/)\
[Aryan Mahindra](https://www.linkedin.com/in/aryanmahindra/)\
[Christin Glorioso](https://www.linkedin.com/in/christin-glorioso-md-phd-39627719/)\
[Harshita Chopra](https://www.linkedin.com/in/harshita-chopra-7ba01a191/)\
[Ishaan Singh](https://www.linkedin.com/in/ishaansingh22/)\
[Kashish Panjvani](https://www.linkedin.com/in/kashishpanjvani/)\
[Krishnendu Dasgupta](https://www.linkedin.com/in/krishdasgupta/)\
[Krutika Misra](https://www.linkedin.com/in/krutika-misra-48bb4817/)\
[Kshitij Patil](https://www.linkedin.com/in/kshitij-patil-252a47166/)\
[Maurizio Arseni](https://www.linkedin.com/in/maurizio-arseni/)\
[Nathan Yap](https://www.linkedin.com/in/nathan-yap-9b8301180/)\
[Parth Patwa](https://www.linkedin.com/in/parth-patwa/)\
[Paul Baier](https://www.linkedin.com/in/paulbaier/)\
[Priya Ramaswamy](https://www.linkedin.com/in/priya-ramaswamy-md-m-eng-47708997/)\
[Priyanshi Katiyar](https://www.linkedin.com/in/priyanshi-katiyar-7222ba186/)\
[Qamil Mirza](https://www.linkedin.com/in/qamil-mirza-a50551183/)\
[Rahul Shirale](https://www.linkedin.com/in/rahulshirale/)\
[Ramesh Raskar](https://www.linkedin.com/in/raskar/)\
[Rishank K](https://www.linkedin.com/in/rishank/)\
[Rohan Sukumaran](https://www.linkedin.com/in/rohan-sukumaran-3271ba145/)\
[Santiago Romero Brufau](https://www.linkedin.com/in/santiago-romero-brufau/)\
[Sethuraman TV](https://www.linkedin.com/in/sethuraman-t-v-64099b137/)\
[Shailesh Advani](https://www.linkedin.com/in/shaileshadvani/)\
[Sheshank Shankar](https://www.linkedin.com/in/sheshank-s/)\
[Shirley Bergin](https://www.linkedin.com/in/shirleybergin/)\
[Tavpritesh Sethi](https://www.linkedin.com/in/tavpritesh/)\
[Thomas Kingsley](https://www.linkedin.com/in/thomas-c-kingsley-18a5a260/)\
Ujjwal Mishra\
[Ullas R Bhat](https://www.linkedin.com/in/ullasism/)\
[Vitor Pamplona](https://www.linkedin.com/in/vitorpamplona/)


# Accolades

* Published more than **30+ papers** on public health and how digital tools can impact it, with more than 100+ citations.
* Invited to the congressional hearings for deploying contact tracing apps
* The first company to launch EN app in the US
* **Released the EN** solution in more than **5 states** and in active conversation with "n" more.
* Awards for Digital Pandemic Response Work
  * Robert Wood Johnson Foundation Emergency Response For The Health Care System Innovation Challenge **Finalist**
  * Facebook COVID-19 Symptom Data Challenge **Finalist**
  * NIST Bluetooth Data Challenge 1st Round **Winner**
  * **Top 10** models in the Xprize Pandemic Response Challenge&#x20;
* Talk at NIST Challenges for Digital Proximity Detection in Pandemics Event
* Host of convening events for COVID-19 digital solutions (<https://pandemic.mit.edu>) and (<https://responsibledata.ai/events/trust>)
* **Largest open source privacy-focused volunteer organization** for COVID-19 digital solutions.

| **Position**                                                                                                                                                                                                                                                                                                                   | Organizers                                                                                                                                                                                                                                                                                                                                                                                                                    |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <p><strong>Finalist, The COVID-19 Symptom Data Challenge</strong> </p><p><strong>(Top 5)</strong></p><p><strong>Facebook, CMU, UMD</strong></p><p><strong>\[</strong><a href="https://www.symptomchallenge.org/"><strong>Link to the Announcement page</strong></a><strong>]</strong></p>                                      | <p></p><p><img src="https://lh3.googleusercontent.com/Y3u92qlYq6VLBegou9_dnYfjYsbI-LxQ47yCniP5vpNnkKNcUJIs9OHz9BebpDFl-pRfsyFS_z3h5C6jbjru4eU_L6DVKapSuZNPyXKmAATo7lDgHaV8oTPVKCXFXhuZ8wHmFIW-" alt=""></p>                                                                                                                                                                                                                   |
| <p><strong>Finalist, COVID-19 Pandemic Response Competition</strong></p><p><strong>XPRIZE, Cognizant</strong></p><p><strong>\[</strong><a href="https://www.xprize.org/articles/pandemic-response-challenge-finalists"><strong>Link to the Announcement Page</strong></a><strong>]</strong></p>                                | <p></p><p><img src="https://lh3.googleusercontent.com/HsgjCBeBx7iGw7WJq4gXptm9ftB-Pm4VHftb7hwhonKQteX-e5W50o76zdqCsS-a_2H3xYGDXAqU2Oh3z7JX9UXxYuhDxbEue9R3bJyqRMRJvgZh4kswzJluExqu13LTsMPQBPta" alt=""></p>                                                                                                                                                                                                                   |
| <p><strong>Winner, Round 1 of Bluetooth Data Challenge and invited talk on Challenges of DCT</strong></p><p><strong>NIST, US Dept. of Commerce</strong></p>                                                                                                                                                                    | <img src="https://lh5.googleusercontent.com/oZlvH7ojphb9sPhwAUmmvkDJ3yUfoobgEoGXTjgS5jHe0Ann562KfuUgeZRhWoLRLo5pOpYJgjSFK7M3fZDuUA6ud_nH5KaUALtiFYQTlU2ZrsNrGgSfwuB6tQK2AMKpatJ8sJIj" alt="" data-size="original">                                                                                                                                                                                                            |
| <p><strong>2nd Place, Emergency Response For The HealthCare System Innovation Challenge</strong> </p><p><strong>Robert Wood Johnson Foundation</strong></p><p><strong>\[</strong><a href="https://www.healthsystemcrisisresponse.com/"><strong>Link to the Announcement Page</strong></a><strong>]</strong></p>                | <img src="https://lh6.googleusercontent.com/iV5WCfIY8OIlHCiJXd6eVg1Xp3v4uMkMpkjZLjDgtRLYyngzjmO-aGtr10glk-KePjh6Gte6y-fAjxpL6C2yM5xFN3Jn879_gOsC8j8j0IG1AKh8XijOqVxNYokPbVaSjAV2zJdG" alt="" data-size="original">                                                                                                                                                                                                            |
| <p><strong>US Congressional Testimony for Contact Tracing</strong></p><p><strong>Ramesh Raskar</strong></p><p><strong>July 2020</strong></p><p><strong>\[</strong><a href="https://financialservices.house.gov/calendar/eventsingle.aspx?EventID=406731"><strong>Link to US House Committee</strong></a><strong>]</strong></p> | <p><img src="https://lh5.googleusercontent.com/Ym8NTBP-M_TYLYFivy16EC7CVlg4nTqoXUdMNKvWOMn66Fzo_W9byEpzws8iY6VvENUX1DbkOKgh-FDg4qVL7sWpV3YX8RdkriKgoP3D12jK_D5UAI8n4pKUCGOVcbAO7C4cTyYf" alt="" data-size="original"></p><p><img src="https://lh4.googleusercontent.com/nI-Ab_c-gSUItdOgN1WTbNrvSR3pPDKo1mPzHrJ1axsNd9ED4qsFn76eVhavh1JyE_7IVwaGCkUBThYtkhm9P_08_NPI6_LQxPvCtyQBEj5iKvwd0__gFli2qQa-O0tYAxVZWpNo" alt=""></p> |


# Frequently Asked Questions (FAQs)

**Q) Adoption: Historically, government health departments have been terrible at consumer marketing. How is your solution any different?**&#x20;

**A)**  (i) Role of Govt and campuses: The ongoing covid pandemic has identified key areas of weakness across levels of society and opportunities for governments to improve. We have seen 20-30% adoption of our PathCheck contact tracing apps within days of official branded launch by governments. We expect the government to promote and incentivize the population with our toolkit. We are not directly responsible for increasing adoption but do help share best practices. Our specific and focused role is to build the toolkit (open source software for app, server, dashboard, and engagement platform). We partner with several regional health IT partners who deliver it to the government or large campuses. This model has worked well during our rollouts over the last year.  We built a vibrant network of regional health-IT companies who want to help government officials and institutions.&#x20;

(ii) Only 5% adoption required: Navigation apps make estimates of traffic density with well under 5% drivers contributing. While the larger goal is to achieve higher adoption rates, we can make statistically significant and meaningful predictions with as little as 5%. <br>

**Q) Is this yet another app?: Healthcare is littered with failed consumer health apps (i.e. low adoption, unknown effectiveness), why is this different?**&#x20;

**A)** Closed-loop with public health: We expect our toolkit to be used for a specific purpose and at a specific time: during the outbreak when stakeholders are pushing it as one of the official and easy way to share data and to stay engaged. Exposure notification (EN) apps as well as other crisis apps usually have very good adoption for that limited window. Some of PathCheck’s EN apps reached 20% - 30% adoption in just days in many of the states. See our work in [Guam](https://www.technologyreview.com/2020/11/30/1012732/us-guam-covid-alert-app-exposure-notification/) and [Minnesota](https://covidawaremn.com/) where public health created a closed-loop system for engaging with infected citizens.

Beyond a diary: We are not building an app that is focused on just UI/UX or health questionnaires. We are building a mechanism for the governments to launch a solution within days of an outbreak and be able to guarantee to their citizens about a solution that is well tested and proven for privacy, security, quality, behavioral factors in addition to epidemiology. We will provide the modular tech stack for NoPeek privacy, crowdsourced data analytics and models.<br>

**Q) Why can’t Google, Apple, or Facebook on their own build this kind of crowdsourcing for the pandemic response?** &#x20;

**A)** Privacy perception and brand risk: Any entity with an expansive view of people’s daily lives can play a positive role in a pandemic. But it is challenging for these for-profit companies to ask for even more sensitive personal health and activity data. Big tech companies care about their brand and are concerned about the implications of their privacy perception, even if algorithmically NoPeek is privacy-preserving.&#x20;

API, not app: Because of this privacy brand risk, for exposure notification, Apple and Google provided the APIs and minimal app skeletons rather than launching a full-fledged company-branded product.  We think they will continue to play this important enabling role. For pandemic response, many new sensor APIs will be required: mask-wearing detected during facial recognition phone unlock to compute risk scores and to estimate aggregate mask-wearing percentage, Oxygen saturation levels (as well as pulse rate) recorded using a finger on the camera by adding another wavelength, improving GPS protocols to create a private diary of GPS trail even when the app is in the background and so on.

Privacy Expectations: We are in touch with these companies, and our team members have worked at these three companies recently including Facebook health and the Apple Privacy team. As far as we know, beyond exposure notification APIs, these companies have yet to implement NoPeek solutions for consumer health. If the big tech companies implement NoPeek for some consumer health, customers may start to expect such privacy guarantees for other services like email, maps, news, social media, and e-commerce. This will challenge their data and targeting-driven business models.&#x20;

Gathering momentum is more important than deciding who delivers it ultimately: In any case, we can develop these NoPeek and engagement technologies in the specific context of pandemic response. We can scale them to the deployment of billions of people or partner/sell the ruggedized solution to big-tech companies. Either way, the world needs a ready-to-deploy privacy-preserving crowdsourcing solution for health crisis response, and we are passionate to get started and play a catalyst role for the whole industry. <br>

**Q) Many symptom trackers now exist with websites or mobile apps. While these are for generic symptoms, why wouldn’t they be a better platform for pandemic alerts rather than a specific-purpose PathCheck solution?**&#x20;

**A)** We are in touch with various for-profit companies in the symptoms-to-guidance space. Self-reported symptom data is one specific aspect of our solution. It can come from these platforms or be integrated with our app to create a seamless experience. Over time, approved apps can access each other’s online ‘health vault’ with NoPeek. We agree that adoption and daily engagement are tricky problems that each jurisdiction will solve on its own. Our EN apps are used by a large fraction of the population within days of the launch so that gives us hope that in a crisis, citizens look for trustworthy solutions recommended by the governments.<br>

**Q) What is your actual evidence of feasibility?**&#x20;

**A)** There are multiple sub-questions to consider:&#x20;

(i) Does citizen data and personalized engagement help in quickly taming a pandemic?: Some Asian countries did show that aggressive test-trace-isolate works to tame the spread within three months with non-pharmaceutical intervention. (However, the citizen data collection was invasive and engagement was coercive). Separately, peer-reviewed papers have demonstrated that one life is saved for approximately every 200 citizens participating in crowdsourced exposure notification and significant benefits at even 15% adoption \[Abueg2020].&#x20;

(ii) Can governments and institutions nudge citizens to do the right thing to achieve pandemic orchestration? Behavior change is challenging. We know it works in Navigation map apps if the solution can achieve the quality of engagement and immediate tangible benefits (or involves compliance fines).&#x20;

(iii) Will citizens trust the privacy guarantees? And will they understand NoPeek is better than consent or pseudo-anonymity?: The awareness about privacy layers and their implications among citizens may be limited. However, we learned from deploying contact tracing apps that governments in democratic countries will deploy an official solution only if it is highly privacy-preserving and has no risks of data breach, security, or fraud.  This awareness among government and public health leaders has led to strong support for our solutions.  We need strong technology and public education as the alternative scenario of no privacy or data centralization is too dire.

(iv) Why will citizens bother downloading or using yet another app? Will there be an immediate tangible benefit?  We have shown for EN, that citizens are very excited to use our new digital solutions in a crisis if recommended by trusted entities and adoption can reach 20-30% within days. The good news with crowdsourcing is that we need a small fraction of the population to participate to create statistically meaningful estimates. For example, with navigation map apps, a small fraction of app users on the road is enough to help us estimate traffic blockage versus a smooth flow.<br>

**Q) What evidence do you have for data analysis?** &#x20;

**A)** We have developed machine learning and deep learning models that can predict prevalence, and forecast outbreaks based on self-reported population-level aggregated symptoms data. Furthermore, we have also developed multi-agent reinforcement learning models for prescribing non-pharmaceutical interventions, given the current state of the disease, economic impact, and more (Glorioso et al., 2021; Patwa et al., 2021; Sukumaran et al., 2021). Our DeepABM system makes use of Graph Neural Networks (GNNs) to scale agent-based models to run simulations with 100,000s agents in less than a few seconds (Chopra et al., 2021). Our outbreak predictions and NPI prescription models were ranked among the top models globally in renowned competitions organized by Facebook and XPRIZE. Members of our team have also been at the forefront of creating privacy-preserved machine learning models. We have published these works as numerous research papers in top-tier AI/ML venues (NeurIPS, ICLR, CVPR, and more).

{% embed url="<http://pathcheck.org/>" %}


# Our research papers and articles

**Privacy**&#x20;

* [**Apps Gone Rogue: Maintaining Personal Privacy in an Epidemic**](https://arxiv.org/abs/2003.08567)
* [**Verifiable Proof of Health using Public Key Cryptography**](https://arxiv.org/abs/2012.02885)
* [**PPContactTracing: A Privacy-Preserving Contact Tracing Protocol for COVID-19 Pandemic**](https://arxiv.org/abs/2008.06648)

**Digital Contact Tracing**&#x20;

* [**Comparing manual contact tracing and digital contact advice**](https://arxiv.org/abs/2008.07325)
* [**Proximity Sensing: Modeling and Understanding Noisy RSSI-BLE Signals and Other Mobile Sensor Data for Digital Contact Tracing**](https://arxiv.org/abs/2009.04991)
* [**Proximity Interference with Wifi-Colocation during the COVID-19 Pandemic**](https://arxiv.org/abs/2009.12699)
* [**Spatial K-anonymity: A Privacy-preserving Method for COVID-19 Related Geo-spatial Technologies**](https://arxiv.org/abs/2101.02556)
* [**COVID-19 Contact-Tracing Mobile Apps: Evaluation and Assessment for Decision Makers**](https://arxiv.org/abs/2006.05812)
* [**Contact Tracing: Holistic Solution beyond Bluetooth**](http://sites.computer.org/debull/A20june/p67.pdf)
* [**Contact Tracing to Manage COVID19 Spread – Balancing Personal Privacy and Public Health**](https://www.mayoclinicproceedings.org/article/S0025-6196\(20\)30424-9/fulltext)
* [**The Architecture of Trust in Contact Tracing**](https://f.hubspotusercontent40.net/hubfs/8097148/documents/evaluating-contact-tracing-apps.pdf)
* [**Adding Location and Global Context to the Google/Apple Exposure Notification Bluetooth API**](https://arxiv.org/abs/2007.02317)

**Equitable Vaccine Distribution and Coordination**

* [**Mobile Apps Prioritizing Privacy, Efficiency and Equity: A Decentralized Approach to COVID-19 Vaccination Coordination**](https://arxiv.org/abs/2102.09372)
* [**Challenges of Equitable Vaccine Distribution in the COVID-19 Pandemic**](https://arxiv.org/abs/2012.12263)&#x20;
* [**Vaccination Worldwide: Strategies, Distribution and Challenges**](https://www.researchgate.net/profile/Sheshank-Shankar-2/publication/349787933_Vaccination_Worldwide_Strategies_Distribution_and_Challenges/links/6041a104a6fdcc9c78122a66/Vaccination-Worldwide-Strategies-Distribution-and-Challenges.pdf)
* [**The Public Health Impact of Delaying a Second Dose of the BNT162b2 or mRNA-1273 COVID-19 Vaccine**](https://www.medrxiv.org/content/10.1101/2021.02.23.21252299v1)

**Vaccine Credentials**&#x20;

* [**MIT SafePaths Card (MiSaCa): Augmenting Paper Based Vaccination Cards with Printed Codes**](https://arxiv.org/abs/2101.07931)
* [**Paper card-based vs application-based vaccine credentials: a comparison**](https://arxiv.org/abs/2102.04512)

**COVID-19 Testing**&#x20;

* [**Clinical landscape of covid-19 testing: Difficult choices**](https://arxiv.org/abs/2011.04202)
* [**COVID-19 Tests Gone Rogue: Privacy, Efficacy, Mismanagement and Misunderstandings**](https://arxiv.org/abs/2101.01693)
* [**Digital Landscape of COVID-19 Testing: Challenges and Opportunities**](https://arxiv.org/abs/2012.01772)

**Pandemic Prediction**&#x20;

* [**COVID-driven Risk Profile**](https://www.researchgate.net/publication/345823059_COVID-driven_Risk_Profile)
* [**Can self-reported symptoms predict daily COVID-19 cases?**](https://arxiv.org/pdf/2105.08321.pdf)
* [**COVID-19 Outbreak Prediction and Analysis using Self Reported Symptoms**](https://arxiv.org/abs/2101.10266)
* [**Analysis of Tata-1mg data for Covid-19 2nd wave prediction in India**](https://scholar.google.com/citations?view_op=view_citation\&hl=en\&user=3NGJNQIAAAAJ\&sortby=pubdate\&citation_for_view=3NGJNQIAAAAJ:UebtZRa9Y70C)
* [**Estimating Active Cases of COVID-19**](https://ui.adsabs.harvard.edu/abs/2021arXiv210803284A/abstract)

**Split Learning and Federated Learning**

* [**Distributed learning of deep neural network over multiple agents, Accepted in Journal of Network and Computer Applications 116**](https://www.sciencedirect.com/science/article/pii/S1084804518301590)
* [**DISCO: Dynamic and Invariant Sensitive Channel Obfuscation, Accepted to CVPR 2021**](https://github.com/splitlearning/splitlearning.github.io/blob/master/DISCO.pdf)
* [**FedML: A Research Library and Benchmark for Federated Machine Learning, (Baidu Best Paper Award at NeurIPS-SpicyFL 2020)**](https://www.media.mit.edu/publications/fedml-a-research-library-and-benchmark-for-federated-machine-learning/)
* [**NoPeek: Information leakage reduction to share activations in distributed deep learning**](https://arxiv.org/abs/2008.09161)
* [**Split learning for health: Distributed deep learning without sharing raw patient data, Accepted to ICLR 2019 Workshop on AI for social good**](https://arxiv.org/pdf/1812.00564.pdf)
* [**Detailed comparison of communication efficiency of split learning and federated learning**](https://arxiv.org/pdf/1909.09145.pdf)
* [**ExpertMatcher: Automating ML Model Selection for Users in Resource Constrained Countries,**](https://arxiv.org/pdf/1910.02312.pdf)
* [**Split Learning for collaborative deep learning in healthcare**](https://arxiv.org/abs/1912.12115)

**Differential Privacy**

* [**Differentially Private Supervised Manifold Learning with Applications like Private Image Retrieval**](https://arxiv.org/pdf/2102.10802.pdf)
* [**DAMS: Meta-estimation of private sketch data structures for differentially private COVID-19 contact tracing, PPML-NeurIPS 2020**](https://github.com/PrivateKit/PrivacyDocuments/blob/master/DAMS_Meta-estimation_of_private_sketch_data_structures_for_differentially_private_contact_tracing.pdf)


