Azure.Health.Deidentification

Azure Health Data Services de-identification service client library for .NET

This package contains a client library for the de-identification service in Azure Health Data Services which enables users to tag, redact, or surrogate health data containing Protected Health Information (PHI). For more on service functionality and important usage considerations, see the de-identification service overview.

Source code | Package (NuGet) | API reference documentation | Product documentation | Samples

Getting started

Prerequisites

• Have an Azure Subscription.
• Deploy the de-identification service.
• Configure Azure role-based access control (RBAC) for the operations you will perform.

Install the package

Install the .NET client library NuGet package:

dotnet add package Azure.Health.Deidentification

Authenticate the client

You will need a service URL to instantiate a client. You can find the service URL for a particular resource in the Azure portal:

You can also find the service URL with Azure CLI:

# Get the service URL for the resource
az deidservice show --name "<resource-name>" --resource-group "<resource-group-name>" --query "properties.serviceUrl"

The Azure Identity package provides the default implementation for authenticating the client. You can use DefaultAzureCredential to automatically find the best credential to use at runtime.

const string serviceEndpoint = "https://example.api.cac001.deid.azure.com";
TokenCredential credential = new DefaultAzureCredential();

DeidentificationClient client = new(
    new Uri(serviceEndpoint),
    credential,
    new DeidentificationClientOptions()
);

Key concepts

Operation Types

Given an input text, the de-identification service can perform three main operations:

• Tag returns the category and location within the text of detected PHI entities.
• Redact returns output text where detected PHI entities are replaced with placeholder text. For example John replaced with [name].
• Surrogate returns output text where detected PHI entities are replaced with realistic replacement values. For example, My name is John Smith could become My name is Tom Jones.
• SurrogateOnly returns output test where user-defined PHI entities are replaced with realistic replacement values.

For more information about customizing the redaction format, see Tutorial: Use a custom redaction format with the de-identification service.

String Encoding

When using the Tag operation, the service will return the locations of PHI entities in the input text. These locations will be represented as offsets and lengths, each of which is a StringIndex containing three properties corresponding to three different text encodings. .NET applications should use the Utf16 property.

For more on text encoding, see Character encoding in .NET.

De-identification Methods

There are two methods of interacting with the de-identification service. You can send text directly, or you can create jobs to de-identify documents in Azure Storage.

You can de-identify text directly using the DeidentificationClient:

DeidentificationContent content = new("Hello, John!");

Response<DeidentificationResult> result = client.DeidentifyText(content);
string outputString = result.Value.OutputText;
Console.WriteLine(outputString); // Hello, Tom!

To de-identify documents in Azure Storage, you'll need a storage account with a container to which the de-identification service has been granted an appropriate role. See Tutorial: Configure Azure Storage to de-identify documents for prerequisites and configuration options. You can upload the files in the test data folder as blobs, like: https://<storageaccount>.blob.core.windows.net/<container>/example_patient_1/doctor_dictation.txt.

You can create jobs to de-identify documents in the source Azure Storage account and container with an optional input prefix. If there's no input prefix, all blobs in the container will be de-identified. Azure Storage blobs can use / in the blob name to emulate a folder or directory layout. For more on blob naming, see Naming and Referencing Containers, Blobs, and Metadata. The files you've uploaded can be de-identified by providing example_patient_1 as the input prefix:

<container>/
├── example_patient_1/
       └──doctor_dictation.txt
       └──row-2-data.txt
       └──visit-summary.txt

Your target Azure Storage account and container where documents will be written can be the same as the source, or a different account or container. In the examples below, the source and target account and container are the same. You can specify an output prefix to indicate where the job's output documents should be written (defaulting to _output). Each document processed by the job will have the same relative blob name with the input prefix replaced by the output prefix:

<container>/
├── example_patient_1/
       └──doctor_dictation.txt
       └──row-2-data.txt
       └──visit-summary.txt
├── _output/
       └──doctor_dictation.txt
       └──row-2-data.txt
       └──visit-summary.txt

Create a job to de-identify documents:

DeidentificationJob job = new()
{
    SourceLocation = new SourceStorageLocation(new Uri(storageAccountUrl), "folder1/"),
    TargetLocation = new TargetStorageLocation(new Uri(storageAccountUrl), "output_folder1/"),
    OperationType = DeidentificationOperationType.Redact,
};

job = client.DeidentifyDocuments(WaitUntil.Started, "my-job-1", job).Value;
Console.WriteLine($"Job status: {job.Status}"); // Job status: NotStarted

Thread safety

All client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

For sample code snippets illustrating common patterns used in the de-identification service, see the samples.

Next steps

• Find a bug, or have feedback? Raise an issue with the Health Deidentification label.

Troubleshooting

• Unable to Access Source or Target Storage
  • Ensure you assign a managed identity to your de-identification service
  • Ensure you assign appropriate permissions to the managed identity to access the storage account

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.