← All resources

Private AI on Azure: A Researcher's Setup Guide

14 min read
private ai on azure
Private AI on Azure: A Researcher's Setup Guide

Private AI on Azure: A Researcher’s Setup Guide

Researcher configuring an Azure private endpoint

Private AI on Azure means four things working together: a model deployment that Microsoft does not train on, a network path that never touches the public internet, identity controls that replace shared keys, and a paper trail you can show an ethics board. Azure can deliver all four. It will not deliver them by default, and some of the gaps (DNS, abuse monitoring, lawful basis) catch researchers who assume “private” is a checkbox.

This guide walks through each layer in the order you would configure it, flags the failure points researchers hit, and then covers the case where the cleanest answer is to keep the analysis on your own machine.

Plotstudio
plotstudio.ai
Keep Sensitive Analysis Local
PlotStudio AI runs analyses on your machine, with code, methods, and results available for review in a reproducible workflow.
Visit PlotStudio AI

TL;DR

  • Microsoft states that prompts, completions, embeddings and training data are not used to train foundation models and are not shared with other customers or model providers.
  • Real network isolation needs private endpoints, private DNS zones, and the supporting services (storage, Key Vault) inside the same boundary.
  • Prefer Microsoft Entra managed identities with Azure RBAC over API keys, and apply for modified abuse monitoring if your data is sensitive.
  • Under UK GDPR, technical controls do not replace a documented lawful basis and an Article 9 condition for special category data.
  • If patient-level data should not be moved to the cloud at all, local agentic analytics such as PlotStudio AI is a simpler route, and it sits alongside Azure rather than against it.

Table of Contents

What “private” actually covers on Azure

The word bundles together separate guarantees. Keep them apart, because each one answers a different reviewer question.

Layer Question it answers Azure control
Model training Will my data improve someone else’s model? Microsoft’s data privacy commitments
Data residency Where is the data processed and stored? Region of the resource, geography boundary
Network Can traffic reach the service from the internet? Private endpoints, private DNS, managed virtual network
Identity Who can call the model? Entra ID, managed identities, Azure RBAC
Retention and review Can a human read my prompts? Abuse monitoring, modified abuse monitoring
Legal Am I allowed to process this data this way? Your lawful basis, DPIA, ethics approval

The first five are configuration. The last is yours alone, and no Azure setting can supply it.

Layer 1: what Microsoft promises about your data

According to Microsoft’s data, privacy and security documentation for Foundry Models sold by Azure, your prompts, completions, embeddings and training data are not used to train the foundation models and are not available to other customers or to model providers such as OpenAI. Customer data is logically separated by resource ID, and it stays within the geography where you deploy the resource.

Two practical consequences follow for a research group:

  1. Choose the region deliberately. The geography boundary is set by where the resource lives. If your data governance plan says EU or UK only, deploy there and write down the region in your protocol.
  2. Read the scope. The commitment applies to models sold by Azure within that environment. A third-party model reached through another route may carry different terms, so check the specific model’s documentation before putting identifiable data near it.

Layer 2: network isolation with private endpoints

A deployment with a public endpoint and a strong password is not private. Microsoft’s cloud adoption guidance on AI networking points to private endpoints, which give the service a private IP address inside your virtual network so traffic does not cross the public internet. Private endpoints need matching private DNS zones to resolve properly, and the services around the model (Azure Storage, Key Vault) should be isolated in the same boundary. A locked-down model reading from an open storage account only moves the exposure.

A minimal checklist

  1. Create the Azure OpenAI or Foundry resource in your chosen region.
  2. Create a private endpoint in your virtual network and link the private DNS zone.
  3. Disable public network access on the resource.
  4. Put storage and Key Vault behind private endpoints too.
  5. Test from a machine inside the network, not from your laptop on home Wi-Fi.

The 403 that is really a DNS problem

The most common frustration is a 403 Forbidden error that looks like a permissions failure. Microsoft’s Q&A threads describe a typical cause: the private endpoint exists, but the application still resolves the public hostname and is rejected, with a message that traffic is not from an approved private endpoint. The fix is to confirm that the hostname resolves to the private IP from the environment where the code runs, using a tool such as nslookup. For Azure Functions, the same thread recommends setting WEBSITE_VNET_ROUTE_ALL to 1 so outbound traffic, DNS included, goes through your virtual network.

Public and private DNS resolution paths

A related symptom is a certificate hostname mismatch when calling through a private link URL. If you see one, check that you are calling the resource’s own endpoint name and that DNS maps it to the private address, rather than disabling certificate checks.

Managed virtual networks: convenient, but fixed

Foundry offers a managed virtual network that automates private connectivity. It is a good default for a team with a single, well-defined deployment. The trade-offs matter for research, though. Once managed network isolation is enabled for a Foundry resource you cannot turn it off, and you cannot bring your own virtual network for that isolation. Outbound access is limited to approved rules, which can get awkward when an analysis pipeline needs a package mirror, a data source, and a notebook environment that were not planned in advance.

If your work is iterative (new packages, new datasets, new methods every week), decide up front whether that fixed boundary suits you.

Layer 3: identity, not keys

API keys are shared secrets. They get pasted into notebooks, committed to repositories, and emailed to collaborators. Microsoft’s AI security best practices recommend private endpoints together with Microsoft Entra authentication and Azure RBAC, so access is tied to identities your institution already manages and every call can be attributed.

In practice:

  • Assign a managed identity to the application or compute that calls the model.
  • Grant the narrowest RBAC role that works, and review it when the project team changes.
  • Turn off key-based access once identity-based access is confirmed working.

One caution from the Q&A archive: a managed identity that works over a public path can still return 403 over a private endpoint if the role assignment, DNS, or network rules differ for the private path. Troubleshooting reports on Azure AI Search behind a virtual network show this pattern, so test each path separately.

Layer 4: abuse monitoring and human review

This is the layer researchers most often miss. Azure runs abuse monitoring that classifies prompts and completions for harmful content. Microsoft’s data privacy page explains that, without a modification, flagged content may be reviewed by automated and human processes to confirm a terms-of-service violation, and that approved customers can apply for modified abuse monitoring to turn off that sampling.

Content classification and review paths

If your prompts could contain patient information, free-text clinical notes, or interview transcripts, apply for the modification before you start, and keep the approval with your project records. Do not assume it is on; verify it for your specific resource and subscription.

Researchers working with health, genetic, or other special category data face an obligation that sits outside the portal. The UK Information Commissioner’s Office states in its guidance on lawfulness in AI that you need an appropriate lawful basis for the processing, plus an Article 9 condition when special category data is involved. It also stresses documenting those determinations before processing begins, and that a general privacy policy is typically not enough.

Three points worth taking seriously:

  • “Incidental” is not an exemption. Data protection law applies whether or not you intended to process personal data.
  • Inferences count. If a model infers something about a sensitive characteristic and you use it, treat that output as potentially special category data.
  • Write it down first. Lawful basis, Article 9 condition, risk assessment, and the reasoning for why this tool is proportionate belong in your protocol and data protection impact assessment before the first prompt.

This is UK guidance. If you work under the EU GDPR, HIPAA, or another regime, the same discipline applies but the specific tests differ, so take advice from your institution’s data protection officer and ethics board. Our own notes on documenting Article 6 and 9 proof for GDPR PII categories cover how to structure that record.

When a private cloud deployment is the wrong tool

Azure private AI is excellent when you are building a service: a chatbot over an institutional document store, a pipeline for many users, an application with an owner and an IT team. Researchers doing statistical analysis often have a different shape of problem:

  • One dataset, one team, a deadline.
  • Patient-level or participant-level data that the governance approval says should not leave a secure machine or institutional drive.
  • A need to show a supervisor or reviewer exactly how a number was produced.
  • No time to administer virtual networks, DNS zones, and RBAC.

In that situation, the lowest-risk architecture is often the one where the data never moves. Our guides on on-device data analysis that keeps provenance intact and secure analytics for healthcare researchers set out that argument in more detail.

Azure private AI versus local agentic analytics

Question Private AI on Azure Local analysis with PlotStudio AI
Where does the data sit during analysis? In your Azure tenant, in the region you choose On the researcher’s machine
Setup effort Networking, DNS, identity, monitoring settings Install and connect a dataset
Best for Multi-user services and applications Individual or group research analyses
Reproducibility record You build it Plan, code, outputs and statistics preserved
Institutional control Your tenant, your policies Your device, your policies

These are not rivals. A hospital trust or university may use Azure for shared infrastructure and still want analysts to work on sensitive extracts locally.

Our perspective: privacy is an architecture decision, then a methods decision

We build PlotStudio AI for researchers, and the pattern we see is that privacy conversations stop too early. Teams settle where the model runs, then discover a reviewer asking how a result was produced, or a supervisor unable to reproduce a table.

Private infrastructure answers the first question. It does not answer the second. A chat answer, however well protected in transit, is not a method. What a reviewer, an ethics board or a collaborator needs is the analytical plan, the code that ran, the intermediate checks, and the final statistics, kept together.

That is why PlotStudio AI is built as agentic analytics for researchers rather than a one-shot chat tool. Specialized agents plan the investigation, write and run real code, inspect intermediate results and synthesize findings. Plan Mode lets you review and edit the proposed methods, assumptions and steps before anything executes. Analyses can run locally on your machine, which suits workflows where datasets should not be uploaded to conventional cloud analytics platforms. Completed work is kept as searchable Analysis Pages and can be exported as notebooks and PDF reports.

We do not see PlotStudio AI replacing R, Python, Stata, SPSS, SAS, Jupyter or RStudio. Those provide the statistical computing. We think of PlotStudio AI as an emerging must-have layer in the modern research toolkit, an agentic layer that plans, executes, inspects, validates and documents complete analyses on top of the tools you already trust. And for organizations that do want a managed footprint, we offer enterprise Azure deployment, so the two approaches can meet.

— Aymen

A practical decision path

Use this order when you are choosing an approach for a specific study.

  1. Read your governance approvals. Do they restrict cloud processing, or specify a region or environment? That settles most of the decision.
  2. Classify the data. Is it special category, identifiable, pseudonymized, or aggregate? Pseudonymized data is generally still personal data under GDPR.
  3. Decide who needs access. One analyst on a secure machine points to local analysis. Many users or an application points to a cloud deployment.
  4. If you use Azure, configure all layers. Region, private endpoint, private DNS, disabled public access, Entra identities, modified abuse monitoring, isolated storage and Key Vault.
  5. Document the legal basis. Lawful basis and Article 9 condition (or your jurisdiction’s equivalent) before processing.
  6. Keep the analysis reproducible. Whatever the infrastructure, store the plan, code and outputs with the project.

Trying PlotStudio AI on your own data

If local, reviewable analysis fits your study, you can start with the Free Trial on the pricing page. Paid options there are Managed Credits at 69.99 USD per month and Bring Your Own Key at 39.99 USD per month. Academic researchers can also take Bring Your Own Key at 399.90 USD per year on the academic page. For organization-wide or managed deployments, including Azure, the enterprise page lists a one-off pilot fee of 500 USD.

For a wider view of how this fits into a thesis or lab workflow, see our guide to AI for academic research and the walkthrough of a reproducible analysis workflow for PhD researchers.

FAQ

Does Azure OpenAI use my research data to train models?

Microsoft states that prompts, completions, embeddings and training data are not used to train its foundation models and are not made available to other customers or to providers like OpenAI. This applies to models sold by Azure within its environment. Confirm the terms for any other model you deploy.

Can Microsoft staff read my prompts?

By default, abuse monitoring can sample prompts and completions, and flagged content may be reviewed by automated and human processes. Microsoft allows eligible customers to apply for modified abuse monitoring to turn this off. If your data is sensitive, apply before you start and keep the approval on file.

Why do I get a 403 error even though my private endpoint is set up?

Usually the application is resolving the public hostname instead of the private IP. Check DNS resolution from the environment where the code runs, and make sure private DNS zones are linked to your virtual network. For Azure Functions, routing all outbound traffic through the virtual network is a documented fix.

Is a private Azure deployment enough for GDPR compliance?

No. Technical controls reduce risk, but you still need a documented lawful basis and, for special category data, an Article 9 condition. The UK ICO says to document these before processing begins. Your data protection officer should sign off on the specifics for your study.

When should I analyze sensitive data locally instead?

When your approvals restrict where data can go, when one analyst or a small team is doing the work, or when you cannot spare time to run cloud networking. Local analysis keeps the dataset on the machine, and tools that preserve the plan, code and outputs keep the work reviewable.

Sources