Azure OpenAI on your own data

With the current previews, the Azure OpenAI services can now also be used for own data. The following example shows the Azure services that are required for this.

Required services:

The following services are required and must be used in combination with each other:

• Azure OpenAI
• Azure Cognitive Search Index

As of June 2023, Azure OpenAI is still in preview and you need to sign up for the preview to use the features. Azure Cognitive Search is available by default.

Azure Cognitiv Search

First, an instance of Azure Cognitive Search needs to be provisioned. How to do this is described here: Create an Azure Cognitive Search service in the portal.

The next step is to configure where the data that should be indexed is located. There are various options for this, all of them results in an index in the newly created Azure Cognitive Search instance. On the Overview page, you will find the link "Connect your data -> Learn more", which provides a good overview of the options:

For testing scenarios, demo data provided by Microsoft can also be used. The "Import" button takes you to the currently available options:

Especially for customers whose data is stored in M365 / SharePoint Online and Teams, the option "SharePoint Online" is very interesting. Currently, this option is still in preview and cannot be used via the UI. How Azure Cognitive Search can currently be used to index data in SharePoint Online is described here: Index data from SharePoint document libraries.

In my example I use the demo data "hotel-sample" that Microsoft provides:

The index contains the following fields:

These fields can now be used in Azure OpenAI.

Azure OpenAI

As soon as an instant of Azure OpenAI is created / available, the Azure OpenAI Studio is also available. This is where you have the options "ChatGPT-Playground":

In the "ChatGPT-Playground" you can now work either with the data from the general Language Model GPT or with your own data:

To use your own data from the first step "Azure Cognitive Search" you have to select the corresponding details:

The "Index data field mapping" in the next step is optional, but recommended because it increases the quality of the answers significantly. In the example with the "hotel-sample" demo data it looks like this:

This can then be used to generate texts based on GPT and the associated data.

Example:

Do we still need metadata when using Azure OpenAI?

Answer: The question is asked incorrectly. The question should be: Does AI have access to all relevant information. This includes metadata. The example shows the problem. Azure OpenAI does not know the information even it is there.

Podcast on the topic of Next Level AI

I was part of two podcasts with Torben Blankertz and Michael Greth on the topic of Next Level AI.

Here are the links to the podcasts:

Einführen von Microsoft Copilot, ChatGPT und AI – mit Nicki Borell:

LINK: https://podcast.blankertz-pm.de/069-einfuehren-von-microsoft-copilot-chatgpt-und-ai-mit-nicki-borell/ 

MikeOnAI DerTalk - S01E02 mit Nicki Borell

LINK: https://youtu.be/vMVmIAL8I5k

Next Level AI

Writing assistance, code generation, and conclusions over data - How machine learning and artificial intelligence generate and understand natural language.

The whitepaper by Dr. Michel Rath and Nicki Borell explains the current state of technology, what OpenAI and Microsoft are doing, and how interested customers can benefit from it.

The first part covers the architecture and technical details of Generative Pre-trained Transformers, or GPT for short. It covers basic concepts such as LLM's - (Large Language Model) and what a Prompt is. The whitepaper explains the difference between OpenAI, Azure OpenAI and the announced Microsoft Copilot feature.

Details:

• Introducing ChatGPT
• Basic terms
• LLM
• Prompt
• GPT3, GPT4 and other models
• The cooperation between Open AI and Microsoft
• Microsoft Copilot
• What is the difference between Microsoft Copilot and Microsoft OpenAI
• Current state of technology - what is available, what is announced?

The second part deals with the legal aspects of the topic. How to create a guideline for employees while respecting the protection of company secrets and what needs to be considered when using ChatGPT with an eye on copyright.

Details:

• Legal Aspects related to ChatGPT & Co.
• ChatGPT and data protection
• Guidelines for employees and the protection of trade secrets
• ChatGPT and Copyright
• The AI Regulation is coming

Download

• Download German version: LINK

• Download English version: LINK

Authors

Dr.Michael Rath is a lawyer, a specialist in information technology law and a partner at Luther Rechtsanwaltsgesellschaft mbH, based in Cologne. He coordinates Luther's Information Tech & Ccommunications practice area. He is also a Certified ISO/IEC 27001 Lead Auditor.

Nicki Borell is co-founder of Experts Inside, a technology consultancy focused on Azure and Microsoft 365, and the head behind the label "Xperts At Work". His focus topics are enterprise collaboration, security and compliance. The Azure OpenAI services and the GPT language model therefore fit perfectly into his work context. They build another integartion between Microsoft 365  and Microsoft's Azure services. Content generation and semantic search are thus also possible for the data within a Microsoft 365 environment in a secure and controllable way.

Introducing Microsoft Security Copilot

Converting questions into actions

Key Functions:

• Simplify the complex 
• Catch what others miss 
• Address the talent gap

Ask Security Copilot questions in natural language and get actionable answers.

Microsoft Security Copilot combines a Large Language Model (LLM) with a Microsoft security-specific model.

When Security Copilot receives a question from a security expert, it uses a security-specific language model to provide answers that can help assess and resolve the incident:

In doing so, the Microsoft Security Copilot response leads to a higher quality of detection and reduces the time needed to resolve the problem:

The solution is thus a kind of SOC as a Service powed by AI.

Security Copilot is a learning system, which means it is continuously improving. Users can directly provide feedback on the answers and solutions suggested by Security Copilot via the integrated interface. The Security Copilot is also able to prepare and report / document incidents:

Source and further details: Introducing Microsoft Security Copilot - Empowering defenders at the speed of AI

Anonymize your Microsoft 365 reports

The topic of data protection in the context of Microsoft 365 is still ongoing and not finally clarified in all details. The handling of user information and reports is not only a point from the GDPR. Other audits and ISO standards also address this point. For this reason, Microsoft 365 has been offering the option to output anonymous user names in reports instead of the actual user names. Settings -> Org Settings -> Services -> Reports:

By default, the function is active and the reports are anonymized. However, the actual log data is not changed, but the data in the reports is displayed anonymized, depending on the setting. The anonymization can thus be switched on or off and the user data in the reports change ahock: 

The setting affects the following reports in Microsoft 365:

• Email Activity
• Mailbox Activity
• OneDrive files
• SharePoint Activity
• SharePoint Site Usage
• Microsoft Teams Activity
• Yammer Activity
• Active users in Microsoft 365 Services and Apps
• Groups Activity

More details: Microsoft 365 reports show anonymous user names instead of actual user names

Microsoft 365 and the DSK (German Data Protection Conference)

 

First of all and very important: I, the author, am not a lawyer and have no legal qualification. This article summarizes the facts on the topic of "Can Microsoft Online Services be used in a privacy-compliant manner following the GDPR?

What is it actually about?

Ulrich Kelber, Chairman of the DSK (German Data Protection Conference), said at a press conference on November 24, 2022, that the use of Microsoft 365 remains contrary to data protection.

Reason, among others: It was still unclear which data was collected, transferred and processed for Microsoft's own purposes.

In summary, the lack of transparency is the cause of concern for the DSK.

The DSK doubts that Microsoft 365 can be used in a data protection-compliant manner "...just like that on a computer without further protective measures". The protection that is meant here refers to the so-called perimeter security. The DSK admits that data protection-compliant use is perhaps possible if techniques such as proxy servers or micro-virtualization are used. On a central proxy server, over which all data traffic is routed, the data flow can then be monitored and controlled in detail.

This approach is not new, but is increasingly being replaced by zero trust architectures because it is simply no longer up to date. In addition, cloud providers generally require that data traffic/access to SaaS, IaaS and PaaS services is direct, i.e., that there is no proxy server or techniques such as packet inspection in between.

Example Microsoft:

• „…Microsoft 365 networking is to optimize the end user experience by enabling the least restrictive access between clients and the closest Microsoft 365 endpoints…“
• https://learn.microsoft.com/en-us/microsoft-365/enterprise/microsoft-365-networking-overview 

Example AWS:

• „…To connect to your WorkSpaces, the network that your WorkSpaces clients are connected to must have certain ports open to the IP address ranges for the various AWS services…“
• https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html 

Options of the customer

The customer itself is not in a able to setup the transparency required by the DSK or to technically influence which data is processed by the contract data processors (Microsoft, AWS, Google, Salesforce, etc.).

In the Microsoft cloud services, the customer can only partially influence the "transparency" by using the Double Key Encryption technology.

However, this does not apply to all types of personal data. For example, it cannot be used to encrypt user data in Azure AD. The "Bring your Own Key" technology, which then also affects tenant encryption as a whole, requires that the own key is uploaded to a Key Vault in Azure. This key is then also stored in the Microsoft Cloud and Microsoft therefore has access to it, at least technically. For details see: Service encryption with Microsoft Purview Customer Key

So what to do?

The DSK…

The DSK (German Data Protection Conference) is a committee of the independent data protection authorities of the German government and the German Federal States. Its focus is on compliance with data protection in the non-public sector. The opinions and guidance published by the DSK are based on the data protection laws of Germany and the German Federal States.

Its focus is on compliance with data protection in the non-public sector. The opinions and guidance provided by the conference are not legally binding. However, they have a de facto impact on the future of data privacy in Germany due to the expertise and authority of the conference members.

DSK says about the use of M365...

Translated quote from: AG DSK „Microsoft-Onlinedienste“ Zusammenfassung der Bewertung der aktuellen Vereinbarung zur Auftragsverarbeitung, November 24, 2022

...data controllers must be in a position at all times to meet their accountability obligations under Article 5 (2) GDPR. When using Microsoft 365, difficulties can still be expected in this regard on the basis of the "data protection supplement", as Microsoft does not fully disclose which processing operations take place in detail. In addition, Microsoft does not fully disclose which processing operations take place on behalf of the customer or which take place for its own purposes. The contract documents are not precise in this respect and, as a result, do not permit conclusively assessable, possibly even extensive processing also for the customer's own purposes...

Microsoft says this...

Quote from: Our Continued Commitment to Meet and Exceed EU Data Privacy Laws, November 25, 2022

… We respectfully disagree with the DSK position as we ensure that our M365 products not only meet, but often exceed, the strong data privacy laws in the European Union. Our customers in Germany and across the EU can confidently use the M365 products in a legally compliant way…

What now?

From the customer's perspective, this is a tricky situation. On the one hand, an institution whose statements are not binding, but are weighty, said NO to M365 for German customers. On the other hand, Microsoft says, "Our customers in Germany and throughout the EU can continue to use M365 products without hesitation and in a legally secure manner.

All in all, the following wording sums it up perfectly for now:

The statement of the DSK is nothing more and nothing less than the legal opinion of a committee of the executive authority. This is not binding. The judiciary, i.e. the German and European courts, has the final word

Or to put it in the words from the Microsoft article:

We look forward to the new framework becoming the basis for a positive European Commission adequacy decision under the GDPR in 2023.

Your file server in the cloud - What is Azure Files?

Azure Files provides fully managed file shares in the cloud, accessible via SMB as well as via NFS.

Azure Files offers all the features expected by a modern file server: Encryption in transit / Encryption at rest, Soft delete, Backup & Recovery as well as monitoring by Microsoft Defender for Storage. A search capability can also be implemented via Azure Cognitive Search and Azure File Indexer. This makes this solution an interesting alternative to the classic file server, which is located in a data center.

SMB file shares in Azure Files can be accessed from Windows, Linux, and macOS clients. NFS shares are available for access from Linux or macOS clients. SMB file shares in Azure Files can also be cached on Windows servers in your own data center, using Azure File Sync. This can ensure fast access for large files.

Key benefits

• Shared access: Azure Files support both SMB and NFS standard protocols. This makes it easy to replace local file servers with Azure Files without worrying about application compatibility. 
• Complete management: Azure Files can be set up without having to worry about the hardware or operating systems. Security upgrades or hardware failures are no longer an ongoing issue to worry about.
• Scripts and Tools: PowerShell cmdlets and the Azure CLI can be used to manage Azure Files. The Azure Portal or Azure Storage Explorer can also be used for management.
• Resilience: Azure Files is designed from the ground up as a highly available solution. Unlike on-premises file servers, failsafe power or networks are not an issue here.
• Access via code: Applications running in Azure can access data in Azure Files via file system I/O APIs. Developers can therefore use their existing code and previously learned skills to migrate existing applications. In addition to system I/O APIs, Azure Storage client libraries or the Azure Storage REST API can also be used.

Typical use cases of Azure Files

• Replace or extend local file servers: Azure Files can replace or extend local file servers or NAS system. Common operating systems such as Windows, macOS, and Linux can directly integrate Azure Files. SMB file shares in Azure can be replicated to Windows servers (either on-premises or in the cloud) via Azure File Sync to provide high performance and distributed caching for large files at the point of use. With the current release of Azure Files AD authentication, SMB file shares in Azure can still be used with the locally hosted AD instance for access control.
• Lift & Shift Applications: Azure Files simplifies Lift & Shift projects to the cloud for applications where file shares are expected to store data. Azure Files enables not only the classic Lift & Shift scenario, where both the application and its associated data are moved to Azure, but also the Lift & Shift hybrid scenario, where the application data is moved to Azure Files and the application continues to run locally.

Indexer in Azure - Cognitive Search for Azure Files

An indexer in Azure Cognitive Search is a crawler that extracts content from cloud data sources and creates a search index using field-to-field mappings between source data and a search index. This approach is also known as the "pull model" because the search service retrieves data without writing any code. Indexers can be run on demand or on a schedule for regular data updates.

Besides the common file formats such as Microsoft Office formats and PDF, the following formats are also supported by the indexer: CSV, EML, EPUB, GZ, HTML, JSON, KML, , ODT, ODS, ODP, TXT, RTF, XML, ZIP.

Microsoft Defender for Storage & Azure Files

Protection for Azure Storage to support Azure Files and Azure Data Lake Storage Gen2 API was introduced by Microsoft in the year 2020. Advanced Threat Protection for Azure Storage provides an additional layer of security intelligence that pushes alerts when unusual and potentially malicious activity is detected.

These security alerts are integrated with Azure Security Center and are also emailed to subscription administrators. Details about the suspicious activity and recommendations on how to investigate and remediate threats are integrated.