Large language models are powerful tools for summarizing big texts or data, which is super useful in areas like cybersecurity. When dealing with cybersecurity incidents, analysts often face huge amounts of messy data. Having a short summary of what happened can be a big help to get started on solving the issue. This is where promptbooks come in handy. They’re tools that let you break down the big task into smaller parts and set up a series of steps or prompts to tackle each part. This way, even if a language model can’t handle too much information at once because of its limits, you can still get a detailed summary by working through the data bit by bit. Plus, you can give specific instructions for each part, making sure the summary covers everything important.

Azure AI Studio

Creating promptbooks is not a hard task today as we are served by Azure AI studio. Azure AI Studio is a powerful tool designed for developers to easily create, deploy, and manage AI projects. It’s a one-stop-shop for working with various AI models, linking up with Azure’s AI services, and building smart AI applications. From the beginning steps of creating AI solutions to putting them into use, Azure AI Studio supports the whole process. It includes ready-to-use and customizable models and APIs for different AI tasks, like generating content, understanding languages, enhancing searches, and more, in areas such as text processing, translating languages, recognizing speech and images, and making decisions. The platform is also built with a focus on responsible AI, meaning it guides users in creating AI solutions that are ethical and consider the broader impacts on society and the environment.

The Azure AI studio is accessible trough ai.azure.com

Creating your first promptbook

In Azure AI Studio, the concept of projects plays a central role. A project can consist of a variety of components, including multiple prompt flows, datasets, deployments, and content filters. To start building your project, navigate to the “prompt flows” option in the menu and click on the “create” button. This action will launch a gallery showcasing a range of pre-designed prompt flows. For creating a summary flow, select the “standard flow” option and proceed to create your flow. Initially, this will generate a new prompt flow designed to tell a joke.

As a first step, it’s advisable to clear the slate by removing all inputs, outputs, and the “nodes” that are part of the prompt flow. This approach gives you a clean starting point to build the flow according to your specific requirements.

Input & Outputs

Let’s begin by setting up the input and output parameters for our prompt flow. Input parameters are essentially the data that needs to be condensed into a summary. Taking a cybersecurity incident as an example, I’ve outlined the following inputs:

The incident title (string)
Incident description (string)
Activity log of the incident (that contains everything that has been done) (string)
Entities (a list of ip addresses, hosts, files etc. that play a role in this incident (string)
The status (string)
The triage result (string)
The riage reason (string)

You also need to define a output. As I only want a summary as output, I have only defined one output called “summary”

If you have sample data availabe for the inputs, you can fill this in at the values column. You can later use this data to test your flow.

Building the Promptflow

Now it is time to build the prompt flow! A promptflow consists of multiple “prompts” chained together. In case of the cyber security incident example, I will first create a summary of the activity log and entities before doing somethign else. These texts are often that long that I will process them using their own prompts. Aside from the large amounts of data, I also would like to summarize them using different instructions.

You can use variables in your promtflow by using {{variable}}. After you have written your prompt, click on “validate and parse input”. In order to validate your prompt, a runtime ins requried. In the top menubar your are able to let Azure AI Studio create a runtime for your. This runtime will automatically shutdown when its not being used for one hour.
After your runtime is deployed and you have parsed your input, you can bind your variable to an input parameter.

To have this flow working I have deployed a Azure OpenAI instance in Microsoft Azure. Within the OpenAI instance, a GPT 3.5 Turbo model is deployed. This model is defined as connection in AzureAI studio. As displayed in the screenshots: in each of my prompts I use a connection to “Default_AzureOpenAI’ with the “gpt-35-turbo” deployment. This is the GPT 3.5 model deployed in my Azure OpenAI instance.

To learn more about connections and the Azure OpenAI service, please check: Microsoft Learn

Chaining prompts

The power of promptflows lies in chaining prompts. Using the output of one prompt as input for the next flow allow you to process your data in setps.

In case of the cyber security incident I would like to generate a conclusion based on the prompt that generates a summary for my activitylog and a prompt that generates a summary about my entities. I also give it inputs that were defined when creating this promptflow.

Generating the summary

The last part will be writing the prompt that creates the summary. This is a prompt that probably needs a little more attention than the previous ones. In this prompt I will define my “summary template”. By using a template, I can make sure that my summary is always well formed and build-up on a standard way. The summary can be easy told the prompt using natural language.

# system:

You act as a senior cyber security analyst. You are an expert in writing incident summaries

# user:

An summary has the following format:
“Summary: [title here]
Event date: [date of incident here]

Description
[description here]

Entities
[entities here]

Activities executed by SOC
[activities]

Conclusion
[conclusion here]

This RCA is generated using AI and may be incorrect“

Act as a senior security analyst and write a full RCA based on:
– Title: “{{title}}”
– Description: “{{description}}”
– Entities: “{{entitySummary}}”
– Activities: “{{activities}}”
– Conclusion: “{{conclusion}}”

The final result

Once you’ve set up and connected all the prompts, you’re ready to generate the initial summary. Simply click the “run” button to activate your promptbook. Upon completion, head over to the summary prompt to review its outputs. Your summary will be waiting for you there, ready for examination and use.

If the summery does not fullfill your requirements, it might be that some of the prompts need to be tuned. In order to do so check the outputs of the other prompts in your flow and adjust the prompt so it will take care of your requirement.

Conclusion

To wrap it up, prompt flows are incredibly powerful tools in the AI world, making it much easier to sort through and summarize big chunks of information, like those we often see in cybersecurity. Using promptflows you can break down the data into manageable pieces, helping the AI model to understand it better. But, to get the most out of prompt flows, some adjustments might be needed. Fine-tuning the prompts helps ensure the information we get is exactly what we need.

The next step in making the promptflow available as REST API with Azure AI Studio. This means the prompt flow can be connected to and used by other software and scripts easily, opening up lots of possibilities for using AI in different ways.

Create complex summaries using promptbooks