This notebook is adapted from the Azure OpenAI Samples Repository that includes notebooks that access Azure OpenAI services.
"Large language models are functions that map text to text. Given an input string of text, a large language model tries to predict the text that will come next"(1). This "quickstart" notebook will introduce users to high-level LLM concepts, core package requirements for getting started with AML, a soft introduction to prompt design, and several short examples of different use cases.
This short exercise will provide a basic introduction for submitting prompts to a model in Github Models for a simple task "summarization".
Steps:
azure-ai-inference library in your python environment, if you haven't done so.azure-ai-inference¶%pip install azure-ai-inference
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential
token = os.environ["GITHUB_TOKEN"]
endpoint = "https://models.inference.ai.azure.com"
client = ChatCompletionsClient(
endpoint=endpoint,
credential=AzureKeyCredential(token),
)
The GPT-3.5-turbo or GPT-4 models can understand and generate natural language.
# Select the General Purpose curie model for text
model_name = "gpt-4o"
"The magic of large language models is that by being trained to minimize this prediction error over vast quantities of text, the models end up learning concepts useful for these predictions. For example, they learn concepts like"(1):
"Of all the inputs to a large language model, by far the most influential is the text prompt(1).
Large language models can be prompted to produce output in a few ways:
Instruction: Tell the model what you want Completion: Induce the model to complete the beginning of what you want Demonstration: Show the model what you want, with either: A few examples in the prompt Many hundreds or thousands of examples in a fine-tuning training dataset"
Show and tell. Make it clear what you want either through instructions, examples, or a combination of the two. If you want the model to rank a list of items in alphabetical order or to classify a paragraph by sentiment, show it that's what you want.
Provide quality data. If you're trying to build a classifier or get the model to follow a pattern, make sure that there are enough examples. Be sure to proofread your examples — the model is usually smart enough to see through basic spelling mistakes and give you a response, but it also might assume this is intentional and it can affect the response.
Check your settings. The temperature and top_p settings control how deterministic the model is in generating a response. If you're asking it for a response where there's only one right answer, then you'd want to set these lower. If you're looking for more diverse responses, then you might want to set them higher. The number one mistake people make with these settings is assuming that they're "cleverness" or "creativity" controls.
Source: https://github.com/Azure/OpenAI/blob/main/How%20to/Completions.md
# Create your first prompt
text_prompt = "Should oxford commas always be used?"
response = client.complete(
model=model_name,
messages = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":text_prompt},])
response.choices[0].message.content
response = client.complete(
model=model_name,
messages = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":text_prompt},])
response.choices[0].message.content
Summarize text by adding a 'tl;dr:' to the end of a text passage. Notice how the model understands how to perform a number of tasks with no additional instructions. You can experiment with more descriptive prompts than tl;dr to modify the model’s behavior and customize the summarization you receive(3).
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something that current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
Tl;dr
prompt = "Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something that current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.\n\nTl;dr"
#Setting a few additional, typical parameters during API Call
response = client.complete(
model=model_name,
messages = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt},])
response.choices[0].message.content
Classify items into categories provided at inference time. In the following example, we provide both the categories and the text to classify in the prompt(*playground_reference).
Customer Inquiry: Hello, one of the keys on my laptop keyboard broke recently and I'll need a replacement:
Classified category:
prompt = "Classify the following inquiry into one of the following: categories: [Pricing, Hardware Support, Software Support]\n\ninquiry: Hello, one of the keys on my laptop keyboard broke recently and I'll need a replacement:\n\nClassified category:"
print(prompt)
#Setting a few additional, typical parameters during API Call
response = client.complete(
model=model_name,
messages = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt},])
response.choices[0].message.content
Create product names from examples words. Here we include in the prompt information about the product we are going to generate names for. We also provide a similar example to show the pattern we wish to receive. We have also set the temperature value high to increase randomness and more innovative responses.
Product description: A home milkshake maker Seed words: fast, healthy, compact. Product names: HomeShaker, Fit Shaker, QuickShake, Shake Maker
Product description: A pair of shoes that can fit any foot size. Seed words: adaptable, fit, omni-fit.
prompt = "Product description: A home milkshake maker\nSeed words: fast, healthy, compact.\nProduct names: HomeShaker, Fit Shaker, QuickShake, Shake Maker\n\nProduct description: A pair of shoes that can fit any foot size.\nSeed words: adaptable, fit, omni-fit."
print(prompt)
#Setting a few additional, typical parameters during API Call
response = client.complete(
model=model_name,
messages = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt}])
response.choices[0].message.content