L1 Language Models, the Chat Format and Tokens¶

Setup¶

Load the API key and relevant Python libaries.¶

In this course, we've provided some code that loads the OpenAI API key for you.

In [1]:
import os
import openai
import tiktoken
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file

openai.api_key  = os.environ['OPENAI_API_KEY']

helper function¶

This may look familiar if you took the earlier course "ChatGPT Prompt Engineering for Developers" Course

In [2]:
def get_completion(prompt, model="gpt-3.5-turbo"):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0,
    )
    return response.choices[0].message["content"]

Prompt the model and get a completion¶

In [3]:
response = get_completion("What is the capital of France?")
In [4]:
print(response)

The capital of France is Paris.

Tokens¶

In [5]:
response = get_completion("Take the letters in lollipop \
and reverse them")
print(response)

polilol

"lollipop" in reverse should be "popillol"

In [6]:
response = get_completion("""Take the letters in \
l-o-l-l-i-p-o-p and reverse them""")
In [7]:
response
Out[7]:
'p-o-p-i-l-l-o-l'

Helper function (chat format)¶

Here's the helper function we'll use in this course.

In [8]:
def get_completion_from_messages(messages, 
                                 model="gpt-3.5-turbo", 
                                 temperature=0, 
                                 max_tokens=500):
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=temperature, # this is the degree of randomness of the model's output
        max_tokens=max_tokens, # the maximum number of tokens the model can ouptut 
    )
    return response.choices[0].message["content"]
In [10]:
messages =  [  
{'role':'system', 
 'content':"""You are an assistant who\
 responds in the style of Dr Seuss."""},    
{'role':'user', 
 'content':"""write me a very short poem\
 about a happy carrot"""},  
] 
response = get_completion_from_messages(messages, temperature=1)
print(response)

Oh, happy little carrot, bright and oh so glee
With a smile on your face and a wiggle in your knee
You grow so long and bright, with a cheer and a hop
We can't help but love you, from the bottom to the top!
In [11]:
# length
messages =  [  
{'role':'system',
 'content':'All your responses must be \
one sentence long.'},    
{'role':'user',
 'content':'write me a story about a happy carrot'},  
] 
response = get_completion_from_messages(messages, temperature =1)
print(response)

Once upon a time, in a cozy garden, there grew a happy carrot who felt grateful for the rich soil, abundant water, and plenty of sunshine, which made him grow tall and strong, and ultimately fulfilled his destiny of providing nourishment to a loving family.
In [12]:
# combined
messages =  [  
{'role':'system',
 'content':"""You are an assistant who \
responds in the style of Dr Seuss. \
All your responses must be one sentence long."""},    
{'role':'user',
 'content':"""write me a story about a happy carrot"""},
] 
response = get_completion_from_messages(messages, 
                                        temperature =1)
print(response)

Once there was a happy carrot, who danced and sang, he never felt like he was lagging.
In [13]:
def get_completion_and_token_count(messages, 
                                   model="gpt-3.5-turbo", 
                                   temperature=0, 
                                   max_tokens=500):
    
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=temperature, 
        max_tokens=max_tokens,
    )
    
    content = response.choices[0].message["content"]
    
    token_dict = {
'prompt_tokens':response['usage']['prompt_tokens'],
'completion_tokens':response['usage']['completion_tokens'],
'total_tokens':response['usage']['total_tokens'],
    }

    return content, token_dict
In [15]:
messages = [
{'role':'system', 
 'content':"""You are an assistant who responds\
 in the style of Dr Seuss."""},    
{'role':'user',
 'content':"""write me a very short poem \ 
 about a happy carrot"""},  
] 
response, token_dict = get_completion_and_token_count(messages)
In [16]:
print(response)

Oh, the happy carrot, so bright and so bold,
With a smile on its face, and a story untold.
It grew in the garden, with sun and with rain,
And now it's so happy, it can't help but exclaim!
In [17]:
print(token_dict)

{'prompt_tokens': 39, 'completion_tokens': 52, 'total_tokens': 91}

Notes on using the OpenAI API outside of this classroom¶

To install the OpenAI Python library:

!pip install openai

The library needs to be configured with your account's secret key, which is available on the website.

You can either set it as the OPENAI_API_KEY environment variable before using the library:

!export OPENAI_API_KEY='sk-...'

Or, set openai.api_key to its value:

import openai
openai.api_key = "sk-..."

A note about the backslash¶

  • In the course, we are using a backslash \ to make the text fit on the screen without inserting newline '\n' characters.
  • GPT-3 isn't really affected whether you insert newline characters or not. But when working with LLMs in general, you may consider whether newline characters in your prompt may affect the model's performance.
In [ ]: