Hi there. The code below might be of some interest to other programmers who are working with the GPT API from OpenAI. The API has been somewhat slow lately, and it is often cumbersome to debug code while waiting for GPT to respond. I have faced a lot of timeouts, particularly with GPT4, which is particularly frustrating when the AI response is not the main focus of my programming session, but just a step in a larger process.
I am about to develop some multithreaded GPT programs that uses different GPT4 instantiations to complete different cognitive tasks. I don’t really want to blow through my token budget on mere debugging, and often the issue is not with the GPT 4 response, but with the surrounding application.
Also, if I am going to have an error-checking thread or a guidance thread in a multithreaded GPT-based cognitive system, it will be useful to be able to force the AI to answer in a certain way to test the various modules, such as the error-checking thread or get the nuts and bolts of the executive guidance system.
With this class, which lets humans stand in for AIs, I can make the “AI” as dumb or as smart as I want, and force it to make specific errors to test the other parts of the multi-threaded program.
The class below, HumanOrAIChatCompletion, written in Python, is a subclass of openai.ChatCompletion, inheriting all of its methods. For just one of those methods, it departs from the parent class by allowing the model to be specified as “human” instead of “gpt-4” or “gpt-3.5-turbo”, and so on. Where a program would normally have a line like response = openai.ChatCompletion.create( … ), the programmer can nor use myCompletion.create( … ). If myCompletion is an instance of HumanOrAIChatCompletion, then the model can be specified as “human”, and the response text can be written or pasted in.
This is already proving useful for a program I am writing where I am getting GPT4 to grade AI responses to an automated cognitive test. I already know the standard AI answers, and I can use this new class to test how GPT4 grades better or worse answers by having a mock AI provide answers I choose. The rest of the code doesn’t need to factor in the fact that the “AI” here is actually a human.
Comments or suggestions for improvements welcome. This is a niche application, obviously, but I know it will prove useful for me so I thought I would share it.
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QPushButton, QTextEdit, QLabel, QHBoxLayout, QDialog
import openai
import time
openai.api_key = 'YOUR_OPEN_AI_KEY'
class HumanOrAIChatCompletion(openai.ChatCompletion):
def __init__(self):
super().__init__()
default_model = "human"
self.result = None
def simplify_messages(self, messages):
simplified_messages = []
for msg in messages:
role = msg['role']
content = msg['content']
if role == 'user':
simplified_msg = f"User: {content}"
elif role == 'assistant':
simplified_msg = f"Assistant: {content}"
else:
simplified_msg = f"{role.capitalize()}: {content}"
simplified_messages.append(simplified_msg)
return '\n\n'.join(simplified_messages)
def get_user_input(self, messages, max_tokens, n, stop, temperature):
app = QApplication([])
dialog = QDialog()
dialog.setWindowTitle("Choose an agent to respond")
# Find the system message
system_message = next((msg for msg in messages if msg['role'] == 'system'), None)
if system_message:
# Create a label for the system message
system_label = QLabel()
system_label.setText(f"System: {system_message['content']}")
# Create a label for the user input
label = QLabel()
label.setText("Human Response:")
# Create buttons
button_gpt35 = QPushButton("GPT3.5")
button_gpt4 = QPushButton("GPT4")
button_ok = QPushButton("Okay")
button_cancel = QPushButton("Cancel")
button_gpt35.clicked.connect(lambda: input_field.setPlainText(button_gpt35.text()))
button_gpt4.clicked.connect(lambda: input_field.setPlainText(button_gpt4.text()))
button_ok.clicked.connect(dialog.accept)
button_cancel.clicked.connect(dialog.reject)
# Create text input field
input_field = QTextEdit()
input_field.setFixedWidth(400)
# Create layout for buttons
button_layout = QHBoxLayout()
button_layout.addWidget(button_ok)
button_layout.addWidget(button_cancel)
message_window = QTextEdit()
message_window.setReadOnly(True)
# Remove the system message from the messages list
messages = [msg for msg in messages if msg['role'] != 'system']
message_string = self.simplify_messages(messages)
message_window.setText(message_string)
# Create main layout and add labels/input field/buttons
layout = QVBoxLayout()
if system_message:
system_layout = QVBoxLayout()
system_layout.addWidget(system_label)
system_layout.setContentsMargins(0, 10, 10, 10) # Adjust the margins as desired
layout.addLayout(system_layout)
layout.addWidget(message_window)
layout.addWidget(label)
layout.addWidget(input_field)
layout.addWidget(button_gpt35)
layout.addWidget(button_gpt4)
layout.addLayout(button_layout)
dialog.setLayout(layout)
# Add options to the dialog
result = dialog.exec_()
print("RESULT:" + str(result))
if (result == QDialog.Rejected):
return None
else:
# Setting up the variables
message_content = input_field.toPlainText()
if (message_content.__eq__("GPT3.5")):
print("Calling AI from dialog...(gpt-3.5-turbo)...")
return super().create(
model="gpt-3.5-turbo",
messages=messages,
max_tokens=max_tokens,
n=n,
stop=stop,
temperature=temperature
)
if (message_content.__eq__("GPT4")):
print("Calling AI from dialog...(gpt-4)...")
return super().create(
model="gpt-4",
messages=messages,
max_tokens=max_tokens,
n=n,
stop=stop,
temperature=temperature
)
message_role = "assistant"
finish_reason = "stop"
message_index = 0
created_time = int(time.time()) # this is a Unix imestamp
completion_id = "human_completion"
model_name = "human"
object_type = "chat.completion"
completion_tokens = 0
prompt_tokens = 0
total_tokens = 0
# Constructing the response dictionary
response = {
"choices": [
{
"finish_reason": finish_reason,
"index": message_index,
"message": {
"content": message_content,
"role": message_role
}
}
],
"created": created_time,
"id": completion_id,
"model": model_name,
"object": object_type,
"usage": {
"completion_tokens": completion_tokens,
"prompt_tokens": prompt_tokens,
"total_tokens": total_tokens
}
}
return response
def create(self, model, messages, max_tokens, n, stop, temperature):
self.messages = messages
if model == "human":
dialog_response = self.get_user_input(messages, max_tokens, n, stop, temperature)
if dialog_response is None:
return None
else:
print("Dialog response:", dialog_response)
return dialog_response
else:
print("Calling AI...(" + model + ")...")
return super().create(
model=model,
messages=messages,
max_tokens=max_tokens,
n=n,
stop=stop,
temperature=temperature
)
if __name__ == "__main__":
myCompletion = HumanOrAIChatCompletion()
#change line below to use a different default model. Use human to check/force specific responses, to save tokens, or to avoid long response times or timeouts
myCompletion.defaultModel = "human" #"gpt-3.5-turbo" #options: gpt-4, human, gpt-3.5-turbo, etc. Note, all models are lower case.
messages = [{"role": "system", "content": "You are a helpful assistant."}]
for i in range(3):
messages.append({"role": "user", "content": "Blah"})
messages.append({"role": "assistant", "content": "As an AI..."})
messages.append({"role": "user", "content": "What is the Hard Problem of Consciousness?"})
response = myCompletion.create(
model=myCompletion.defaultModel,
messages=messages,
max_tokens=4000,
n=1,
stop=None,
temperature=0.5
)
if response is None:
print("Cancelled")
else:
response_text = response['choices'][0]['message']['content']
print("CONTENT OF RESPONSE:\n" + response_text)
Coded with the help of GPT4, this bit of code creates a new class that can employ a “human” model as a substitute version of the GPT API.
Works even without an API key, but the human might need coffee from time to time.
What?
Maybe you could explain wtf you’re doing?
Sure… I just posted the code, which will make sense to other coders.
I am about to add more context. Do you code?