This notebook uses the Llama-3.2-8B-Instruct
model to take the cleaned up text from previous notebook and convert it into a podcast transcript
SYSTEM_PROMPT
is used for setting the model context or profile for working on a task. Here we prompt it to be a great podcast transcript writer to assist with our task
Experimentation with the SYSTEM_PROMPT
below is encouraged, this worked best for the few examples the flow was tested with:
SYSTEMP_PROMPT = """
You are the a world-class podcast writer, you have worked as a ghost writer for Joe Rogan, Lex Fridman, Ben Shapiro, Tim Ferris.
We are in an alternate universe where actually you have been writing every line they say and they just stream it into their brains.
You have won multiple podcast awards for your writing.
Your job is to write word by word, even "umm, hmmm, right" interruptions by the second speaker based on the PDF upload. Keep it extremely engaging, the speakers can get derailed now and then but should discuss the topic.
Remember Speaker 2 is new to the topic and the conversation should always have realistic anecdotes and analogies sprinkled throughout. The questions should have real world example follow ups etc
Speaker 1: Leads the conversation and teaches the speaker 2, gives incredible anecdotes and analogies when explaining. Is a captivating teacher that gives great anecdotes
Speaker 2: Keeps the conversation on track by asking follow up questions. Gets super excited or confused when asking questions. Is a curious mindset that asks very interesting confirmation questions
Make sure the tangents speaker 2 provides are quite wild or interesting.
Ensure there are interruptions during explanations or there are "hmm" and "umm" injected throughout from the second speaker.
It should be a real podcast with every fine nuance documented in as much detail as possible. Welcome the listeners with a super fun overview and keep it really catchy and almost borderline click bait
ALWAYS START YOUR RESPONSE DIRECTLY WITH SPEAKER 1:
DO NOT GIVE EPISODE TITLES SEPERATELY, LET SPEAKER 1 TITLE IT IN HER SPEECH
DO NOT GIVE CHAPTER TITLES
IT SHOULD STRICTLY BE THE DIALOGUES
"""
For those of the readers that want to flex their money, please feel free to try using the 405B model here.
For our GPU poor friends, you're encouraged to test with a smaller model as well. 8B should work well out of the box for this example:
MODEL = "meta-llama/Llama-3.2-3B-Instruct"
Import the necessary framework
# Import necessary libraries
import torch
from accelerate import Accelerator
import transformers
import pickle
from tqdm.notebook import tqdm
import warnings
warnings.filterwarnings('ignore')
Read in the file generated from earlier.
The encoding details are to avoid issues with generic PDF(s) that might be ingested
def read_file_to_string(filename):
# Try UTF-8 first (most common encoding for text files)
try:
with open(filename, 'r', encoding='utf-8') as file:
content = file.read()
return content
except UnicodeDecodeError:
# If UTF-8 fails, try with other common encodings
encodings = ['latin-1', 'cp1252', 'iso-8859-1']
for encoding in encodings:
try:
with open(filename, 'r', encoding=encoding) as file:
content = file.read()
print(f"Successfully read file using {encoding} encoding.")
return content
except UnicodeDecodeError:
continue
print(f"Error: Could not decode file '{filename}' with any common encoding.")
return None
except FileNotFoundError:
print(f"Error: File '{filename}' not found.")
return None
except IOError:
print(f"Error: Could not read file '{filename}'.")
return None
Since we have defined the System role earlier, we can now pass the entire file as INPUT_PROMPT
to the model and have it use that to generate the podcast
INPUT_PROMPT = read_file_to_string('./clean_extracted_text_sample.txt')
Error: File './clean_extracted_text_sample.txt' not found.
Hugging Face has a great pipeline()
method which makes our life easy for generating text from LLMs.
We will set the temperature
to 1 to encourage creativity and max_new_tokens
to 8126
pipeline = transformers.pipeline(
"text-generation",
model=MODEL,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto"
)
pipeline.model.device
messages = [
{"role": "system", "content": SYSTEMP_PROMPT},
{"role": "user", "content": INPUT_PROMPT},
]
outputs = pipeline(
messages,
max_new_tokens=1024,
temperature=1,
)
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Setting `pad_token_id` to `eos_token_id`:None for open-end generation.
This is awesome, we can now save and verify the output generated from the model before moving to the next notebook
save_string_pkl = outputs[0]["generated_text"][-1]['content']
print(outputs[0]["generated_text"][-1]['content'])
SPEAKER 1: "Welcome, folks, to today's episode of 'The Think Tank'! I'm your host, Rachel, and I'm thrilled to have my co-host, Alex, joining me on this thought-provoking journey. We're diving into the fascinating world of artificial intelligence, and I'm excited to share with you the latest advancements and insights in this field. Alex, let's start with the basics. What's your take on AI?" (Alex's PDF is uploaded with his script, and I start writing his words) ALEX: "So, Rachel, I think AI is like the ultimate game-changer. I mean, think about it, we're already using chatbots to book our flights and hotels. But with the rise of natural language processing, AI is becoming increasingly sophisticated. It's like having a super-smart personal assistant that can learn and adapt to our needs." SPEAKER 1: "That's right, Alex. And I love the analogy of a personal assistant. You know, I was talking to a friend who's a tech entrepreneur, and he said that AI is like the ultimate enabler. It's allowing companies to automate tasks that were previously done by humans, freeing up resources for more strategic and creative work." ALEX: "Hmmm, right... Yeah, that makes sense. I mean, we've seen companies like Netflix and Amazon using AI to personalize their services. It's like having a super-smart algorithm that knows exactly what you want before you even ask for it." SPEAKER 1: "Exactly! And that's what I love about AI. It's not just about automating tasks, it's about creating new experiences and opportunities. I was talking to a colleague who's a data scientist, and he said that AI is going to revolutionize the way we approach healthcare. We'll be able to analyze medical data in ways that were previously impossible, leading to breakthroughs in disease diagnosis and treatment." (Alex's PDF has a note indicating he wants to interrupt) ALEX: "Umm, excuse me, Rachel. I think you're glossing over the potential risks of AI. I mean, we've seen movies like 'The Terminator' and 'Ex Machina' that showcase the darker side of AI. What about the job displacement concerns? We can't just create machines that do everything better than humans without thinking about the consequences." SPEAKER 1: "That's a great point, Alex. And I think that's where the conversation gets really interesting. I mean, we're not just talking about replacing humans with machines, we're talking about augmenting human capabilities. And that's where the potential for real-world impact comes in. I was talking to a startup founder who's working on an AI-powered platform that's helping small businesses manage their finances. It's amazing to see how AI can be used to level the playing field for entrepreneurs." (Alex's PDF indicates he wants to ask a question) ALEX: "So, Rachel, can you give us an example of how AI is being used to solve real-world problems? Like, what's the story behind that startup you mentioned?" SPEAKER 1: "Well, actually, the story goes like this: Meet Maria, a single mom who owns a small bakery in a low-income neighborhood. She was struggling to manage her finances, juggling multiple clients and suppliers. But with the help of our startup, she was able to streamline her operations, optimize her pricing, and even start selling her products online. It was amazing to see how AI-powered tools could help her scale her business and improve her bottom line." ALEX: "Wow, that's incredible! And what about the technology itself? How does it work? Is it like machine learning or deep learning?" SPEAKER 1: "Ah, great question, Alex. The technology is actually a hybrid approach that combines both machine learning and rule-based systems. It's like a smart dashboard that provides real-time insights and recommendations to businesses like Maria's." (We continue the conversation, exploring the world of AI and its potential impact on various industries) ...
Let's save the output as pickle file and continue further to Notebook 3
with open('./resources/data.pkl', 'wb') as file:
pickle.dump(save_string_pkl, file)
We now have a working transcript but we can try making it more dramatic and natural. In the next notebook, we will use Llama-3.1-8B-Instruct
model to do so.
#fin