Quick Sho{r}t Exercises
Ex1. Building a Local LLM and extract information
ollama run llama3.1
from langchain_community.llms import Ollama
#ollama run llama3.1
llm = Ollama(model="llama3.1")
response = llm("Tell me about Mahatma Gandhi")
print(response)
Response
A great figure in history! Mahatma Gandhi (1869-1948) was a Indian
independence activist, leader of the non-violent resistance movement
against British rule in India, and a key figure in the country's struggle
for freedom. He is widely regarded as one of the most influential
leaders of the 20th century.
**Early Life**
Gandhi was born on October 2, 1869, in Porbandar, a coastal town in the
state of Gujarat, India. His father, Karamchand Gandhi, was a local
leader and businessman, while his mother, Putlibai, was a devout Hindu
from a family of farmers. Gandhi's early life was marked by a strong
emphasis on spirituality and self-discipline.
**Education and Career**
Gandhi studied law at the University College London and later
practiced as a lawyer in India. However, he soon became disillusioned
with the British colonial system and its treatment of Indians.
In 1893, he traveled to South Africa to work as a lawyer for an Indian
firm, but his experience there exposed him to the harsh realities of
racial segregation and discrimination.
**Non-Violent Resistance**
Gandhi's experiences in South Africa influenced his approach to politics
and led him to develop the philosophy of non-violent resistance.
He believed that peaceful protests, civil disobedience, and boycotts
could be effective tools for achieving social change without resorting
to violence.
**Indian Independence Movement**
In 1915, Gandhi returned to India and joined the Indian National
Congress (INC), a party fighting for Indian independence from British
rule. He became one of its key leaders and used his non-violent
resistance methods to challenge the British government's authority.
Some notable examples include:
1. **Salt March** (1930): A 24-day march from Ahmedabad to Dandi,
where Gandhi and thousands of followers protested against the
British Salt Act by making salt themselves.
2. **Non-Cooperation Movement** (1920-1922): Gandhi called for
Indians to boycott British goods, schools, and courts, which led
to widespread protests and arrests.
3. **Quit India Movement** (1942): A campaign that urged the British
to leave India immediately, but was met with brutal suppression.
**Philosophy and Legacy**
Gandhi's philosophy of non-violent resistance, known as "Satyagraha"
(holding onto truth), emphasized:
1. **Non-violence**: The rejection of violence in all forms, including
physical and verbal aggression.
2. **Simple living**: Emphasis on simplicity, self-sufficiency,
and a commitment to reducing one's material needs.
3. **Love and compassion**: A focus on promoting empathy, understanding,
and reconciliation.
Gandhi's influence extends far beyond India:
1. **Civil Rights Movement** (USA): Gandhi's philosophy inspired leaders
like Martin Luther King Jr., who adopted similar non-violent resistance
methods in the American civil rights movement.
2. **International diplomacy**: Gandhi's approach has influenced global
efforts towards peaceful conflict resolution and disarmament.
**Assassination**
Tragically, Gandhi was assassinated on January 30, 1948, by a Hindu
nationalist named Nathuram Godse. The news of his death sent shockwaves
around the world, but it also cemented his legacy as a champion of peace
and non-violence.
Today, Gandhi's birthday, October 2, is celebrated as International
Day of Non-Violence, and he remains an iconic figure in history,
inspiring countless people worldwide with his message of love, compassion,
and peaceful resistance. """
Ex2: Private Chatbot
from langchain_ollama import OllamaLLM
from langchain_core.prompts import ChatPromptTemplate
template = """
Answer the Queries below
Here is the Chat History: {history}
Question: {querWHat y}
Answer :
"""
model = OllamaLLM(model="llama3.1")
prompt = ChatPromptTemplate.from_template(template)
chain = prompt | model
def answer_questions():
history = ""
print(" Welcome to iMMSS Super AI Bot! type 'quit' to exit")
while True:
user_input =input("You: ")
if user_input.lower() == "quit":
break
result = chain.invoke({"history": history, "query": user_input})
print(f"Bot: {result}")
history += f"iMMSS Bot: {result}\n"
if __name__ == "__main__":
answer_questions()
Steps:
ollama pull llama3.1
pulling manifest
pulling 948af2743fc7... 100% ▕▏ 1.5 KB
pulling 0ba8f0e314b4... 100% ▕▏ 12 KB
pulling 56bb8bd477a5... 100% ▕▏ 96 B
pulling 455f34728c9b... 100% ▕▏ 487 B
verifying sha256 digest
writing manifest
success
Prompt:
PS F:\ollama>python 1.py
Welcome to iMMSS Super AI Bot! type 'quit' to exit
You: Tell about Mahatma Gandhiji?
Bot: Mahatma Gandhi was an Indian independence activist who played a key
role in leading the country to freedom from British rule.Born on October 2,
1869, in Porbandar, India, he is widely regarded as one of the most
influential leaders of the 20th century.
Gandhi's early life was marked by his education in law and his
experiences in South Africa, where he fought against racial segregation
and discrimination. He returned to India in 1915 and began advocating
for Indian independence from British rule through non-violent civil
disobedience.
Gandhi led numerous movements and protests, including the
Non-Cooperation Movement (1920-1922) and the Salt March (1930),
which drew international attention to India's struggle for freedom.
His philosophy of non-violence and civil disobedience inspired leaders
across the world, including Martin Luther King Jr. in the United States.
Gandhi was assassinated on January 30, 1948, by a Hindu nationalist
who opposed his tolerance towards Muslims. Despite his tragic death,
Gandhi's legacy continues to inspire people worldwide with his
message of peace, love, and non-violence.
Some of his notable contributions include:
* Leading India to independence from British rule
* Promoting non-violent civil disobedience as a means of social change
* Fostering Hindu-Muslim unity in India through his tolerance and
acceptance
* Advocating for women's rights and empowerment
* Encouraging sustainable living and simple living principles
Gandhi's birthday, October 2, is celebrated as Gandhi Jayanti,
a national holiday in India. His life and teachings continue to
be a source of inspiration for people across the globe.
You:
EX3. Simple Usage
ollama ls
mistral:latest f974a74358d6 4.1 GB 19 minutes ago
mistral:latest f974a74358d6 4.1 GB 19 minutes ago
import requests
import json
# Set up the base URL for the local Ollama API
url = "http://localhost:11434/api/chat"
# Define the payload (your input prompt)
payload = {
"model": "mistral", # Replace with the model name you're using
"messages": [{"role": "user", "content": "Where is Annamalai
University?"}]
}
# Send the HTTP POST request with streaming enabled
response = requests.post(url, json=payload, stream=True)
# Check the response status
if response.status_code == 200:
print("Streaming response from Ollama:")
for line in response.iter_lines(decode_unicode=True):
if line: # Ignore empty lines
try:
# Parse each line as a JSON object
json_data = json.loads(line)
# Extract and print the assistant's message content
if "message" in json_data and "content" in
json_data["message"]:
print(json_data["message"]["content"], end="")
except json.JSONDecodeError:
print(f"\nFailed to parse line: {line}")
print() # Ensure the final output ends with a newline
else:
print(f"Error: {response.status_code}")
print(response.text)
"""
#Response :
Streaming response from Ollama:
Annamalai University is located in Chidambaram, Tamil Nadu, India.
It was established on September 26, 1947, and is one of the largest
universities in India by campus area. The university was founded by
Rajah Sir Raja Annamalai Chettiar."""
Ex4 :Client
import ollama
# Initialize the Ollama client
client = ollama.Client()
# Define the model and the input prompt
model = "mistral" # Replace with your model name
prompt = "What is python?"
# Send the query to the model
response = client.generate(model=model, prompt=prompt)
# Print the response from the model
print("Response from Ollama:")
print(response.response)
"""
Response from Ollama:
Python is a high-level, interpreted programming language that was
created by Guido van Rossum and first released in 1991. It is known
for its simplicity and readability, making it a great choice for
beginners to learn programming. Python supports multiple programming
paradigms, including procedural, object-oriented, and functional
programming.
Python has a large standard library that includes various modules
for tasks such as file input/output, web scraping, network programming,
GUI development, and data analysis, making it versatile for many different
types of projects.
It's also widely used in scientific computing, artificial intelligence,
machine learning, and data science due to its extensive set of libraries
like NumPy, Pandas, Matplotlib, and Scikit-learn.
Python can run on various operating systems such as Windows, Linux, and
macOS, and it's also available for mobile platforms (Python for Android
and iOS) and web browsers (Pyodide). The language is open-source and
free to use under the Python Software Foundation License.
"""
Ex5: Chat with PDF
from langchain.embeddings import GPT4AllEmbeddings
from langchain.llms import Ollama
from langchain.chains import RetrievalQA
from langchain.document_loaders import PyMuPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import
StreamingStdOutCallbackHandler
import time as timer
pdf_path = "Gandhi.pdf" # from Gandhi.PDF
loader = PyMuPDFLoader(pdf_path)
data = loader.load()
llm = Ollama(
base_url='http://localhost:11434',
model='mistral',
callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])
)
# Split the text into smaller chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=20,
chunk_overlap=0)
all_splits = text_splitter.split_documents(data)
# Define vectostore
vectorstore = Chroma.from_documents(documents=all_splits,
embedding=GPT4AllEmbeddings())
# Define RetrievalQA chain
chain = RetrievalQA.from_chain_type(
llm,
retriever=vectorstore.as_retriever(),
verbose=True
)
# Define prompt
query = "Explain about Prabu Siliwangi, an historical character
from West Java"
print(f"Query: {query}")
# docs = vectorstore.similarity_search(query)
# print(f"Docs (similarity search results): {docs}")
# Run the chain
start_t = timer.time()
response = chain({"query": query})
elapsed_t = timer.time() - start_t
print(f"\n\nElapsed time: {elapsed_t}")
"""
Response :
I'm sorry for the confusion, but it seems there's been a mix-up as
Prabu Siliwangi is not related to the context provided about
Mahatma Gandhi. Prabu Siliwangi was a legendary Sundanese king
who reigned over the ancient Sundanese kingdom of Pakuan or Pajajaran,
which is located in West Java, Indonesia during the 14th century.
He was known for his military prowess and successful defense against
foreign invasions, particularly that of the Majapahit Empire.
However, as I'm an AI model focused on assisting with questions
about Mahatma Gandhi, I don't have extensive knowledge about
Prabu Siliwangi or other historical figures outside of
the given context.
> Finished chain.
Elapsed time: 54.046849727630615
"""
#query = "Explain about Mahatma Gandhi and Non Violence"
"""
Response:
Mahatma Gandhi was a significant political and spiritual leader in
India during the 20th century, known for his advocacy of nonviolence
as a means to achieve independence and social justice. Born on
October 2, 1869, in Porbandar, Gujarat, India, Gandhi's early life
was influenced by various religious and ethical traditions,
including Hinduism, Jainism, and Christianity. These teachings
played a significant role in shaping his philosophy of nonviolence,
which he famously called Satyagraha (truth force).
The core of Gandhi's philosophy centered around the belief that
truth is eternal, unchangeable, and absolute, while violence is a
human creation that can lead to suffering and injustice.
He believed that an individual's duty was to seek truth,
respect all life forms, strive for self-purification, and
promote equality, justice, and peace. These principles formed
the foundation of his nonviolent resistance strategies, which
emphasized peaceful protests, civil disobedience, and active
suffering as a means of challenging oppressive systems and
achieving social change.
Gandhi's most well-known use of nonviolence was during
India's struggle for independence from British colonial rule.
Through various campaigns such as the Salt March (1930),
Quit India Movement (1942), and noncooperation movements,
Gandhi and his followers demonstrated their commitment
to nonviolence by engaging in peaceful protests, boycotting
British goods, and refusing to cooperate with the colonial government.
These campaigns played a crucial role in galvanizing public
support for independence and eventually led to India's
independence in 1947. Gandhi's influence extended far beyond India,
as his philosophy of nonviolence continues to inspire movements
for peace, justice, and human rights around the world.
He is often referred to as Mahatma (great soul) and remains
one of the most revered figures in modern history.
> Finished chain.
Elapsed time: 73.50113415718079
"""
Ex 6.
start-1.requirements.txt
# for start1-py
ollama
chromadb
pdfplumber
langchain
langchain-core
langchain-ollama
langchain-community
langchain_text_splitters
unstructured
unstructured[all-docs]
fastembed
pdfplumber
sentence-transformers
elevenlabs
#pip install start-1.requirements.txt
start-1.py
###Begin1-py
#ollama pull mistral
import requests
import json
url = "http://localhost:11434/api/generate"
data = {
"model": "mistral",
"prompt": "tell me a short story of Panachathanthra .
Make it only within 200 words. ",
}
response = requests.post(
url, json=data, stream=True
) # remove the stream=True to get the full response
# check the response status
if response.status_code == 200:
print("Generated Text:", end=" ", flush=True)
# Iterate over the streaming response
for line in response.iter_lines():
if line:
# Decode the line and parse the JSON
decoded_line = line.decode("utf-8")
result = json.loads(decoded_line)
# Get the text from the response
generated_text = result.get("response", "")
print(generated_text, end="", flush=True)
else:
print("Error:", response.status_code, response.text)
# Check
"""ollama ls
NAME ID SIZE MODIFIED
mistral:latest f974a74358d6 4.1 GB 14 hours ago
llama3.1:latest 46e0c10c039e 4.9 GB 24 hours ago
nomic-embed-text:latest 0a109f422b47 274 MB 30 hours ago
phi3:latest 4f2222927938 2.2 GB 32 hours ago"""
"""
###Response
"""
Generated Text: In the mystical kingdom of Vellai-Vanam, a wise and humble
woodcarver named Panachathanthra lived. His skills with wood were legendary,
but he was equally admired for his kindness and humility.
One day, a greedy ruler sought Panachathanthra's help in crafting an
idol of the Goddess Lakshmi, hoping to win wealth beyond measure.
Panachathanthra, knowing the true nature of the ruler, decided to
create not an idol, but a mirror instead.
The mirror was delivered to the king, who, upon seeing his own greedy
reflection, was shocked. He realized his error and sought forgiveness
from both the woodcarver and himself. The king then dedicated his kingdom
to acts of kindness and charity, becoming a benevolent ruler.
Panachathanthra, though poor in material wealth, lived contentedly.
His wisdom and compassion had turned an ungrateful ruler into a just king,
proving that even the simplest actions can have profound impacts on
the world. And so, Panachathanthra continued to carve, teach, and inspire,
leaving a lasting legacy of love and humility in Vellai-Vanam.
"""
start-2.py - Chat
import ollama
response = ollama.list()
# print(response)
# == Chat example ==
res = ollama.chat(
model="mistral",
messages=[
{"role": "user", "content": "why is the sky blue?"},
],
)
print(res["message"]["content"])
#response
"""
The blue color of the sky is primarily due to a process called Rayleigh scattering.
As sunlight reaches Earth, it is made up of different colors that travel as
waves with varying wavelengths. Shorter wavelength colors, like violet and blue,
are scattered in all directions more than longer wavelength colors, such as red,
yellow, and green. However, our eyes are more sensitive to blue light and less blue
light needs to be scattered for us to see it compared to violet. Furthermore,
the ozone layer selectively absorbs violet light, allowing blue light to penetrate
deeper into the atmosphere before being scattered back to our eyes, making the
sky appear blue
"""
start-3.py Chat with streaming
#start-3.py
import ollama
# == Chat example streaming ==
res = ollama.chat(
model="mistral",
messages=[
{
"role": "user",
"content": "why is the ocean so salty? Answer in short",
},
],
stream=True, ## Note True
)
for chunk in res:
print(chunk["message"]["content"], end="", flush=True)
#response
"""
The ocean is salty because water from rivers and rainfall contains
dissolved minerals like sodium, chloride (salt), magnesium, and
calcium. Over millions of years, more salt has gone into the
ocean than has come out due to evaporation. This process continues
today, making the ocean salty.
"""
Creating model file
# candy_modelfile.modelfile
FROM mistral
SYSTEM """You are very Clever assistant who knows everything about sky.
You are very succinct and informative."""
PARAMETER temperature 0.1
ollama create Candy --file candy_modelfile.modelfile
start4-py - working with customized new model named as Candy
#start-4.py
# == Generate example with model file ==
import ollama
res = ollama.generate(
model="mistral",
prompt="why is the sky blue?",
)
# show
# print(ollama.show("mistral"))
# Create a new model with modelfile
modelfile = """
FROM mistral
SYSTEM You are very smart assistant who knows everything about sky.
You are very succinct and informative.
PARAMETER temperature 0.1
"""
## Create a customized model names as Candy
# ollama.create(model="Candy", modelfile=modelfile)
# print(ollama.list())
res = ollama.generate(model="Candy", prompt="why is the ocean so salty?")
print(res["response"])
#Response:
"""
The ocean's salinity is primarily due to two factors:
1. **Evaporation**: When water evaporates from the surface, it leaves
behind salt and other minerals, increasing the concentration of dissolved
salts in the remaining seawater.
2. **Geological processes**: The Earth's crust is rich in salt deposits,
which are eroded and carried into the oceans through rivers and underground
flows.
On average, the ocean contains about 3.5% salt by weight, with some areas
having higher concentrations (up to 4%) and others lower (around 2.5%).
"""
# print(ollama.list())
# # delete model
# ollama.delete("Candy")
# print(ollama.list())
# ollama rm Candy from terminal
Next Big thing is UI. A lot are there. Let us(e) pickup msty.app for easiest way to loacl and online AI
#download from https://msty.app/ for your os. (use case windows - CPU only) Install. See the Desktop version and Play and have fun.
Ex 7. RAG with PDF
# app.py
import streamlit as st
import os
import logging
from langchain_community.document_loaders import UnstructuredPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_ollama import OllamaEmbeddings
from langchain.prompts import ChatPromptTemplate, PromptTemplate
from langchain_ollama import ChatOllama
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain.retrievers.multi_query import MultiQueryRetriever
import ollama
# Configure logging
logging.basicConfig(level=logging.INFO)
# Constants
DOC_PATH = "gandhi.pdf"
MODEL_NAME = "llama3.2"
EMBEDDING_MODEL = "nomic-embed-text"
VECTOR_STORE_NAME = "simple-rag"
PERSIST_DIRECTORY = "./chroma_db"
def ingest_pdf(doc_path):
"""Load PDF documents."""
if os.path.exists(doc_path):
loader = UnstructuredPDFLoader(file_path=doc_path)
data = loader.load()
logging.info("PDF loaded successfully.")
return data
else:
logging.error(f"PDF file not found at path: {doc_path}")
st.error("PDF file not found.")
return None
def split_documents(documents):
"""Split documents into smaller chunks."""
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1200, chunk_overlap=300)
chunks = text_splitter.split_documents(documents)
logging.info("Documents split into chunks.")
return chunks
@st.cache_resource
def load_vector_db():
"""Load or create the vector database."""
# Pull the embedding model if not already available
ollama.pull(EMBEDDING_MODEL)
embedding = OllamaEmbeddings(model=EMBEDDING_MODEL)
if os.path.exists(PERSIST_DIRECTORY):
vector_db = Chroma(
embedding_function=embedding,
collection_name=VECTOR_STORE_NAME,
persist_directory=PERSIST_DIRECTORY,
)
logging.info("Loaded existing vector database.")
else:
# Load and process the PDF document
data = ingest_pdf(DOC_PATH)
if data is None:
return None
# Split the documents into chunks
chunks = split_documents(data)
vector_db = Chroma.from_documents(
documents=chunks,
embedding=embedding,
collection_name=VECTOR_STORE_NAME,
persist_directory=PERSIST_DIRECTORY,
)
vector_db.persist()
logging.info("Vector database created and persisted.")
return vector_db
def create_retriever(vector_db, llm):
"""Create a multi-query retriever."""
QUERY_PROMPT = PromptTemplate(
input_variables=["question"],
template="""You are an AI language model assistant. Your task is to generate five
different versions of the given user question to retrieve relevant documents from
a vector database. By generating multiple perspectives on the user question, your
goal is to help the user overcome some of the limitations of the distance-based
similarity search. Provide these alternative questions separated by newlines.
Original question: {question}""",
)
retriever = MultiQueryRetriever.from_llm(
vector_db.as_retriever(), llm, prompt=QUERY_PROMPT
)
logging.info("Retriever created.")
return retriever
def create_chain(retriever, llm):
"""Create the chain with preserved syntax."""
# RAG prompt
template = """Answer the question based ONLY on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
chain = (
{"context": retriever, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
logging.info("Chain created with preserved syntax.")
return chain
def main():
st.title("Document Assistant")
# User input
user_input = st.text_input("Enter your question:", "")
if user_input:
with st.spinner("Generating response..."):
try:
# Initialize the language model
llm = ChatOllama(model=MODEL_NAME)
# Load the vector database
vector_db = load_vector_db()
if vector_db is None:
st.error("Failed to load or create the vector database.")
return
# Create the retriever
retriever = create_retriever(vector_db, llm)
# Create the chain
chain = create_chain(retriever, llm)
# Get the response
response = chain.invoke(input=user_input)
st.markdown("**Assistant:**")
st.write(response)
except Exception as e:
st.error(f"An error occurred: {str(e)}")
else:
st.info("Please enter a question to get started.")
if __name__ == "__main__":
main()
WIP ....................
I get some Build-Error
Run this in vs code terminal (i am trying vscode) to overcome:
pip install --no-cache-dir llama-cpp-python==0.2.77 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124
STill some DLL persists. Any body faced this, kindly throw some light to me.... Thanks🙏
No comments:
Post a Comment