[Llama] Using Llama 3.2 11B Vision with Ollame in your local machine

After you install ollama, you can pull the model.

ollama pull llama3.2-vision

Then, start ollama. It will start server can respond you with local AI which is Llama 3.2 11B here. Ollama provide Python and JavaScript Libraries you can install with pip and npm respectively. The details can be found https://ollama.com/blog/python-javascript-libraries Some other examples are also available from https://github.com/ollama/ollama-python

pip install ollama
npm install ollama

I ran Ollama and started to code from Jupyter Notebook. The first prompt is “Hello World!” for LLM AI. “‘”Why is the sky blue?”. It can be done as below.

import ollama
response = ollama.chat(model='x/llama3.2-vision:latest', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])

Next, I asked to describe one of math problem image. This can be used for alt text in the web page for web accessibility compliance.

with open('21A10.png', 'rb') as file:
  response = ollama.chat(
    model='x/llama3.2-vision:latest',
    messages=[
      {
        'role': 'user',
        'content': 'Can you describe this image?',
        'images': [file.read()],
      },
    ],
  )
print(response['message']['content'])

Lastly, I asked to read OMR card, but it did not go well. However, It does OCR much better. So, using OCR to collect any information needed rather using old OMR technology. I wouldn’t recommend below but just keeping as a record how it went.

import ollama
import datetime

# Get the current date and time
current_datetime = datetime.datetime.now()
# Print the current date and time
print(current_datetime)

with open('PCF_page2.png', 'rb') as file:
  response = ollama.chat(
    model='x/llama3.2-vision:latest',
    messages=[
      {
        'role': 'user',
        'content': '''Can you read Contest, Gender, Grade, Age, Last Name, First Name and Answers of question 1 to question 25 that are marked with bubbles and their information is usually written next to it in this image? 
                    Contest is one from Pascal Caley or Fermat. They should be coded as P, C or F. Select bubbled contest.
                    Gender is one from Female, Male or Other. They should be coded as F, M or O. Select bubbled gender.
                    Grade is one from 08, 09, 10, 11 or 12. They can be coded as they are. Select bubbled grade.
                    Last Name and First Name bubbles are labled from space, A, B, C to X, Y,Z. 
                    There are 25 answers for question 1 to question 25 vertically. Answer bubbles are labled from A to E horizontally. They can be coded as 1 for A, 2 for B, 3 for C, 4 for D or 5 for E. Select bubbled answers. 
                    Some answers are not answered which should be blank. 
                    Please return reponse in JSON format''',
        'images': [file.read()],
      },
    ],
  )
# They can be coded as 1 for A, 2 for B, 3 for C, 4 for D or 5 for E.    
print(response['message']['content'])

# Get the current date and time
current_datetime = datetime.datetime.now()
# Print the current date and time
print(current_datetime)