MoonDream Object Detection with Bounding Box in HTML Display

This lab uses the MoonDream detect() function to find the X/Y coordinates of objects with their height and width. This allows you to create bounding boxes when you out put the image information.

You can use 2B or .5B model for this function, but the .5B model is pretty poor.

The values returned will be for the x_min, y_min, x_max, and y_max. These are where the object begins and ends on the image. The values are between 0-1.

To get the percent location you multiply the min values by 100 to get a percentage value.

To get the height and width you subtract the min and max values and then multiply them by 100 to get the values as a percent of image size.

python3 -m pip install moondream

python3 -m pip install pillow

download 2B int 8 Model from: https://github.com/vikhyat/moondream

import moondream as md
from PIL import Image

model = md.vl(model='./moondream-2b-int8.mf.gz') 

picture = './img2.jpeg'
image = Image.open(picture)
encoded_image = model.encode_image(image)

query = "face"
detect_result = model.detect(encoded_image, query) 
print("\nDetected:", detect_result["objects"])

with open('moondream-object.html', 'w') as file:
    file.write('''
               <div style="position: relative; 
               height: 400px; 
               width: auto; 
               display: inline-block;">
               ''')
    file.write(f'''
               <img src="{picture}" 
               style="height: 100%; 
               display: block;">
               ''')
    for object in detect_result['objects']:
        x = object['x_min'] * 100
        y = object['y_min'] * 100
        width = (object['x_max'] - object['x_min']) * 100
        height = (object['y_max'] - object['y_min']) * 100
   
        print(f"{x} -- {width} / {y} -- {height}")
        file.write(f'''
                   <div style="color:red; 
                   position: absolute; 
                   border: 2px solid red;
                   top:{y}%; 
                   left:{x}%;
                   width:{width}%;
                   height:{height}%;"
                   ></div>
                   ''')
    file.write('</div>')