Learn How to Identify and Track Hands With OpenCV and Python - MUO - MakeUseOf

Hand tracking is the process of utilizing machine imaginativeness to observe and travel the movements of a person's manus successful existent time. The astir ascendant exertion of manus tracking is successful virtual world headsets. The headsets let you to usage your hands arsenic input successful spot of interaction controllers. This successful crook makes the acquisition much immersive.

Find retired however to way a person’s hands utilizing Python, OpenCV for machine vision, and MediaPipe.

Google developed the MediaPipe framework, which contains galore machine-learning solutions. One of the solutions is the manus and digit tracking solution called MediaPipe Hands. To way hands, MediaPipe Hands performs 2 processes: thenar detection and landmark detection.

Hand Palm Detection

MediaPipe begins by identifying wherever the palms are successful the input image. Since estimating bounding boxes for stiff objects is simpler than identifying hands with jointed fingers.

Hand Landmarks Detection

After thenar detection, MediaPipe performs manus landmarks detection. The manus landmark exemplary tin foretell 21 precise coordinates of the determination of each manus landmark.

The numbers correspond a unsocial identifier for each landmark.

Setting Up Your Environment

To travel on with this project, you should beryllium acquainted with the basics of Python. Install the pursuing libraries successful your environment:

• OpenCV: You volition usage this room for machine imaginativeness and to execute representation processing techniques connected the input image.
• MediaPipe: You volition usage this room to execute manus detection and tracking connected the input image.
• imutils: You volition this room to resize the video framework of the input.

Run the pursuing bid connected your terminal to instal the OpenCV, MediaPipe, and imutils libraries. Install pip—the Python bundle manager—if you request to. Ensure you walk the libraries arsenic a space-delimited list.

pip instal OpenCV-Python MediaPipe imutils

When the update is implicit the situation is acceptable for you to commencement coding.

The afloat root codification for this task is disposable successful its GitHub repository.

Importing the Required Libraries

You’ll request to import the libraries you installed truthful you tin usage them. Open immoderate Python IDE, make a Python file, and adhd the pursuing imports:

import cv2
import mediapipe as mp
import imutils

Ensure you import OpenCV arsenic cv2 and MediaPipe successful lowercase. Failing to bash truthful volition propulsion an error.

You volition usage mpHands to telephone the MediaPipe hands solution, and the hands entity to observe and way the manus input. You’ll usage the mpDraw entity to gully the connections betwixt the landmarks of the identified hands.

mpHands = mp.solutions.hands
hands = mpHands.Hands()
mpDraw = mp.solutions.drawing_utils

You tin fine-tune the MediaPipe hands exemplary by passing assorted parameters to the Hands() constructor. The default values are bully capable for this project, but you tin experimentation with them to spot however they impact the model:

You should permission the static_image_mode arsenic False to guarantee the exemplary detects the hands erstwhile earlier it begins tracking them. It lone repeats the tracking process if the detection assurance falls little than the declared parameter, making wide input processing faster.

Performing Hand Tracking

You request 3 functions to execute manus tracking: 1 to process the input, 1 to gully the manus landmark connections, and a main relation to power the programme flow.

Input Processing Function

This relation takes the input, converts it to grayscale, and passes it to the MediaPipe hands exemplary to observe and way the hands successful the input.

def process_image(img):
    
    gray_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    results = hands.process(gray_image)

    
    return results

The relation returns the results connected whether determination were immoderate detected hands connected the input.

The Hand Landmark Connections Drawing Function

This relation checks whether the input processing relation detected immoderate hands. If determination are immoderate detected hands, it loops implicit each landmark and draws a ellipse astir it, keeping way of the landmark utilizing Python’s enumerate function. It past draws the connections betwixt the landmarks connected the archetypal video input.

def draw_hand_connections(img, results):
    if results.multi_hand_landmarks:
        for handLms in results.multi_hand_landmarks:
            for id, lm in enumerate(handLms.landmark):
                h, w, c = img.shape

                
                cx, cy = int(lm.x * w), int(lm.y * h)

                
                
                print(id, cx, cy)

                
                cv2.circle(img, (cx, cy), 10, (0, 255, 0),
                           cv2.FILLED)
                
                mpDraw.draw_landmarks(img, handLms,
                                      mpHands.HAND_CONNECTIONS)

        return img

The relation starts by circling each landmark:

It past draws the manus connections:

It yet returns its output to the calling function.

The Main Function

Create a main relation that volition power the travel of your program. It volition instrumentality the input and resize the video framework to guarantee the consistency of the output. Pass the input to the processing relation which volition past observe and way the hands. Take the returned results to the manus landmarks transportation drafting relation which volition gully the transportation connected the archetypal video input. It volition yet show the output to the user.

def main():
   
   
    cap = cv2.VideoCapture(0)

    while True:
        
        success, representation = cap.read()
        image = imutils.resize(image, width=500, height=500)
        results = process_image(image)
        draw_hand_connections(image, results)

        
        cv2.imshow("Hand tracker", image)

        
        if cv2.waitKey(1) == ord('q'):
            cap.release()
            cv2.destroyAllWindows()

The past measurement is moving your program. The codification beneath ensures that erstwhile you tally the program, the main relation runs first.

if __name__ == "__main__":
    main()

When the programme runs, it produces output similar this:

The programme tracks the hands successful existent time.

Hand Tracking for Immersive Virtual Reality

Hand tracking successful virtual world makes the exertion much enticing. Virtual world headsets person started to present manus tracking, bringing a consciousness of heightened world to the virtual world. The headsets let the idiosyncratic to input commands utilizing a virtual hand.

Hand tracking successful virtual headsets is conscionable 1 exertion of this technology. You tin incorporated manus tracking successful immoderate applicable country of your liking.