Apple has won another Key Hand-and-Eye Tracking System Patent relating to Interacting with 3D Environments - Patently Apple

Yesterday Patently Apple posted a study titled "Apple's Mixed Reality Headset volition enactment with a 3D-Like mentation of iOS, usage Patented Eye and Hand-Tracking technologies and more." The study noted Bloomberg's Mark Gurman noting that "Apple’s extremity is to bring thing caller to the table.  Eye and hand-tracking capabilities (covered successful patents 01 and 02) volition beryllium a large selling constituent for the device, according to radical acquainted with the product." Today, Apple was granted yet different important patent relating to manus and oculus tracking technology.

GUI's for Interacting with 3D Environments

Apple's patent mostly relates to machine systems with a show procreation constituent and 1 oregon much input devices that supply machine generated world (CGR) experiences, including but not constricted to physics devices that supply virtual world and mixed world experiences via a show specified arsenic a Mixed Reality Headset. A cardinal facet of the patent focuses connected manus and eye-tracking systems.

Apple's patent FIG. 1 beneath is simply a artifact diagram illustrating an operating situation of a machine strategy for providing CGR experiences; FIG. 4 is simply a artifact diagram illustrating a manus tracking portion of a machine strategy that is configured to seizure motion inputs of the user. Although hand-tracking began with a absorption connected desktops and televisions via PrimeSense exertion for the Xbox Kinect device, Apple acquired PrimeSense and the exertion volition beryllium utilized successful their aboriginal HMD and handhelds.

Apple's patent FIG. 5 above is simply a artifact diagram illustrating an oculus tracking portion of a machine strategy (HMD) that is configured to seizure regard inputs of the user.

Apple's patent FIG. 6 beneath is simply a flowchart illustrating a glint-assisted regard tracking pipeline.

Beyond the instauration of providing Hand and Eye-Tracking technology, the patent takes a heavy dive into interacting with 3D Environments. Apple notes that machine systems with show procreation components are provided with improved methods and interfaces for interacting with a three-dimensional situation and facilitating the user's usage of the machine systems erstwhile interacting with the three-dimensional environment, thereby expanding the effectiveness, efficiency, and idiosyncratic information and restitution with specified machine systems.

Such methods and interfaces whitethorn complement oregon regenerate accepted methods for interacting with a three-dimensional situation and facilitating the user's usage of the machine systems erstwhile interacting with the three-dimensional environment.

In immoderate embodiments, the machine strategy changes the level of immersion with which a computer-generated acquisition (e.g., ocular experience, audio-visual experience, virtual world experience, augmented world experience, etc.) is presented to a idiosyncratic successful accordance with biometric information corresponding to the user.

For example, erstwhile the idiosyncratic is adjusting their carnal and affectional states, e.g., proactively oregon nether the power of the computer-generated content, aft the computer-generated acquisition is started, the machine strategy whitethorn observe changes successful the biometric information (e.g., bosom rate, humor pressure, breathing rate, etc.) corresponding to the user. In accordance with the changes successful the biometric information comparative to respective sets of preset criteria associated with antithetic levels of immersion, the machine strategy increases oregon decreases the level of immersion with which the computer-generated acquisition is provided to the idiosyncratic (e.g., by changing the ocular prominence (e.g., including spatial extent, ocular depth, colour saturation, ocular contrast, etc.) of virtual contented comparative to the ocular prominence of the practice of the carnal situation (e.g., by enhancing complexity, spatial extent, and/or ocular characteristics of the virtual content, and/or reducing the ocular clarity, blur radius, opacity, colour saturation, etc. of the practice of the carnal environment, etc.). 

Adjusting the level of immersion with which a computer-generated acquisition is provided to a idiosyncratic based connected changes successful the biometric information corresponding to the idiosyncratic helps the machine strategy to supply a smoother modulation betwixt a little immersive acquisition and a much immersive acquisition that amended corresponds to the perceptive authorities of the idiosyncratic for the computer-generated experience, thereby reducing idiosyncratic disorder and improving the efficacy of the computer-generated experience.

For much details, reappraisal Apple's granted patent US 11562528 B2.