WO2016118111A1

WO2016118111A1 - System and method for associating device input to users via skeletal tracking

Info

Publication number: WO2016118111A1
Application number: PCT/US2015/011984
Authority: WO
Inventors: Robert William GRUBBS; Justin Varkey JOHN
Original assignee: General Electric Company
Priority date: 2015-01-20
Filing date: 2015-01-20
Publication date: 2016-07-28

Abstract

Systems and methods are described for associating device input to users via skeletal tracking is disclosed. The system and methods include an input device and a tracking device. One or more aspects of the system can be controlled by tracking one or more skeletal features of one or more users interacting with the system through the device, which can be, for example, a touch screen.

Description

SYSTEM AND METHOD FOR ASSOCIATING DEVICE INPUT TO USERS VIA

SKELETAL TRACKING

BACKGROUND

[0001] User interface devices for electronics have evolved from keyboards and similar hardware peripherals to more interactive means such as touch screens and the like. While keyboards have been widely adopted, they suffer from some limitations, such their size, cumbersome design, lack of cleanliness, and learning-curve to mastery. Further, there has generally not been any correlation between a user and the input device. For example, a user may log onto a system using a password and username entered via a keyboard or other input device; however, if that user were to leave the area, there is nothing to prevent a second user from accessing the system if the first user failed to log off or lock the system.

[0002] Historically, computing applications used remotes, keyboards, mice, and the like to allow users to control the application. More recently, cameras and software gesture recognition engines are employing a natural user interface ("NUI"). With NUI, user gestures can be detected, interpreted and used for application control. NUI systems can for example, identify users in the field of view of a tracking device, and the positions of the user's bodily skeletal functions within the field of view using body-tracking methods.

[0003] Generally, as noted above, a challenge with security in computer software is that users can walk away from devices while they are logged in with elevated privileges. This is particularly true on large form factor touchscreens that are used by multiple users and in casual situations where any given user may simply walk away from the device. [0004] Therefore, what is desired are systems and methods that overcome challenges in the art such as by using NUI systems to monitor the physical locations and actions of users and correlate them to software activity.

SUMMARY

[0005] The disclosure relates generally to systems employing an input device such as a mouse, keyboard, or touchscreen displays and skeletal tracking devices, and more particularly, to systems and methods that identify users and user gestures with or in proximity to the input device.

[0006] In one aspect, a system is described. The system may include at least an input device such as a mouse, keyboard, a touch screen and the like and a tracking device in communication with a processor. The system can be controlled by tracking one or more skeletal features of one or more users interacting with the system through the input device. Moreover, the input device comprising a touch screen can have vertical and horizontal dimensions comparable to size of the users. The tracking can be done by skeletal tracking. The one or more users can be uniquely identified using the skeletal features. A measurement of skeletal features of the users can be performed and can include a measurement of one or more of a digit size, a hand size, a height, an arm length, a leg length, and a head size, and the like. Access to the system can be controlled based on the user identification. The level of authorization can additionally be controlled based on the user identification. The identification of users can be used to initiate a logon event to the system when an identified user enters a field of view of the tracking device or a logoff event when an identified user exits a field of view of the tracking device. The identification of the one or more users can be used to initiate a change in a level of authorization of the system when an identified user enters or exits the field of view of the tracking device. A calibration of a location of the input device to the tracking device can be performed. Tracking bodily skeletal features of users interacting with the system through the input device can include tracking a user's gestures. A portion of pre-defined user interactions may not inputted to the input device. For example, an elbow as compared to a finger that interacts with an input device comprising a touch screen may be an accidental interaction, and therefore, be not inputted to the input device. Finally, the user input to the input device and the tracking one or more skeletal features can be correlated and associated with a user. A calibration of the tracking device with respect to the location of the one or more users can be performed based on the one or more users interacting with the tracking device in a specified way. Furthmore, the intersection of the locations of the input device and one or more skeletal features can used to determine a probability for the correctness of the identification of the one or more users.

[0007] In another aspect, a method is disclosed comprising tracking one or more skeletal features of one or more users interacting with a system through a input device such as a mouse, keyboard, a touch screen and the like and controlling one or more aspects of the system by the tracking of the one or more skeletal features of one or more users interacting with the system through the input device. The input device can comprise a touch screen having a vertical and horizontal dimensions comparable to size of the one or more users. The input device can be located in a public setting. The tracking can comprise skeletal tracking. The one or more users can be uniquely identified using the one or more skeletal features. The one or more skeletal features of the users can comprise a measurement of a height, an arm length, a leg length, a head size, and the like. Access to the system can be controlled based on the user identification. A level of authorization can be controlled based on the user identification. The identification of the users can be used to initiate a logon event to the system when an identified user enters a field of view of the tracking device or a logoff event from the system when an identified user exits the field of view of the tracking device. The identification of the users can be used to initiate a change in a level of authorization of the system when an identified user enters or exits a field of view of the tracking device. A calibration of a location of the input device to the tracking device can be performed. Tracking skeletal features of the users interacting with the system through the input device can comprise tracking gestures of users. A portion of pre-defined user interactions may not inputted to the input device. For example, an elbow as compared to a finger that interacts with an input device comprising a touch screen may be an accidental interaction, and therefore, be not inputted to the input device. Finally, the input to the input device and the tracking one or more skeletal features can be correlated and associated with a user. A calibration of the tracking device with respect to the location of the one or more users can be performed based on the one or more users interacting with the tracking device in a specified way. Furthmore, the intersection of the locations of the input device and one or more skeletal features can used to determine a probability for the correctness of the identification of the one or more users.

[0008] Other systems, methods, features and/or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features and/or advantages be included within this description and be protected by the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.

[0010] FIG. 1 illustrates an exemplary overview system for associating input to an input device to one or more users via skeletal tracking.

[0011] FIG. 2 is a flow diagram illustrating example operations for associating input to an input device to one or more users via skeletal tracking.

[0012] FIG. 3 is a block diagram of an example computing device upon which embodiments of the invention may be implemented.

[0013] FIG. 4 illustrates an exemplary overview system for associating input to an input device to multiple users via skeletal tracking.

[0014] FIG. 5 illustrates an example use case of the systems and methods of the disclosure.

[0015] FIG. 6 is a flow diagram illustrating example operations for different levels of authorization to users based on skeletal tracking.

DETAILED DESCRIPTION

[0016] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present disclosure. [0017] As used in the specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from "about" one particular value, and/or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent "about," it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.

[0018] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0019] Throughout the description and claims of this specification, the word "comprise" and variations of the word, such as "comprising" and "comprises," means "including but not limited to," and is not intended to exclude, for example, other additives, components, integers or steps. "Exemplary" means "an example of and is not intended to convey an indication of a preferred or ideal embodiment. "Such as" is not used in a restrictive sense, but for explanatory purposes.

[0020] Disclosed are components that can be used to perform the disclosed methods and systems. These and other components are disclosed herein, and it is understood that when combinations, subsets, interactions, groups, etc. of these components are disclosed that while specific reference of each various individual and collective combinations and permutation of these may not be explicitly disclosed, each is specifically contemplated and described herein, for all methods and systems. This applies to all aspects of this application including, but not limited to, steps in disclosed methods. Thus, if there are a variety of additional steps that can be performed it is understood that each of these additional steps can be performed with any specific embodiment or combination of embodiments of the disclosed methods.

[0021] The present methods and systems may be understood more readily by reference to the following detailed description of preferred embodiments and the Examples included therein and to the Figures and their previous and following description.

[0022] Fig. 1 illustrates an exemplary overview system for associating inputs to an input device to one or more users via skeletal tracking. The system may include hardware and software elements, which work together to track skeletal movements of users, create skeletal maps, associate a recognized skeletal movement (e.g. gesture) with a specific command, and execute the command within an application or otherwise control the system. The system may comprise a tracking device 102, a computer system 110, and an input device 120. As shown in FIG. 1, in one embodiment of the system the tracking device 102 can be camera, such as a webcam. The camera can comprise a digital camera, but can be an analog device equipped with or in communication with an appropriate analog/digital converter. The tracking device 102 may also be a scanner, recorder, or any other device capable of capturing a still image or a video. In one aspect, the tracking device 102 can be a display or scanner equipped with tracking device technology. For example, the tracking device 102 can be a display equipped with Microsoft PixelSense™ technology (Microsoft Corporation, Redmond, Washington USA), or any similar technology. The tracking devices can include a depth sensor and multi-array microphone to capture three-dimensional skeletal feature data, in addition to having facial recognition and voice recognition capabilities. The depth sensor can comprise an infrared laser projector combined with a monochrome CMOS sensor, in order to capture video data in three-dimensions with different lighting conditions. The users' physical environment can be accounted for, e.g. for the presence of equipment, furniture, or other obstacles. The camera can further comprise a depth- aware camera, or stereo cameras. Depth-aware cameras can comprise structured light or time-of- flight cameras that create depth maps of what is being seen through the camera at a short range, and use this data to approximate a three-dimensional representation of what is being seen. Stereo cameras can comprise using two or more cameras whose position to one another is known. A three-dimensional representation can be approximated by the output of the stereo cameras. Moreover, audio may be captured by a microphone or by two or more stereophonic microphones. For example, the tracking device 102 may comprise one or more image tracking devices and/or microphones. In addition, the tracking device 102 may use other forms of energy detection, including but not limited to infrared (passive and active sensors), radio frequency energy (radar, microwave and tomographic motion detection), sound (microphones and acoustic sensors), vibration (triboelectric, seismic, and inertia-switch sensors), and magnetism (magnetic sensors and magnetometers). In one aspect, the tracking device 102 can be an optical scanner that is integrated into a touchscreen display. As non- limiting examples, the tracking device 102 can be motion detection devices such as Microsoft™ Kinect (Microsoft Corporation, Redmond, Washington USA), Leap™ Motion controllers (Leap Motion, Inc., San Francisco, California, USA), and the like.

[0023] The tracking device 102 can be in communication with the computing device 1 10. Generally, the computing device 1 10 comprises at least a processor 104 and a memory 108. The tracking device 102 can communicate with the processor 104 through, for example, a network (wired (including fiber optic), wireless or a combination of wired and wireless) or a direct-connect cable (e.g., using a universal serial bus (USB) connection, IEEE 1394 "Firewire" connections, and the like).

[0024] The system of FIG. 1 can further comprise an input device 120. Generally, the input device is also in communication with the processor 104 in the same or similar manner as the tracking device 102. The input device 120 can be a keyboard, mouse, trackball, a touchscreen display, and the like. The touchscreen display can be a display equipped with Microsoft PixelSense™ technology (Microsoft Corporation, Redmond, Washington USA), or any similar technology. Moreover, the touchscreen display can have various form factors (e.g. a large form factor, a small form factor, etc.). For example, it can have vertical and horizontal dimensions comparable to size of the users. The display can be located in a public setting, such as a common conference room, a control room, a bus station, a park, library, train station, commercial mall, and the like. In one aspect, each skeletal feature may be described in three- dimensional Cartesian space in a frame of reference with respect to the tracking device 102 or any other reference point. Skeletal mapping of users may be generated from the tracking device 102. A variety of skeletal features can be measured and identified. For example, a measurement of one or more of a digit size, a hand size, a height, an arm length, a leg length, and a head size and the like may be performed. Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes. It is understood that one or more of the features may be omitted and/or others may be added. Moreover, a skeletal mapping is one example of a computer model of a user, and other computer models are contemplated. Aspects of the disclosure can provide tracking system with the ability to recognize and interpret skeletal features, e.g. gestures. Moreover, prototype skeletal feature (e.g. gesture) data representative may be employed in a training process and then used to classify skeletal features performed by users. Data may be evaluated in an accurate, scalable and robust manner.

[0025] It should be understood that at least some of the steps of associating input from an input device to users via skeletal tracking can be at least partially performed by at least one processor (described above and below). Additionally, at least some of the steps for associating input from an input device to users via skeletal tracking can optionally be implemented within a cloud computing environment, for example, in order to decrease the time needed to perform the algorithms, which can facilitate visualization of the prior analysis on skeletal features. Cloud computing is well-known in the art. Cloud computing enables network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be provisioned and released with minimal interaction. It promotes high availability, on- demand self-services, broad network access, resource pooling and rapid elasticity.

[0026] It should be appreciated that the logical operations described herein with respect to the various figures may be implemented (1) as a sequence of computer implemented acts or program modules (i.e., software) running on a computing device, (2) as interconnected machine logic circuits or circuit modules (i.e., hardware) within the computing device and/or (3) a combination of software and hardware of the computing device. Thus, the logical operations discussed herein are not limited to any specific combination of hardware and software. The implementation is a matter of choice dependent on the performance and other requirements of the computing device. Accordingly, the logical operations described herein are referred to variously as operations, structural devices, acts, or modules. These operations, structural devices, acts and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in a different order than those described herein.

[0027] Referring now to Fig. 2, a flow diagram illustrating example operations 200 for performing the steps of associating touchscreen input to users via skeletal tracking is shown. At 202, the system can receive skeletal features of one or more users interacting with an input device, e.g. a touchscreen. This is done, for example, with the tracking device elaborated on in Fig. 1. Next in 205, the skeletal features of the users is tracked, comprising for example, skeletal features such as gestures and users' positions of one or more users interacting with the system through the input device, e.g. the touch screen. One or more aspects of the system can be controlled by determining the skeletal feature of the users interacting with the input device.

[0028] User profiles based on the skeletal features can then be generated in 210. In one aspect, these profiles can be stored in a database for organization, future updating, and retrieval. In another aspect, these features of authorized (or unauthorized) users have been prerecorded and stored in the database. As such access to the system can be controlled based on the user identification. In 220, the one or more users can be uniquely identified based on the skeletal features. This may involve processing algorithms and communication between the skeletal features being tracked and the database of users' profiles. In 230, the system monitors users' interactions with the input device, e.g. touch screen. This may occur simultaneously with steps 202-220, or occur at a separate time. In 240, the users' interactions with the input device and the users' skeletal features (e.g. gestures) are correlated. Specific input events on the input device can be linked to a user, and all inputs are associated with a skeletally tracked user. For an input device comprising, for example, a mouse or keyboard, this might be done by tuning the tracking to locate the mouse or keyboard and find skeletal "hands" that intersect those locations. For touchscreens, the specific location of the touch on the screen can be calculated and correlated to specific touch events. Once a correlation has been identified, the input event can be assigned an identification label for the life of the input event, along with a probability rating for the correctness of the identification. Software on the computing device can use that information in different ways; for example, a button on a touchscreen that stops a turbine might require a high probability of the identification label being the currently logged in user, while simply moving a window on the screen might not require any identification label at all or a low probability. Probability can be calculated, for example, as a combination of the distance between the skeletal hand and the input device location, the intrinsic probability rating from the skeletal tracking device, and a "residual probability" that is an integral of the past probability for that particular identification label, weighted for recent activity. Constants for these components of probability can be tuned for optimal reliability. A calibration of the tracking device may be performed with the location of the input device. During initial setup of the system (or later, if the input device is moved), the one or more users can initiate a calibration. During calibration, the one or more user may be asked to interact with the input device in a specified way so that the input device location relative to the tracking device can be calculated. For a touch screen, this may consist for example, of tapping on the four corners of the screen, and at each touch event the skeletal "hand's" (an example of a skeletal feature) coordinates are recorded, providing the baseline for later finding intersections of the skeletal "hand" and the input device. As such, touch screen input and the tracking one or more skeletal features is correlated and associated with a user. At step 250, the system can be controlled based on the correlation performed in 240.

[0029] When the logical operations described herein are implemented in software, the process may execute on any type of computing architecture or platform. As noted herein, the computing device may comprise the secure system that a person is attempting to access. For example, referring to Fig. 3, an example computing device upon which embodiments of the invention may be implemented is illustrated. In particular, at least one processing device described above may be a computing device, such as computing device 300 shown in Fig. 3. The computing device 300 may include a bus or other communication mechanism for communicating information among various components of the computing device 300. In its most basic configuration, computing device 300 typically includes at least one processing unit 306 and system memory 304. Depending on the exact configuration and type of computing device, system memory 304 may be volatile (such as random access memory (RAM)), nonvolatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in Fig. 3 by dashed line 302. The processing unit 306 may be a standard programmable processor that performs arithmetic and logic operations necessary for operation of the computing device 300.

[0030] Computing device 300 may have additional features/functionality. For example, computing device 300 may include additional storage such as removable storage 308 and non-removable storage 310 including, but not limited to, magnetic or optical disks or tapes. Computing device 300 may also contain network connection(s) 316 that allow the device to communicate with other devices. Computing device 300 may also have input device(s) 314 such as a keyboard, mouse, trackball, touch screen, etc. Output device(s) 312 such as a display, speakers, printer, etc. may also be included. The additional devices may be connected to the bus in order to facilitate communication of data among the components of the computing device 300. All these devices are well known in the art and need not be discussed at length here. [0031] The processing unit 306 may be configured to execute program code encoded in tangible, computer-readable media. Computer-readable media refers to any media that is capable of providing data that causes the computing device 300 (i.e., a machine) to operate in a particular fashion. Various computer-readable media may be utilized to provide instructions to the processing unit 306 for execution. Common forms of computer-readable media include, for example, magnetic media, optical media, physical media, memory chips or cartridges, a carrier wave, or any other medium from which a computer can read. Example computer-readable media may include, but is not limited to, volatile media, non-volatile media and transmission media. Volatile and non-volatile media may be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data and common forms are discussed in detail below. Transmission media may include coaxial cables, copper wires and/or fiber optic cables, as well as acoustic or light waves, such as those generated during radio-wave and infra-red data communication. Example tangible, computer- readable recording media include, but are not limited to, an integrated circuit (e.g., field- programmable gate array or application-specific IC), a hard disk, an optical disk, a magneto- optical disk, a floppy disk, a magnetic tape, a holographic storage medium, a solid-state device, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices.

[0032] In an example implementation, the processing unit 306 may execute program code stored in the system memory 304. For example, the bus may carry data to the system memory 304, from which the processing unit 306 receives and executes instructions. The data received by the system memory 304 may optionally be stored on the removable storage 308 or the non-removable storage 310 before or after execution by the processing unit 606.

[0033] Computing device 300 typically includes a variety of non-transitory computer- readable media. Computer-readable media can be any available media that can be accessed by device 300 and includes both volatile and non-volatile media, removable and non-removable media. Computer storage media include volatile and non-volatile, and removable and nonremovable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. System memory 304, removable storage 308, and non-removable storage 310 are all examples of computer storage media. Computer storage media include, but are not limited to, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 300. Any such computer storage media may be part of computing device 300.

[0034] It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination thereof. Thus, the methods and apparatuses of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computing device, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, e.g., through the use of an application programming interface (API), reusable controls, or the like. Such programs may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language and it may be combined with hardware implementations.

[0035] Fig. 4 illustrates another aspect of the system for associating input from an input device (e.g. a touchscreen, keyboard, mouse, etc.) 420 to users via skeletal tracking, similar to that shown in Fig. 1. Here one embodiment of the system comprises several tracking devices 402 (only two are shown in the figure). In one aspect, the tracking devices 402 can be in direct communication with a system 410 comprising at least a processor 404 and a memory 408 through, for example, a network (wired (including fiber optic), wireless or a combination of wired and wireless) or a direct-connect cable (e.g., using a universal serial bus (USB) connection, IEEE 1394 "Firewire" connections, and the like). The users can interact with an input device 420 such as a display. For example, the display can be a touchscreen display. Moreover, the display can have a large form factor. For example, it can have vertical and horizontal dimensions comparable to size of the users. The display can be located in a public setting, such as a common conference room, a bus station, a park, library, train station, commercial mall, and the like. [0036] For example, Fig. 5 shows an aspect of the disclosure where at least two users A and B (505) interact with one input device, e.g. a touch screen (500) on at least two applications, A' and B' (510). In this scenario, it may be desirable to grant user A authorization to make changes to application A' as well as B' . However, it may be undesirable to grant user B authorization to changes to application A' . The system would be able to use the skeletal tracking device to identify the different users and to correlate the users' interaction with the touchscreen 500 to implement this rule. Other extensions of these aspects are contemplated wherein multiple users interact with multiple applications with various levels of authorization both between applications and within a given application.

Examples

[0037] Referring again to FIG. 4, in one aspect, the system that users are attempting to access can be a secure computer system 410 such as, for example, a control system in an industrial setting. A profile of users associated with the logon can be created, combining measurements of a digit size, a hand size, a height, an arm length, a leg length, and a head size and the like. For example, the identification of one or more users 406 can be used to initiate a logon event to the system 410 when an identified user enters a field of view of the tracking device 402 or a logoff event from the system 410 when an identified user exits the field of view of the tracking device.

[0038] Additionally, it may be desirable to grant different levels of authorization to users that is within the field of view of a tracking device 402 associated with the control system. For example, when one or more users steps 406 into the field of view of the tracking device 402, the system 410 can automatically grant at least one of the one or more users 406 higher levels of authorization; alternatively, when the one or more users step out of the field of view of the tracking device 402, the system 410 can automatically grant the remaining one or more users lower levels of authorization.

[0039] Moreover, in subsequent logons, individuals not matching profile measurements can be flagged in the system for administrative review; the system could be configured to reject the logon attempt. As such, access to the system or level of authorization can be controlled based on the user identification.

[0040] Referring now to FIG. 6, a flow diagram illustrating example operations 600 for performing the steps of user identification and authorization is shown. At 602, a processor 104 receives skeletal features of users interacting with system through the input device in field of view of tracking device. In response, at 604, the processor 104 of the tracking device tracks by executing one or more of the recognition algorithms encoded as computer-executable instructions and stored in the memory 108, skeletal features of the users. The processor 104 in communication with the database stored in the memory 108 and executing the recognition algorithms stored in the memory 108 form a threshold module. At step 606, skeletal features of users are compared, by the processor 104, to the skeletal features of authorized users that are stored in the database. At step 608, the processor 104 assigns a confidence factor based on the comparison of characteristics as determined from skeletal features of the users to the skeletal features stored in the database. The more skeletal features that are the same or similar to the skeletal features stored in the database of a particular authorized user (a positive comparison), the higher the confidence factor. At step 610, if the confidence factor meets or exceeds a threshold, then at step 612, the user(s) attempting to access the secure system is (are) prompted for a second form of authentication (e.g., a password, biometric information, a code, an electronic key, etc.). For example, the user(s) attempting to access the secure system can be prompted for a second form of identification using input-output devices in communication with the processor 104. For example, the prompt can be shown on a display and a password can be entered using a keyboard, though other input-output devices are contemplated within the scope of the invention.

[0041] If the confidence factor fails to meet or exceed the threshold, then at step

614 the person attempting to access the secure system is denied access. In various aspects, the threshold can be a pre-established value that is stored in the memory 108 or, it can be a dynamically-adjusted value based on positive comparisons of certain of the skeletal features. For example, if certain high-confidence characteristics are positively identified from the skeletal features, the threshold value can be lowered, thus requiring fewer positive comparisons to meet or exceed the threshold. Alternatively, the threshold value could be static, but the characteristics from the skeletal features (e.g. specific gestures) have varying (weighted) confidence values. Such comparisons, adjustment of the threshold, or weighting of the characteristics can be performed by a threshold module. The threshold module comprises computer-readable instructions stored in the memory 108 that are executed by the processor 104. The computer- readable instructions cause the processor to dynamically adjust the threshold based on positive comparisons of certain skeletal features or, in another embodiment, the threshold module comprises computer-readable instructions stored in the memory 108 that are executed by the processor 104 that cause the processor 104 to assign weights to each of skeletal features, wherein all of the assigned weights are not equal, so that fewer or greater positive comparisons of certain skeletal features are required to meet or exceed the threshold.

[0042] Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED;

1. A system comprising:

an input device;

a tracking device; and

a processor in communication with the input device and the tracking device, wherein one or more aspects of the system are controlled by tracking one or more skeletal features of one or more users interacting with the system through the input device.

2. The system of claim 1, wherein the input device is a touch screen.

3. The system of claim 1, wherein the one or more users are uniquely identified using the one or more skeletal features.

4. The system of claim 3, wherein the one or more skeletal features of the one or more users comprises a measurement of one or more of a digit size, a hand size, a height, an arm length, a leg length, and a head size.

5. The system of claim 3, wherein an intersection of the locations of the input device and one or more skeletal features is used to determine a probability for correctness of the identification of the one or more users.

6. The system of claim 3, wherein access to the system is controlled based on the user identification, or wherein a level of authorization to the system is controlled based on the user identification.

7. The system of claim 5, wherein the identification of the one or more users is used to initiate a logon event to the system when an identified user enters a field of view of the tracking device or wherein the identification of the one or more users is used to initiate a logoff event from the system when an identified user exits the field of view of the tracking device.

8. The system of claim 5, wherein the identification of the one or more users is used to initiate a change in a level of authorization of the system when an identified user enters or exits a field of view of the tracking device.

9. The system of claim 1, wherein the one or more aspects of the system that are controlled are determined by the skeletal feature interacting with the input device.

10. The system of claim 1, wherein a calibration of the tracking device with respect to a location of the one or more users is performed based on the one or more users interacting with the tracking device in a specified way.

11. A method comprising:

tracking one or more skeletal features of one or more users interacting with a system through an input device; and

controlling one or more aspects of the system by the tracking the one or more skeletal features of one or more users interacting with the system through the input device.

12. The method of claim 11, wherein the input device comprises a touch screen.

13. The method of claim 111, further comprising uniquely identifying the one or more users using the one or more skeletal features.

14. The method of claim 13, wherein the one or more skeletal features of the one or more users comprises a measurement of one or more of a digit size, a hand size, a height, an arm length, a leg length, and a head size.

15. The method of claim 13, wherein an intersection of locations of the input device and one or more skeletal features is used to determine a probability of correctness for the identification of the one or more users.

16. The method of claim 13, wherein access to the system or a level of authorization is

controlled based on the user identification.

17. The method of claim 16, wherein the identification of the one or more users is used to initiate a logon event to the system when an identified user enters a field of view of the tracking device or wherein the identification of the one or more users is used to initiate a logoff event from the system when an identified user exits the field of view of the tracking device.

18. The method of claim 16, wherein the identification of the one or more users is used to initiate a change in a level of authorization of the system when an identified user enters or exits a field of view of the tracking device.

19. The method of claim 11, wherein controlling the system one or more aspects of the

system by the tracking the one or more skeletal features of one or more users interacting with the system through the input device comprises determining the one or more aspects of the system that are controlled by the skeletal feature interacting with the input device.

20. The method claim 11, further comprising calibrating the tracking device with respect to a location of the one or more users interacting with the system through the input device, wherein the calibration of the tracking device with respect to the location of the one or more users is performed based on the one or more users interacting with the tracking device in a specified way.