Description IMPROVED GESTURE RECOGNITION FOR POINTING DEVICES
[1] This application claims priority from US appl. no. 60/481,240, filed August 15, 2003, which application is incorporated herein by reference for all purposes. Background of Invention
[2] Modern electronic devices often make use of a cursor seen on the display, or control some parameter or position of a mechanical actuator, by means of a pointing device. One or more buttons or switches are usually associated with a pointing device, controlling the required actions once the cursor or actuator is positioned in an appropriate location. However, for the users, it is highly advantageous not to employ such buttons or switches, and affect an appropriate action utilizing the pointing device itself. This is desirable since the controlling digit or limb is already acting on the pointing device, and does not need to be moved to the button or switch (so that the operation can be conducted without the user actually looking at the pointing device), or the second digit or appendage does not need to come into play.
[3] Numerous previous art apparatus and patents describe means for achieving such functionality utilizing variety of pointing devices. However, each method is typically applicable only to a specific pointing device type, and is usually sensitive to a multiplicity of "noise" sources, be it internal electrical noise, or aberrations in the functioning of the human digits or limbs. Summary of Invention
[4] The objective of the current invention is to provide a robust method and an algorithm for determination of various gestures carried out on the pointing device, simultaneously applicable to a multiplicity of the pointing device types.
[5] For example, gesture determination on a touch pad requires timely and reliable information that the touch event is taking place and the duration of such touch event.
[6] According to the current invention, the desired resistance to noise sources (e.g. probability of "false positive" activations and probability of "false negative" behavior) can be readily adjusted and can be made variable in time, if different degrees of noise resistance are required at different moments in time.
[7] The determination of a gesture is made using past history of the data, with the amount of data (length of observation) suitable to the desired resistance to noise. Variable threshold levels are utilized when the higher/lower type of comparisons are made. The values of the thresholds are determined and constantly adjusted in time based on the amount of noise (electrical, mechanical or human behavior), expected in the future and observed in the past (for an appropriate period of time and including the present conditions).
[8] The algorithm can be further characterized, using the language of mathematics, as
utilizing the cross-correlations of the parts of the data itself, and correlations of the data to variable templates based on the previous history of the data, human motor abilities, and specific mechanical configuration of the particular pointing device.
[9] Data from all mutually independent sources (for example X-position, Y-position, X-force, Y-force, Z-force, duration of Z-force event, Z-proximity, actions on other controls and/or keys/keyboard of the host device) are accumulated and factor into the final decision in order to reduce the possibility of erroneous and/or undesirable operation.
[10] The invention provides for means to prevent erroneous motion of the cursor or actuator while the determination of the gesture is taking place.
[11] The burden of logical and arithmetical calculations can be arbitrarily divided between the local controller of the pointing device and the driver processing the data from the pointing device, which is running on the host apparatus (if such exists).
[12] An arbitrarily small subset of the method and algorithm of the invention can be utilized, suitable to the operating conditions, specific mechanical configuration and desired functionality of a particular pointing device. In some of such cases, operations may be equivalent to the previously described in prior art and/or patents.
[13] Brief Description of Drawings
[14] Figure 1 shows a representation of information flow in the pointing device.
[15] Figure 2 is a depiction of Probability Density Function ("PDF" or "Bell Curve") and Determination of Threshold Value.
[16] Figure 3 illustrates Typical Gesture on Touch Pad Hold.
[17] Figure 4 illustrates Typical Gesture on Touch Pad Click.
[18] Figure 5 illustrates Typical Gesture on Touch Pad Double Click.
[ 19] Figure 6 illustrates Typical Gesture on Touch Pad Click and Hold. Detailed Description
[20] Any variable that requires a threshold to detect activation or de-activation (such as Z-force or Z-proximity for detection of touch, for example) is processed and a histogram (a representation of Probability Density Function or PDF) is accumulated. The data in the histogram includes past history and the current conditions. As the time progresses, some old data is removed from the histogram, and new data is added. These operations are carried out using an optimized algorithm that does not require recalculation of the complete array of data.
[21] Looking at Figure 2, once the median value 21 (V ) is determined, several values 22 around V are used to determine the shape of the Probability Distribution (with the Normal or Gaussian Distribution being the worst case). A threshold value 23 (V ) is then determined based on the acceptable error probability as described below.
[22] The total area under the PDF curve 20 includes all possible values of the variable and thus represents a probability of 1 (100%). A portion of the area 24 represents the
probability of erroneous activation for a given threshold 23 (V ) for an arbitrarily selected case when the activation event requires the variable to exceed the threshold. The same considerations apply when the desired activation will result from the variable being smaller than the threshold, except the threshold value will probably be located to the left of the median value V o .
[23] Using Gaussian Distribution as the worst possible case, a value for threshold 23 can be calculated or selected from a table of values such that the probability of error is always smaller than or equal to the error represented by area 24.
[24] When appropriate, the determination of the PDF may also be done during a training session when the user sets various parameters according to personal abilities, and the system learns the user's behavior.
[25] For example, looking at Figure 3 through Figure 6, a typical gesture on a touch pad may require setting of value thresholds for parameters Z-force and Z-proximity (which are used for creation of logical variable "Processed Touch Event") and time thresholds for time intervals designated 40, 41, 50, 51, 52, 60 and 61. The time thresholds may at first be set to default values, then they may be further refined during a training session, and then they may be even further refined during the actual operations of a pointing device. The algorithm for refinement is the same as described above (building of a histogram, checking the shape of the distribution, and calculating a threshold according to the required probability of error).
[26] It will be appreciated that the benefits of the invention offer themselves with any of a multitude of types of pointing devices, among them force sticks, movable joy-sticks, FSR (force sensor resistance), and touch pads. Touch pad technologies with which the invention can offer its benefits include capacitive, resistive, infrared, or any other technology.
[27] Those skilled in the art will have no difficulty whatsoever devising myriad obvious variants and improvements of the invention, all of which are intended to be encompassed within the claims which follow.