US20100188357A1 - Adaptive Keyboard - Google Patents

Adaptive Keyboard Download PDF

Info

Publication number
US20100188357A1
US20100188357A1 US12/727,217 US72721710A US2010188357A1 US 20100188357 A1 US20100188357 A1 US 20100188357A1 US 72721710 A US72721710 A US 72721710A US 2010188357 A1 US2010188357 A1 US 2010188357A1
Authority
US
United States
Prior art keywords
icons
contact
symbol
touch
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/727,217
Inventor
Kenneth Kocienda
Scott Herz
Richard Williamson
Gregory Novick
Virgil Scott King
Chris Blumenberg
Marcel van Os
Bas Ording
Scott Forstall
Imran Chaudhri
Greg Christie
Stephen O. Lemay
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US12/727,217 priority Critical patent/US20100188357A1/en
Publication of US20100188357A1 publication Critical patent/US20100188357A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0238Programmable keyboards
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Definitions

  • the disclosed embodiments relate to user interfaces, and in particular, to user interfaces that include a touch screen keyboard.
  • the user interface is the gateway through which users receive not only content but also responses to user actions or behaviors, including user attempts to access a device's features or tools.
  • Some portable electronic devices e.g., mobile phones
  • These conventional user interfaces often result in complicated key sequences and menu hierarchies that must be memorized by the user.
  • the proximity of neighboring buttons often makes it difficult for users to activate a desired pushbutton.
  • a method includes displaying a plurality of icons on a touch-sensitive display.
  • a respective icon in at least a subset of the plurality of icons corresponds to two or more symbols.
  • a contact by a user with the touch-sensitive display that corresponds to the respective icon is detected.
  • a respective symbol in the two or more symbols to which the contact further corresponds is determined.
  • the displayed respective icon is modified to indicate that the contact corresponds to the respective symbol.
  • the respective symbol may be selected when the user breaks contact with the respective icon.
  • the respective symbol may be capitalized when contact is maintained for a time interval exceeding a pre-determined value.
  • Modifying may include changing a shape of the respective icon. Changing the shape may include an asymmetric distortion of the shape. An initial shape of the respective icon may include an arc.
  • Detecting may include detecting rolling of a finger over a region that corresponds to the respective symbol.
  • the contact may include a gesture that is selected from the group consisting of one or more taps, a swipe and a rolling of a finger.
  • the two or more symbols for the respective icon may be determined in accordance with a lexicography model.
  • the lexicography model may correspond to a user usage history. The user usage history may occur prior to the establishment of the contact.
  • the lexicography model may correspond to a frequency of usage of symbols in a language.
  • the respective symbol is displayed in a region within the shape of the respective icon and outside of a region corresponding to the contact.
  • a visual indicator corresponding to the respective symbol is provided.
  • the visual indicator may include visual illumination proximate to the respective icon.
  • the visual illumination may include a band around at least a portion of the respective icon.
  • the visual indicator may be in accordance with a user usage history that occurs prior to the detecting of the contact.
  • a method includes displaying a plurality of icons on a touch-sensitive display. Two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display. A space greater than a pre-determined value is included between adjacent rows. A contact by a user with the touch-sensitive display that corresponds to a respective icon is detected. A symbol corresponding to the respective icon is displayed in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.
  • the symbol may be the respective icon.
  • the symbol may be magnified relative to the respective icon.
  • the neighboring row may be above the respective row.
  • a plurality of icons are displayed on a touch-sensitive display.
  • a contact by a user with the touch-sensitive display that corresponds to the respective icon is determined.
  • a symbol corresponding to the respective icon is displayed superimposed over one or more additional icons in the plurality of icons while the contact is maintained.
  • a plurality of icons are displayed on a touch-sensitive display. Two or more subsets of the plurality of icons are arranged in corresponding rows. A contact by a user with the touch-sensitive display that corresponds to the respective icon is determined. The displayed plurality of icons are modified to include a space greater than a pre-determined value between a row corresponding to the respective icon and an adjacent row on the touch-sensitive display while the contact is maintained. A symbol corresponding to the respective icon is displayed in the space while the contact is maintained.
  • a method includes displaying a plurality of icons on a touch-sensitive display.
  • a respective icon in the plurality of icons corresponds to at least one symbol.
  • One or more recommended words are displayed.
  • the one or more recommended words are in accordance with a user history.
  • the one or more recommended words are displayed prior to detecting any contacts by a user corresponding to symbol selection in a current application session.
  • a contact by the user with the touch-sensitive display is detected.
  • the contact includes a gesture.
  • a respective recommended word corresponding to the gesture is selected.
  • the gesture may include a swipe motion.
  • the swipe motion may include a horizontal component with displacement from left to right or from right to left along the touch-sensitive display.
  • the swipe motion may include a vertical component with displacement downward or upward along the touch-sensitive display.
  • the gesture may include one or more taps.
  • a respective tap may include making contact with the touch-sensitive display for a time interval less than a pre-determined value.
  • the gesture may include a rolling motion of the contact.
  • the rolling motion may be from left to right or from right to left along the touch-sensitive display.
  • a method includes displaying a plurality of icons on a touch-sensitive display.
  • a respective icon in at least a subset of the plurality of icons corresponds to two or more symbols.
  • a contact by a user with the touch-sensitive display that corresponds to a selection of the respective icon, wherein the contact includes a respective gesture, is detected.
  • a respective symbol in the two or more symbols for the respective icon to which the contact further corresponds is determined.
  • the respective symbol is a first symbol in the two or more symbols if the respective gesture includes a continuous contact and the respective symbol is a second symbol in the two or more symbols if the respective gesture includes a discontinuous contact.
  • the continuous contact may include a swipe motion.
  • the swipe motion may include a horizontal component with displacement from left to right or from right to left along the touch-sensitive display.
  • the swipe motion may include a vertical component with displacement downward or with displacement upward along the touch-sensitive display.
  • the continuous contact may include a rolling motion of the contact. The rolling motion may be from left to right or from right to left along the touch-sensitive display.
  • the discontinuous contact may include one or more taps.
  • a respective tap may include contact with the touch-sensitive display for a time interval less than a first pre-determined value. Two or more consecutive taps may correspond to the second symbol if a time interval between two or more corresponding contacts is less than a second pre-determined value.
  • the first symbol may be included in a first subset of symbols and the second symbol may be included in a second subset of symbols.
  • the first subset of symbols may have a probability of occurrence that is greater than a first pre-determined value and the second subset of symbols may have a probability of occurrence that is less than the first pre-determined value.
  • the probability of occurrence may be in accordance with a user history.
  • the probability of occurrence may be in accordance with a lexicography model.
  • the lexicography model may include a frequency of usage of symbols in a language.
  • the second symbol for the respective icon has a probability of occurrence immediately following the first symbol for the respective icon that is less than a second pre-determined value. In some embodiments, the first symbol for the respective icon has a probability of occurrence immediately following the second symbol for the respective icon that is less than a second pre-determined value.
  • the displayed respective icon is modified to indicate that the contact corresponds to a respective symbol.
  • a visual indicator corresponding to a respective symbol is provided.
  • the visual indicator may include visual illumination proximate to the respective icon.
  • the visual illumination may include a band around at least a portion of the respective icon.
  • a method includes displaying a plurality of icons on a touch-sensitive display.
  • a respective icon in at least a subset of the plurality of icons corresponds to two or more symbols.
  • a first symbol in the two or more symbols belongs to a first subset of symbols and a second symbol in the two or more symbols belongs to a second subset of symbols.
  • the first symbol has a probability of occurrence that is greater than a first pre-determined value and the second symbol has a probability of occurrence that is less than the first pre-determined value.
  • a contact by a user with the touch-sensitive display that corresponds to a selection of the respective icon is detected.
  • the contact includes a respective gesture.
  • a respective symbol in the two or more symbols for the respective icon to which the contact further corresponds is determined.
  • the probability of occurrence may be in accordance with a user history.
  • the probability of occurrence may be in accordance with lexicography model.
  • the lexicography model may include a frequency of usage of symbols in a language.
  • the second symbol may have a probability of occurrence immediately following the first symbol that is less than a second pre-determined value.
  • the first symbol may be selected using one or more tap gestures and the second symbol may be selected using a swipe gesture.
  • a respective tap may include making contact with the touch-sensitive display for a time interval less than a second pre-determined value. Two or more consecutive taps may correspond to the second symbol if a time interval between two or more corresponding contacts is less than a third pre-determined value.
  • the displayed respective icon is modified to indicate that the contact corresponds to the respective symbol.
  • a visual indicator corresponding to the respective symbol is provided.
  • the visual indicator may include visual illumination proximate to the respective icon.
  • the visual illumination may include a band around at least a portion of the respective icon.
  • the first subset of symbols includes e, t, a, o, i, n, s, r and h. In some embodiments, the first subset of symbols includes q, e, u, I, o, a, d, g, j, l, z, c, b, n and m. In some embodiments, the first subset of symbols includes q, c, e, h, I, l, n, o, r, t, u, w and y.
  • the second subset of symbols includes w, y and j. In some embodiments, the second subset of symbols includes w, y, p, g and j. In some embodiments, the second subset of symbols includes w, r, t, y, p, s, f, h, k, x and v. In some embodiments, the second subset of symbols includes j, v, x and z. In some embodiments, the second subset of symbols includes b, d, f, g, j, k, m, p, q, s, v, x and z.
  • the aforementioned methods may be performed by a portable electronic device having a touch-sensitive display with a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing these methods.
  • GUI graphical user interface
  • the portable electronic device provides a plurality of functions, including wireless communication.
  • Instructions for performing the aforementioned methods may be included in a computer program product configured for execution by one or more processors.
  • FIG. 1 is a block diagram illustrating an embodiment of an architecture for a portable electronic device.
  • FIG. 2 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 4 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 5 is a block diagram illustrating an embodiment of a character set data structure.
  • FIG. 6A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6D is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 7 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 8 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 9 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 10A is a block diagram illustrating an embodiment of a user word history data structure.
  • FIG. 10B is a block diagram illustrating an embodiment of a language data structure system.
  • FIG. 11A is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 11B is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 11C is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 12A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12D is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12E is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12F is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12G is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 13 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 14 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 15 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 16 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 17 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 18 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 19 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • the device may be a portable communications device.
  • the user interface may include a click wheel and/or touch screen.
  • a click wheel is a physical user-interface device that may provide navigation commands based on an angular displacement of the wheel or a point of contact with the wheel by a user of the device.
  • a click wheel may also be used to provide a user command corresponding to selection of one or more items, for example, when the user of the device presses down on at least a portion of the wheel.
  • a portable communications device e.g., a cellular telephone that may also contain other functions, such as SMS, PDA and/or music player functions
  • a touch screen is used as an exemplary embodiment.
  • the user interfaces and associated processes may be applied to other devices, such as personal computers and laptops, that may include one or more other physical user-interface devices, such as a click wheel, a keyboard, a mouse and/or a joystick.
  • the device may support a variety of applications, such as a telephone, text messaging, word processing, email and a music player.
  • the music player may be compatible with one or more file formats, such as MP3 and/or AAC.
  • the device includes an iPod music player (trademark of Apple Computer, Inc.).
  • the various applications that may be executed on the device may use at least one common physical user-interface device, such as the touch screen.
  • a common physical architecture such as the click wheel
  • the device may support the variety of applications with user interfaces that are intuitive and transparent.
  • the user interfaces may include one or more keyboard embodiments.
  • the keyboard embodiments may include standard (qwerty) and/or non-standard configurations of symbols on the displayed icons of the keyboard.
  • the keyboard embodiments may include a reduced number of icons (or soft keys) relative to the number of keys in existing physical keyboards, such as that for a typewriter. This may make it easier for users to select one or more icons in the keyboard, and thus, one or more corresponding symbols.
  • the keyboard embodiments may be adaptive. For example, displayed icons may be modified in accordance with user actions, such as selecting one or more icons and/or one or more corresponding symbols.
  • One or more applications on the portable device may utilize common and/or different keyboard embodiments. Thus, the keyboard embodiment used may be tailored to at least some of the applications.
  • one or more keyboard embodiments may be tailored to a respective user. For example, based on a word usage history (lexicography, slang, individual usage) of the respective user. Some of the keyboard embodiments may be adjusted to reduce a probability of a user error when selecting one or more icons, and thus one or more symbols, when using the keyboard embodiments.
  • word usage history lexicography, slang, individual usage
  • FIG. 1 is a block diagram illustrating an architecture for a portable electronic device 100 , according to some embodiments of the invention.
  • the device 100 may include a memory 102 (which may include one or more computer readable storage mediums), a memory controller 122 , one or more processing units (CPU's) 120 , a peripherals interface 118 , RF circuitry 108 , audio circuitry 110 , a speaker 111 , a microphone 113 , an input/output (I/O) subsystem 106 , a display system 112 (which may include a touch screen), a click wheel 114 , other input or control devices 116 , and an external port 124 .
  • the device 100 may be any portable electronic device, including but not limited to a handheld computer, a tablet computer, a mobile phone, a media player, a personal digital assistant (PDA), or the like, including a combination of two or more of these items. In other embodiments, the device 100 may not be portable, such as a personal computer.
  • PDA personal digital assistant
  • the device 100 is only one example of a portable electronic device 100 , and that the device 100 may have more or fewer components than shown, may combine two or more components, or a may have a different configuration or arrangement of the components.
  • the various components shown in FIG. 1 may be implemented in hardware, software or a combination of both hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • the memory 102 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices.
  • the memory 102 may further include storage remotely located from the one or more processors 120 , for instance network attached storage accessed via the RF circuitry 108 or the external port 124 and a communications network (not shown) such as the Internet, intranet(s), Local Area Networks (LANs), Wide Local Area Networks (WLANs), Storage Area Networks (SANs) and the like, or any suitable combination thereof.
  • Access to the memory 102 by other components of the device 100 such as the CPU 120 and the peripherals interface 118 , may be controlled by the memory controller 122 .
  • the peripherals interface 118 couples the input and output peripherals of the device to the CPU 120 and the memory 102 .
  • the one or more processors 120 run or execute various software programs and/or sets of instructions stored in the memory 102 to perform various functions for the device 100 and to process data.
  • the peripherals interface 118 , the CPU 120 , and the memory controller 122 may be implemented on a single chip, such as a chip 104 . In some other embodiments, they may be implemented on separate chips.
  • the RF (radio frequency) circuitry 108 receives and sends electromagnetic waves.
  • the RF circuitry 108 converts electrical signals to/from electromagnetic waves and communicates with communications networks and other communications devices via the electromagnetic waves.
  • the RF circuitry 108 may include well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth.
  • an antenna system an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth.
  • SIM subscriber identity module
  • the RF circuitry 108 may communicate with the networks, such as the Internet, also referred to as the World Wide Web (WWW), an Intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication.
  • networks such as the Internet, also referred to as the World Wide Web (WWW), an Intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication.
  • the networks such as the Internet, also referred to as the World Wide Web (WWW), an Intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication.
  • WLAN wireless local area network
  • MAN metropolitan area network
  • the wireless communication may use any of a plurality of communications standards, protocols and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for email, instant messaging, and/or Short Message Service (SMS)), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.
  • GSM Global System for Mobile Communications
  • EDGE Enhanced Data GSM Environment
  • W-CDMA wideband code division multiple access
  • CDMA code division multiple access
  • TDMA time division multiple access
  • Bluetooth Bluetooth
  • Wi-Fi e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g
  • the audio circuitry 110 , the speaker 111 , and the microphone 113 provide an audio interface between a user and the device 100 .
  • the audio circuitry 110 receives audio data from the peripherals interface 118 , converts the audio data to an electrical signal, and transmits the electrical signal to the speaker 111 .
  • the speaker 111 converts the electrical signal to human-audible sound waves.
  • the audio circuitry 110 also receives electrical signals converted by the microphone 113 from sound waves.
  • the audio circuitry 110 converts the electrical signal to audio data and transmits the audio data to the peripherals interface 118 for processing. Audio data may be may be retrieved from and/or transmitted to the memory 102 and/or the RF circuitry 108 by the peripherals interface 118 .
  • the audio circuitry 110 also includes a headset jack (not shown).
  • the headset jack provides an interface between the audio circuitry 110 and removable audio input/output peripherals, such as output-only headphones or a headset with both output (headphone for one or both ears) and input (microphone).
  • the I/O subsystem 106 provides the interface between input/output peripherals on the device 100 , such as the display system 112 , the click wheel 114 and other input/control devices 116 , and the peripherals interface 118 .
  • the I/O subsystem 106 may include a display controller 156 , a click wheel controller 158 and one or more input controllers 160 for other input or control devices.
  • the one or more input controllers 160 receive/send electrical signals from/to other input or control devices 160 .
  • the other input/control devices 160 may include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, sticks, and so forth.
  • the display system 112 provides an output interface and/or an input interface between the device and a user.
  • the display controller 156 receives and/or sends electrical signals from/to the display system 112 .
  • the display system 112 displays visual output to the user.
  • the visual output may include text, icons, graphics, video, and any combination thereof. In some embodiments, some or all of the visual output may correspond to user-interface objects, further details of which are described below.
  • the display system 112 also accepts input from the user based on haptic and/or tactile contact.
  • the display system 112 forms a touch-sensitive surface that accepts user input.
  • the display system 112 and the display controller 156 (along with any associated modules and/or sets of instructions in the memory 102 ) detect contact (and any movement or breaking of the contact) on the display system 112 and converts the detected contact into interaction with user-interface objects, such as one or more soft keys, that are displayed on a touch screen.
  • a point of contact between a touch screen in the display system 112 and the user corresponds to one or more digits of the user.
  • the touch screen in the display system 112 may use LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology, although other display technologies may be used in other embodiments.
  • a touch screen in the display system 112 and the display controller 156 may detect contact and any movement or breaking thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with a touch screen in the display system 112 .
  • a touch-sensitive display in some embodiments of the display system 112 may be analogous to the multi-touch sensitive tablets described in the following U.S. Pat. No.
  • a touch screen in the display system 112 displays visual output from the portable device 100 , whereas touch sensitive tablets do not provide visual output.
  • the touch screen in the display system 112 may have a resolution in excess of 100 dpi. In an exemplary embodiment, the touch screen in the display system may have a resolution of approximately 168 dpi.
  • the user may make contact with the touch screen in the display system 112 using any suitable object or appendage, such as a stylus, finger, and so forth.
  • the device 100 may include a touchpad (not shown) for activating or deactivating particular functions.
  • the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output.
  • the touchpad may be a touch-sensitive surface that is separate from the touch screen in the display system 112 or an extension of the touch-sensitive surface formed by the touch screen.
  • the device 100 may include a click wheel 114 .
  • a user may navigate among one or more graphical objects (henceforth referred to as icons) displayed in the display system 112 by rotating the click wheel 114 or by moving (e.g., angular displacement) of a point of contact with the click wheel 114 .
  • the click wheel 114 may also be used to select one or more of the displayed icons. For example, the user may press down on at least a portion of the click wheel 114 or an associated physical button.
  • User commands and navigation commands provided by the user via the click wheel 114 may be processed by the click wheel controller 158 as well as one or more of the modules and/or sets of instructions in the memory 102 .
  • the device 100 also includes a power system 162 for powering the various components.
  • the power system 162 may include a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.
  • a power management system e.g., one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.
  • power sources e.g., battery, alternating current (AC)
  • AC alternating current
  • a recharging system
  • the software components stored in the memory 102 may include an operating system 126 , a communication module (or set of instructions) 128 , a contact/motion module (or set of instructions) 130 , a graphics module (or set of instructions) 132 , one or more applications (or set of instructions) 136 , a timer module (or set of instructions) 144 , a word prediction module (or set of instructions) 146 , an address book 148 , a user word history 150 , one or more character sets 152 , and one or more lexicography models 154 .
  • the graphics module 132 may include an icon effects module (or set of instructions) 134 .
  • the applications module 136 may include a telephone module (or set of instructions) 138 , a text messaging module (or set of instructions) 140 and/or a music player module (or set of instructions) 142 .
  • the operating system 126 e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks
  • the operating system 126 includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
  • the communication module 128 facilitates communication with other devices over one or more external ports 124 and also includes various software components for handling data received by the RF circuitry 108 and/or the external port 124 .
  • the external port 124 e.g., Universal Serial Bus (USB), FIREWIRE, etc.
  • USB Universal Serial Bus
  • FIREWIRE FireWire
  • the external port 124 is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.).
  • the contact/motion module 130 may detect contact with the click wheel 114 and/or a touch screen in the display system 112 (in conjunction with the display controller 156 ).
  • the contact/motion module 130 includes various software components for performing various operations related to detection of contact, such as determining if contact has occurred, determining if there is movement of the contact and tracking the movement across the click wheel 114 and/or a touch screen in the display system 112 , and determining if the contact has been broken (i.e., if the contact has ceased). Determining movement of the point of contact may include determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (including magnitude and/or direction) of the point of contact.
  • the contact/motion module 130 and the display controller 156 also detect contact on a touchpad.
  • the graphics module 132 includes various known software components for rendering and displaying graphics on the display system 112 .
  • graphics includes any object that can be displayed to a user, including without limitation text, web pages, icons (such as user-interface objects including soft keys), digital images, videos, animations and the like.
  • the graphics module 132 includes the icon effects module 134 .
  • the icon effects module 134 may modify a displayed position of one or more icons on the display system 112 (in conjunction with the display controller 156 ) based on user actions (such as detecting a contact corresponding to at least one icon). In some embodiments, the modification of the displayed icon(s) may be based on an animation sequence.
  • the one or more applications 136 may include any applications installed on the device 100 , including without limitation, a browser, the address book 148 , contact list, email, instant messaging, word processing, keyboard emulation, widgets, JAVA-enabled applications, encryption, digital rights management, voice recognition, voice replication, location determination capability (such as that provided by the global positioning system (GPS)), etc.
  • a browser the address book 148
  • contact list email
  • instant messaging word processing
  • keyboard emulation widgets
  • JAVA-enabled applications JAVA-enabled applications
  • encryption digital rights management
  • voice recognition voice replication
  • location determination capability such as that provided by the global positioning system (GPS)
  • the telephone module 138 may be used to enter a sequence of characters corresponding to a telephone number, access one or more telephone numbers in the address book 148 , modify a telephone number that has been entered, dial a respective telephone number, conduct a conversation and disconnect or hang up when the conversation is completed.
  • the text messaging module 140 may be used to enter a sequence of characters corresponding to a text message, to modify previously entered characters, to transmit a respective text message (for example, using a Short Message Service or SMS protocol), to receive text messages and to view received text messages.
  • transmitted and/or received text messages may include graphics, photos, audio files, video files and/or other attachments as are supported in a Multimedia Message Service (MMS) and/or an Enhanced Messaging Service (EMS).
  • MMS Multimedia Message Service
  • EMS Enhanced Messaging Service
  • the music player module 142 allows the user to play back recorded music stored in one or more files, such as MP3 or AAC files.
  • the device 100 may include the functionality of an MP3 player, such as an iPod (trademark of Apple Computer, Inc.).
  • the device 100 may, therefore, include a 36-pin connector that is compatible with the iPod.
  • the timer module 144 may provide a time reference and/or time stamps for user commands received by the device 100 , for example, using the click wheel 114 and the click wheel controller 158 .
  • the word prediction module 146 may be used in conjunction with one or more of the applications 136 , such as the text messaging module 140 .
  • the word prediction module 146 may suggest one or more words or symbols (such as punctuation marks, pronunciation marks or spaces) in accordance with a context.
  • the context may be based on one or more of the lexicography models 154 (for example, grammatical and/or syntax rules associated with one or more languages) and/or a user word history 150 .
  • the context may include one or more previously entered words, characters, and/or symbols.
  • the context may depend on which of the applications 136 is being used. For example, there may be different contexts for an email application as opposed to a word processing application.
  • a user interface and associated process that include recommended words from the word prediction module 146 are discussed further below with reference to FIGS. 8 and 9 .
  • the user word history 150 may include static content (such as that associated with a dictionary) and/or dynamic content (such as that associated with characters, symbols and/or words that are routinely and/or recently used by the user).
  • the user word history 150 may include a static dictionary built up by scanning a user's address book, emails, and other documents.
  • the user word history 150 may include weighted scores or probabilities for predicted words based on a set of characters, symbols and/or words that are provided by the user to the device 100 , for example, using the display system 112 , the click wheel 114 and the click wheel controller 158 .
  • the user word history 150 may also include use statistics (e.g., time of use and/or frequency of use) of one or more characters, symbols and/or words that are provided by the user. The user word history 150 is discussed further below with reference to FIGS. 10A and 10B .
  • the character sets 152 may include one or more sets of characters corresponding to numbers, letters and/or symbols. The letters and/or symbols may correspond to one or more languages.
  • the character sets 152 may be used by one or more of the applications 136 , such as the text messaging module 140 .
  • a data structure associated with the one or more character sets (which may be used in one or more of the keyboard embodiments) is discussed further below with reference to FIG. 5 .
  • the device 100 may include one or more optional optical sensors (not shown), such as CMOS or CCD image sensors, for use in imaging applications.
  • optional optical sensors such as CMOS or CCD image sensors, for use in imaging applications.
  • the device 100 is a device where operation of a predefined set of functions on the device is performed exclusively through a touch screen in the display system 112 and/or a touchpad.
  • a touch screen and/or a touchpad as the primary input/control device for operation of the device 100 , the number of physical input/control devices (such as push buttons, dials, and the like) on the device 100 may be reduced.
  • the device 100 includes a touch screen, a touchpad, a push button for powering the device on/off and locking the device, a volume adjustment rocker button and a slider switch for toggling ringer profiles.
  • the push button may be used to turn the power on/off on the device by depressing the button and holding the button in the depressed state for a predefined time interval, or may be used to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed.
  • the device 100 also may accept verbal input for activation or deactivation of some functions through the microphone 113 .
  • the predefined set of functions that are performed exclusively through a touch screen and/or a touchpad include navigation between user interfaces.
  • the touchpad when touched by the user, navigates the device 100 to a main, home, or root menu from any user interface that may be displayed on the device 100 .
  • the touchpad may be referred to as a “menu button.”
  • the menu button may be a physical push button or other physical input/control device instead of a touchpad.
  • the device 100 is a device where operation of a predefined set of functions on the device is performed exclusively or primarily through the click wheel 114 .
  • the click wheel 114 as the primary input/control device for operation of the device 100 , the number of other physical input/control devices (such as push buttons, dials, and the like) on the device 100 may be reduced.
  • FIG. 2 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 200 .
  • the device 200 includes a touch screen 208 .
  • the touch screen may display one or more trays.
  • a tray is a region within a graphical user interface.
  • One tray may include a user entry interface, such as a keyboard 210 that includes a plurality of icons.
  • the icons may include one or more symbols.
  • a user may select one or more of the icons, and thus, one or more of the corresponding symbols, by making contact or touching the keyboard 210 , for example, with one or more fingers 212 (not drawn to scale in the figure).
  • the contact may correspond to the one or more icons.
  • selection of one or more icons occurs when the user breaks contact with the one or more icons.
  • the contact may include a gesture, such as one or more taps, one or more swipes (from left to right, right to left, upward and/or downward) and/or a rolling of a finger (from right to left, left to right, upward and/or downward) that has made contact with the device 200 .
  • in advertent contact with an icon may not select a corresponding symbol. For example, a swipe gesture with an icon may not select a corresponding symbol when the gesture corresponding to selection is a tap.
  • the device 200 may include a display tray 214 .
  • the display tray 214 may display one or more of the characters and/or symbols that are selected by the user.
  • the device 200 may also include one or more physical buttons, such as the clear, hold and menu buttons shown in FIG. 2 .
  • the menu button may be used to navigate within a hierarchy of applications that may be executed on the device 200 .
  • the clear, hold, and/or menu buttons are implemented as soft keys in a GUI in touch screen 208 .
  • FIGS. 3A-3C are schematic diagrams illustrating an embodiment of a user interface for a portable electronic device 300 .
  • the user interface includes a keyboard 310 that includes a plurality of icons.
  • the icons include three symbols each. In other embodiments, the icons include two symbols each. In other embodiments, different icons on the same keyboard may include one, two, or three symbols each (e.g., some icons may contain one symbol while other icons contain two or three symbols).
  • the symbols on the icons are in a non-standard configuration, i.e., non-qwerty.
  • the total number of icons in the keyboard 310 is less than the number of physical keys in a standard keyboard.
  • the symbols in the icons in the keyboard 310 may be determined using a lexicography model, such as a language.
  • the lexicography model may include a frequency of use of symbols in a language. For example, characters or symbols that are unlikely to occur immediately proximate to one another or immediately after one another in a set of symbols that the user may enter may be grouped on a respective icon 312 ( FIG. 3B ).
  • a language may include slang as well as individual usage (for example, words that are commonly used by the user).
  • the lexicography model may correspond to a user usage or word history that occurs prior to the user making contact with the device 300 , i.e., a past usage.
  • the shape of the respective icon 312 is modified. This provides information to the user as to which icon and which symbol the contact 314 currently corresponds. This may be useful since the contact 314 may obscure at least a portion of the respective icon 312 making it difficult for the user to see the respective symbol he or she is currently positioned on.
  • the icons in the keyboard 310 may at least in part include an arc.
  • the shape of the respective icon 312 may be asymmetrically distorted and the respective symbol that the contact 314 currently corresponds to may be displayed within the shape of the respective icon 312 and outside of the contact 314 .
  • the user may select the respective symbol by making the contact 314 with the respective icon 312 and rolling a finger over a region within the respective icon 312 that corresponds to the respective symbol. If the user determines, based on the modified shape of the respective icon 312 and/or the displayed symbol within the modified shape that the wrong symbol is currently contacted, the user may roll their finger to a different position within the respective icon 312 that corresponds to the correct symbol. Once the contact 314 has been positioned over or proximate to the correct symbol, the user may select this symbol by breaking the contact 314 with the respective icon 312 . The selected symbol (such as the letter ‘a’) may then be displayed in the display tray 214 . In some embodiments, if the contact 314 is maintained by the user for a time interval that is more than a first pre-determined value, such as 0.5, 1 or 2 s, before the contact 314 is broken, the respective symbol may be capitalized.
  • a first pre-determined value such as 0.5, 1 or 2 s
  • the user may clear the entire display tray 214 using a clear icon or may delete a most recently selected symbol using a delete icon.
  • a set of symbols such as a message
  • the user may accept the set of symbols (which may store and/or send the set of symbols depending on the application executing on the device 300 ) using an accept icon.
  • an additional visual indicator corresponding to the respective icon 312 may be provided on the display 208 .
  • the visual indicator may be proximate to the respective icon 312 .
  • the visual indicator may include a band 318 around at least a portion of the respective icon 312 .
  • a shape of the respective icon 312 may not be modified in response to the contact 314 . Instead, an icon 316 corresponding to the respective symbol 316 may be displayed proximate to the respective icon 312 .
  • the modifying of the shape of the respective icon 312 and/or the displaying of the visual indicator, such as the band 318 and/or the icon 316 , may be included in at least some of the embodiments discussed further below.
  • the device 300 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed.
  • the keyboard 310 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboard 310 .
  • FIG. 4 is a flow diagram of an embodiment of a symbol entry process 400 . While the symbol entry process 400 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 400 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 410 ).
  • a respective icon may correspond to two or more symbols. Contact by a user with the display that corresponds to the respective icon may be detected ( 412 ). The displayed respective icon may be modified to indicate that the contact corresponds to a respective symbol in the two or more symbols ( 414 ). The respective symbol may be optionally displayed in a region within the shape of the respective icon and outside of a region corresponding to the contact ( 416 ). A visual indicator corresponding to the respective symbol may be optionally provided ( 418 ). The respective symbol may be optionally capitalized when contact is maintained for a time interval exceeding a pre-determined value ( 420 ). The respective symbol may be selected when the user breaks contact with the respective icon ( 422 ).
  • FIG. 5 is a block diagram illustrating an embodiment of a character set data structure 500 .
  • the character sets 152 may include multiple sets 512 of characters and/or symbols.
  • a respective set such as the set 512 - 1 , may include one or more symbols 514 and one or more probabilities 516 .
  • the probabilities may include frequencies of occurrence of use, as well as conditional probabilities (such as the probability of a given symbol occurring given one or more symbols that have already occurred).
  • the character set data structure 500 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • FIGS. 6A-6D are schematic diagrams illustrating an embodiment of a user interface for a portable electronic device 600 .
  • the device 600 includes a keyboard 610 that has a plurality of icons arranged in rows. A given row includes a subset of the plurality of icons. Adjacent rows are separated by a space greater than a second pre-determined value, such as a height of one of the icons.
  • an icon 614 may be displayed in the space between two adjacent rows.
  • the icon may correspond to a respective symbol that corresponds to the respective icon that the user has contacted 612 .
  • the icon 614 may correspond to the character ‘u’.
  • the user may receive feedback that the respective icon (and thus, the respective symbol) is currently contacted. This may be useful because the contact 612 may obscure the respective icon, and thus, the respective symbol, that has been selected in the rows of icons.
  • the icon 614 may be displayed above a respective row in which the contact 612 has occurred. In some embodiments, the icon 614 may be magnified, i.e., larger, than the respective icon.
  • the icon 614 may be displayed while the contact 612 is maintained. When the user breaks the contact 612 with the respective icon, the respective symbol may be selected. In some embodiments, the respective symbol may be displayed in the display tray 214 .
  • a keyboard 616 may be displayed with rows of icons. Initially, the rows of icons may not include a significant space between adjacent rows, e.g., the space may be less than the second pre-determined value.
  • the displayed keyboard 616 may be modified to include a space greater the second pre-determined value and the icon 614 may be displayed. This modified configuration or layout of the keyboard 616 may be maintained while the contact 612 is maintained by the user.
  • a keyboard 618 may include rows of icons.
  • an icon 620 may be displayed superimposed over at least one or more additional icons in the keyboard 618 .
  • the device 600 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed.
  • the keyboards 610 , 616 and/or 618 may include fewer or additional icons.
  • a different character set and/or different groups of symbols may be used on the icons in the keyboards 610 , 616 and/or 618 .
  • FIG. 7 is a flow diagram of an embodiment of a symbol entry process 700 . While the symbol entry process 700 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 700 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 710 ). Two or more subsets of the plurality of icons may be arranged in rows. A contact by a user with the display that corresponds to a respective icon may be detected ( 712 ). A symbol corresponding to the respective icon may be optionally displayed between a row corresponding to the respective icon and a neighboring row ( 714 ). A symbol corresponding to the respective icon may be optionally displayed superimposed over one or more additional icons in the plurality of icons ( 716 ).
  • FIG. 8 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 800 .
  • the device 800 may include a tray 812 that includes one or more recommended words 810 .
  • the one or more recommended words 810 may be determined using a user word history. This is discussed further below with reference to FIGS. 10A and 10B .
  • the one or more recommended words 810 are displayed prior to detecting any contacts corresponding to text input (symbol selection) by the user in a current application session.
  • the one or more recommended words 810 may be displayed when the user initially opens an application, such as email, on the device 800 .
  • the one or more recommended words 810 therefore, may be determined based on a user word or usage history that may be application specific.
  • the one or more recommended words 810 may change dynamically in response to contacts corresponding to text input by the user during the application session.
  • the user may select one or more of the recommended words 810 by making contact with the display 208 .
  • one or more of the recommended words 810 such as a phrase (“How are you?”) may be selected with a single contact.
  • the contact may include a gesture, such as one or more taps, one or more swipes, and/or a rolling motion of a finger that makes the contact.
  • the one or more taps may have a duration that is less than a third pre-determined value, such as 0.1, 0.5 or 1 s.
  • the device 800 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed.
  • the keyboard 210 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboard 210 .
  • FIG. 9 is a flow diagram of an embodiment of a symbol entry process 900 . While the symbol entry process 900 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 900 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 910 ).
  • a respective icon may correspond to at least one symbol.
  • One or more recommended words may be displayed ( 912 ). The one or more recommended words may be in accordance with a user history prior to detecting any contacts corresponding to text input (symbol selection) by the user in a current application session.
  • a contact by the user with the display may be detected ( 914 ). The contact may include a gesture.
  • a respective recommended word that corresponds to the gesture may be selected ( 916 ).
  • FIG. 10A is a block diagram illustrating an embodiment of a user word history data structure 1000 .
  • the user word history 150 may include a deleted word stack 1010 and multiple words 1016 .
  • the words 1016 may include one or more characters and/or one or more symbols.
  • the deleted word stack 1010 includes one or more words 1014 in a sequential order in which the one or more words 1014 were deleted by the user in an application, such as the text messaging module 140 ( FIG. 1 ).
  • a respective word in the words 1016 may include multiple records.
  • a respective record may include a time-weighted score 1018 , use statistics 1020 (such as a time of use and/or a frequency of use), a context 1022 and one or more applications 1024 .
  • the time-weighted score 1018 may indicate a probability that the word 1016 -M is a next predicted word based on the context 1022 (one or more characters, symbols and/or words that have previously been provided by the user) and/or the application 1024 .
  • the time-weighted score 1018 may therefore be different for email than for the text messaging module 140 ( FIG. 1 ).
  • the time-weighted score 1018 may be computed to favorably weight (e.g., give a higher probability) to words that are used recently. For example, the time-weighted score 1018 may give favorable weighting to words 1016 that are used within the last 24 hours or week. Words 1016 used on longer time scales (e.g., more than a day or a week ago) may have their corresponding time-weighted scores 1018 reduced by a pre-determined ratio (such as 0.9) for each additional time interval (e.g., each day or week) since the words 1016 were last used.
  • a pre-determined ratio such as 0.9
  • the user history data structure 1000 may include static information (for example, corresponding to a dictionary and/or grammatical and syntax rules for one or more languages) as well as dynamic information (based on recent usage statistics and/or patterns). Thus, the user history data structure 1000 may be dynamically updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user.
  • the user history data structure 1000 may include a static dictionary built up by scanning a user's address book, emails, and other documents.
  • the user history data structure 1000 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • FIG. 10B is a block diagram illustrating an embodiment of a language data structure system 1050 .
  • the language data structure system 1050 may be used to provide recommended words in the device 800 ( FIG. 8 ).
  • a sequence of symbols 1062 (including one or more characters, symbols and/or words) may be provided by the user.
  • a set of symbols 1062 corresponding to a context 1022 - 1 may be processed by a context map 1060 .
  • the context 1022 - 1 may be a null set, i.e., one or more recommended words are provided before the user provides any symbols 1062 (e.g., when an application is first opened).
  • the context 1022 - 1 may include one or more previously entered or provided words as well as one or more symbols, such as the first one, two or three letters in a current word that the user is providing.
  • the context map 1060 may include a select and hashing module 1064 and a hash map 1066 .
  • the hash map 1066 may select one or more appropriate entries in an application-specific dictionary 1068 .
  • the entries in the application-specific dictionary 1068 may include contexts 1070 , predicted words 1072 , and time-weighted scores 1074 .
  • the application-specific dictionary 1068 may utilize the records in the user history data structure 1000 . As a consequence, the application-specific dictionary 1068 may be dynamically updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user.
  • the language data structure system 1050 may be used to provide one or more recommended words based on the context 1022 - 1 .
  • the context map may find a top-5 or top-10 best context 1070 matches.
  • the corresponding predicted words 1072 may be recommended to the user in accordance with the time-weighted scores 1074 .
  • only a subset of the predicted words 1072 corresponding to the best context 1070 matches may be presented to the user (e.g., just the top-1, top-2, or top-3 predicted words).
  • the language data structure system 1050 may provide one or more recommended words in accordance with a state machine (corresponding to a Markov sequence or process) that corresponds to a language.
  • a state machine corresponding to a Markov sequence or process
  • the application-specific dictionary 1068 may be based on a stochastic model of the relationships among letters, characters, symbols and/or words in a language.
  • a path memory (such as up to three characters in a word that is currently being entered and/or two or three previously entered words) of the probabilistic model represents a tradeoff between accuracy and the processing and power capabilities (for example, battery life) of the portable electronic device 100 ( FIG. 1 ).
  • a probabilistic model may be based on a lexicography and usage that is user-specific and/or, as discussed previously, even application specific. For example, user emails, address book and/or other documents may be analyzed to determine an appropriate probabilistic model for that user based on the syntax and/or lexicography (including names and slang) that are employed by the user.
  • the probabilistic model may be updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user.
  • the probabilistic model may be based on one or more mistakes made by the user when using the click wheel 114 ( FIG. 1 ) and/or a touch-sensitive display in the display system 112 ( FIG. 1 ). For example, if the user accidentally selects the wrong icon when typing a respective word, the probabilistic model may be updated to account for such errors in the future. In an exemplary embodiment, a mistake may be determined based on a user activation of an icon corresponding to the delete function.
  • This adaptability of the portable electronic device 100 ( FIG. 1 ) may allow correction of user interface errors (such as parallax and/or left-right symmetry) associated with which finger(s) the user is using and how the user is holding the portable electronic device 100 ( FIG. 1 ) while using it. This functionality is discussed further below with reference to FIG. 14 .
  • the language data structure system 1050 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • FIG. 11A is a flow diagram of an embodiment of a symbol entry process 1100 . While the symbol entry process 1100 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1100 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1110 ).
  • a respective icon may correspond to two or more symbols.
  • a contact by a user with the display that corresponds to selection of the respective icon may be detected ( 1112 ).
  • a symbol in the two or more symbols for which the contact further corresponds may be determined ( 1114 ).
  • FIG. 11B is a flow diagram of an embodiment of a symbol entry process 1130 . While the symbol entry process 1130 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1130 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1132 ).
  • a respective icon may correspond to two or more symbols.
  • a first symbol may belong to a first subset of symbols and a second symbol may belong to a second subset of symbols.
  • the first symbol may have a probability of occurrence greater than the second symbol.
  • a contact by a user with the display that corresponds to selection of the respective icon may be detected ( 1134 ).
  • a symbol in the two or more symbols for which the contact further corresponds may be determined ( 1136 ).
  • FIG. 11C is a flow diagram of an embodiment of a symbol entry process 1150 . While the symbol entry process 1150 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1150 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1152 ).
  • a respective icon may correspond to two or more symbols.
  • a first symbol may belong to a first subset of symbols and a second symbol may belong to a second subset of symbols.
  • the second symbol may have a probability of occurrence immediately following the first symbol that is less than a pre-determined value.
  • a contact by a user with the display that corresponds to selection of the respective icon may be detected ( 1154 ).
  • a symbol in the two or more symbols for which the contact further corresponds may be determined ( 1156 ).
  • FIGS. 12A-12G are schematic diagrams illustrating embodiments of a user interface for a portable electronic device 1200 . These embodiments may utilize the symbol entry processes 1100 ( FIG. 11A ), 1130 ( FIG. 11B ) and/or 1150 ( FIG. 11C ) described previously.
  • the device 1200 may include a keyboard 1210 with a plurality of icons.
  • a respective icon may include two or more symbols.
  • a first symbol for a respective icon may be selected by the user using a first gesture.
  • a second symbol for a respective icon may be selected by the user using a second gesture.
  • the first gesture may include a continuous contact with the display 208 and the second gesture may include a discontinuous contact with the display 208 .
  • the continuous contact may include a swipe and/or a rolling motion of the contact.
  • the discontinuous contact may include one or more consecutive taps.
  • a respective tap may include contact with the display 208 for a time interval that is less than a fourth pre-determined value, such as 0.1, 0.5 or 1 s.
  • two or more consecutive taps may correspond to a second symbol if a time interval between the two or more consecutive taps is less than a fifth pre-determined value, such as 0.1, 0.5 or 1 s.
  • the first symbol is in a first subset of the symbols in the character set displayed in the keyboard 1210 and the second symbol is in a second subset of the symbols in the character set displayed in the keyboard 1210 .
  • the first subset may have a probability of occurrence that is greater than a sixth pre-determined value and the second subset may have a probability of occurrence that is less than the sixth pre-determined value.
  • the first subset may include symbols that are more likely to occur, for example, in a language (using a lexicography model) and/or based on a user history.
  • the gesture used to select the first symbol may, therefore, be easier or quicker for the user to make.
  • the first gesture may be a tap gesture and the second gesture may be a swipe gesture.
  • the gestures needed to select corresponding symbols for a respective icon may be indicated on the icon.
  • a dot on the icon may correspond to a tap and a horizontal line on the icon may correspond to a dash.
  • This ‘tap-dash’ embodiment is an example of a two-gesture keyboard. Additional examples are discussed below.
  • the first symbol may have a probability of occurrence immediately after the second symbol that is less than a seventh pre-determined value.
  • the second symbol may have a probability of occurrence immediately after the first symbol that is less than a seventh pre-determined value.
  • FIGS. 12B-12G illustrate additional multi-gesture keyboards.
  • a first symbol for a respective icon in these keyboards may be selected with a first gesture (for example, a single tap) and a second symbol for the respective icon may be selected using a second gesture (for example, two consecutive taps).
  • the keyboard 1222 in FIG. 12G includes some icons that correspond to more than two symbols. These symbols may be selected by making additional gestures, such as three consecutive taps.
  • a second or third symbol for the respective icon may be selected by the user by first contacting a meta key, such as a shift key, and then contacting and/or breaking contact with the respective icon.
  • the device 1200 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed.
  • the keyboards 1210 , 1212 , 1214 , 1216 , 1218 , 1220 and/or 1222 may include fewer or additional icons.
  • a different character set and/or different groups of symbols may be used on the icons in the keyboard 1210 , 1212 , 1214 , 1216 , 1218 , 1220 and/or 1222 .
  • the user selects symbols by breaking a contact with one or more icons on the display 208 .
  • the user may select one or more symbols without breaking contact with the display 208 .
  • the user may pause or maintain contact over the respective icon for a time interval longer than an eighth pre-determined value (such as 0.1, 0.5 or 1 s) before moving on to the next icon and corresponding symbol.
  • an eighth pre-determined value such as 0.1, 0.5 or 1 s
  • the user may maintain contact with the display.
  • selection of the respective icon and corresponding symbol may occur by increasing a contact pressure with the display 208 while maintaining the contact with the display.
  • FIG. 13 A flow chart for a symbol entry process 1300 corresponding to embodiments where contact is not broken is shown in FIG. 13 . While the symbol entry process 1300 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1300 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1310 ).
  • a respective icon may correspond to at least one symbol.
  • a contact by a user with the display may be detected ( 1312 ).
  • Positions of the contact corresponding to a sequence of icons may be determined ( 1314 ).
  • the at least one symbol may be selected when a respective position of the contact corresponds to the respective icon for a time interval exceeding a pre-determined value ( 1316 ).
  • the device 100 may, therefore, adapt an offset between an estimated contact and an actual contact in accordance with such errors.
  • Feedback may be provided by the user activating an icon corresponding to a delete key.
  • the offset may be applied to one or more icons. In some embodiments, there may be more than one offset and a respective offset may be applied to a respective subset that includes one or more icons in a plurality of the icons in a keyboard or other user interface.
  • the adaptation may occur continuously, after a pre-determined time interval and/or if an excessive number of user errors occur (e.g., as evidenced by a frequency of use of the delete icon). The adaptation may occur during a normal mode of operation of the device 100 ( FIG. 1 ), rather than requiring the user to implement a separate keyboard training/adaptation mode.
  • FIG. 14 A flow chart for a symbol entry process 1400 corresponding to such embodiments is shown in FIG. 14 . While the symbol entry process 1400 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1400 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1410 ).
  • a respective icon may correspond to at least one symbol.
  • a contact by a user with the display may be detected ( 1412 ).
  • An estimated contact that corresponds to the respective icon and the at least one symbol may be determined in accordance with the actual contact and pre-determined offset ( 1414 ).
  • One or more corrections for one or more errors in one or more selected symbols may be received ( 1416 ).
  • the offset for at least the respective icon may be modified in accordance with the one or more received corrections ( 1418 ).
  • FIG. 15 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1500 .
  • the device 1500 includes a keyboard 1510 with a plurality of icons. Different spacings (“guard bands”) are used between the icons.
  • the guard bands between icons visually encourage a user to touch the center of an adjacent icon, although user contact in the guard band region may also activate the nearest icon to the contact.
  • icons near the center of the display 208 may have a smaller guard band between adjacent icons than icons near an edge of the display. This may reduce errors when using the display 208 if it is easier for a user to select or contact a respective icon near the center of the display 208 .
  • the guard band near the edge of the display 208 may be larger than that near the center of the display 208 .
  • icons near the center of the display 208 may have a larger guard band between adjacent icons than icons near an edge of the display. This may reduce errors when using the display 208 if it is easier for a user to select or contact a respective icon near the edge of the display 208 .
  • the guard band near the edge of the display 208 may be smaller than that near the center of the display 208 .
  • icons near the center of the display 208 may be larger than icons near the edge of the display 208 .
  • icons at the edge of the display are about half the size of the other icons because it is easier to identify contacts corresponding to edge icons.
  • either the size of the icons or the size of the guard bands between icons could incrementally vary between the edge of the display and the center of the display (e.g., from small icons at the edge to large icons in the center or from small guard bands at the edge to large guard bands in the center).
  • FIG. 16 A flow chart for a symbol entry process 1600 corresponding to such embodiments is shown in FIG. 16 . While the symbol entry process 1600 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1600 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1610 ).
  • the plurality of icons may be arranged in rows in a first dimension of the display.
  • a first guard band in the first dimension between adjacent icons in a first subset of the icons may be greater than a pre-determined value and a second guard band in the first dimension between adjacent icons in a second subset of the icons may be less than a pre-determined value.
  • a contact by the user with the display that corresponds to selection of the respective icon may be detected ( 1612 ).
  • a symbol corresponding to the respective icon may be displayed ( 1614 ).
  • FIG. 17 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1700 .
  • the device 1700 includes a keyboard 1710 that has a plurality of icons.
  • a respective icon corresponds to two or more symbols. Some symbols may be selected by contacting two or more icons simultaneously.
  • a respective symbol that is selected may be displayed in the display tray 214 .
  • a letter ‘e’ may be selected by contacting and breaking contact with the first icon in the first row.
  • a letter ‘l’ may be selected by contacting and breaking contact with the first and the second icons in the first row.
  • the icons include visual information indicating the combinations of contacts with icons (also referred to as chords) that correspond to given symbols.
  • Keyboard 1710 is sometimes referred to as a hop-scotch keyboard.
  • FIG. 18 A flow chart for a symbol entry process 1800 corresponding to such embodiments is shown in FIG. 18 . While the symbol entry process 1800 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1800 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • a plurality of icons may be displayed on a touch-sensitive display ( 1810 ).
  • a first icon and a second icon each correspond to two or more symbols.
  • a contact by a user with the display that corresponds to the first icon and the second icon is detected ( 1812 ).
  • a respective symbol in the two or more symbols to which the contact corresponds may be determined ( 1814 ).
  • a visual indicator corresponding to the respective symbol is displayed ( 1816 ).
  • FIG. 19 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1900 .
  • a keyboard 1910 does not include fixed icons. Instead symbols are displayed.
  • a nearest group of symbols such as three letters in a region 1912 , are selected in accordance with a user contact with the display 208 .
  • the region 1912 may include two or more symbols or characters.
  • a correct set of symbols may be determined using a lexicography model or system, such as that shown in FIG. 10A , in accordance with a sequence of groups of symbols that correspond to a sequence of contacts by the user. As more contacts occur, a tree of possible words or sets of symbols corresponding to the groups of symbols that have been selected may be pruned until a correct or highest likelihood word or set of symbols is determined.
  • a respective user may play a game that is used to determine a smallest acceptable key size for a user interface, such as a keyboard.
  • the smallest key size may be in accordance with a user's manual dexterity, age, health, finger size and vision. Errors made in using the icons in a keyboard during the game may help determine a minimum icon size for the respective user.
  • icons in the embodiments of the user interfaces may have an effective contact area or a strike area that is larger than the displayed icon size.
  • the effective contact area or strike area may be larger than the displayed icon size in at least one dimension of the display 208 surface.

Abstract

A plurality of icons are displayed on a touch-sensitive display. Two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display. A space greater than a pre-determined value is included between adjacent rows. A contact by a user with the touch-sensitive display that corresponds to a respective icon is detected. A symbol corresponding to the respective icon is displayed in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.

Description

    RELATED APPLICATIONS
  • This application is a divisional of U.S. patent application Ser. No. 11/459,606, filed Jul. 24, 2006, entitled “Keyboards for Portable Electronic Devices,” which claims priority to U.S. Provisional Patent Application No. 60/756,890, filed Jan. 5, 2006, entitled “Keyboards for Portable Electronic Devices,” which applications are incorporated by reference herein in their entirety.
  • This application is related to U.S. patent application Ser. No. 11/459,615, filed Jul. 24, 2006, entitled “Touch Screen Keyboards for Portable Electronic Devices,” which application is incorporated by reference herein in its entirety.
  • This application is related to U.S. patent application Ser. No. 11/228,700, filed Sep. 16, 2005, entitled “Operation of a Computer with Touch Screen Interface,” which application is incorporated by reference herein in its entirety.
  • This application is related to U.S. patent application Ser. No. ______, filed ______, entitled “User Interface Including Word Recommendations,” Attorney Docket No. P4150USD2/63266-5271US), which application is incorporated by reference herein in its entirety.
  • This application is related to U.S. patent application Ser. No. ______, filed ______, entitled “Keyboard with Multi-Symbol Icons,” Attorney Docket No. P4150USD3/63266-5272US), which application is incorporated by reference herein in its entirety.
  • This application is related to U.S. patent application Ser. No. ______, filed ______, entitled “Keyboard with Multi-Symbol Icons,” Attorney Docket No. P4150USD4/63266-5273US), which application is incorporated by reference herein in its entirety.
  • TECHNICAL FIELD
  • The disclosed embodiments relate to user interfaces, and in particular, to user interfaces that include a touch screen keyboard.
  • BACKGROUND
  • As portable devices become more compact, and the amount of information to be processed and stored increases, it has become a significant challenge to design a user interface that allows users to easily interact with the device. This is unfortunate since the user interface is the gateway through which users receive not only content but also responses to user actions or behaviors, including user attempts to access a device's features or tools. Some portable electronic devices (e.g., mobile phones) have resorted to adding more pushbuttons, increasing a density of push buttons, overloading the functions of pushbuttons, or using complex menu systems to allow a user to access, store and manipulate data. These conventional user interfaces often result in complicated key sequences and menu hierarchies that must be memorized by the user. In addition, as the number of pushbuttons has increased the proximity of neighboring buttons often makes it difficult for users to activate a desired pushbutton.
  • Many conventional user interfaces, such as those that include physical pushbuttons, are also inflexible. This is unfortunate since it may prevent a user interface from being configured and/or adapted by either an application running on the portable device or by users. When coupled with the time consuming requirement to memorize multiple key sequences and menu hierarchies, and the difficulty in activating a desired pushbutton, such inflexibility is frustrating to most users.
  • Accordingly, there is a need for more transparent and intuitive user interfaces for portable electronic devices that are easy to use, configure, and/or adapt.
  • SUMMARY OF EMBODIMENTS
  • The above deficiencies and other problems associated with user interfaces for portable devices are reduced or eliminated by the disclosed touch screen keyboards and their methods of use.
  • In some embodiments, a method includes displaying a plurality of icons on a touch-sensitive display. A respective icon in at least a subset of the plurality of icons corresponds to two or more symbols. A contact by a user with the touch-sensitive display that corresponds to the respective icon is detected. A respective symbol in the two or more symbols to which the contact further corresponds is determined. The displayed respective icon is modified to indicate that the contact corresponds to the respective symbol.
  • The respective symbol may be selected when the user breaks contact with the respective icon. The respective symbol may be capitalized when contact is maintained for a time interval exceeding a pre-determined value.
  • Modifying may include changing a shape of the respective icon. Changing the shape may include an asymmetric distortion of the shape. An initial shape of the respective icon may include an arc.
  • Detecting may include detecting rolling of a finger over a region that corresponds to the respective symbol. The contact may include a gesture that is selected from the group consisting of one or more taps, a swipe and a rolling of a finger.
  • The two or more symbols for the respective icon may be determined in accordance with a lexicography model. The lexicography model may correspond to a user usage history. The user usage history may occur prior to the establishment of the contact. The lexicography model may correspond to a frequency of usage of symbols in a language.
  • In some embodiments, the respective symbol is displayed in a region within the shape of the respective icon and outside of a region corresponding to the contact.
  • In some embodiments, a visual indicator corresponding to the respective symbol is provided. The visual indicator may include visual illumination proximate to the respective icon. The visual illumination may include a band around at least a portion of the respective icon. The visual indicator may be in accordance with a user usage history that occurs prior to the detecting of the contact.
  • In some embodiments, a method includes displaying a plurality of icons on a touch-sensitive display. Two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display. A space greater than a pre-determined value is included between adjacent rows. A contact by a user with the touch-sensitive display that corresponds to a respective icon is detected. A symbol corresponding to the respective icon is displayed in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.
  • The symbol may be the respective icon. The symbol may be magnified relative to the respective icon. The neighboring row may be above the respective row.
  • In another embodiment, a plurality of icons are displayed on a touch-sensitive display. A contact by a user with the touch-sensitive display that corresponds to the respective icon is determined. A symbol corresponding to the respective icon is displayed superimposed over one or more additional icons in the plurality of icons while the contact is maintained.
  • In another embodiment, a plurality of icons are displayed on a touch-sensitive display. Two or more subsets of the plurality of icons are arranged in corresponding rows. A contact by a user with the touch-sensitive display that corresponds to the respective icon is determined. The displayed plurality of icons are modified to include a space greater than a pre-determined value between a row corresponding to the respective icon and an adjacent row on the touch-sensitive display while the contact is maintained. A symbol corresponding to the respective icon is displayed in the space while the contact is maintained.
  • In some embodiments, a method includes displaying a plurality of icons on a touch-sensitive display. A respective icon in the plurality of icons corresponds to at least one symbol. One or more recommended words are displayed. The one or more recommended words are in accordance with a user history. The one or more recommended words are displayed prior to detecting any contacts by a user corresponding to symbol selection in a current application session. A contact by the user with the touch-sensitive display is detected. The contact includes a gesture. A respective recommended word corresponding to the gesture is selected.
  • The gesture may include a swipe motion. The swipe motion may include a horizontal component with displacement from left to right or from right to left along the touch-sensitive display. The swipe motion may include a vertical component with displacement downward or upward along the touch-sensitive display.
  • The gesture may include one or more taps. A respective tap may include making contact with the touch-sensitive display for a time interval less than a pre-determined value.
  • The gesture may include a rolling motion of the contact. The rolling motion may be from left to right or from right to left along the touch-sensitive display.
  • In some embodiments, a method includes displaying a plurality of icons on a touch-sensitive display. A respective icon in at least a subset of the plurality of icons corresponds to two or more symbols. A contact by a user with the touch-sensitive display that corresponds to a selection of the respective icon, wherein the contact includes a respective gesture, is detected. A respective symbol in the two or more symbols for the respective icon to which the contact further corresponds is determined. The respective symbol is a first symbol in the two or more symbols if the respective gesture includes a continuous contact and the respective symbol is a second symbol in the two or more symbols if the respective gesture includes a discontinuous contact.
  • The continuous contact may include a swipe motion. The swipe motion may include a horizontal component with displacement from left to right or from right to left along the touch-sensitive display. The swipe motion may include a vertical component with displacement downward or with displacement upward along the touch-sensitive display. The continuous contact may include a rolling motion of the contact. The rolling motion may be from left to right or from right to left along the touch-sensitive display.
  • The discontinuous contact may include one or more taps. A respective tap may include contact with the touch-sensitive display for a time interval less than a first pre-determined value. Two or more consecutive taps may correspond to the second symbol if a time interval between two or more corresponding contacts is less than a second pre-determined value.
  • The first symbol may be included in a first subset of symbols and the second symbol may be included in a second subset of symbols. The first subset of symbols may have a probability of occurrence that is greater than a first pre-determined value and the second subset of symbols may have a probability of occurrence that is less than the first pre-determined value.
  • The probability of occurrence may be in accordance with a user history. The probability of occurrence may be in accordance with a lexicography model. The lexicography model may include a frequency of usage of symbols in a language.
  • In some embodiments, the second symbol for the respective icon has a probability of occurrence immediately following the first symbol for the respective icon that is less than a second pre-determined value. In some embodiments, the first symbol for the respective icon has a probability of occurrence immediately following the second symbol for the respective icon that is less than a second pre-determined value.
  • In some embodiments, the displayed respective icon is modified to indicate that the contact corresponds to a respective symbol. In some embodiments, a visual indicator corresponding to a respective symbol is provided. The visual indicator may include visual illumination proximate to the respective icon. The visual illumination may include a band around at least a portion of the respective icon.
  • In some embodiments, a method includes displaying a plurality of icons on a touch-sensitive display. A respective icon in at least a subset of the plurality of icons corresponds to two or more symbols. A first symbol in the two or more symbols belongs to a first subset of symbols and a second symbol in the two or more symbols belongs to a second subset of symbols. The first symbol has a probability of occurrence that is greater than a first pre-determined value and the second symbol has a probability of occurrence that is less than the first pre-determined value. A contact by a user with the touch-sensitive display that corresponds to a selection of the respective icon is detected. The contact includes a respective gesture. A respective symbol in the two or more symbols for the respective icon to which the contact further corresponds is determined.
  • The probability of occurrence may be in accordance with a user history. The probability of occurrence may be in accordance with lexicography model. The lexicography model may include a frequency of usage of symbols in a language. The second symbol may have a probability of occurrence immediately following the first symbol that is less than a second pre-determined value.
  • The first symbol may be selected using one or more tap gestures and the second symbol may be selected using a swipe gesture. A respective tap may include making contact with the touch-sensitive display for a time interval less than a second pre-determined value. Two or more consecutive taps may correspond to the second symbol if a time interval between two or more corresponding contacts is less than a third pre-determined value.
  • In some embodiments, the displayed respective icon is modified to indicate that the contact corresponds to the respective symbol. In some embodiments, a visual indicator corresponding to the respective symbol is provided. The visual indicator may include visual illumination proximate to the respective icon. The visual illumination may include a band around at least a portion of the respective icon.
  • In some embodiments, the first subset of symbols includes e, t, a, o, i, n, s, r and h. In some embodiments, the first subset of symbols includes q, e, u, I, o, a, d, g, j, l, z, c, b, n and m. In some embodiments, the first subset of symbols includes q, c, e, h, I, l, n, o, r, t, u, w and y.
  • In some embodiments, the second subset of symbols includes w, y and j. In some embodiments, the second subset of symbols includes w, y, p, g and j. In some embodiments, the second subset of symbols includes w, r, t, y, p, s, f, h, k, x and v. In some embodiments, the second subset of symbols includes j, v, x and z. In some embodiments, the second subset of symbols includes b, d, f, g, j, k, m, p, q, s, v, x and z.
  • The aforementioned methods may be performed by a portable electronic device having a touch-sensitive display with a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing these methods. In some embodiments, the portable electronic device provides a plurality of functions, including wireless communication.
  • Instructions for performing the aforementioned methods may be included in a computer program product configured for execution by one or more processors.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • For a better understanding of the aforementioned embodiments of the invention as well as additional embodiments thereof, reference should be made to the Description of Embodiments below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.
  • FIG. 1 is a block diagram illustrating an embodiment of an architecture for a portable electronic device.
  • FIG. 2 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 3C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 4 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 5 is a block diagram illustrating an embodiment of a character set data structure.
  • FIG. 6A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 6D is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 7 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 8 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 9 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 10A is a block diagram illustrating an embodiment of a user word history data structure.
  • FIG. 10B is a block diagram illustrating an embodiment of a language data structure system.
  • FIG. 11A is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 11B is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 11C is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 12A is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12B is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12C is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12D is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12E is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12F is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 12G is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 13 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 14 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 15 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 16 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 17 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • FIG. 18 is a flow diagram of an embodiment of a symbol entry process.
  • FIG. 19 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device.
  • DESCRIPTION OF EMBODIMENTS
  • Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
  • Embodiments of user interfaces and associated processes for using a device are described. In some embodiments, the device may be a portable communications device. The user interface may include a click wheel and/or touch screen. A click wheel is a physical user-interface device that may provide navigation commands based on an angular displacement of the wheel or a point of contact with the wheel by a user of the device. A click wheel may also be used to provide a user command corresponding to selection of one or more items, for example, when the user of the device presses down on at least a portion of the wheel. For simplicity, in the discussion that follows, a portable communications device (e.g., a cellular telephone that may also contain other functions, such as SMS, PDA and/or music player functions) that includes a touch screen is used as an exemplary embodiment. It should be understood, however, that the user interfaces and associated processes may be applied to other devices, such as personal computers and laptops, that may include one or more other physical user-interface devices, such as a click wheel, a keyboard, a mouse and/or a joystick.
  • The device may support a variety of applications, such as a telephone, text messaging, word processing, email and a music player. The music player may be compatible with one or more file formats, such as MP3 and/or AAC. In an exemplary embodiment, the device includes an iPod music player (trademark of Apple Computer, Inc.).
  • The various applications that may be executed on the device may use at least one common physical user-interface device, such as the touch screen. In embodiments that include a click wheel, one or more functions of the click wheel as well as corresponding information displayed on the device may be adjusted and/or varied from one application to the next and/or within a respective application. In this way, a common physical architecture (such as the click wheel) of the device may support the variety of applications with user interfaces that are intuitive and transparent.
  • The user interfaces may include one or more keyboard embodiments. The keyboard embodiments may include standard (qwerty) and/or non-standard configurations of symbols on the displayed icons of the keyboard. The keyboard embodiments may include a reduced number of icons (or soft keys) relative to the number of keys in existing physical keyboards, such as that for a typewriter. This may make it easier for users to select one or more icons in the keyboard, and thus, one or more corresponding symbols. The keyboard embodiments may be adaptive. For example, displayed icons may be modified in accordance with user actions, such as selecting one or more icons and/or one or more corresponding symbols. One or more applications on the portable device may utilize common and/or different keyboard embodiments. Thus, the keyboard embodiment used may be tailored to at least some of the applications. In some embodiments, one or more keyboard embodiments may be tailored to a respective user. For example, based on a word usage history (lexicography, slang, individual usage) of the respective user. Some of the keyboard embodiments may be adjusted to reduce a probability of a user error when selecting one or more icons, and thus one or more symbols, when using the keyboard embodiments.
  • Attention is now directed towards embodiments of the device. FIG. 1 is a block diagram illustrating an architecture for a portable electronic device 100, according to some embodiments of the invention. The device 100 may include a memory 102 (which may include one or more computer readable storage mediums), a memory controller 122, one or more processing units (CPU's) 120, a peripherals interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input/output (I/O) subsystem 106, a display system 112 (which may include a touch screen), a click wheel 114, other input or control devices 116, and an external port 124. These components may communicate over the one or more communication buses or signal lines 103. The device 100 may be any portable electronic device, including but not limited to a handheld computer, a tablet computer, a mobile phone, a media player, a personal digital assistant (PDA), or the like, including a combination of two or more of these items. In other embodiments, the device 100 may not be portable, such as a personal computer.
  • It should be appreciated that the device 100 is only one example of a portable electronic device 100, and that the device 100 may have more or fewer components than shown, may combine two or more components, or a may have a different configuration or arrangement of the components. The various components shown in FIG. 1 may be implemented in hardware, software or a combination of both hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • The memory 102 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices. In some embodiments, the memory 102 may further include storage remotely located from the one or more processors 120, for instance network attached storage accessed via the RF circuitry 108 or the external port 124 and a communications network (not shown) such as the Internet, intranet(s), Local Area Networks (LANs), Wide Local Area Networks (WLANs), Storage Area Networks (SANs) and the like, or any suitable combination thereof. Access to the memory 102 by other components of the device 100, such as the CPU 120 and the peripherals interface 118, may be controlled by the memory controller 122.
  • The peripherals interface 118 couples the input and output peripherals of the device to the CPU 120 and the memory 102. The one or more processors 120 run or execute various software programs and/or sets of instructions stored in the memory 102 to perform various functions for the device 100 and to process data.
  • In some embodiments, the peripherals interface 118, the CPU 120, and the memory controller 122 may be implemented on a single chip, such as a chip 104. In some other embodiments, they may be implemented on separate chips.
  • The RF (radio frequency) circuitry 108 receives and sends electromagnetic waves. The RF circuitry 108 converts electrical signals to/from electromagnetic waves and communicates with communications networks and other communications devices via the electromagnetic waves. The RF circuitry 108 may include well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth. The RF circuitry 108 may communicate with the networks, such as the Internet, also referred to as the World Wide Web (WWW), an Intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication. The wireless communication may use any of a plurality of communications standards, protocols and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for email, instant messaging, and/or Short Message Service (SMS)), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.
  • The audio circuitry 110, the speaker 111, and the microphone 113 provide an audio interface between a user and the device 100. The audio circuitry 110 receives audio data from the peripherals interface 118, converts the audio data to an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal to human-audible sound waves. The audio circuitry 110 also receives electrical signals converted by the microphone 113 from sound waves. The audio circuitry 110 converts the electrical signal to audio data and transmits the audio data to the peripherals interface 118 for processing. Audio data may be may be retrieved from and/or transmitted to the memory 102 and/or the RF circuitry 108 by the peripherals interface 118. In some embodiments, the audio circuitry 110 also includes a headset jack (not shown). The headset jack provides an interface between the audio circuitry 110 and removable audio input/output peripherals, such as output-only headphones or a headset with both output (headphone for one or both ears) and input (microphone).
  • The I/O subsystem 106 provides the interface between input/output peripherals on the device 100, such as the display system 112, the click wheel 114 and other input/control devices 116, and the peripherals interface 118. The I/O subsystem 106 may include a display controller 156, a click wheel controller 158 and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive/send electrical signals from/to other input or control devices 160. The other input/control devices 160 may include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, sticks, and so forth.
  • The display system 112 provides an output interface and/or an input interface between the device and a user. The display controller 156 receives and/or sends electrical signals from/to the display system 112. The display system 112 displays visual output to the user. The visual output may include text, icons, graphics, video, and any combination thereof. In some embodiments, some or all of the visual output may correspond to user-interface objects, further details of which are described below.
  • In some embodiments, such as those that include a touch screen, the display system 112 also accepts input from the user based on haptic and/or tactile contact. In embodiments with a touch screen, the display system 112 forms a touch-sensitive surface that accepts user input. In these embodiments, the display system 112 and the display controller 156 (along with any associated modules and/or sets of instructions in the memory 102) detect contact (and any movement or breaking of the contact) on the display system 112 and converts the detected contact into interaction with user-interface objects, such as one or more soft keys, that are displayed on a touch screen. In an exemplary embodiment, a point of contact between a touch screen in the display system 112 and the user corresponds to one or more digits of the user.
  • In embodiments with a touch screen, the touch screen in the display system 112 may use LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology, although other display technologies may be used in other embodiments. A touch screen in the display system 112 and the display controller 156 may detect contact and any movement or breaking thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with a touch screen in the display system 112. A touch-sensitive display in some embodiments of the display system 112 may be analogous to the multi-touch sensitive tablets described in the following U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat. No. 6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932 (Westerman), and/or U.S. Patent Publication 2002/0015024A1, each of which is hereby incorporated by reference. However, a touch screen in the display system 112 displays visual output from the portable device 100, whereas touch sensitive tablets do not provide visual output. The touch screen in the display system 112 may have a resolution in excess of 100 dpi. In an exemplary embodiment, the touch screen in the display system may have a resolution of approximately 168 dpi. The user may make contact with the touch screen in the display system 112 using any suitable object or appendage, such as a stylus, finger, and so forth.
  • In some embodiments, in addition to touch screen, the device 100 may include a touchpad (not shown) for activating or deactivating particular functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad may be a touch-sensitive surface that is separate from the touch screen in the display system 112 or an extension of the touch-sensitive surface formed by the touch screen.
  • The device 100 may include a click wheel 114. A user may navigate among one or more graphical objects (henceforth referred to as icons) displayed in the display system 112 by rotating the click wheel 114 or by moving (e.g., angular displacement) of a point of contact with the click wheel 114. The click wheel 114 may also be used to select one or more of the displayed icons. For example, the user may press down on at least a portion of the click wheel 114 or an associated physical button. User commands and navigation commands provided by the user via the click wheel 114 may be processed by the click wheel controller 158 as well as one or more of the modules and/or sets of instructions in the memory 102.
  • The device 100 also includes a power system 162 for powering the various components. The power system 162 may include a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.
  • In some embodiments, the software components stored in the memory 102 may include an operating system 126, a communication module (or set of instructions) 128, a contact/motion module (or set of instructions) 130, a graphics module (or set of instructions) 132, one or more applications (or set of instructions) 136, a timer module (or set of instructions) 144, a word prediction module (or set of instructions) 146, an address book 148, a user word history 150, one or more character sets 152, and one or more lexicography models 154. The graphics module 132 may include an icon effects module (or set of instructions) 134. The applications module 136 may include a telephone module (or set of instructions) 138, a text messaging module (or set of instructions) 140 and/or a music player module (or set of instructions) 142.
  • The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
  • The communication module 128 facilitates communication with other devices over one or more external ports 124 and also includes various software components for handling data received by the RF circuitry 108 and/or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.).
  • The contact/motion module 130 may detect contact with the click wheel 114 and/or a touch screen in the display system 112 (in conjunction with the display controller 156). The contact/motion module 130 includes various software components for performing various operations related to detection of contact, such as determining if contact has occurred, determining if there is movement of the contact and tracking the movement across the click wheel 114 and/or a touch screen in the display system 112, and determining if the contact has been broken (i.e., if the contact has ceased). Determining movement of the point of contact may include determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (including magnitude and/or direction) of the point of contact. In some embodiments, the contact/motion module 130 and the display controller 156 also detect contact on a touchpad.
  • The graphics module 132 includes various known software components for rendering and displaying graphics on the display system 112. Note that the term “graphics” includes any object that can be displayed to a user, including without limitation text, web pages, icons (such as user-interface objects including soft keys), digital images, videos, animations and the like.
  • In some embodiments, the graphics module 132 includes the icon effects module 134. The icon effects module 134 may modify a displayed position of one or more icons on the display system 112 (in conjunction with the display controller 156) based on user actions (such as detecting a contact corresponding to at least one icon). In some embodiments, the modification of the displayed icon(s) may be based on an animation sequence.
  • In addition to the telephone module 138, the text messaging module 140 and/or the music player module 142, the one or more applications 136 may include any applications installed on the device 100, including without limitation, a browser, the address book 148, contact list, email, instant messaging, word processing, keyboard emulation, widgets, JAVA-enabled applications, encryption, digital rights management, voice recognition, voice replication, location determination capability (such as that provided by the global positioning system (GPS)), etc.
  • In conjunction with the RF circuitry 108, the audio circuitry 110, the speaker 111, the microphone 113, the display system 112, the display controller 156, the click wheel 114 and/or the click wheel controller 158, the telephone module 138 may be used to enter a sequence of characters corresponding to a telephone number, access one or more telephone numbers in the address book 148, modify a telephone number that has been entered, dial a respective telephone number, conduct a conversation and disconnect or hang up when the conversation is completed.
  • In conjunction with the display system 112, the display controller 156, the click wheel 114 and/or the click wheel controller 158, the text messaging module 140 may be used to enter a sequence of characters corresponding to a text message, to modify previously entered characters, to transmit a respective text message (for example, using a Short Message Service or SMS protocol), to receive text messages and to view received text messages. In some embodiments, transmitted and/or received text messages may include graphics, photos, audio files, video files and/or other attachments as are supported in a Multimedia Message Service (MMS) and/or an Enhanced Messaging Service (EMS). Embodiments of user interfaces and associated processes corresponding to the symbol entry, such as with the text messaging module 140, and more generally, to text entry and communication are described further below with reference to FIGS. 2-4, 6-9 and 11-20.
  • In conjunction with the display system 112, the display system controller 156, the click wheel 114, the click wheel controller 158, the audio circuitry 110, the speaker 111 and/or the microphone 113, the music player module 142 allows the user to play back recorded music stored in one or more files, such as MP3 or AAC files. In some embodiments, the device 100 may include the functionality of an MP3 player, such as an iPod (trademark of Apple Computer, Inc.). The device 100 may, therefore, include a 36-pin connector that is compatible with the iPod.
  • The timer module 144 may provide a time reference and/or time stamps for user commands received by the device 100, for example, using the click wheel 114 and the click wheel controller 158.
  • The word prediction module 146 may be used in conjunction with one or more of the applications 136, such as the text messaging module 140. The word prediction module 146 may suggest one or more words or symbols (such as punctuation marks, pronunciation marks or spaces) in accordance with a context. The context may be based on one or more of the lexicography models 154 (for example, grammatical and/or syntax rules associated with one or more languages) and/or a user word history 150. The context may include one or more previously entered words, characters, and/or symbols. The context may depend on which of the applications 136 is being used. For example, there may be different contexts for an email application as opposed to a word processing application. A user interface and associated process that include recommended words from the word prediction module 146 are discussed further below with reference to FIGS. 8 and 9.
  • The user word history 150 may include static content (such as that associated with a dictionary) and/or dynamic content (such as that associated with characters, symbols and/or words that are routinely and/or recently used by the user). The user word history 150 may include a static dictionary built up by scanning a user's address book, emails, and other documents. The user word history 150 may include weighted scores or probabilities for predicted words based on a set of characters, symbols and/or words that are provided by the user to the device 100, for example, using the display system 112, the click wheel 114 and the click wheel controller 158. The user word history 150 may also include use statistics (e.g., time of use and/or frequency of use) of one or more characters, symbols and/or words that are provided by the user. The user word history 150 is discussed further below with reference to FIGS. 10A and 10B.
  • The character sets 152 may include one or more sets of characters corresponding to numbers, letters and/or symbols. The letters and/or symbols may correspond to one or more languages. The character sets 152 may be used by one or more of the applications 136, such as the text messaging module 140. A data structure associated with the one or more character sets (which may be used in one or more of the keyboard embodiments) is discussed further below with reference to FIG. 5.
  • In some embodiments, the device 100 may include one or more optional optical sensors (not shown), such as CMOS or CCD image sensors, for use in imaging applications.
  • In some embodiments, the device 100 is a device where operation of a predefined set of functions on the device is performed exclusively through a touch screen in the display system 112 and/or a touchpad. By using a touch screen and/or a touchpad as the primary input/control device for operation of the device 100, the number of physical input/control devices (such as push buttons, dials, and the like) on the device 100 may be reduced. In one embodiment, the device 100 includes a touch screen, a touchpad, a push button for powering the device on/off and locking the device, a volume adjustment rocker button and a slider switch for toggling ringer profiles. The push button may be used to turn the power on/off on the device by depressing the button and holding the button in the depressed state for a predefined time interval, or may be used to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed. In an alternative embodiment, the device 100 also may accept verbal input for activation or deactivation of some functions through the microphone 113.
  • The predefined set of functions that are performed exclusively through a touch screen and/or a touchpad include navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 to a main, home, or root menu from any user interface that may be displayed on the device 100. In such embodiments, the touchpad may be referred to as a “menu button.” In some other embodiments, the menu button may be a physical push button or other physical input/control device instead of a touchpad.
  • In some embodiments, the device 100 is a device where operation of a predefined set of functions on the device is performed exclusively or primarily through the click wheel 114. By using the click wheel 114 as the primary input/control device for operation of the device 100, the number of other physical input/control devices (such as push buttons, dials, and the like) on the device 100 may be reduced.
  • Attention is now directed towards embodiments of user interfaces and associated processes that may be implemented on the device 100. FIG. 2 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 200. The device 200 includes a touch screen 208. The touch screen may display one or more trays. A tray is a region within a graphical user interface. One tray may include a user entry interface, such as a keyboard 210 that includes a plurality of icons. The icons may include one or more symbols. In this embodiment, as well as others described below, a user may select one or more of the icons, and thus, one or more of the corresponding symbols, by making contact or touching the keyboard 210, for example, with one or more fingers 212 (not drawn to scale in the figure). The contact may correspond to the one or more icons. In some embodiments, selection of one or more icons occurs when the user breaks contact with the one or more icons. In some embodiments, the contact may include a gesture, such as one or more taps, one or more swipes (from left to right, right to left, upward and/or downward) and/or a rolling of a finger (from right to left, left to right, upward and/or downward) that has made contact with the device 200. In some embodiments, in advertent contact with an icon may not select a corresponding symbol. For example, a swipe gesture with an icon may not select a corresponding symbol when the gesture corresponding to selection is a tap.
  • The device 200 may include a display tray 214. The display tray 214 may display one or more of the characters and/or symbols that are selected by the user. The device 200 may also include one or more physical buttons, such as the clear, hold and menu buttons shown in FIG. 2. As described previously, the menu button may be used to navigate within a hierarchy of applications that may be executed on the device 200. Alternatively, in some embodiments, the clear, hold, and/or menu buttons are implemented as soft keys in a GUI in touch screen 208.
  • FIGS. 3A-3C are schematic diagrams illustrating an embodiment of a user interface for a portable electronic device 300. The user interface includes a keyboard 310 that includes a plurality of icons. The icons include three symbols each. In other embodiments, the icons include two symbols each. In other embodiments, different icons on the same keyboard may include one, two, or three symbols each (e.g., some icons may contain one symbol while other icons contain two or three symbols). The symbols on the icons are in a non-standard configuration, i.e., non-qwerty. In addition, the total number of icons in the keyboard 310 is less than the number of physical keys in a standard keyboard.
  • The symbols in the icons in the keyboard 310 may be determined using a lexicography model, such as a language. The lexicography model may include a frequency of use of symbols in a language. For example, characters or symbols that are unlikely to occur immediately proximate to one another or immediately after one another in a set of symbols that the user may enter may be grouped on a respective icon 312 (FIG. 3B). A language may include slang as well as individual usage (for example, words that are commonly used by the user). The lexicography model may correspond to a user usage or word history that occurs prior to the user making contact with the device 300, i.e., a past usage.
  • As shown in FIG. 3B, when a user makes contact 314 with the touch screen 208 in the device 300 corresponding to the respective icon 312 and a respective symbol (in this case a letter ‘a’), the shape of the respective icon 312 is modified. This provides information to the user as to which icon and which symbol the contact 314 currently corresponds. This may be useful since the contact 314 may obscure at least a portion of the respective icon 312 making it difficult for the user to see the respective symbol he or she is currently positioned on.
  • In an exemplary embodiment, the icons in the keyboard 310 may at least in part include an arc. In response to the contact 314, the shape of the respective icon 312 may be asymmetrically distorted and the respective symbol that the contact 314 currently corresponds to may be displayed within the shape of the respective icon 312 and outside of the contact 314.
  • In some embodiments, the user may select the respective symbol by making the contact 314 with the respective icon 312 and rolling a finger over a region within the respective icon 312 that corresponds to the respective symbol. If the user determines, based on the modified shape of the respective icon 312 and/or the displayed symbol within the modified shape that the wrong symbol is currently contacted, the user may roll their finger to a different position within the respective icon 312 that corresponds to the correct symbol. Once the contact 314 has been positioned over or proximate to the correct symbol, the user may select this symbol by breaking the contact 314 with the respective icon 312. The selected symbol (such as the letter ‘a’) may then be displayed in the display tray 214. In some embodiments, if the contact 314 is maintained by the user for a time interval that is more than a first pre-determined value, such as 0.5, 1 or 2 s, before the contact 314 is broken, the respective symbol may be capitalized.
  • If an error has been made, the user may clear the entire display tray 214 using a clear icon or may delete a most recently selected symbol using a delete icon. Once a set of symbols (such as a message) has been entered, the user may accept the set of symbols (which may store and/or send the set of symbols depending on the application executing on the device 300) using an accept icon.
  • As shown in FIG. 3C, in some embodiments an additional visual indicator corresponding to the respective icon 312 may be provided on the display 208. The visual indicator may be proximate to the respective icon 312. The visual indicator may include a band 318 around at least a portion of the respective icon 312.
  • As is also shown in FIG. 3C, in some embodiments a shape of the respective icon 312 may not be modified in response to the contact 314. Instead, an icon 316 corresponding to the respective symbol 316 may be displayed proximate to the respective icon 312.
  • The modifying of the shape of the respective icon 312 and/or the displaying of the visual indicator, such as the band 318 and/or the icon 316, may be included in at least some of the embodiments discussed further below.
  • While the device 300 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed. For example, the keyboard 310 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboard 310.
  • FIG. 4 is a flow diagram of an embodiment of a symbol entry process 400. While the symbol entry process 400 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 400 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (410). A respective icon may correspond to two or more symbols. Contact by a user with the display that corresponds to the respective icon may be detected (412). The displayed respective icon may be modified to indicate that the contact corresponds to a respective symbol in the two or more symbols (414). The respective symbol may be optionally displayed in a region within the shape of the respective icon and outside of a region corresponding to the contact (416). A visual indicator corresponding to the respective symbol may be optionally provided (418). The respective symbol may be optionally capitalized when contact is maintained for a time interval exceeding a pre-determined value (420). The respective symbol may be selected when the user breaks contact with the respective icon (422).
  • Attention is now directed towards embodiments of a character set data structure that may be used in implementing the user interface in the device 300 (FIG. 3) and/or user interfaces described further below. FIG. 5 is a block diagram illustrating an embodiment of a character set data structure 500. The character sets 152 may include multiple sets 512 of characters and/or symbols. A respective set, such as the set 512-1, may include one or more symbols 514 and one or more probabilities 516. The probabilities may include frequencies of occurrence of use, as well as conditional probabilities (such as the probability of a given symbol occurring given one or more symbols that have already occurred). In some embodiments the character set data structure 500 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • Attention is now directed towards additional embodiments of user interfaces and associated processes that may be implemented on the device 100 (FIG. 1). FIGS. 6A-6D are schematic diagrams illustrating an embodiment of a user interface for a portable electronic device 600. The device 600 includes a keyboard 610 that has a plurality of icons arranged in rows. A given row includes a subset of the plurality of icons. Adjacent rows are separated by a space greater than a second pre-determined value, such as a height of one of the icons.
  • As shown in FIG. 6B, when the user makes a contact 612 with the display 208 corresponding to a respective icon in the keyboard 610, an icon 614 may be displayed in the space between two adjacent rows. The icon may correspond to a respective symbol that corresponds to the respective icon that the user has contacted 612. For example, if the user contacts or is proximate to an icon for the character ‘u’ in the keyboard 610, the icon 614 may correspond to the character ‘u’. In this way, the user may receive feedback that the respective icon (and thus, the respective symbol) is currently contacted. This may be useful because the contact 612 may obscure the respective icon, and thus, the respective symbol, that has been selected in the rows of icons.
  • In some embodiments, the icon 614 may be displayed above a respective row in which the contact 612 has occurred. In some embodiments, the icon 614 may be magnified, i.e., larger, than the respective icon.
  • The icon 614 may be displayed while the contact 612 is maintained. When the user breaks the contact 612 with the respective icon, the respective symbol may be selected. In some embodiments, the respective symbol may be displayed in the display tray 214.
  • As shown in FIG. 6C, in some embodiments a keyboard 616 may be displayed with rows of icons. Initially, the rows of icons may not include a significant space between adjacent rows, e.g., the space may be less than the second pre-determined value. When the user makes the contact 612 with the display 208, however, the displayed keyboard 616 may be modified to include a space greater the second pre-determined value and the icon 614 may be displayed. This modified configuration or layout of the keyboard 616 may be maintained while the contact 612 is maintained by the user.
  • As shown in FIG. 6D, in some embodiments a keyboard 618 may include rows of icons. When the contact 612 is made, an icon 620 may be displayed superimposed over at least one or more additional icons in the keyboard 618.
  • While the device 600 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed. For example, the keyboards 610, 616 and/or 618 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboards 610, 616 and/or 618.
  • FIG. 7 is a flow diagram of an embodiment of a symbol entry process 700. While the symbol entry process 700 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 700 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (710). Two or more subsets of the plurality of icons may be arranged in rows. A contact by a user with the display that corresponds to a respective icon may be detected (712). A symbol corresponding to the respective icon may be optionally displayed between a row corresponding to the respective icon and a neighboring row (714). A symbol corresponding to the respective icon may be optionally displayed superimposed over one or more additional icons in the plurality of icons (716).
  • FIG. 8 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 800. The device 800 may include a tray 812 that includes one or more recommended words 810. The one or more recommended words 810 may be determined using a user word history. This is discussed further below with reference to FIGS. 10A and 10B.
  • In some embodiments, the one or more recommended words 810 are displayed prior to detecting any contacts corresponding to text input (symbol selection) by the user in a current application session. For example, the one or more recommended words 810 may be displayed when the user initially opens an application, such as email, on the device 800. The one or more recommended words 810, therefore, may be determined based on a user word or usage history that may be application specific. After the device 800 receives contacts corresponding to text input, the one or more recommended words 810 may change dynamically in response to contacts corresponding to text input by the user during the application session.
  • The user may select one or more of the recommended words 810 by making contact with the display 208. In some embodiments, one or more of the recommended words 810, such as a phrase (“How are you?”), may be selected with a single contact. The contact may include a gesture, such as one or more taps, one or more swipes, and/or a rolling motion of a finger that makes the contact. The one or more taps may have a duration that is less than a third pre-determined value, such as 0.1, 0.5 or 1 s.
  • While the device 800 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed. For example, the keyboard 210 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboard 210.
  • FIG. 9 is a flow diagram of an embodiment of a symbol entry process 900. While the symbol entry process 900 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 900 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (910). A respective icon may correspond to at least one symbol. One or more recommended words may be displayed (912). The one or more recommended words may be in accordance with a user history prior to detecting any contacts corresponding to text input (symbol selection) by the user in a current application session. A contact by the user with the display may be detected (914). The contact may include a gesture. A respective recommended word that corresponds to the gesture may be selected (916).
  • Attention is now directed towards embodiments of data structure systems that may be implementing in the device 100 (FIG. 1). FIG. 10A is a block diagram illustrating an embodiment of a user word history data structure 1000. The user word history 150 may include a deleted word stack 1010 and multiple words 1016. The words 1016 may include one or more characters and/or one or more symbols. The deleted word stack 1010 includes one or more words 1014 in a sequential order in which the one or more words 1014 were deleted by the user in an application, such as the text messaging module 140 (FIG. 1).
  • A respective word in the words 1016, such as word 1016-M, may include multiple records. A respective record may include a time-weighted score 1018, use statistics 1020 (such as a time of use and/or a frequency of use), a context 1022 and one or more applications 1024. The time-weighted score 1018 may indicate a probability that the word 1016-M is a next predicted word based on the context 1022 (one or more characters, symbols and/or words that have previously been provided by the user) and/or the application 1024. For example, the time-weighted score 1018 may therefore be different for email than for the text messaging module 140 (FIG. 1). The time-weighted score 1018 may be computed to favorably weight (e.g., give a higher probability) to words that are used recently. For example, the time-weighted score 1018 may give favorable weighting to words 1016 that are used within the last 24 hours or week. Words 1016 used on longer time scales (e.g., more than a day or a week ago) may have their corresponding time-weighted scores 1018 reduced by a pre-determined ratio (such as 0.9) for each additional time interval (e.g., each day or week) since the words 1016 were last used.
  • The user history data structure 1000 may include static information (for example, corresponding to a dictionary and/or grammatical and syntax rules for one or more languages) as well as dynamic information (based on recent usage statistics and/or patterns). Thus, the user history data structure 1000 may be dynamically updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user. The user history data structure 1000 may include a static dictionary built up by scanning a user's address book, emails, and other documents. In some embodiments the user history data structure 1000 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • FIG. 10B is a block diagram illustrating an embodiment of a language data structure system 1050. The language data structure system 1050 may be used to provide recommended words in the device 800 (FIG. 8). A sequence of symbols 1062 (including one or more characters, symbols and/or words) may be provided by the user. A set of symbols 1062 corresponding to a context 1022-1 may be processed by a context map 1060. In some embodiments, the context 1022-1 may be a null set, i.e., one or more recommended words are provided before the user provides any symbols 1062 (e.g., when an application is first opened). In other embodiments, the context 1022-1 may include one or more previously entered or provided words as well as one or more symbols, such as the first one, two or three letters in a current word that the user is providing. The context map 1060 may include a select and hashing module 1064 and a hash map 1066. The hash map 1066 may select one or more appropriate entries in an application-specific dictionary 1068. The entries in the application-specific dictionary 1068 may include contexts 1070, predicted words 1072, and time-weighted scores 1074. The application-specific dictionary 1068 may utilize the records in the user history data structure 1000. As a consequence, the application-specific dictionary 1068 may be dynamically updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user.
  • The language data structure system 1050 may be used to provide one or more recommended words based on the context 1022-1. The context map may find a top-5 or top-10 best context 1070 matches. The corresponding predicted words 1072 may be recommended to the user in accordance with the time-weighted scores 1074. In some embodiments, only a subset of the predicted words 1072 corresponding to the best context 1070 matches may be presented to the user (e.g., just the top-1, top-2, or top-3 predicted words).
  • In some embodiments, the language data structure system 1050 may provide one or more recommended words in accordance with a state machine (corresponding to a Markov sequence or process) that corresponds to a language. For example, the application-specific dictionary 1068 may be based on a stochastic model of the relationships among letters, characters, symbols and/or words in a language.
  • A path memory (such as up to three characters in a word that is currently being entered and/or two or three previously entered words) of the probabilistic model represents a tradeoff between accuracy and the processing and power capabilities (for example, battery life) of the portable electronic device 100 (FIG. 1). In some embodiments, such a probabilistic model may be based on a lexicography and usage that is user-specific and/or, as discussed previously, even application specific. For example, user emails, address book and/or other documents may be analyzed to determine an appropriate probabilistic model for that user based on the syntax and/or lexicography (including names and slang) that are employed by the user. The probabilistic model may be updated continuously, after pre-determined time intervals, or when a new word or syntax is employed by the user.
  • In some embodiments, the probabilistic model may be based on one or more mistakes made by the user when using the click wheel 114 (FIG. 1) and/or a touch-sensitive display in the display system 112 (FIG. 1). For example, if the user accidentally selects the wrong icon when typing a respective word, the probabilistic model may be updated to account for such errors in the future. In an exemplary embodiment, a mistake may be determined based on a user activation of an icon corresponding to the delete function. This adaptability of the portable electronic device 100 (FIG. 1) may allow correction of user interface errors (such as parallax and/or left-right symmetry) associated with which finger(s) the user is using and how the user is holding the portable electronic device 100 (FIG. 1) while using it. This functionality is discussed further below with reference to FIG. 14.
  • In some embodiments the language data structure system 1050 may include fewer or more components. Two or more components may be combined and an order of two or more components may be changed.
  • Attention is now directed towards additional embodiments of user interfaces and associated processes that may be implemented on the device 100 (FIG. 1). FIG. 11A is a flow diagram of an embodiment of a symbol entry process 1100. While the symbol entry process 1100 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1100 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1110). A respective icon may correspond to two or more symbols. A contact by a user with the display that corresponds to selection of the respective icon may be detected (1112). A symbol in the two or more symbols for which the contact further corresponds may be determined (1114).
  • FIG. 11B is a flow diagram of an embodiment of a symbol entry process 1130. While the symbol entry process 1130 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1130 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1132). A respective icon may correspond to two or more symbols. A first symbol may belong to a first subset of symbols and a second symbol may belong to a second subset of symbols. The first symbol may have a probability of occurrence greater than the second symbol. A contact by a user with the display that corresponds to selection of the respective icon may be detected (1134). A symbol in the two or more symbols for which the contact further corresponds may be determined (1136).
  • FIG. 11C is a flow diagram of an embodiment of a symbol entry process 1150. While the symbol entry process 1150 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1150 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1152). A respective icon may correspond to two or more symbols. A first symbol may belong to a first subset of symbols and a second symbol may belong to a second subset of symbols. The second symbol may have a probability of occurrence immediately following the first symbol that is less than a pre-determined value. A contact by a user with the display that corresponds to selection of the respective icon may be detected (1154). A symbol in the two or more symbols for which the contact further corresponds may be determined (1156).
  • FIGS. 12A-12G are schematic diagrams illustrating embodiments of a user interface for a portable electronic device 1200. These embodiments may utilize the symbol entry processes 1100 (FIG. 11A), 1130 (FIG. 11B) and/or 1150 (FIG. 11C) described previously. As shown in FIG. 12A, the device 1200 may include a keyboard 1210 with a plurality of icons. A respective icon may include two or more symbols. A first symbol for a respective icon may be selected by the user using a first gesture. A second symbol for a respective icon may be selected by the user using a second gesture. The first gesture may include a continuous contact with the display 208 and the second gesture may include a discontinuous contact with the display 208.
  • The continuous contact may include a swipe and/or a rolling motion of the contact. The discontinuous contact may include one or more consecutive taps. A respective tap may include contact with the display 208 for a time interval that is less than a fourth pre-determined value, such as 0.1, 0.5 or 1 s. In some embodiments, two or more consecutive taps may correspond to a second symbol if a time interval between the two or more consecutive taps is less than a fifth pre-determined value, such as 0.1, 0.5 or 1 s.
  • In some embodiments, the first symbol is in a first subset of the symbols in the character set displayed in the keyboard 1210 and the second symbol is in a second subset of the symbols in the character set displayed in the keyboard 1210. The first subset may have a probability of occurrence that is greater than a sixth pre-determined value and the second subset may have a probability of occurrence that is less than the sixth pre-determined value. Thus, the first subset may include symbols that are more likely to occur, for example, in a language (using a lexicography model) and/or based on a user history. The gesture used to select the first symbol may, therefore, be easier or quicker for the user to make. For example, the first gesture may be a tap gesture and the second gesture may be a swipe gesture. This is illustrated in FIG. 12A. The gestures needed to select corresponding symbols for a respective icon may be indicated on the icon. For example, a dot on the icon may correspond to a tap and a horizontal line on the icon may correspond to a dash. This ‘tap-dash’ embodiment is an example of a two-gesture keyboard. Additional examples are discussed below.
  • In some embodiments, the first symbol may have a probability of occurrence immediately after the second symbol that is less than a seventh pre-determined value. In some embodiments, the second symbol may have a probability of occurrence immediately after the first symbol that is less than a seventh pre-determined value. This arrangement or grouping of the symbols displayed on the icons may reduce errors when using the keyboard 1210 because the user will be less likely to make a first gesture for the first symbol corresponding to a respective icon and then make the second gesture for the second symbol corresponding to the respective icon (or vice versa). Gestures for different symbols on the respective icon may, therefore, be separated by a time interval that is large enough to reduce a likelihood of inadvertently selecting a respective symbol using consecutive gestures for symbols corresponding to the respective icon.
  • FIGS. 12B-12G illustrate additional multi-gesture keyboards. For the icons in keyboards 1212, 1214, 1216, 1218, 1220 and 1222, a first symbol for a respective icon in these keyboards may be selected with a first gesture (for example, a single tap) and a second symbol for the respective icon may be selected using a second gesture (for example, two consecutive taps). The keyboard 1222 in FIG. 12G includes some icons that correspond to more than two symbols. These symbols may be selected by making additional gestures, such as three consecutive taps. In some embodiments, a second or third symbol for the respective icon may be selected by the user by first contacting a meta key, such as a shift key, and then contacting and/or breaking contact with the respective icon.
  • While the device 1200 has been illustrated with certain components and a particular arrangement of these components, it should be understood that there may be fewer or more components, two or more components may be combined, and positions of one or more components may be changed. For example, the keyboards 1210, 1212, 1214, 1216, 1218, 1220 and/or 1222 may include fewer or additional icons. In some embodiments, a different character set and/or different groups of symbols may be used on the icons in the keyboard 1210, 1212, 1214, 1216, 1218, 1220 and/or 1222.
  • In some embodiments, the user selects symbols by breaking a contact with one or more icons on the display 208. In other embodiments, however, the user may select one or more symbols without breaking contact with the display 208. For example, the user may pause or maintain contact over the respective icon for a time interval longer than an eighth pre-determined value (such as 0.1, 0.5 or 1 s) before moving on to the next icon and corresponding symbol. In the process, the user may maintain contact with the display. In other embodiments, selection of the respective icon and corresponding symbol may occur by increasing a contact pressure with the display 208 while maintaining the contact with the display.
  • A flow chart for a symbol entry process 1300 corresponding to embodiments where contact is not broken is shown in FIG. 13. While the symbol entry process 1300 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1300 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1310). A respective icon may correspond to at least one symbol. A contact by a user with the display may be detected (1312). Positions of the contact corresponding to a sequence of icons may be determined (1314). The at least one symbol may be selected when a respective position of the contact corresponds to the respective icon for a time interval exceeding a pre-determined value (1316).
  • As discussed previously, the user may make errors when using a touch screen in the display system 112 (FIG. 1). The device 100 (FIG. 1) may, therefore, adapt an offset between an estimated contact and an actual contact in accordance with such errors. Feedback may be provided by the user activating an icon corresponding to a delete key. The offset may be applied to one or more icons. In some embodiments, there may be more than one offset and a respective offset may be applied to a respective subset that includes one or more icons in a plurality of the icons in a keyboard or other user interface. The adaptation may occur continuously, after a pre-determined time interval and/or if an excessive number of user errors occur (e.g., as evidenced by a frequency of use of the delete icon). The adaptation may occur during a normal mode of operation of the device 100 (FIG. 1), rather than requiring the user to implement a separate keyboard training/adaptation mode.
  • A flow chart for a symbol entry process 1400 corresponding to such embodiments is shown in FIG. 14. While the symbol entry process 1400 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1400 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1410). A respective icon may correspond to at least one symbol. A contact by a user with the display may be detected (1412). An estimated contact that corresponds to the respective icon and the at least one symbol may be determined in accordance with the actual contact and pre-determined offset (1414). One or more corrections for one or more errors in one or more selected symbols may be received (1416). The offset for at least the respective icon may be modified in accordance with the one or more received corrections (1418).
  • FIG. 15 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1500. The device 1500 includes a keyboard 1510 with a plurality of icons. Different spacings (“guard bands”) are used between the icons. The guard bands between icons visually encourage a user to touch the center of an adjacent icon, although user contact in the guard band region may also activate the nearest icon to the contact. In some embodiments, icons near the center of the display 208 may have a smaller guard band between adjacent icons than icons near an edge of the display. This may reduce errors when using the display 208 if it is easier for a user to select or contact a respective icon near the center of the display 208. In some embodiments, the guard band near the edge of the display 208 may be larger than that near the center of the display 208. Conversely, in some embodiments (opposite to what is shown in FIG. 15), icons near the center of the display 208 may have a larger guard band between adjacent icons than icons near an edge of the display. This may reduce errors when using the display 208 if it is easier for a user to select or contact a respective icon near the edge of the display 208. In some embodiments, the guard band near the edge of the display 208 may be smaller than that near the center of the display 208. In some embodiments, icons near the center of the display 208 may be larger than icons near the edge of the display 208. In some embodiments, icons at the edge of the display are about half the size of the other icons because it is easier to identify contacts corresponding to edge icons.
  • In some embodiments, either the size of the icons or the size of the guard bands between icons could incrementally vary between the edge of the display and the center of the display (e.g., from small icons at the edge to large icons in the center or from small guard bands at the edge to large guard bands in the center).
  • A flow chart for a symbol entry process 1600 corresponding to such embodiments is shown in FIG. 16. While the symbol entry process 1600 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1600 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1610). The plurality of icons may be arranged in rows in a first dimension of the display. A first guard band in the first dimension between adjacent icons in a first subset of the icons may be greater than a pre-determined value and a second guard band in the first dimension between adjacent icons in a second subset of the icons may be less than a pre-determined value. A contact by the user with the display that corresponds to selection of the respective icon may be detected (1612). A symbol corresponding to the respective icon may be displayed (1614).
  • FIG. 17 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1700. The device 1700 includes a keyboard 1710 that has a plurality of icons. A respective icon corresponds to two or more symbols. Some symbols may be selected by contacting two or more icons simultaneously. A respective symbol that is selected may be displayed in the display tray 214. For example, a letter ‘e’ may be selected by contacting and breaking contact with the first icon in the first row. A letter ‘l’ may be selected by contacting and breaking contact with the first and the second icons in the first row. The icons include visual information indicating the combinations of contacts with icons (also referred to as chords) that correspond to given symbols. Keyboard 1710 is sometimes referred to as a hop-scotch keyboard.
  • A flow chart for a symbol entry process 1800 corresponding to such embodiments is shown in FIG. 18. While the symbol entry process 1800 described below includes a number of operations that appear to occur in a specific order, it should be apparent that the process 1800 can include more or fewer operations, which can be executed serially or in parallel (e.g., using parallel processors or a multi-threading environment), an order of two or more operations may be changed and/or two or more operations may be combined into a single operation.
  • A plurality of icons may be displayed on a touch-sensitive display (1810). A first icon and a second icon each correspond to two or more symbols. A contact by a user with the display that corresponds to the first icon and the second icon is detected (1812). A respective symbol in the two or more symbols to which the contact corresponds may be determined (1814). A visual indicator corresponding to the respective symbol is displayed (1816).
  • FIG. 19 is a schematic diagram illustrating an embodiment of a user interface for a portable electronic device 1900. A keyboard 1910 does not include fixed icons. Instead symbols are displayed. A nearest group of symbols, such as three letters in a region 1912, are selected in accordance with a user contact with the display 208. In other embodiments, the region 1912 may include two or more symbols or characters. A correct set of symbols may be determined using a lexicography model or system, such as that shown in FIG. 10A, in accordance with a sequence of groups of symbols that correspond to a sequence of contacts by the user. As more contacts occur, a tree of possible words or sets of symbols corresponding to the groups of symbols that have been selected may be pruned until a correct or highest likelihood word or set of symbols is determined.
  • In other embodiments not shown, a respective user may play a game that is used to determine a smallest acceptable key size for a user interface, such as a keyboard. The smallest key size may be in accordance with a user's manual dexterity, age, health, finger size and vision. Errors made in using the icons in a keyboard during the game may help determine a minimum icon size for the respective user.
  • In some embodiments, icons in the embodiments of the user interfaces, such as the keyboards described above, may have an effective contact area or a strike area that is larger than the displayed icon size. In other embodiments, the effective contact area or strike area may be larger than the displayed icon size in at least one dimension of the display 208 surface.
  • The foregoing descriptions of specific embodiments of the present invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Rather, it should be appreciated that many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.

Claims (15)

1. A method, comprising:
at a portable electronic device with a touch-sensitive display:
displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display, and wherein a space greater than a pre-determined value is included between adjacent rows;
detecting a contact by a user with the touch-sensitive display that corresponds to a respective icon; and
displaying a symbol corresponding to the respective icon in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.
2. The method of claim 1, wherein the symbol is the respective icon, and wherein the symbol is magnified relative to the respective icon.
3. The method of claim 1, wherein the neighboring row is above the respective row.
4. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device with a touch-sensitive display, cause the portable electronic device to perform a method comprising:
displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display, and wherein a space greater than a pre-determined value is included between adjacent rows;
detecting a contact by a user with the touch-sensitive display that corresponds to a respective icon; and
displaying a symbol corresponding to the respective icon in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.
5. The computer readable storage medium of claim 4, wherein the symbol is the respective icon, and wherein the symbol is magnified relative to the respective icon.
6. The computer readable storage medium of claim 4, wherein the neighboring row is above the respective row.
7. A graphical user interface on a portable electronic device with a touch-sensitive display, a memory, and one or more processors to execute one or more programs stored in the memory, the graphical user interface comprising:
a plurality of icons displayed on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display, and wherein a space greater than a pre-determined value is included between adjacent rows; and
a symbol corresponding to a respective icon displayed in the space between a respective row corresponding to the respective icon and a neighboring row, wherein the symbol is displayed when a contact by a user with the touch-sensitive display that corresponds to the respective icon is established.
8. The graphical user interface of claim 7, wherein the symbol is the respective icon, and wherein the symbol is magnified relative to the respective icon.
9. The graphical user interface of claim 7, wherein the neighboring row is above the respective row.
10. A portable electronic device, comprising:
a touch-sensitive display;
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including:
instructions for displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows on the touch-sensitive display, and wherein a space greater than a pre-determined value is included between adjacent rows;
instructions for detecting a contact by a user with the touch-sensitive display that corresponds to the respective icon; and
instructions for displaying a symbol corresponding to the respective icon in the space between a respective row corresponding to the respective icon and a neighboring row while the contact is maintained.
11. The portable electronic device of claim 10, wherein the symbol is the respective icon, and wherein the symbol is magnified relative to the respective icon.
12. The portable electronic device of claim 10, wherein the neighboring row is above the respective row.
13. A method, comprising:
at a portable electronic device with a touch-sensitive display:
displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows;
detecting a contact by a user with the touch-sensitive display that corresponds to the respective icon;
modifying the displaying of the plurality of icons to include a space greater than a pre-determined value between a row corresponding to the respective icon and an adjacent row on the touch-sensitive display while the contact is maintained; and
displaying a symbol corresponding to the respective icon in the space while the contact is maintained.
14. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device with a touch-sensitive display, cause the portable electronic device to perform a method comprising:
displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows;
detecting a contact by a user with the touch-sensitive display that corresponds to the respective icon;
modifying the displaying of the plurality of icons to include a space greater than a pre-determined value between a row corresponding to the respective icon and an adjacent row on the touch-sensitive display while the contact is maintained; and
displaying a symbol corresponding to the respective icon in the space while the contact is maintained.
15. A portable electronic device, comprising:
a touch-sensitive display;
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including:
instructions for displaying a plurality of icons on the touch-sensitive display, wherein two or more subsets of the plurality of icons are arranged in corresponding rows;
instructions for detecting a contact by a user with the touch-sensitive display that corresponds to the respective icon;
instructions for modifying the displaying of the plurality of icons to include a space greater than a pre-determined value between a row corresponding to the respective icon and an adjacent row on the touch-sensitive display while the contact is maintained; and
instructions for displaying a symbol corresponding to the respective icon in the space while the contact is maintained.
US12/727,217 2006-01-05 2010-03-18 Adaptive Keyboard Abandoned US20100188357A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/727,217 US20100188357A1 (en) 2006-01-05 2010-03-18 Adaptive Keyboard

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US75689006P 2006-01-05 2006-01-05
US11/459,606 US7694231B2 (en) 2006-01-05 2006-07-24 Keyboards for portable electronic devices
US12/727,217 US20100188357A1 (en) 2006-01-05 2010-03-18 Adaptive Keyboard

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/459,606 Division US7694231B2 (en) 2005-09-16 2006-07-24 Keyboards for portable electronic devices

Publications (1)

Publication Number Publication Date
US20100188357A1 true US20100188357A1 (en) 2010-07-29

Family

ID=37964578

Family Applications (5)

Application Number Title Priority Date Filing Date
US11/459,606 Active 2028-10-21 US7694231B2 (en) 2005-09-16 2006-07-24 Keyboards for portable electronic devices
US12/727,219 Abandoned US20100188358A1 (en) 2006-01-05 2010-03-18 User Interface Including Word Recommendations
US12/727,221 Abandoned US20100177056A1 (en) 2006-01-05 2010-03-18 Keyboard with Multi-Symbol Icons
US12/727,217 Abandoned US20100188357A1 (en) 2006-01-05 2010-03-18 Adaptive Keyboard
US12/727,220 Abandoned US20100192086A1 (en) 2006-01-05 2010-03-18 Keyboard with Multi-Symbol Icons

Family Applications Before (3)

Application Number Title Priority Date Filing Date
US11/459,606 Active 2028-10-21 US7694231B2 (en) 2005-09-16 2006-07-24 Keyboards for portable electronic devices
US12/727,219 Abandoned US20100188358A1 (en) 2006-01-05 2010-03-18 User Interface Including Word Recommendations
US12/727,221 Abandoned US20100177056A1 (en) 2006-01-05 2010-03-18 Keyboard with Multi-Symbol Icons

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/727,220 Abandoned US20100192086A1 (en) 2006-01-05 2010-03-18 Keyboard with Multi-Symbol Icons

Country Status (6)

Country Link
US (5) US7694231B2 (en)
EP (1) EP1969453A2 (en)
JP (1) JP4981066B2 (en)
KR (1) KR101110501B1 (en)
DE (1) DE112007000122T5 (en)
WO (1) WO2007082139A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8866762B2 (en) 2011-07-01 2014-10-21 Pixart Imaging Inc. Method and apparatus for arbitrating among contiguous buttons on a capacitive touchscreen
US9280266B2 (en) 2010-11-12 2016-03-08 Kt Corporation Apparatus and method for displaying information as background of user interface
US10551987B2 (en) 2011-05-11 2020-02-04 Kt Corporation Multiple screen mode in mobile terminal
US10592100B2 (en) 2007-01-05 2020-03-17 Apple Inc. Method, system, and graphical user interface for providing word recommendations
US11079933B2 (en) 2008-01-09 2021-08-03 Apple Inc. Method, device, and graphical user interface providing word recommendations for text input

Families Citing this family (445)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9722766D0 (en) 1997-10-28 1997-12-24 British Telecomm Portable computers
US9292111B2 (en) 1998-01-26 2016-03-22 Apple Inc. Gesturing with a multipoint sensing device
US9239673B2 (en) 1998-01-26 2016-01-19 Apple Inc. Gesturing with a multipoint sensing device
US8479122B2 (en) 2004-07-30 2013-07-02 Apple Inc. Gestures for touch sensitive input devices
US7760187B2 (en) * 2004-07-30 2010-07-20 Apple Inc. Visual expander
US7614008B2 (en) 2004-07-30 2009-11-03 Apple Inc. Operation of a computer with touch screen interface
US7469381B2 (en) * 2007-01-07 2008-12-23 Apple Inc. List scrolling and document translation, scaling, and rotation on a touch-screen display
US8645137B2 (en) 2000-03-16 2014-02-04 Apple Inc. Fast, language-independent method for user authentication by voice
US8381135B2 (en) 2004-07-30 2013-02-19 Apple Inc. Proximity detector in handheld device
US20060232565A1 (en) * 2005-04-11 2006-10-19 Drevnig Arthur L Electronic media reader that splits into two pieces
US9152238B2 (en) * 2005-08-01 2015-10-06 Wai-Lin Maw Asymmetric shuffle keyboard
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US20080098331A1 (en) * 2005-09-16 2008-04-24 Gregory Novick Portable Multifunction Device with Soft Keyboards
US7694231B2 (en) * 2006-01-05 2010-04-06 Apple Inc. Keyboards for portable electronic devices
US7958456B2 (en) 2005-12-23 2011-06-07 Apple Inc. Scrolling list with floating adjacent index symbols
US7509588B2 (en) 2005-12-30 2009-03-24 Apple Inc. Portable electronic device with interface reconfiguration mode
KR100823083B1 (en) * 2006-02-09 2008-04-18 삼성전자주식회사 Apparatus and method for correcting document of display included touch screen
US8683362B2 (en) 2008-05-23 2014-03-25 Qualcomm Incorporated Card metaphor for activities in a computing device
US8296684B2 (en) 2008-05-23 2012-10-23 Hewlett-Packard Development Company, L.P. Navigating among activities in a computing device
US8325356B2 (en) * 2006-07-31 2012-12-04 Fujifilm North America Corporation Method of sending a photograph electronically from a self-service digital photograph processing station to a remote printing facility
US8014760B2 (en) 2006-09-06 2011-09-06 Apple Inc. Missed telephone call management for a portable multifunction device
US10313505B2 (en) 2006-09-06 2019-06-04 Apple Inc. Portable multifunction device, method, and graphical user interface for configuring and displaying widgets
US20080055263A1 (en) * 2006-09-06 2008-03-06 Lemay Stephen O Incoming Telephone Call Management for a Portable Multifunction Device
US8564544B2 (en) 2006-09-06 2013-10-22 Apple Inc. Touch screen device, method, and graphical user interface for customizing display of content category icons
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
CN201348962Y (en) 2006-09-25 2009-11-18 捷讯研究有限公司 Hand-held wireless communication device
US7793228B2 (en) * 2006-10-13 2010-09-07 Apple Inc. Method, system, and graphical user interface for text entry with partial word display
US8090087B2 (en) * 2006-10-26 2012-01-03 Apple Inc. Method, system, and graphical user interface for making conference calls
US8570278B2 (en) 2006-10-26 2013-10-29 Apple Inc. Portable multifunction device, method, and graphical user interface for adjusting an insertion point marker
US7856605B2 (en) 2006-10-26 2010-12-21 Apple Inc. Method, system, and graphical user interface for positioning an insertion marker in a touch screen display
US8006190B2 (en) 2006-10-31 2011-08-23 Yahoo! Inc. Social namespace addressing for non-unique identifiers
US8970503B2 (en) * 2007-01-05 2015-03-03 Apple Inc. Gestures for devices having one or more touch sensitive surfaces
US8144129B2 (en) * 2007-01-05 2012-03-27 Apple Inc. Flexible touch sensing circuits
US7957955B2 (en) * 2007-01-05 2011-06-07 Apple Inc. Method and system for providing word recommendations for text input
US8519964B2 (en) 2007-01-07 2013-08-27 Apple Inc. Portable multifunction device, method, and graphical user interface supporting user navigations of graphical objects on a touch screen display
US20080168402A1 (en) 2007-01-07 2008-07-10 Christopher Blumenberg Application Programming Interfaces for Gesture Operations
US7844915B2 (en) 2007-01-07 2010-11-30 Apple Inc. Application programming interfaces for scrolling operations
US8451232B2 (en) * 2007-01-07 2013-05-28 Apple Inc. Portable multifunction device, method, and graphical user interface for interacting with user input elements in displayed content
US7975242B2 (en) * 2007-01-07 2011-07-05 Apple Inc. Portable multifunction device, method, and graphical user interface for conference calling
US20080168478A1 (en) 2007-01-07 2008-07-10 Andrew Platzer Application Programming Interfaces for Scrolling
US8821799B2 (en) 2007-01-26 2014-09-02 Palo Alto Research Center Incorporated Method and system implementing spatially modulated excitation or emission for particle characterization with enhanced sensitivity
US9164037B2 (en) * 2007-01-26 2015-10-20 Palo Alto Research Center Incorporated Method and system for evaluation of signals received from spatially modulated excitation and emission to accurately determine particle positions and distances
US8599801B2 (en) * 2007-02-01 2013-12-03 Yahoo! Inc. Collecting implicit information for determining context of event actions
KR100896711B1 (en) 2007-02-08 2009-05-11 삼성전자주식회사 Method for executing function according to tap in mobile terminal with touch screen
US8370349B2 (en) * 2007-02-28 2013-02-05 Yahoo! Inc. Instant contact searching and presentation by category
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US9423996B2 (en) * 2007-05-03 2016-08-23 Ian Cummings Vehicle navigation user interface customization methods
US9933937B2 (en) 2007-06-20 2018-04-03 Apple Inc. Portable multifunction device, method, and graphical user interface for playing online videos
US9772751B2 (en) 2007-06-29 2017-09-26 Apple Inc. Using gestures to slide between user interfaces
US9043725B2 (en) * 2007-07-27 2015-05-26 Sony Corporation User interface with enlarged icon display of key function
KR101339499B1 (en) * 2007-08-16 2013-12-10 엘지전자 주식회사 Mobile communication terminal with touch screen and control method thereof
US20090051661A1 (en) * 2007-08-22 2009-02-26 Nokia Corporation Method, Apparatus and Computer Program Product for Providing Automatic Positioning of Text on Touch Display Devices
US8619038B2 (en) 2007-09-04 2013-12-31 Apple Inc. Editing interface
US11126321B2 (en) 2007-09-04 2021-09-21 Apple Inc. Application menu user interface
US9619143B2 (en) 2008-01-06 2017-04-11 Apple Inc. Device, method, and graphical user interface for viewing application launch icons
WO2009036293A1 (en) * 2007-09-12 2009-03-19 Macfarlane Scott S Highly compact keyboards
US8122384B2 (en) * 2007-09-18 2012-02-21 Palo Alto Research Center Incorporated Method and apparatus for selecting an object within a user interface by performing a gesture
US20090079702A1 (en) * 2007-09-25 2009-03-26 Nokia Corporation Method, Apparatus and Computer Program Product for Providing an Adaptive Keypad on Touch Display Devices
JP5207699B2 (en) 2007-09-28 2013-06-12 京セラ株式会社 Character input device, character input method, and character input program
CN101174190B (en) * 2007-09-30 2013-06-12 上海触乐信息科技有限公司 Software keyboard entry method for implementing composite key on screen of electronic equipments
KR101422837B1 (en) * 2007-10-02 2014-08-13 엘지전자 주식회사 Touch screen device and Character input method thereof
US8631358B2 (en) * 2007-10-10 2014-01-14 Apple Inc. Variable device graphical user interface
US20090102818A1 (en) * 2007-10-22 2009-04-23 Motorola, Inc. Method and device for error-free keypad input
KR20090054831A (en) * 2007-11-27 2009-06-01 삼성전자주식회사 Method for character input and electronic device using the same
US8610671B2 (en) 2007-12-27 2013-12-17 Apple Inc. Insertion marker placement on touch sensitive display
US8863037B2 (en) * 2007-12-28 2014-10-14 Blackberry Limited Keypad navigation selection and method on mobile device
EP2077486A1 (en) 2007-12-28 2009-07-08 Research In Motion Limited Keypad navigation selection and method on mobile device
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
EP2085865A1 (en) 2008-01-30 2009-08-05 Research In Motion Limited Electronic device and method of controlling the same
US8629981B2 (en) * 2008-02-01 2014-01-14 Palo Alto Research Center Incorporated Analyzers with time variation based on color-coded spatial modulation
US9092134B2 (en) * 2008-02-04 2015-07-28 Nokia Technologies Oy User touch display interface providing an expanded selection area for a user selectable object
US8205157B2 (en) 2008-03-04 2012-06-19 Apple Inc. Methods and graphical user interfaces for conducting searches on a portable multifunction device
US8650507B2 (en) * 2008-03-04 2014-02-11 Apple Inc. Selecting of text using gestures
US8645827B2 (en) 2008-03-04 2014-02-04 Apple Inc. Touch event model
US8717305B2 (en) 2008-03-04 2014-05-06 Apple Inc. Touch event model for web pages
US8416196B2 (en) 2008-03-04 2013-04-09 Apple Inc. Touch event model programming interface
US8201109B2 (en) 2008-03-04 2012-06-12 Apple Inc. Methods and graphical user interfaces for editing on a portable multifunction device
TW200941293A (en) * 2008-03-20 2009-10-01 E Ten Information Sys Co Ltd Virtual key input method and its applications
KR100900087B1 (en) * 2008-03-31 2009-06-01 박병진 Button extension type letter input system for touch screen
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
TW200945171A (en) * 2008-04-25 2009-11-01 Htc Corp Operation method of user interface and computer readable and accessable medium and portable device
US20110010622A1 (en) * 2008-04-29 2011-01-13 Chee Keat Fong Touch Activated Display Data Entry
US8159469B2 (en) * 2008-05-06 2012-04-17 Hewlett-Packard Development Company, L.P. User interface for initiating activities in an electronic device
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US9030418B2 (en) * 2008-06-24 2015-05-12 Lg Electronics Inc. Mobile terminal capable of sensing proximity touch
US8570279B2 (en) 2008-06-27 2013-10-29 Apple Inc. Touch screen device, method, and graphical user interface for inserting a character from an alternate keyboard
US8504946B2 (en) * 2008-06-27 2013-08-06 Apple Inc. Portable device, method, and graphical user interface for automatically scrolling to display the top of an electronic document
US20100008031A1 (en) * 2008-07-08 2010-01-14 Emblaze Mobile Ltd Ergonomic handheld device
KR101070943B1 (en) * 2008-07-10 2011-10-06 삼성메디슨 주식회사 Ultrasound system having virtual keyboard and method of controlling the same
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
JP4752887B2 (en) 2008-09-12 2011-08-17 ソニー株式会社 Information processing apparatus, information processing method, and computer program
TW201013431A (en) * 2008-09-17 2010-04-01 Mitac Int Corp Local search method, local search system, program product, portable miniature electronic device, and input interface
US20100066764A1 (en) * 2008-09-18 2010-03-18 Microsoft Corporation Selective character magnification on touch screen devices
US8284170B2 (en) 2008-09-30 2012-10-09 Apple Inc. Touch screen device, method, and graphical user interface for moving on-screen objects without using a cursor
US20100088654A1 (en) * 2008-10-08 2010-04-08 Research In Motion Limited Electronic device having a state aware touchscreen
US20100110002A1 (en) * 2008-11-06 2010-05-06 Sony Ericsson Mobile Communications Ab Communication device with combined input and display device
US9959870B2 (en) 2008-12-11 2018-05-01 Apple Inc. Speech recognition involving a mobile device
US8289286B2 (en) * 2008-12-19 2012-10-16 Verizon Patent And Licensing Inc. Zooming keyboard/keypad
US9311112B2 (en) 2009-03-16 2016-04-12 Apple Inc. Event recognition
US8370736B2 (en) 2009-03-16 2013-02-05 Apple Inc. Methods and graphical user interfaces for editing on a multifunction device with a touch screen display
US8566045B2 (en) 2009-03-16 2013-10-22 Apple Inc. Event recognition
US8689128B2 (en) 2009-03-16 2014-04-01 Apple Inc. Device, method, and graphical user interface for moving a current position in content at a variable scrubbing rate
US8285499B2 (en) 2009-03-16 2012-10-09 Apple Inc. Event recognition
US9684521B2 (en) 2010-01-26 2017-06-20 Apple Inc. Systems having discrete and continuous gesture recognizers
US8589374B2 (en) 2009-03-16 2013-11-19 Apple Inc. Multifunction device with integrated search and application selection
CN102356374A (en) * 2009-03-17 2012-02-15 日本电气株式会社 Input device for portable electronic device, control method therfor, and program
US20100251176A1 (en) * 2009-03-24 2010-09-30 Microsoft Corporation Virtual keyboard with slider buttons
US8294680B2 (en) * 2009-03-27 2012-10-23 Sony Mobile Communications Ab System and method for touch-based text entry
GB201108200D0 (en) * 2011-05-16 2011-06-29 Touchtype Ltd User input prediction
KR101542136B1 (en) 2009-03-31 2015-08-05 삼성전자 주식회사 Method for inputting character message and mobile terminal using the same
KR101553842B1 (en) * 2009-04-21 2015-09-17 엘지전자 주식회사 Mobile terminal providing multi haptic effect and control method thereof
US20100271312A1 (en) * 2009-04-22 2010-10-28 Rachid Alameh Menu Configuration System and Method for Display on an Electronic Device
US8340969B2 (en) * 2009-04-24 2012-12-25 Research In Motion Limited Method and mobile communication device for generating dual-tone multi-frequency (DTMF) commands on a mobile communication device having a touchscreen
US20100275126A1 (en) * 2009-04-27 2010-10-28 Scott David Lincke Automatic On-Screen Keyboard
US20100293457A1 (en) * 2009-05-15 2010-11-18 Gemstar Development Corporation Systems and methods for alphanumeric navigation and input
US10705692B2 (en) 2009-05-21 2020-07-07 Sony Interactive Entertainment Inc. Continuous and dynamic scene decomposition for user interface
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US20120311585A1 (en) 2011-06-03 2012-12-06 Apple Inc. Organizing task items that represent tasks to perform
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
SG177156A1 (en) 2009-06-16 2012-01-30 Intel Corp Camera applications in a handheld device
JP5532499B2 (en) 2009-06-16 2014-06-25 インテル・コーポレーション Adaptive virtual keyboard for handheld devices
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
KR20110015811A (en) * 2009-08-10 2011-02-17 삼성전자주식회사 Method for displaying letter in terminal with touch screen and apparatus thereof
US8531410B2 (en) * 2009-08-18 2013-09-10 Fuji Xerox Co., Ltd. Finger occlusion avoidance on touch display devices
CN101655771B (en) * 2009-09-07 2011-07-20 上海合合信息科技发展有限公司 Method and system for inputting multi-contact characters
WO2011037558A1 (en) 2009-09-22 2011-03-31 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
US9223590B2 (en) * 2010-01-06 2015-12-29 Apple Inc. System and method for issuing commands to applications based on contextual information
US8832585B2 (en) 2009-09-25 2014-09-09 Apple Inc. Device, method, and graphical user interface for manipulating workspace views
US8766928B2 (en) * 2009-09-25 2014-07-01 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
US8799826B2 (en) * 2009-09-25 2014-08-05 Apple Inc. Device, method, and graphical user interface for moving a calendar entry in a calendar application
US8624933B2 (en) 2009-09-25 2014-01-07 Apple Inc. Device, method, and graphical user interface for scrolling a multi-section document
US8780069B2 (en) 2009-09-25 2014-07-15 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
JPWO2011043045A1 (en) * 2009-10-07 2013-03-04 パナソニック株式会社 Imaging device, solid-state imaging device, image generation method, and program
US8665227B2 (en) * 2009-11-19 2014-03-04 Motorola Mobility Llc Method and apparatus for replicating physical key function with soft keys in an electronic device
JP5211019B2 (en) * 2009-11-26 2013-06-12 京セラドキュメントソリューションズ株式会社 Display device, image forming apparatus equipped with the same, and electronic apparatus
US8358281B2 (en) * 2009-12-15 2013-01-22 Apple Inc. Device, method, and graphical user interface for management and manipulation of user interface elements
US8621391B2 (en) * 2009-12-16 2013-12-31 Apple Inc. Device, method, and computer readable medium for maintaining a selection order in a displayed thumbnail stack of user interface elements acted upon via gestured operations
US8347238B2 (en) * 2009-12-16 2013-01-01 Apple Inc. Device, method, and graphical user interface for managing user interface content and user interface elements by dynamic snapping of user interface elements to alignment guides
US8381125B2 (en) 2009-12-16 2013-02-19 Apple Inc. Device and method for resizing user interface content while maintaining an aspect ratio via snapping a perimeter to a gridline
US20110145739A1 (en) * 2009-12-16 2011-06-16 Peter Glen Berger Device, Method, and Graphical User Interface for Location-Based Data Collection
US8698845B2 (en) 2010-01-06 2014-04-15 Apple Inc. Device, method, and graphical user interface with interactive popup views
US8806362B2 (en) * 2010-01-06 2014-08-12 Apple Inc. Device, method, and graphical user interface for accessing alternate keys
US8736561B2 (en) 2010-01-06 2014-05-27 Apple Inc. Device, method, and graphical user interface with content display modes and display rotation heuristics
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US8677268B2 (en) * 2010-01-26 2014-03-18 Apple Inc. Device, method, and graphical user interface for resizing objects
US8539385B2 (en) * 2010-01-26 2013-09-17 Apple Inc. Device, method, and graphical user interface for precise positioning of objects
US8209630B2 (en) * 2010-01-26 2012-06-26 Apple Inc. Device, method, and graphical user interface for resizing user interface content
US8539386B2 (en) * 2010-01-26 2013-09-17 Apple Inc. Device, method, and graphical user interface for selecting and moving objects
US8683363B2 (en) * 2010-01-26 2014-03-25 Apple Inc. Device, method, and graphical user interface for managing user interface content and user interface elements
KR101097332B1 (en) * 2010-02-10 2011-12-21 삼성모바일디스플레이주식회사 Display module having haptic function
US8782556B2 (en) 2010-02-12 2014-07-15 Microsoft Corporation User-centric soft keyboard predictive technologies
US20110231796A1 (en) * 2010-02-16 2011-09-22 Jose Manuel Vigil Methods for navigating a touch screen device in conjunction with gestures
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US20110210850A1 (en) * 2010-02-26 2011-09-01 Phuong K Tran Touch-screen keyboard with combination keys and directional swipes
US8686955B2 (en) * 2010-03-11 2014-04-01 Apple Inc. Device, method, and graphical user interface for performing character entry
US9823831B2 (en) 2010-04-07 2017-11-21 Apple Inc. Device, method, and graphical user interface for managing concurrently open software applications
US8291344B2 (en) 2010-04-07 2012-10-16 Apple Inc. Device, method, and graphical user interface for managing concurrently open software applications
US9513801B2 (en) 2010-04-07 2016-12-06 Apple Inc. Accessing electronic notifications and settings icons with gestures
CN101833420B (en) * 2010-05-19 2012-08-29 鸿富锦精密工业(深圳)有限公司 Electronic device with touch panel
JP5010714B2 (en) 2010-05-21 2012-08-29 株式会社東芝 Electronic device, input control program, and input control method
US9542091B2 (en) 2010-06-04 2017-01-10 Apple Inc. Device, method, and graphical user interface for navigating through a user interface using a dynamic object selection indicator
US10216408B2 (en) 2010-06-14 2019-02-26 Apple Inc. Devices and methods for identifying user interface objects based on view hierarchy
US9081494B2 (en) 2010-07-30 2015-07-14 Apple Inc. Device, method, and graphical user interface for copying formatting attributes
US8972879B2 (en) 2010-07-30 2015-03-03 Apple Inc. Device, method, and graphical user interface for reordering the front-to-back positions of objects
US9098182B2 (en) 2010-07-30 2015-08-04 Apple Inc. Device, method, and graphical user interface for copying user interface objects between content regions
US9411413B2 (en) 2010-08-04 2016-08-09 Apple Inc. Three dimensional user interface effects on a display
US8913056B2 (en) 2010-08-04 2014-12-16 Apple Inc. Three dimensional user interface effects on a display by using properties of motion
JP5593972B2 (en) 2010-08-30 2014-09-24 ソニー株式会社 Information processing apparatus, stereoscopic display method, and program
JP5289399B2 (en) * 2010-09-06 2013-09-11 三菱電機株式会社 Touch panel device
US10026058B2 (en) 2010-10-29 2018-07-17 Microsoft Technology Licensing, Llc Enterprise resource planning oriented context-aware environment
US20120108172A1 (en) * 2010-10-29 2012-05-03 Microsoft Corporation Personal digital context
CN102479046B (en) * 2010-11-30 2013-09-25 英业达股份有限公司 Touch device and operation method thereof
US9285836B2 (en) 2010-12-13 2016-03-15 Blackberry Limited Portable electronic device including touch-sensitive display
US9244606B2 (en) 2010-12-20 2016-01-26 Apple Inc. Device, method, and graphical user interface for navigation of concurrently open software applications
US10762293B2 (en) 2010-12-22 2020-09-01 Apple Inc. Using parts-of-speech tagging and named entity recognition for spelling correction
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US9298363B2 (en) 2011-04-11 2016-03-29 Apple Inc. Region activation for touch sensitive surface
US9767195B2 (en) 2011-04-21 2017-09-19 Touchstream Technologies, Inc. Virtualized hosting and displaying of content using a swappable media player
US8904289B2 (en) * 2011-04-21 2014-12-02 Touchstream Technologies, Inc. Play control of content on a display device
WO2012159249A1 (en) * 2011-05-20 2012-11-29 Microsoft Corporation Advaced prediction
US20120304121A1 (en) * 2011-05-25 2012-11-29 Componentart Holdings Inc. Method, processing device, and article of manufacture for providing instructions for displaying time-dependent information and for allowing user selection of time ranges
US9389764B2 (en) * 2011-05-27 2016-07-12 Microsoft Technology Licensing, Llc Target disambiguation and correction
US9032338B2 (en) 2011-05-30 2015-05-12 Apple Inc. Devices, methods, and graphical user interfaces for navigating and editing text
US8719695B2 (en) 2011-05-31 2014-05-06 Apple Inc. Devices, methods, and graphical user interfaces for document manipulation
US9471560B2 (en) * 2011-06-03 2016-10-18 Apple Inc. Autocorrecting language input for virtual keyboards
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US9029800B2 (en) 2011-08-09 2015-05-12 Palo Alto Research Center Incorporated Compact analyzer with spatial modulation and multiple intensity modulated excitation sources
US8723140B2 (en) 2011-08-09 2014-05-13 Palo Alto Research Center Incorporated Particle analyzer with spatial modulation and long lifetime bioprobes
US8971572B1 (en) 2011-08-12 2015-03-03 The Research Foundation For The State University Of New York Hand pointing estimation for human computer interaction
US8806369B2 (en) 2011-08-26 2014-08-12 Apple Inc. Device, method, and graphical user interface for managing and interacting with concurrently open software applications
US8994660B2 (en) 2011-08-29 2015-03-31 Apple Inc. Text correction processing
US8631357B2 (en) 2011-10-31 2014-01-14 Apple Inc. Dual function scroll wheel input
US8490008B2 (en) * 2011-11-10 2013-07-16 Research In Motion Limited Touchscreen keyboard predictive display and generation of a set of characters
EP2590062A1 (en) * 2011-11-03 2013-05-08 Dassault Systèmes Method and system for designing a modeled assembly of at least one object in a computer-aided design system
US10082950B2 (en) 2011-11-09 2018-09-25 Joseph T. LAPP Finger-mapped character entry systems
US9310889B2 (en) 2011-11-10 2016-04-12 Blackberry Limited Touchscreen keyboard predictive display and generation of a set of characters
US9122672B2 (en) 2011-11-10 2015-09-01 Blackberry Limited In-letter word prediction for virtual keyboard
JP5829506B2 (en) * 2011-12-13 2015-12-09 株式会社ソニー・コンピュータエンタテインメント Portable electronic device and history display method
US9152323B2 (en) 2012-01-19 2015-10-06 Blackberry Limited Virtual keyboard providing an indication of received input
US8884928B1 (en) * 2012-01-26 2014-11-11 Amazon Technologies, Inc. Correcting for parallax in electronic displays
JP2013161208A (en) * 2012-02-03 2013-08-19 Sony Corp Information processing apparatus, information processing method, and program
TW201333752A (en) * 2012-02-14 2013-08-16 Thlight Co Ltd Receiver apparatus, host apparatus and control method
US9910588B2 (en) 2012-02-24 2018-03-06 Blackberry Limited Touchscreen keyboard providing word predictions in partitions of the touchscreen keyboard in proximate association with candidate letters
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9292192B2 (en) 2012-04-30 2016-03-22 Blackberry Limited Method and apparatus for text selection
US20130285916A1 (en) * 2012-04-30 2013-10-31 Research In Motion Limited Touchscreen keyboard providing word predictions at locations in association with candidate letters
EP3401773A1 (en) 2012-05-09 2018-11-14 Apple Inc. Device, method, and graphical user interface for displaying additional information in response to a user contact
WO2013169851A2 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Device, method, and graphical user interface for facilitating user interaction with controls in a user interface
CN107728906B (en) 2012-05-09 2020-07-31 苹果公司 Device, method and graphical user interface for moving and placing user interface objects
WO2013169865A2 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input
WO2013169875A2 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Device, method, and graphical user interface for displaying content associated with a corresponding affordance
WO2013169843A1 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Device, method, and graphical user interface for manipulating framed graphical objects
JP6082458B2 (en) 2012-05-09 2017-02-15 アップル インコーポレイテッド Device, method, and graphical user interface for providing tactile feedback of actions performed within a user interface
AU2013259637B2 (en) 2012-05-09 2016-07-07 Apple Inc. Device, method, and graphical user interface for selecting user interface objects
JP6002836B2 (en) 2012-05-09 2016-10-05 アップル インコーポレイテッド Device, method, and graphical user interface for transitioning between display states in response to a gesture
WO2013169842A2 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Device, method, and graphical user interface for selecting object within a group of objects
WO2013169849A2 (en) 2012-05-09 2013-11-14 Industries Llc Yknots Device, method, and graphical user interface for displaying user interface objects corresponding to an application
JP6182207B2 (en) 2012-05-09 2017-08-16 アップル インコーポレイテッド Device, method, and graphical user interface for providing feedback for changing an activation state of a user interface object
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US9261961B2 (en) 2012-06-07 2016-02-16 Nook Digital, Llc Accessibility aids for users of electronic devices
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US9116552B2 (en) 2012-06-27 2015-08-25 Blackberry Limited Touchscreen keyboard providing selection of word predictions in partitions of the touchscreen keyboard
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US20140026101A1 (en) 2012-07-20 2014-01-23 Barnesandnoble.Com Llc Accessible Menu Navigation Techniques For Electronic Devices
US9063653B2 (en) 2012-08-31 2015-06-23 Blackberry Limited Ranking predictions based on typing speed and typing confidence
US9524290B2 (en) 2012-08-31 2016-12-20 Blackberry Limited Scoring predictions based on prediction length and typing speed
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US20140267050A1 (en) * 2013-03-15 2014-09-18 Logitech Europe S.A. Key layout for an input device
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
GB2504157B (en) * 2012-10-26 2014-09-24 Glynn Andrew Reynolds Secure user interface
US9591339B1 (en) 2012-11-27 2017-03-07 Apple Inc. Agnostic media delivery system
US9411510B2 (en) 2012-12-07 2016-08-09 Apple Inc. Techniques for preventing typographical errors on soft keyboards
US9075462B2 (en) * 2012-12-10 2015-07-07 Sap Se Finger-specific input on touchscreen devices
US9774917B1 (en) 2012-12-10 2017-09-26 Apple Inc. Channel bar user interface
US10200761B1 (en) 2012-12-13 2019-02-05 Apple Inc. TV side bar user interface
US9477382B2 (en) 2012-12-14 2016-10-25 Barnes & Noble College Booksellers, Inc. Multi-page content selection technique
US9134892B2 (en) 2012-12-14 2015-09-15 Barnes & Noble College Booksellers, Llc Drag-based content selection technique for touch screen UI
US9134893B2 (en) 2012-12-14 2015-09-15 Barnes & Noble College Booksellers, Llc Block-based content selecting technique for touch screen UI
US9448719B2 (en) 2012-12-14 2016-09-20 Barnes & Noble College Booksellers, Llc Touch sensitive device with pinch-based expand/collapse function
US8963865B2 (en) 2012-12-14 2015-02-24 Barnesandnoble.Com Llc Touch sensitive device with concentration mode
US9001064B2 (en) 2012-12-14 2015-04-07 Barnesandnoble.Com Llc Touch sensitive device with pinch-based archive and restore functionality
US9134903B2 (en) 2012-12-14 2015-09-15 Barnes & Noble College Booksellers, Llc Content selecting technique for touch screen UI
US9030430B2 (en) 2012-12-14 2015-05-12 Barnesandnoble.Com Llc Multi-touch navigation mode
US9532111B1 (en) 2012-12-18 2016-12-27 Apple Inc. Devices and method for providing remote control hints on a display
USD735238S1 (en) 2012-12-26 2015-07-28 Lookout, Inc. Display screen with graphical user interface with interleaved rows of text and reduced error key layout short reach
USD735215S1 (en) 2012-12-31 2015-07-28 Lookout, Inc. Display screen or portion thereof with graphical user interface with interleaved rows of text and reduced error key layout short reach
USD738886S1 (en) 2012-12-31 2015-09-15 Lookout, Inc. Display screen or portion thereof with graphical user interface with interleaved rows of text and reduced error key layout short reach
US9367240B2 (en) * 2012-12-26 2016-06-14 Lookout, Inc. Graphical user interface for improved text character entry
AU2013368441B2 (en) 2012-12-29 2016-04-14 Apple Inc. Device, method, and graphical user interface for forgoing generation of tactile output for a multi-contact gesture
CN104903834B (en) 2012-12-29 2019-07-05 苹果公司 For equipment, method and the graphic user interface in touch input to transition between display output relation
CN104885050B (en) 2012-12-29 2017-12-08 苹果公司 For determining the equipment, method and the graphic user interface that are rolling or selection content
WO2014105277A2 (en) 2012-12-29 2014-07-03 Yknots Industries Llc Device, method, and graphical user interface for moving a cursor according to a change in an appearance of a control icon with simulated three-dimensional characteristics
CN109375853A (en) 2012-12-29 2019-02-22 苹果公司 To equipment, method and the graphic user interface of the navigation of user interface hierarchical structure
US10521188B1 (en) 2012-12-31 2019-12-31 Apple Inc. Multi-user TV user interface
US9836154B2 (en) 2013-01-24 2017-12-05 Nook Digital, Llc Selective touch scan area and reporting techniques
US9298360B2 (en) 2013-01-25 2016-03-29 Apple Inc. Accessibility techinques for presentation of symbolic expressions
US9971495B2 (en) 2013-01-28 2018-05-15 Nook Digital, Llc Context based gesture delineation for user interaction in eyes-free mode
CN103076924B (en) * 2013-02-06 2014-08-13 东莞市傲思电子科技有限公司 Gesture keyboard application method
KR20230137475A (en) 2013-02-07 2023-10-04 애플 인크. Voice trigger for a digital assistant
US9766723B2 (en) 2013-03-11 2017-09-19 Barnes & Noble College Booksellers, Llc Stylus sensitive device with hover over stylus control functionality
US9760187B2 (en) 2013-03-11 2017-09-12 Barnes & Noble College Booksellers, Llc Stylus with active color display/select for touch sensitive devices
US9367161B2 (en) 2013-03-11 2016-06-14 Barnes & Noble College Booksellers, Llc Touch sensitive device with stylus-based grab and paste functionality
US9189084B2 (en) 2013-03-11 2015-11-17 Barnes & Noble College Booksellers, Llc Stylus-based user data storage and access
US9626008B2 (en) 2013-03-11 2017-04-18 Barnes & Noble College Booksellers, Llc Stylus-based remote wipe of lost device
US9632594B2 (en) 2013-03-11 2017-04-25 Barnes & Noble College Booksellers, Llc Stylus sensitive device with stylus idle functionality
US9946365B2 (en) 2013-03-11 2018-04-17 Barnes & Noble College Booksellers, Llc Stylus-based pressure-sensitive area for UI control of computing device
US9891722B2 (en) 2013-03-11 2018-02-13 Barnes & Noble College Booksellers, Llc Stylus-based notification system
US9785259B2 (en) 2013-03-11 2017-10-10 Barnes & Noble College Booksellers, Llc Stylus-based slider functionality for UI control of computing device
US9448643B2 (en) 2013-03-11 2016-09-20 Barnes & Noble College Booksellers, Llc Stylus sensitive device with stylus angle detection functionality
US9261985B2 (en) 2013-03-11 2016-02-16 Barnes & Noble College Booksellers, Llc Stylus-based touch-sensitive area for UI control of computing device
US9600053B2 (en) 2013-03-11 2017-03-21 Barnes & Noble College Booksellers, Llc Stylus control feature for locking/unlocking touch sensitive devices
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
US9658740B2 (en) 2013-03-15 2017-05-23 Apple Inc. Device, method, and graphical user interface for managing concurrently open software applications
WO2014144579A1 (en) 2013-03-15 2014-09-18 Apple Inc. System and method for updating an adaptive speech recognition model
US9922642B2 (en) 2013-03-15 2018-03-20 Apple Inc. Training an at least partial voice command system
US9477404B2 (en) 2013-03-15 2016-10-25 Apple Inc. Device, method, and graphical user interface for managing concurrently open software applications
US9146672B2 (en) 2013-04-10 2015-09-29 Barnes & Noble College Booksellers, Llc Multidirectional swipe key for virtual keyboard
US8963869B2 (en) 2013-04-23 2015-02-24 Barnesandnoble.Com Llc Color pattern unlocking techniques for touch sensitive devices
US8966617B2 (en) 2013-04-23 2015-02-24 Barnesandnoble.Com Llc Image pattern unlocking techniques for touch sensitive devices
US9152321B2 (en) 2013-05-03 2015-10-06 Barnes & Noble College Booksellers, Llc Touch sensitive UI technique for duplicating content
US9612740B2 (en) 2013-05-06 2017-04-04 Barnes & Noble College Booksellers, Inc. Swipe-based delete confirmation for touch sensitive devices
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
WO2014197336A1 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US10019153B2 (en) 2013-06-07 2018-07-10 Nook Digital, Llc Scrapbooking digital content in computing devices using a swiping gesture
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9733716B2 (en) 2013-06-09 2017-08-15 Apple Inc. Proxy gesture recognizer
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
EP3937002A1 (en) 2013-06-09 2022-01-12 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
KR101809808B1 (en) 2013-06-13 2017-12-15 애플 인크. System and method for emergency calls initiated by voice command
US9400601B2 (en) 2013-06-21 2016-07-26 Nook Digital, Llc Techniques for paging through digital content on touch screen devices
US9244603B2 (en) 2013-06-21 2016-01-26 Nook Digital, Llc Drag and drop techniques for discovering related content
US9423932B2 (en) 2013-06-21 2016-08-23 Nook Digital, Llc Zoom view mode for digital content including multiple regions of interest
US9483176B2 (en) * 2013-07-08 2016-11-01 Samsung Display Co., Ltd. Method and apparatus to reduce display lag of soft keyboard presses
KR102313353B1 (en) * 2013-07-29 2021-10-18 삼성전자주식회사 Character inputting method and display apparatus
JP6163266B2 (en) 2013-08-06 2017-07-12 アップル インコーポレイテッド Automatic activation of smart responses based on activation from remote devices
US20150051901A1 (en) * 2013-08-16 2015-02-19 Blackberry Limited Methods and devices for providing predicted words for textual input
US9575948B2 (en) 2013-10-04 2017-02-21 Nook Digital, Llc Annotation of digital content via selective fixed formatting
CN105722768B (en) 2013-10-07 2019-02-19 蒙诺苏尔有限公司 Water-soluble sustained release capsule, correlation technique and correlated product
JP6691479B2 (en) 2013-10-07 2020-04-28 モノソル リミテッド ライアビリティ カンパニー Water-soluble delayed release capsules, related methods and related articles
USD772278S1 (en) * 2013-12-18 2016-11-22 Apple Inc. Display screen or portion thereof with animated graphical user interface
US10620796B2 (en) 2013-12-19 2020-04-14 Barnes & Noble College Booksellers, Llc Visual thumbnail scrubber for digital content
US10331777B2 (en) 2013-12-31 2019-06-25 Barnes & Noble College Booksellers, Llc Merging annotations of paginated digital content
US9588979B2 (en) 2013-12-31 2017-03-07 Barnes & Noble College Booksellers, Llc UI techniques for navigating a file manager of an electronic computing device
USD766257S1 (en) * 2013-12-31 2016-09-13 Beijing Qihoo Technology Co. Ltd Display screen with a graphical user interface
USD766256S1 (en) * 2013-12-31 2016-09-13 Beijing Qihoo Technology Co. Ltd Display screen with a graphical user interface
US10534528B2 (en) 2013-12-31 2020-01-14 Barnes & Noble College Booksellers, Llc Digital flash card techniques
US9367212B2 (en) 2013-12-31 2016-06-14 Barnes & Noble College Booksellers, Llc User interface for navigating paginated digital content
USD767591S1 (en) * 2013-12-31 2016-09-27 Beijing Qihoo Technology Co., Ltd. Display screen with a graphical user interface
US9792272B2 (en) 2013-12-31 2017-10-17 Barnes & Noble College Booksellers, Llc Deleting annotations of paginated digital content
USD767592S1 (en) * 2013-12-31 2016-09-27 Beijing Qihoo Technology Co., Ltd. Display screen with a graphical user interface
US9367208B2 (en) 2013-12-31 2016-06-14 Barnes & Noble College Booksellers, Llc Move icon to reveal textual information
USD765667S1 (en) * 2013-12-31 2016-09-06 Beijing Qihoo Technology Co. Ltd Display screen with a graphical user interface
US10915698B2 (en) 2013-12-31 2021-02-09 Barnes & Noble College Booksellers, Llc Multi-purpose tool for interacting with paginated digital content
US9424241B2 (en) 2013-12-31 2016-08-23 Barnes & Noble College Booksellers, Llc Annotation mode including multiple note types for paginated digital content
US9652044B2 (en) 2014-03-04 2017-05-16 Microsoft Technology Licensing, Llc Proximity sensor-based interactions
JP1518853S (en) * 2014-04-07 2015-03-09
JP1519059S (en) * 2014-04-07 2015-03-09
USD770487S1 (en) * 2014-04-30 2016-11-01 Tencent Technology (Shenzhen) Company Limited Display screen or portion thereof with graphical user interface
USD770488S1 (en) * 2014-04-30 2016-11-01 Tencent Technology (Shenzhen) Company Limited Portion of a display screen with graphical user interface
CN109922122B (en) * 2014-05-05 2022-03-01 创新先进技术有限公司 Method and device for interacting and acquiring user information
US9620105B2 (en) 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US10592095B2 (en) 2014-05-23 2020-03-17 Apple Inc. Instantaneous speaking of content on touch devices
US9502031B2 (en) 2014-05-27 2016-11-22 Apple Inc. Method for supporting dynamic grammars in WFST-based ASR
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US10289433B2 (en) 2014-05-30 2019-05-14 Apple Inc. Domain specific language for encoding assistant dialog
EP3480811A1 (en) 2014-05-30 2019-05-08 Apple Inc. Multi-command single utterance input method
US9898162B2 (en) 2014-05-30 2018-02-20 Apple Inc. Swiping functions for messaging applications
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9971500B2 (en) 2014-06-01 2018-05-15 Apple Inc. Displaying options, assigning notification, ignoring messages, and simultaneous user interface displays in a messaging application
TWI585673B (en) 2014-06-24 2017-06-01 蘋果公司 Input device and user interface interactions
CN111782128B (en) 2014-06-24 2023-12-08 苹果公司 Column interface for navigating in a user interface
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
JP6089181B2 (en) * 2014-08-21 2017-03-08 国立大学法人 大分大学 Data input system, information processing apparatus, information processing program, and method thereof
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10095396B2 (en) 2015-03-08 2018-10-09 Apple Inc. Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9645732B2 (en) 2015-03-08 2017-05-09 Apple Inc. Devices, methods, and graphical user interfaces for displaying and using menus
US10048757B2 (en) 2015-03-08 2018-08-14 Apple Inc. Devices and methods for controlling media presentation
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9785305B2 (en) 2015-03-19 2017-10-10 Apple Inc. Touch input cursor manipulation
US9808710B2 (en) * 2015-03-31 2017-11-07 King.Com Ltd. User interface
US20170045981A1 (en) 2015-08-10 2017-02-16 Apple Inc. Devices and Methods for Processing Touch Inputs Based on Their Intensities
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US9674426B2 (en) 2015-06-07 2017-06-06 Apple Inc. Devices and methods for capturing and interacting with enhanced digital images
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US9860451B2 (en) 2015-06-07 2018-01-02 Apple Inc. Devices and methods for capturing and interacting with enhanced digital images
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
USD846563S1 (en) * 2016-02-12 2019-04-23 Yonggui Li Display screen or portion thereof with keyboard graphical user interface
USD846564S1 (en) * 2016-02-12 2019-04-23 Yonggui Li Display screen or portion thereof with keyboard graphical user interface
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179309B1 (en) 2016-06-09 2018-04-23 Apple Inc Intelligent automated assistant in a home environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10620812B2 (en) 2016-06-10 2020-04-14 Apple Inc. Device, method, and graphical user interface for managing electronic communications
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK201670582A1 (en) 2016-06-12 2018-01-02 Apple Inc Identifying applications on which content is available
DK201670581A1 (en) 2016-06-12 2018-01-08 Apple Inc Device-level authorization for viewing content
AU2016410752A1 (en) 2016-06-22 2019-01-31 Huawei Technologies Co., Ltd. Method and device for displaying candidate words, and graphical user interface
USD916735S1 (en) * 2016-07-03 2021-04-20 Yonggui Li Display screen or portion thereof with keyboard graphical user interface
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
EP3511813B1 (en) * 2016-09-23 2023-07-26 Huawei Technologies Co., Ltd. Force touch method and terminal
WO2018081112A1 (en) * 2016-10-25 2018-05-03 Apple Inc. Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
US11237709B2 (en) 2016-10-25 2022-02-01 Apple Inc. Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
DK179492B1 (en) 2016-10-25 2019-01-08 Apple Inc. Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
US20180113579A1 (en) 2016-10-26 2018-04-26 Apple Inc. User interfaces for browsing content from multiple content applications on an electronic device
US11281993B2 (en) 2016-12-05 2022-03-22 Apple Inc. Model and ensemble compression for metric learning
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
CN106843737B (en) * 2017-02-13 2020-05-08 北京新美互通科技有限公司 Text input method and device and terminal equipment
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. User interface for correcting recognition errors
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. Low-latency intelligent automated assistant
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US11086975B2 (en) 2017-05-16 2021-08-10 Huawei Technologies Co., Ltd. Input method and electronic device
EP3625652B1 (en) * 2017-05-19 2022-09-14 Michael William Murphy An interleaved character selection interface
WO2019049317A1 (en) * 2017-09-08 2019-03-14 三菱電機株式会社 Position correction device and position correction method
KR20190065746A (en) * 2017-12-04 2019-06-12 삼성전자주식회사 Electronic apparatus, method for controlling thereof and the computer readable recording medium
US10776006B2 (en) 2018-06-03 2020-09-15 Apple Inc. Systems and methods for activating and using a trackpad at an electronic device with a touch-sensitive display and no force sensors
US11669243B2 (en) 2018-06-03 2023-06-06 Apple Inc. Systems and methods for activating and using a trackpad at an electronic device with a touch-sensitive display and no force sensors
US11683565B2 (en) 2019-03-24 2023-06-20 Apple Inc. User interfaces for interacting with channels that provide content that plays in a media browsing application
US11467726B2 (en) 2019-03-24 2022-10-11 Apple Inc. User interfaces for viewing and accessing content on an electronic device
EP3928194A1 (en) 2019-03-24 2021-12-29 Apple Inc. User interfaces including selectable representations of content items
USD984459S1 (en) * 2019-03-28 2023-04-25 Yonggui Li Display screen or portion thereof with keyboard graphical user interface
US11863837B2 (en) 2019-05-31 2024-01-02 Apple Inc. Notification of augmented reality content on an electronic device
CN113906380A (en) 2019-05-31 2022-01-07 苹果公司 User interface for podcast browsing and playback applications
US11194467B2 (en) 2019-06-01 2021-12-07 Apple Inc. Keyboard management user interfaces
US11843838B2 (en) 2020-03-24 2023-12-12 Apple Inc. User interfaces for accessing episodes of a content series
US11899895B2 (en) 2020-06-21 2024-02-13 Apple Inc. User interfaces for setting up an electronic device
JP2023535212A (en) 2020-07-24 2023-08-16 アジリス アイズフリー タッチスクリーン キーボーズ エルティディ Adaptable touch screen keypad with dead zone
USD965013S1 (en) * 2020-09-28 2022-09-27 Samsung Electronics Co., Ltd. Display screen or portion thereof with transitional graphical user interface
US11720229B2 (en) 2020-12-07 2023-08-08 Apple Inc. User interfaces for browsing and presenting content
EP4298501A1 (en) * 2022-05-20 2024-01-03 Google LLC Predictive input interface having improved robustness for processing low precision inputs

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5736974A (en) * 1995-02-17 1998-04-07 International Business Machines Corporation Method and apparatus for improving visibility and selectability of icons
US5748927A (en) * 1996-05-10 1998-05-05 Apple Computer, Inc. Graphical user interface with icons having expandable descriptors
US5963671A (en) * 1991-11-27 1999-10-05 International Business Machines Corporation Enhancement of soft keyboard operations using trigram prediction
US6169538B1 (en) * 1998-08-13 2001-01-02 Motorola, Inc. Method and apparatus for implementing a graphical user interface keyboard and a text buffer on electronic devices
US6359572B1 (en) * 1998-09-03 2002-03-19 Microsoft Corporation Dynamic keyboard
US6573844B1 (en) * 2000-01-18 2003-06-03 Microsoft Corporation Predictive keyboard
US20040160419A1 (en) * 2003-02-11 2004-08-19 Terradigital Systems Llc. Method for entering alphanumeric characters into a graphical user interface
US20050093826A1 (en) * 2003-10-29 2005-05-05 Samsung Electronics Co., Ltd. Apparatus and method for inputting character using touch screen in portable terminal
US20070046641A1 (en) * 2005-09-01 2007-03-01 Swee Ho Lim Entering a character into an electronic device
US7194699B2 (en) * 2003-01-14 2007-03-20 Microsoft Corporation Animating images to reflect user selection
US7382358B2 (en) * 2003-01-16 2008-06-03 Forword Input, Inc. System and method for continuous stroke word-based text input

Family Cites Families (101)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5305205A (en) * 1990-10-23 1994-04-19 Weber Maria L Computer-assisted transcription apparatus
US5128672A (en) * 1990-10-30 1992-07-07 Apple Computer, Inc. Dynamic predictive keyboard
DE69422323T2 (en) 1993-04-01 2000-06-21 Ibm Dynamic adjustment device for touch indicator buttons
US5469722A (en) * 1993-06-24 1995-11-28 Ellefsen; Robert J. Hasp for a cylinder lock
JP3546337B2 (en) * 1993-12-21 2004-07-28 ゼロックス コーポレイション User interface device for computing system and method of using graphic keyboard
US5748512A (en) 1995-02-28 1998-05-05 Microsoft Corporation Adjusting keyboard
JPH08249122A (en) * 1995-03-15 1996-09-27 Matsushita Electric Ind Co Ltd Display device
JPH08272787A (en) * 1995-03-29 1996-10-18 Sharp Corp Character processor
JPH0981320A (en) * 1995-09-20 1997-03-28 Matsushita Electric Ind Co Ltd Pen input type selection input device and method therefor
US5801941A (en) 1996-08-12 1998-09-01 International Business Machines Corporation Mobile client computer programmed to establish soft keyboard targeting sensitivity
JP4316687B2 (en) * 1995-11-07 2009-08-19 善也 加藤 Screen touch input device
KR100260760B1 (en) 1996-07-31 2000-07-01 모리 하루오 Information display system with touch panel
US5818451A (en) 1996-08-12 1998-10-06 International Busienss Machines Corporation Computer programmed soft keyboard system, method and apparatus having user input displacement
US5953541A (en) 1997-01-24 1999-09-14 Tegic Communications, Inc. Disambiguating system for disambiguating ambiguous input sequences by displaying objects associated with the generated input sequences in the order of decreasing frequency of use
US6073036A (en) 1997-04-28 2000-06-06 Nokia Mobile Phones Limited Mobile station with touch input having automatic symbol magnification function
US6049326A (en) 1997-05-12 2000-04-11 Siemens Information And Communication Networks, Inc. System and method for dual browser modes
US6803905B1 (en) * 1997-05-30 2004-10-12 International Business Machines Corporation Touch sensitive apparatus and method for improved visual feedback
CN100334530C (en) * 1997-09-25 2007-08-29 蒂吉通信系统公司 Reduced keyboard disambiguating systems
US5896321A (en) * 1997-11-14 1999-04-20 Microsoft Corporation Text completion system for a miniature computer
GB2332293A (en) 1997-12-11 1999-06-16 British Telecomm An Input Device
US7614008B2 (en) 2004-07-30 2009-11-03 Apple Inc. Operation of a computer with touch screen interface
EP1717684A3 (en) 1998-01-26 2008-01-23 Fingerworks, Inc. Method and apparatus for integrating manual input
JP2938420B2 (en) * 1998-01-30 1999-08-23 インターナショナル・ビジネス・マシーンズ・コーポレイション Function selection method and apparatus, storage medium storing control program for selecting functions, object operation method and apparatus, storage medium storing control program for operating objects, storage medium storing composite icon
KR100327209B1 (en) * 1998-05-12 2002-04-17 윤종용 Software keyboard system using the drawing of stylus and method for recognizing keycode therefor
US6212412B1 (en) * 1998-06-09 2001-04-03 Qualcomm Incorporated System and method for character case control in a wireless communication device
US6271835B1 (en) * 1998-09-03 2001-08-07 Nortel Networks Limited Touch-screen input device
JP2000112636A (en) * 1998-10-07 2000-04-21 Kanazawa Engineering Systems:Kk Kana (japanese syllabary) input device
US7712053B2 (en) * 1998-12-04 2010-05-04 Tegic Communications, Inc. Explicit character filtering of ambiguous text entry
US7679534B2 (en) * 1998-12-04 2010-03-16 Tegic Communications, Inc. Contextual prediction of user words and user actions
US6259436B1 (en) 1998-12-22 2001-07-10 Ericsson Inc. Apparatus and method for determining selection of touchable items on a computer touchscreen by an imprecise touch
US6643824B1 (en) 1999-01-15 2003-11-04 International Business Machines Corporation Touch screen region assist for hypertext links
GB2388938B (en) * 1999-02-22 2004-03-17 Nokia Corp A communication terminal having a predictive editor application
US7286115B2 (en) * 2000-05-26 2007-10-23 Tegic Communications, Inc. Directional input system with automatic correction
US7030863B2 (en) 2000-05-26 2006-04-18 America Online, Incorporated Virtual keyboard system with automatic correction
US7434177B1 (en) * 1999-12-20 2008-10-07 Apple Inc. User interface for providing consolidation and access
US6654733B1 (en) 2000-01-18 2003-11-25 Microsoft Corporation Fuzzy keyboard
US6597345B2 (en) * 2000-03-03 2003-07-22 Jetway Technologies Ltd. Multifunctional keypad on touch screen
US6714221B1 (en) * 2000-08-03 2004-03-30 Apple Computer, Inc. Depicting and setting scroll amount
US20020015064A1 (en) * 2000-08-07 2002-02-07 Robotham John S. Gesture-based user interface to multi-level and multi-modal sets of bit-maps
GB0019540D0 (en) * 2000-08-10 2000-09-27 Plasplugs Ltd Display hanger
JP4197220B2 (en) 2000-08-17 2008-12-17 アルパイン株式会社 Operating device
JP2002091677A (en) * 2000-09-14 2002-03-29 Sanyo Electric Co Ltd Input device and information terminal equipment
TW486657B (en) 2000-10-26 2002-05-11 Animeta Systems Inc Browser interface operation device and its browsing method
AU2002226886A1 (en) * 2000-11-09 2002-05-21 Change Tools, Inc. A user definable interface system, method and computer program product
JP2002222039A (en) * 2001-01-25 2002-08-09 Sony Computer Entertainment Inc Information input processing program, computer-readable recording medium with recorded information input processing program, program performing device performing information input processing program, and device and method for information input
US6677932B1 (en) 2001-01-28 2004-01-13 Finger Works, Inc. System and method for recognizing touch typing under limited tactile feedback conditions
US20020135615A1 (en) 2001-01-31 2002-09-26 Microsoft Corporation Overlaid display for electronic devices
US6570557B1 (en) 2001-02-10 2003-05-27 Finger Works, Inc. Multi-touch system and method for emulating modifier keys via fingertip chords
JP3597141B2 (en) * 2001-04-03 2004-12-02 泰鈞 温 Information input device and method, mobile phone and character input method of mobile phone
TW504916B (en) * 2001-04-24 2002-10-01 Inventec Appliances Corp Method capable of generating different input values by pressing a single key from multiple directions
US7168046B2 (en) 2001-04-26 2007-01-23 Lg Electronics Inc. Method and apparatus for assisting data input to a portable information terminal
US20050024341A1 (en) * 2001-05-16 2005-02-03 Synaptics, Inc. Touch screen with user interface enhancement
US7730401B2 (en) * 2001-05-16 2010-06-01 Synaptics Incorporated Touch screen with user interface enhancement
FI116591B (en) 2001-06-29 2005-12-30 Nokia Corp Method and apparatus for performing a function
EP1457864A1 (en) 2001-09-21 2004-09-15 International Business Machines Corporation INPUT APPARATUS, COMPUTER APPARATUS, METHOD FOR IDENTIFYING INPUT OBJECT, METHOD FOR IDENTIFYING INPUT OBJECT IN KEYBOARD, AND COMPUTER PROGRAM
US7254775B2 (en) 2001-10-03 2007-08-07 3M Innovative Properties Company Touch panel system and method for distinguishing multiple touch inputs
GB2380583A (en) 2001-10-04 2003-04-09 Ilam Samson Touch pad/screen for electronic equipment
US7113172B2 (en) * 2001-11-09 2006-09-26 Lifescan, Inc. Alphanumeric keypad and display system and method
US6604840B2 (en) 2001-11-27 2003-08-12 Marion H. Watson Lighting apparatus for a sign
US7083342B2 (en) 2001-12-21 2006-08-01 Griffin Jason T Keyboard arrangement
US20030197736A1 (en) * 2002-01-16 2003-10-23 Murphy Michael W. User interface for character entry using a minimum number of selection keys
JP4073215B2 (en) * 2002-01-28 2008-04-09 富士通株式会社 Character input device
US20030149978A1 (en) * 2002-02-07 2003-08-07 Bruce Plotnick System and method for using a personal digital assistant as an electronic program guide
JP2003271294A (en) * 2002-03-15 2003-09-26 Canon Inc Data input device, data input method and program
US7038659B2 (en) * 2002-04-06 2006-05-02 Janusz Wiktor Rajkowski Symbol encoding apparatus and method
US20030193481A1 (en) * 2002-04-12 2003-10-16 Alexander Sokolsky Touch-sensitive input overlay for graphical user interface
WO2003098417A2 (en) 2002-05-21 2003-11-27 Koninklijke Philips Electronics N.V. Object entry into an electronic device
FI20021162A0 (en) 2002-06-14 2002-06-14 Nokia Corp Electronic device and a method for administering its keypad
FI112119B (en) 2002-06-25 2003-10-31 Nokia Corp Touch screen control command interpreting method for electronic device e.g. mobile station, involves interpreting contact area larger than area before touch, as same area when area has been touched for release of touch
CN1672141A (en) * 2002-08-16 2005-09-21 姜润基 Method of inputting a character using a software keyboard
US8042044B2 (en) * 2002-11-29 2011-10-18 Koninklijke Philips Electronics N.V. User interface with displaced representation of touch area
US6927763B2 (en) * 2002-12-30 2005-08-09 Motorola, Inc. Method and system for providing a disambiguated keypad
US7103852B2 (en) 2003-03-10 2006-09-05 International Business Machines Corporation Dynamic resizing of clickable areas of touch screen applications
US20040183833A1 (en) 2003-03-19 2004-09-23 Chua Yong Tong Keyboard error reduction method and apparatus
CA2426867A1 (en) 2003-04-30 2004-10-30 Naviform Holdings Ltd. Customizable keyboard
US7057607B2 (en) * 2003-06-30 2006-06-06 Motorola, Inc. Application-independent text entry for touch-sensitive display
US20050190970A1 (en) 2004-02-27 2005-09-01 Research In Motion Limited Text input system for a mobile electronic device and methods thereof
US20050196256A1 (en) * 2004-03-08 2005-09-08 Fmc Technologies, Inc. Method and system for over-steer avoidance
US7571111B2 (en) * 2004-03-29 2009-08-04 United Parcel Service Of America, Inc. Computer system for monitoring actual performance to standards in real time
JP5132028B2 (en) * 2004-06-11 2013-01-30 三菱電機株式会社 User interface device
US7786980B2 (en) 2004-06-29 2010-08-31 Koninklijke Philips Electronics N.V. Method and device for preventing staining of a display device
US20060007174A1 (en) 2004-07-06 2006-01-12 Chung-Yi Shen Touch control method for a drag gesture and control module thereof
US7508324B2 (en) * 2004-08-06 2009-03-24 Daniel Suraqui Finger activated reduced keyboard and a method for performing text input
US7489306B2 (en) 2004-12-22 2009-02-10 Microsoft Corporation Touch screen accuracy
US8552984B2 (en) * 2005-01-13 2013-10-08 602531 British Columbia Ltd. Method, system, apparatus and computer-readable media for directing input associated with keyboard-type device
US7788248B2 (en) * 2005-03-08 2010-08-31 Apple Inc. Immediate search feedback
US20060244733A1 (en) 2005-04-28 2006-11-02 Geaghan Bernard O Touch sensitive device and method using pre-touch information
US20060246955A1 (en) * 2005-05-02 2006-11-02 Mikko Nirhamo Mobile communication device and method therefor
US8036878B2 (en) * 2005-05-18 2011-10-11 Never Wall Treuhand GmbH Device incorporating improved text input mechanism
US7886233B2 (en) * 2005-05-23 2011-02-08 Nokia Corporation Electronic text input involving word completion functionality for predicting word candidates for partial word inputs
US7737999B2 (en) * 2005-08-26 2010-06-15 Veveo, Inc. User interface for visual cooperation between text input and display device
US7873356B2 (en) * 2005-09-16 2011-01-18 Microsoft Corporation Search interface for mobile devices
US7694231B2 (en) * 2006-01-05 2010-04-06 Apple Inc. Keyboards for portable electronic devices
US7644054B2 (en) * 2005-11-23 2010-01-05 Veveo, Inc. System and method for finding desired results by incremental search using an ambiguous keypad with the input containing orthographic and typographic errors
DE102006037156A1 (en) * 2006-03-22 2007-09-27 Volkswagen Ag Interactive operating device and method for operating the interactive operating device
US9552349B2 (en) * 2006-08-31 2017-01-24 International Business Machines Corporation Methods and apparatus for performing spelling corrections using one or more variant hash tables
US7683886B2 (en) * 2006-09-05 2010-03-23 Research In Motion Limited Disambiguated text message review function
US7793228B2 (en) * 2006-10-13 2010-09-07 Apple Inc. Method, system, and graphical user interface for text entry with partial word display
US7957955B2 (en) * 2007-01-05 2011-06-07 Apple Inc. Method and system for providing word recommendations for text input
US8074172B2 (en) * 2007-01-05 2011-12-06 Apple Inc. Method, system, and graphical user interface for providing word recommendations
US8232973B2 (en) * 2008-01-09 2012-07-31 Apple Inc. Method, device, and graphical user interface providing word recommendations for text input

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5963671A (en) * 1991-11-27 1999-10-05 International Business Machines Corporation Enhancement of soft keyboard operations using trigram prediction
US5736974A (en) * 1995-02-17 1998-04-07 International Business Machines Corporation Method and apparatus for improving visibility and selectability of icons
US5748927A (en) * 1996-05-10 1998-05-05 Apple Computer, Inc. Graphical user interface with icons having expandable descriptors
US6169538B1 (en) * 1998-08-13 2001-01-02 Motorola, Inc. Method and apparatus for implementing a graphical user interface keyboard and a text buffer on electronic devices
US6359572B1 (en) * 1998-09-03 2002-03-19 Microsoft Corporation Dynamic keyboard
US6573844B1 (en) * 2000-01-18 2003-06-03 Microsoft Corporation Predictive keyboard
US7194699B2 (en) * 2003-01-14 2007-03-20 Microsoft Corporation Animating images to reflect user selection
US7382358B2 (en) * 2003-01-16 2008-06-03 Forword Input, Inc. System and method for continuous stroke word-based text input
US20040160419A1 (en) * 2003-02-11 2004-08-19 Terradigital Systems Llc. Method for entering alphanumeric characters into a graphical user interface
US20050093826A1 (en) * 2003-10-29 2005-05-05 Samsung Electronics Co., Ltd. Apparatus and method for inputting character using touch screen in portable terminal
US20070229476A1 (en) * 2003-10-29 2007-10-04 Samsung Electronics Co., Ltd. Apparatus and method for inputting character using touch screen in portable terminal
US20070257896A1 (en) * 2003-10-29 2007-11-08 Samsung Electronics Co. Ltd. Apparatus and Method for Inputting Character Using Touch Screen in Portable Terminal
US20070046641A1 (en) * 2005-09-01 2007-03-01 Swee Ho Lim Entering a character into an electronic device
US7443316B2 (en) * 2005-09-01 2008-10-28 Motorola, Inc. Entering a character into an electronic device

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10592100B2 (en) 2007-01-05 2020-03-17 Apple Inc. Method, system, and graphical user interface for providing word recommendations
US11112968B2 (en) 2007-01-05 2021-09-07 Apple Inc. Method, system, and graphical user interface for providing word recommendations
US11416141B2 (en) 2007-01-05 2022-08-16 Apple Inc. Method, system, and graphical user interface for providing word recommendations
US11079933B2 (en) 2008-01-09 2021-08-03 Apple Inc. Method, device, and graphical user interface providing word recommendations for text input
US11474695B2 (en) 2008-01-09 2022-10-18 Apple Inc. Method, device, and graphical user interface providing word recommendations for text input
US9280266B2 (en) 2010-11-12 2016-03-08 Kt Corporation Apparatus and method for displaying information as background of user interface
US10551987B2 (en) 2011-05-11 2020-02-04 Kt Corporation Multiple screen mode in mobile terminal
US8866762B2 (en) 2011-07-01 2014-10-21 Pixart Imaging Inc. Method and apparatus for arbitrating among contiguous buttons on a capacitive touchscreen
US9594465B2 (en) 2011-07-01 2017-03-14 Pixart Imaging, Inc. Method and apparatus for arbitrating among contiguous buttons on a capacitive touchscreen

Also Published As

Publication number Publication date
JP4981066B2 (en) 2012-07-18
US20100188358A1 (en) 2010-07-29
KR101110501B1 (en) 2012-02-09
DE112007000122T5 (en) 2009-01-29
WO2007082139A3 (en) 2008-05-29
US20070152978A1 (en) 2007-07-05
US7694231B2 (en) 2010-04-06
EP1969453A2 (en) 2008-09-17
US20100192086A1 (en) 2010-07-29
WO2007082139A2 (en) 2007-07-19
US20100177056A1 (en) 2010-07-15
JP2009522697A (en) 2009-06-11
KR20080091199A (en) 2008-10-09

Similar Documents

Publication Publication Date Title
US7694231B2 (en) Keyboards for portable electronic devices
US20070152980A1 (en) Touch Screen Keyboards for Portable Electronic Devices
US8918736B2 (en) Replay recommendations in a text entry interface
US7574672B2 (en) Text entry interface for a portable communication device
US7860536B2 (en) Telephone interface for a portable communication device
US11416141B2 (en) Method, system, and graphical user interface for providing word recommendations
US7956846B2 (en) Portable electronic device with content-dependent touch sensitivity
US7793228B2 (en) Method, system, and graphical user interface for text entry with partial word display
US7667148B2 (en) Method, device, and graphical user interface for dialing with a click wheel
US8179371B2 (en) Method, system, and graphical user interface for selecting a soft keyboard
US8605039B2 (en) Text input
US8610669B2 (en) Apparatus and method for inputting character using touch screen in portable terminal
JP4797104B2 (en) Electronic device and method for symbol input
US20080098331A1 (en) Portable Multifunction Device with Soft Keyboards
US20080167083A1 (en) Method, Device, and Graphical User Interface for Location-Based Dialing
JP2014120833A (en) Information processing device and program

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION