Handheld electronic device and associated distributed multi-display system

• US 10,004,985 B2
• Filed: 10/11/2015
• Issued: 06/26/2018
• Est. Priority Date: 10/14/2012
• Status: Active Grant

• Current Assignee: Neonode, Inc.
• Original Assignee: Neonode, Inc.
• Inventors: Holmgren, Stefan, Iyer, Sairam, Berglind, Richard, Nordstrm, Karl Erik Patrik, Sparf, Lars, Rosengren, Per, Rosengren, Erik, Karlsson, John, Eriksson, Thomas, Jubner, Alexander, Behdasht, Remo, Fellin, Simon, man, Robin Kjell, Shain, Joseph
• Primary Examiner(s): MYHR, JUSTIN L
• Application Number:
  US14/880,231
• Publication Number:
  US 20160067602A1
• Time in Patent Office: 989 Days
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/31 Communication aspects specific to video games, e.g. between several handheld game devices at close range
• CPC Class Codes: A63F 13/33 using wide area network [WAN] connections
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2009/245 visual
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32
• CPC Class Codes: G01J 1/0411 using focussing or collimating elements, i.e. lenses or mirrors; Aberration correction
• CPC Class Codes: G01J 1/42 using electric radiation detectors optical or mechanical part G01J1/04; by comparison with a reference light or electric value G01J1/10
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/48 Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 7/4811 common to transmitter and receiver
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/169 : the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/04809 : Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/021 : Arrangements integrating additional peripherals in a keyboard, e.g. card or barcode reader, optical scanner
• CPC Class Codes: G06F 3/0219 : Special purpose keyboards
• CPC Class Codes: G06F 3/0304 : Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/03547 : Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0421 : by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0428 : by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text

Abstract

A system including two handheld electronic devices, each device including a housing, a communicator mounted in the housing continuously receiving images from an internet server, a display mounted in the housing, and a sensor mounted in the housing and connected to the communicator, configured to detect sizes of objects in an adjacent detection zone, and including an analyzer recognizing the other handheld electronic device when the sensor detects that an object in the adjacent detection zone is as long as an edge of the housing, wherein the server receives outputs from the analyzer and partitions each of the images into two half-images, and continuously transmits one of each half-image to the handheld electronic device and the other of each half-image to the other handheld device, such that each full image spans the two device displays.

Claims

1. A system comprising two handheld electronic devices, each device comprising:

a housing;

a communicator mounted in said housing continuously receiving images from an internet server;

a display mounted in said housing rendering the received images; and

a sensor mounted in said housing and connected to said communicator, configured to detect sizes of objects in an adjacent detection zone, the sensor comprising an analyzer recognizing the other handheld electronic device when said sensor detects that an object in the adjacent detection zone is as long as an edge of said housing, wherein the server receives outputs from said analyzer and partitions each of the images into two half-images, and continuously transmits one of each half-image to the handheld electronic device and the other of each half-image to the other handheld device, such that each full image spans the two device displays.

2. The system of claim 1 wherein each said analyzer recognizes a user input gesture comprising translating an object in the sensor detection zone, and the server interprets the input gesture as a command to adjust a parameter of both handheld electronic devices, the parameter having a range of values.

3. The system of claim 2 wherein said analyzer determines two-dimensional locations of the object performing the gesture within the sensor detection zone, and wherein said communicator communicates the object locations to the server, and the server uses the locations to adjust the parameter.

4. The system of claim 1, wherein each said sensor comprises an array of alternating light emitters and light detectors distributed along an edge of said housing, and detects the size of the other handheld electronic device based on each sensor detecting light from the sensor on the other device.

5. The system of claim 4, wherein an activation pattern of at least one of the light emitters in each handheld electronic device indicates to the other handheld electronic device the location of the at least one emitter within the handheld electronic device, which then determines the relative orientation of the two handheld electronic devices to one another.

6. A distributed multi-display system comprising:

an internet server continuously streaming images to a plurality of handheld electronic devices, each device comprising;

a housing;

a communicator communicating with said server, and receiving images from said server;

a display mounted in said housing rendering the received images; and

a plurality of sensors mounted along edges of said housing and connected to said communicator, each sensor being configured to detect a size of an object in a respective detection zone adjacent to its edge and comprising an analyzer recognizing a neighboring handheld electronic device in the detection zone when the sensor detects that the object is at least as long as an edge of said housing, wherein said server receives outputs from said analyzer via said communicator, divides images into image portions according to the number of recognized handheld electronic devices, and transmits the image portions to respective ones of the recognized handheld electronic devices such that the full images span all of the device displays.

7. The distributed multi-display system of claim 6, wherein said server further determines the dimensions of each image portion based on the number of recognized handheld devices.

8. The distributed multi-display system of claim 6, wherein said server maps positions of the recognized devices relative to one another, and wherein said server is further configured to transmit the image portions to the handheld devices, according to the devices'"'"' positions relative to one another.

9. The distributed multi-display system of claim 8, wherein said analyzer identifies how neighboring handheld electronic device displays are rotated relative to one another, and sends the rotation information to said server via said communicator, and wherein said server rotates subsequent image portions based on the identified rotation to provide image orientation consistency across the neighboring device displays.

10. The distributed multi-display system of claim 6, comprising four of said handheld devices arranged as a 2×

2 grid.

11. The distributed multi-display system of claim 6, wherein said sensor further comprises an array of alternating light emitters and light detectors, and wherein neighboring ones of the handheld electronic devices directly handshake with each other and establish a communication channel in response to the detectors on one device detecting a specific light pattern from the emitters on a neighboring device.

12. The distributed multi-display system of claim 11, wherein locations of the light emitters within each device are indicated by corresponding activation patterns of those emitters, thus enabling said analyzers of neighboring devices to determine the neighboring devices'"'"' relative orientation to one another.

13. A distributed multi-display system comprising:

an internet server continuously streaming images to a contiguous plurality of handheld electronic devices, by dividing each image into image portions according to the number of handheld electronic devices, and transmitting each image portion to a respective one of the handheld electronic devices such that the full image spans the devices'"'"' displays; and

the plurality of handheld electronic devices receiving respective image portions from said server, each handheld electronic device comprising;

a housing;

a communicator communicating with said server;

a display mounted in said housing rendering image portions received from said server; and

at least one sensor mounted in said housing and connected to said communicator, being operative to identify how an adjacent handheld electronic device display is rotated relative to the handheld device, and sending the rotation information to said server via said communicator,

wherein said server rotates subsequent image portions based on the received rotation information to provide orientation consistency of the image portions across contiguous handheld electronic device displays.

14. The distributed multi-display system of claim 13, wherein said server further determines the dimensions of the image portions based on the number of handheld electronic devices.

15. The distributed multi-display system of claim 13, wherein said server is further configured to map the positions of said handheld electronic devices relative to one another, based on the sensor outputs, and wherein said server is further configured to transmit the image portions to said handheld electronic devices according to their positions relative to one another.

16. The distributed multi-display system of claim 13, wherein said at least one sensor comprises a one-dimensional array of alternating light emitters and light detectors that are activatable in an activation pattern, and wherein said at least one sensor of one handheld electronic device identifies how an adjacent handheld electronic device display is rotated, based on the sensor detectors of the one handheld electronic device detecting an activation pattern of the light emitters on the adjacent handheld electronic device.