Display apparatus and contents display method

• US 9,594,435 B2
• Filed: 10/17/2012
• Issued: 03/14/2017
• Est. Priority Date: 10/17/2011
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Park, Ji Hyung, Yoon, Dong Wook, Lee, Joong Ho
• Primary Examiner(s): Patel, Sanjiv D
• Application Number:
  US14/351,797
• Publication Number:
  US 20140232648A1
• Time in Patent Office: 1,609 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2380/14 Electronic books and readers

Abstract

Disclosed are a display apparatus and a contents display method. The display apparatus includes: a mobile device tracing information processing unit for receiving position information or orientation information of a mobile device and generating tracing information of the mobile device based on the received position information or orientation information of the mobile device; a gesture processing unit for receiving input information of the mobile device and generating gesture information to change an output format of contents by using the received input information; a rendering unit for generating a predetermined contents image based on the tracing information of the mobile device and the gesture information; and a display unit for displaying the generated contents image.

Claims

1. A display apparatus, comprising:

a mobile device tracing information processing unit for receiving position information or orientation information of a mobile device and generating tracing information of the mobile device based on the received position information or orientation information of the mobile device;

a gesture processing unit for receiving input information of the mobile device and generating gesture information to change an output format of contents by using the received input information;

a rendering unit for generating a predetermined contents image based on the tracing information of the mobile device and the gesture information; and

a display unit for displaying the generated contents image,

wherein the predetermined contents is an e-book contents including a plurality of pages,

wherein the gesture processing unit includes a continuous zooming ratio determination unit for generating first gesture information which changes a size of the e-book contents image at a first predetermined ratio based on the position information of the mobile device and eyes of the user who uses the mobile device,

wherein the gesture processing unit includes a bookmark determination unit for storing a third page of the e-book contents image by receiving predetermined third input information with respect to the mobile device, and, after the page is stored, for generating third gesture information to display the stored third page by receiving predetermined fourth input information,

wherein a plurality of pages can be stored and subsequently displayed by the bookmark determination unit receiving respective predetermined fourth input information, and

wherein the gesture processing unit includes a book flipping-through determination unit for shifting from a general mode to an e-book flipping-through mode by receiving predetermined incline input information with respect to the mobile device, and generating fourth gesture information to continuously turn over a plurality pages of the e-book contents image by receiving predetermined fifth input information.

2. The display apparatus according to claim 1, further comprising a mobile device tracing unit for generating the position information of the mobile device by tracing a position of the mobile device from a position of the display device.

3. The display apparatus according to claim 2, wherein the orientation information of the mobile device is generated from at least one of an acceleration sensor, a magnetic sensor and a gyroscope in the mobile device.

4. The display apparatus according to claim 3, wherein the mobile device tracing information processing unit includes:

a noise attenuation unit for removing noise of the orientation information of the mobile device; and

a 3-dimensional transformation information calculating unit for generating 3-dimensional rotational transformation information about movement from an initial orientation of the mobile device to a present orientation based on the orientation information of the mobile device from which noise is removed,

wherein the tracing information of the mobile device is generated based on the position information and the generated 3-dimensional rotational transformation information.

5. The display apparatus according to claim 2, wherein the mobile device tracing information processing unit includes:

a 3-dimensional transformation information calculating unit for generating 3-dimensional rotational transformation information by calculating movement from an initial orientation of the mobile device to a present orientation based on the orientation information of the mobile device; and

a noise attenuation unit for removing noise of the 3-dimensional rotational transformation information,

wherein the tracing information of the mobile device is generated based on the position information and the generated 3-dimensional rotational transformation information.

6. The display apparatus according to claim 1, wherein the rendering unit includes:

a 3-dimensional image transforming unit for generating 3-dimensional image transformation information by transforming the predetermined contents based on the tracing information of the mobile device and the gesture information;

a rendering control unit for controlling properties of the contents to be rendered and rendering characteristics; and

a rendering calculation unit for generating and transmitting the contents image based of the 3-dimensional image transformation information, the properties of the contents and the rendering characteristics.

7. The display apparatus according to claim 1, wherein the gesture processing unit includes a page-turning determination unit for receiving predetermined second input information with respect to the mobile device and generating second gesture information where a first page of the e-book contents image is shifted to a second page of the e-book contents image.

8. The display apparatus according to claim 1, wherein the book flipping-through determination unit shifts to the general mode again by receiving predetermined sixth input information in the shifted e-book flipping-through mode.

9. The display apparatus according to claim 1, further comprising a feedback control unit for controlling a feedback unit of the mobile device to generate feedback control information, which generates sound or vibration feedback, based on the gesture information, and transmit the feedback control information to the mobile device.

10. The display apparatus according to claim 1, further comprising:

a sound feedback unit for generating a sound feedback according to the gesture information; and

a feedback control unit for controlling the feedback unit by generating feedback control information which controls the feedback unit of the display device to generate a sound feedback, based on the gesture information.

11. A contents display system, comprising:

a display apparatus which includes;

a mobile device tracing information processing unit for receiving movement information of a mobile device and generating tracing information of the mobile device based on the received movement information of the mobile device;

a gesture processing unit for receiving input information with respect to the mobile device and changing an output format of contents by using the received input information;

a rendering unit for generating predetermined contents image, wherein the predetermined contents is an e-book contents including a plurality of pages, based on the tracing information of the mobile device and the gesture information;

a display unit for displaying the generated contents image; and

the mobile device which communicates with the display apparatus,

wherein the gesture processing unit includes a continuous zooming ratio determination unit for generating first gesture information which changes a size of the e-book contents image at a first predetermined ratio based on the position information of the mobile device and eyes of the user who uses the mobile device,

wherein the gesture processing unit includes a bookmark determination unit for storing a third page of the e-book contents image by receiving predetermined third input information with respect to the mobile device, and, after the me is stored, for generating third gesture information to display the stored third page by receiving predetermined fourth input information,

wherein a plurality of pages can be stored and subsequently displayed by the bookmark determination unit receiving respective predetermined fourth input information, and

wherein the gesture processing unit includes a book flipping-through determination unit for shilling from a general mode to an e-book flipping-through mode by receiving predetermined incline input information with respect to the mobile device, and generating fourth gesture information to continuously turn over a plurality pages of the e-book contents image by receiving predetermined fifth input information.

12. A contents display method, comprising:

receiving position information or orientation information of a mobile device;

generating tracing information of the mobile device based on the received position information or orientation information of the mobile device;

receiving input information with respect to the mobile device;

generating gesture information to change an output format of contents by using the received input information;

generating a predetermined contents image based on the tracing information of the mobile device and the gesture information; and

displaying the generated contents image,

wherein the predetermined contents is an e-book contents including a plurality of pages,

wherein said generating of gesture information further includes generating first gesture information to change a size of the e-book contents image according to a first predetermined ratio based on position information of the mobile device and eyes of the user who uses the mobile device,

wherein said generating of gesture information further includes storing a third page of the e-book contents image by receiving predetermined third input information with respect to the mobile device, and, after the page is stored, generating third gesture information to display the stored third page by receiving predetermined fourth input information,

wherein a plurality of pages are stored and subsequently displayed,

wherein said generating of gesture information further includes shifting to an e-book flipping-through mode by receiving predetermined incline input information with respect to the mobile device, and generating fourth gesture information to continuously turn over a plurality pages of the e-book contents image by receiving predetermined fifth input information.

13. The contents display method according to claim 12, wherein said displaying of the contents image further includes displaying the contents image to overlay the mobile device.

14. The contents display method according to claim 12, wherein said receiving of position information or orientation information further includes receiving movement information of the mobile device, which is generated by tracing a position of the mobile device from a position of the display device.

15. The contents display method according to claim 12, wherein said generating of tracing information of the mobile device further includes:

generating 3-dimensional rotational transformation information by calculating movement information from an initial orientation of the mobile device to a present orientation based on the orientation information of the mobile device;

removing noise of the tracing information of the mobile device; and

generating tracing information of the mobile device based on the position information and the generated 3-dimensional rotational transformation information.

16. The contents display method according to claim 12, wherein said generating of a predetermined contents image further includes:

generating 3-dimensional image transformation information by transforming the predetermined contents based on the tracing information of the mobile device and the gesture information;

controlling properties of the contents to be rendered and rendering characteristics; and

generating and transmitting the contents image based on the 3-dimensional image transformation information, the properties of the contents and the rendering characteristics.

17. The contents display method according to claim 12, wherein said generating of gesture information further includes receiving predetermined second input information with respect to the mobile device and generating second gesture information where a first page of the e-book contents image is shifted to a second page of the e-book contents image.

18. The contents display method according to claim 12, further comprising controlling a feedback unit of the mobile device to generate feedback control information, which generates sound or vibration feedback, based on the gesture information, and transmit the feedback control information to the mobile device.

19. The contents display method according to claim 12, further comprising controlling by generating feedback control information which controls the display device to generate a sound feedback, based on the gesture information.