METHOD AND SYSTEM FOR AN AUTOMATIC SENSING, ANALYSIS, COMPOSITION AND DIRECTION OF A 3D SPACE, SCENE, OBJECT, AND EQUIPMENT

• US 20160088286A1
• Filed: 09/18/2015
• Published: 03/24/2016
• Est. Priority Date: 09/19/2014
• Status: Abandoned Application

• Current Assignee: Hamish Forsythe
• Original Assignee: Hamish Forsythe
• Inventors: FORSYTHE, HAMISH, Cecil, Alexander
• Application Number:
  US14/858,901
• Publication Number:
  US 20160088286A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/67 by interactive indications to the user
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

Method and system for automatic composition and orchestration of a 3D space or scene using networked devices and computer vision to bring ease of use and autonomy to a range of compositions. A scene, its objects, subjects and background are identified and classified, and relationships and behaviors are deduced through analysis. Compositional theories are applied, and context attributes (for example location, external data, camera metadata, and the relative positions of subjects and objects in the scene) are considered automatically to produce optimal composition and allow for direction of networked equipment and devices. Events inform the capture process, for example, a video recording initiated when a rock climber waves her hand, an autonomous camera automatically adjusting to keep her body in frame throughout the sequence of moves. Model analysis allows for direction, including audio tones to indicate proper form for the subject and instructions sent to equipment ensure optimal scene orchestration.

Claims

1. A method, comprising:

Capturing a 2D image in a specific format of an object, subject, and scene using a device;

Sensing an object, subject, and scene automatically and continuously using the device;

Analyzing the 2D image of the object, subject, and scene captured to determine the most relevant composition and direction model;

Transforming an object, subject, and scene into a 3D model using existing reference composition/architecture model; and

Storing the 3D model of the scene in a database for use and maintaining it in a feedback loop.

2. The method of claim 1, further comprising:

Performing continuous contextual analysis of an image and its resulting 3D model to provide an update to subsequent 3D modeling processes; and

Dynamically updating and responding to contextual analytics performed.

3. The method of claim 2, further comprising:

Coordinating accurate tracking of objects and subjects in a scene by orchestrating autonomous equipment movements using a feedback loop.

4. The method of claim 3, further comprising:

Controlling the direction of a scene and its subjects via devices using a feedback loop

5. The method of claim 4, further comprising:

Creating and dynamically modifying in real-time the 2D or 3D model for the subject, object, scene, and equipment in any spatial orientation and Providing immediate feedback in a user interface.

6. The method of claim 1, wherein the device is at least one of a camera, wearable device, desktop computer, laptop computer, phone, tablet, and other mobile computer.

7. A system, comprising:

A processing unit that can exist on a user device, on-premise, or as an off-premise service to house the following modules;

A sensing module that can understand the subjects and context of a scene over time via models;

An analytics module that can analyze sensed scenes and subjects to determine the most relevant composition and direction models or create them if necessary;

A composition/architecture module that can simultaneously store the direction of multiple subjects or objects of a scene according to one or more composition models;

A direction/control module that can provide direction and control to each subject, object, and equipment individually and relative to a scene model; and

A database that can store models for use and maintain them in a feedback loop with the above modules.