Virtual data objects

• US 10,372,692 B2
• Filed: 01/25/2016
• Issued: 08/06/2019
• Est. Priority Date: 01/25/2015
• Status: Active Grant

• Current Assignee: Iguazio Systems Ltd.
• Original Assignee: Iguazio Systems Ltd.
• Inventors: Haviv, Yaron, Modai, Ori, Nissan-Messing, Orit
• Primary Examiner(s): Lewis, Cheryl
• Application Number:
  US15/006,105
• Publication Number:
  US 20160217161A1
• Time in Patent Office: 1,289 Days
• CPC Class Codes: G06F 12/0897 with two or more cache hierarchy levels with multilevel cache hierarchies G06F12/0811
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 2212/60 Details of cache memory
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Systems, methods, and related technologies for application-centric object storage and virtual data objects are described. In certain aspects, a processing device can process a first data object in a first format to generate a virtual data object, the virtual data object including a representation of the first data object in a second format. The processing device can receive a request with respect to the first data object from an application that is configured with respect to the second format. The processing device can complete the request with respect to the first data object and the application via the virtual data object.

Claims

1. A method comprising:

receiving a streaming data object in a streaming data format;

receiving a request to access the streaming data object in a database format;

based on a receipt of the request to access the streaming data object in the database format, processing, by a processing device, the streaming data object to generate a virtual data object, the virtual data object comprising a representation of a portion of the streaming data object in the database format;

receiving a command with respect to the streaming data object as represented in the virtual data object from an application configured with respect to the database format; and

using the virtual data object, executing the command, as received from the application configured with respect to the database format, with respect to the streaming data object.

2. The method of claim 1, wherein processing the streaming object in the streaming data format comprises:

based on a receipt of the streaming data object in the streaming data format, processing the streaming data object in the streaming data format to generate the virtual data object.

3. The method of claim 1, further comprising receiving a command to initialize the streaming data object in the database format.

4. The method of claim 3, wherein processing the streaming data object in the streaming data format comprises:

based on a receipt of the command to initialize the streaming data object in the database format, processing the streaming data object in the streaming data format to generate the virtual data object.

5. The method of claim 1, wherein the virtual data object comprises an object identifier and one or more pointers to one or more elements of the streaming data object.

6. The method of claim 1, wherein the virtual data object comprises an object identifier and one or more parameters that pertain to one or more elements of the streaming data object.

7. The method of claim 1, wherein the virtual data object comprises an object identifier and one or more elements of the streaming data object.

8. The method of claim 1, wherein the virtual data object comprises an object identifier, one or more pointers to one or more elements of the streaming data object, and one or more pointers to one or more elements of a second data object.

9. The method of claim 1, wherein the virtual data object comprises an object identifier, one or more parameters that pertain to one or more elements of the streaming data object, and one or more parameters that pertain to one or more elements of a second data object.

10. The method of claim 1, wherein the virtual data object comprises an object identifier, one or more elements of the streaming data object, and one or more elements of a second data object.

11. The method of claim 1, wherein the virtual data object comprises an object identifier, the streaming data object, and a second data object.

19. The method of claim 1, wherein the virtual data object defines a view of the streaming data object.

12. A system comprising:

a processing device; and

a memory coupled to the processing device and storing instructions that, when executed by the processing device, cause the system to perform operations comprising;

receiving a streaming data object in a streaming data format;

receiving a request to access the streaming data object in a database format;

based on a receipt of the request to access the streaming data object in the database format, processing the streaming data object to generate a virtual data object, the virtual data object comprising a representation of the streaming data object in the database format;

receiving a command with respect to the streaming data object as represented in the virtual data object from an application configured with respect to the database format; and

using the virtual data object, executing the command, as received from the application configured with respect to the database format, with respect to the streaming data object.

13. The system of claim 12, wherein processing the streaming data object in the streaming data format comprises processing the streaming data object in the streaming data format based on a receipt of the streaming data object in the streaming data format, to generate the virtual data object.

14. The system of claim 12, wherein the virtual data object comprises an object identifier and one or more pointers to one or more elements of the streaming data object.

15. The system of claim 12, wherein the virtual data object comprises an object identifier, one or more elements of the streaming data object, and one or more elements of a second data object.

16. The system of claim 12, wherein receiving the streaming data object comprises receiving a stream of sensor inputs.

17. The system of claim 12, wherein processing the streaming data object comprises processing the streaming data object to generate (a) a first virtual data object that corresponds to a first portion of the streaming data object and (b) a second virtual data object that corresponds to a second portion of the streaming data object.

18. A non-transitory computer-readable medium having instructions encoded thereon that, when executed by a processing device, cause the processing device to perform operations comprising:

receive a streaming data object in a streaming data format;

receiving a request to access a first chronological interval of the streaming data object;

based on a receipt of the request to access the first chronological interval of the streaming data object, processing the streaming data object to generate a virtual data object, the virtual data object comprising a representation of the first chronological interval of the streaming data object in the second data format;

receiving a command with respect to the streaming data object as represented in the virtual data object from an application configured with respect to the second data format; and

using the virtual data object, executing the command, as received from the application configured with respect to the second data format, with respect to the streaming data object.

20. The non-transitory computer-readable medium of claim 18, wherein the virtual data object defines a virtual file that corresponds to a portion of the streaming data object.