Information processing apparatus, information processing method, program for implementing information processing method, information processing system, and method for information processing system

• US 7,487,151 B2
• Filed: 11/15/2004
• Issued: 02/03/2009
• Est. Priority Date: 12/02/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamamoto, Noriyuki
• Primary Examiner(s): Holmes; Michael B
• Application Number:
  US10/987,158
• Publication Number:
  US 20050165782A1
• Time in Patent Office: 1,541 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/45 Clustering; Classification
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

An information processing apparatus selects a proper content that well matches preference of a user and recommends it. A matrix calculator acquires M (one or more) feature vectors CCV whose elements are given by weight values assigned to a total of N (two or more) pieces of content meta information and context information. The matrix calculator produces a matrix CCM whose columns are given by the M feature vectors CCV and converts it into an approximate matrix CCM* by modifying the weight values of the respective elements of the M feature vectors CCV such that correlations of elements among the M feature vectors CCV are emphasized. Based on the approximate matrix CCM*, a user preference vector (UPV) generator produces a user preference vector UPV*. A matching unit calculates similarity between the user preference vector UPV* and a feature vector CCV produced from new content meta information or context information.

Claims

1. An information processing apparatus comprising:

modification means for acquiring M information sets each including N pieces of individual information and modifying at least partially the N pieces of individual information of each of the M information sets such that correlations among the N pieces of individual information are emphasized, where N is an integer equal to or greater than 2 and M is an integer equal to or greater than 1;

generation means for generating a reference information set including N pieces of individual information for use as a reference in a calculation of similarity, from the M information sets each including N pieces of individual information modified by the modification means; and

similarity calculation means for acquiring, as a comparative information set, a new information set including N individual information elements and calculating the similarity of the comparative information set with respect to the reference information set produced by the generation means.

2. An image processing apparatus according to claim 1, wherein the N pieces of individual information of each of the M information sets acquired via the modification means includes at least one or more pieces of content meta information indicating a particular content or a part of the particular content or indicating an attribute of the content and also includes at least one or more pieces of context information indicating an internal or external state in which the content was used by the user.

3. An image processing apparatus according to claim 2, wherein the context information includes a value or a normalized value thereof assigned as a weight to location information indicating a spatial location of the user.

4. An image processing apparatus according to claim 2, wherein the context information includes a value or a normalized value assigned as a weight to emotion information indicating emotion of the user.

5. An image processing apparatus according to claim 2, wherein the context information includes a value acquired by the user or by a sensor disposed in the vicinity of the user or a normalized value thereof.

6. An image processing apparatus according to claim 2, wherein the modification means includes:

vectorization means for, each time a content is used by the user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information; and

matrix generation means for, when M feature vectors are produced by the vectorization means, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized, wherein

the generation means produces a reference vector including N elements as the reference information set, based on the M columns of the second matrix produced by the matrix generation means; and

the similarity calculation means produces a comparative vector including N elements from the comparative information set and calculates the similarity between the comparative vector and the reference vector produced by the generation means.

7. An image processing apparatus according to claim 6, wherein the generation means selects at least one or more columns of the M columns of the second matrix and adds together elements in each row over all selected columns thereby producing the reference vector.

8. An image processing apparatus according to claim 7, wherein the generation means selects, from the M columns of the second matrix, columns having high similarity to each other in terms of an element corresponding to context information and adds together elements in each row over all selected columns thereby producing the reference vector.

9. An image processing apparatus according to claim 7, wherein the generation means selects, from the M columns of the first matrix, columns having high similarity to each other in terms of an element corresponding to context information, and further selects, from the second matrix, columns corresponding to the selected columns of the first matrix and adds together elements in each row over all columns selected from the second matrix thereby producing the reference vector.

10. An image processing apparatus according to claim 6, wherein the generation means produces reference vectors by directly employing the respective M columns of the second matrix produced by the matrix generation means.

11. An image processing apparatus according to claim 6, wherein the vectorization means assigns the weight value to each of the N pieces of individual information such that the weight value is equal to the number of times that a corresponding piece of individual information appears in the content or equal to a normalized value thereof.

12. An image processing apparatus according to claim 6, wherein the vectorization means produces a feature vector whose elements are respectively given by products of the weight values assigned to the respective N pieces of individual information and N variable coefficients.

13. An image processing apparatus according to claim 6, wherein the similarity calculation means acquires, as the comparative information, an information set including N pieces of individual information associated with a new content, assigns a weight value to each of the N pieces of individual information, produces a comparative vector whose elements are given by the weight values assigned to the N pieces of individual information, and calculates the similarity between the comparative vector and the reference vector, and wherein

the image processing apparatus further comprises presentation means for, if the similarity calculated by the similarity calculation means is equal to or higher than a threshold value, presenting the new content as a recommended content to the user.

14. An information processing apparatus according to claim 13, wherein when the presentation means presents the recommended content to the user, the presentation means also presents a reason for the recommendation based on the reference vector.

15. An information processing apparatus according to claim 13, wherein the similarity calculation means calculates the similarity by:

before the comparative vector is produced, further acquiring an information set including N pieces of individual information associated with a content being currently used by the user;

assigning weight values to the respective N pieces of individual information, and producing a feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information for the content being currently used by the user;

calculating similarity between the produced feature vector and one or more reference vectors produced by the generation means;

producing a new reference vector based on a reference vector having similarity equal to or higher than a threshold value; and

after the comparative vector is produced, calculating similarity between the reference vector and the comparative vector.

16. An information processing apparatus according to claim 13, wherein the similarity calculation means calculates the similarity by:

before the comparative vector is produced, further acquiring one or more pieces of context information indicating a current context in which the user is;

assigning weight values to the respective pieces of context information;

producing a current context feature vector whose elements corresponding to the respective pieces of context information are given by the assigned weight values and the other elements are given by 0;

calculating similarity between the current context feature vector and one or more reference vectors produced by the generation means;

producing a new reference vector based on a reference vector having similarity equal to or higher than a threshold value; and

after the comparative vector is produced, calculating similarity between the reference vector and the comparative vector.

17. An image processing apparatus according to claim 6, wherein the matrix generation means produces the second matrix by:

decomposing the first matrix into the product of a first component matrix with N rows and N columns, a second component matrix with N rows and M columns, and a transpose of a third component matrix with M rows and M columns;

producing a fourth component matrix with N rows and K columns from the first component matrix (where K is an integer less than the rank of the first matrix), producing a fifth component matrix with K rows and K columns from the second component matrix, and producing a sixth component matrix with M rows and K columns from the third component matrix; and

producing the second matrix given by the product of the fourth component matrix, the fifth component matrix, and the transpose of the sixth component matrix.

18. An information processing apparatus according to claim 17, wherein the similarity calculation means calculates the similarity by acquiring, as the comparative information, an information set including N pieces of individual information associated with a new content, assigning weight values to the respective N pieces of individual information, producing the comparative feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information, converting the comparative vector and the reference vector into vectors whose dimension is reduced to K by multiplying the comparative vector and the reference vector by the transpose of the fourth component matrix from left, and calculating similarity between the comparative vector with dimensions reduced to K and the reference vector with dimensions reduced to K, and wherein

the image processing apparatus further comprises presentation means for, if the similarity calculated by the similarity calculation means is equal to or higher than a threshold value, presenting the new content as a recommended content to the user.

19. An information processing apparatus according to claim 18, wherein when the presentation means presents the recommended content to the user, the presentation means also presents a reason for the recommendation based on the reference vector.

20. An information processing apparatus according to claim 18, wherein the similarity calculation means calculates the similarity by:

before the comparative vector is produced, further acquiring an information set including N pieces of individual information associated with a content being currently used by the user, assigning weight values to the respective N pieces of individual information, and producing a feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information for the content being currently used by the user;

converting the produced feature vector and one or more reference vectors into vectors whose dimension is reduced to K by multiplying the feature vector and the one or more reference vectors by the transpose of the fourth component matrix from left;

calculating similarity between the feature vector with dimensions reduced to K and the one or more reference vectors with dimensions reduced to K, and producing a new K-dimensional reference vector based on a reference vector with reduced dimensions having similarity equal to or higher than a threshold value;

after the comparative vector is produced, converting the comparative vector into a vector whose dimension is reduced to K by multiplying the comparative vector by the transpose of the fourth component matrix from left, and

calculating similarity between the new K-dimensional reference vector and the comparative vector with dimensions reduced to K.

21. An information processing apparatus according to claim 18, wherein the similarity calculation means calculates the similarity by:

before the comparative vector is produced, further acquiring one or more pieces of context information indicating a current context in which the user is, assigning weight values to the respective pieces of context information, and producing a current context feature vector whose elements corresponding to the respective pieces of context information are given by the assigned weight values and the other elements are given by 0;

converting the current context feature vector and one or more reference vectors produced by the generation means into vectors whose dimension is reduced to K by multiplying the current context feature vector and the one or more reference vectors by the transpose of the fourth component matrix from left;

calculating similarity between the current context feature vector with dimensions reduced to K and the one or more reference vectors with dimensions reduced to K, and producing a new K-dimensional reference vector based on a reference vector with reduced dimensions having similarity equal to or higher than a threshold value;

after the comparative vector is produced, converting the comparative vector into a vector whose dimension is reduced to K by multiplying the comparative vector by the transpose of the fourth component matrix from left; and

calculating similarity between the new K-dimensional reference vector and the comparative vector with dimensions reduced to K.

22. An image processing apparatus according to claim 6, wherein each time a feature vector is produced, the vectorization means classifies the produced feature vector as either a positive-preference feature vector indicating a positive preference factor of the user or a negative-preference feature vector indicating a negative preference factor of the user.

23. An information processing apparatus according to claim 22, wherein the matrix generation means produces the second matrix using the M positive-preference feature vectors produced by the vectorization means.

24. An information processing apparatus according to claim 23, further comprising:

learning means for producing a classifier by learning classification using, as training data, columns of the second matrix produced by the matrix generation means; and

classification means for making classification by acquiring an information set including N pieces of individual information associated with a new content, assigning weight values to the respective N pieces of individual information, producing a feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information for the new content, and classifying, using the classifier produced by the learning means, the feature vector associated with the new content as either a positive-preference feature vector or a negative-preference feature vector.

25. An information processing apparatus according to claim 24, wherein the similarity calculation means employs as the comparative vector the feature vector classified by the classification means as the positive-preference feature vector, and calculates similarity between the comparative vector and the reference vector.

26. An information processing apparatus according to claim 23, further comprising registration means for selecting one or more particular elements of columns of the second matrix produced by the matrix generation means, and registers a word identifying each selected element as a word of interest to the user or a word of non-interest to the user in a personal dictionary of the user.

27. An information processing apparatus according to claim 23, wherein:

when M negative-preference feature vectors are produced by the vectorization means, the matrix generation means further produces a second matrix using the M negative-preference feature vectors;

the generation means produces a first reference vector from the second matrix produced by the matrix generation means based on the M positive-preference feature vectors produced by the matrix generation means, and further produces a second reference vector from the second matrix produced by the matrix generation means based on the negative-preference feature vectors; and

the similarity calculation means calculates first similarity between the comparative vector and the first reference vector and also calculates second similarity between the comparative vector and the second reference vector.

28. An information processing apparatus according to claim 27, wherein:

the similarity calculation means acquires, as the comparative information, an information set including N pieces of individual information associated with a new content, assigns a weight value to each of the N pieces of individual information, produces a comparative vector whose elements are given by the weight values assigned to the N pieces of individual information, and calculates the first similarity and the second similarity; and

the information processing apparatus further comprises presentation means for determining whether the new content is suitable as a content to be recommended based on the first similarity and the second similarity calculated by the similarity calculation means, and, if the new content is determined to be suitable, presenting the new content as the recommended content to the user.

29. An information processing apparatus according to claim 28, wherein when the presentation means presents the recommended content to the user, the presentation means also presents a reason for the recommendation based on the reference vector.

30. An information processing apparatus according to claim 28, wherein if the first similarity is equal to or higher than a first threshold value and the second similarity is lower than a second threshold value, the presentation means determines that the new content is suitable as the content to be recommended and presents the new content as the recommended content to the user.

31. An information processing apparatus according to claim 28, wherein if the first similarity is equal to or higher than the second similarity and the second similarity is lower than a threshold value, the presentation means determines that the new content is suitable as the content to be recommended and presents the new content as the recommended content to the user.

32. An information processing method comprising:

a modification step of acquiring M information sets each including N pieces of individual information and modifying at least partially the N pieces of individual information of each of the M information sets such that correlations among the N pieces of individual information are emphasized, where N is an integer equal to or greater than 2 and M is an integer equal to or greater than 1;

a generation step of generating a reference information set including N pieces of individual information for use as a reference in a calculation of similarity, from the M information sets each including N pieces of individual information modified in the modification step; and

a similarity calculation step of acquiring, as a comparative information set, a new information set including N individual information elements and calculating the similarity of the comparative information set with respect to the reference information set produced in the generation step.

33. A program executed by a computer, comprising:

a modification step of acquiring M information sets each including N pieces of individual information and modifying at least partially the N pieces of individual information of each of the M information sets such that correlations among the N pieces of individual information are emphasized, where N is an integer equal to or greater than 2 and M is an integer equal to or greater than 1;

a generation step of generating a reference information set including N pieces of individual information for use as a reference in a calculation of similarity, from the M information sets each including N pieces of individual information modified in the modification step; and

a similarity calculation step of acquiring, as a comparative information set, a new information set including N individual information elements and calculating the similarity of the comparative information set with respect to the reference information set produced in the generation step.

34. An information processing apparatus comprising:

vectorization means for, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

matrix generation means for, when M feature vectors are produced by the vectorization means, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized;

reference vector generation means for producing a reference vector including N elements for use as a reference in calculation of similarity, from the M columns of the second matrix produced by the matrix generation means;

candidate acquisition means for, each time a reference vector is produced by the reference vector generation means, acquiring a candidate for a content to be recommended to a user, based on the reference vector;

similarity calculation means for acquiring an information set including N pieces of individual information associated with a new content, assigning a weight value to each of the N pieces of individual information, produces a comparative vector whose elements are given by the weight values assigned to the N pieces of individual information, and calculates the similarity between the comparative vector and one or more reference vectors produced by the reference vector generation means; and

presentation means for selecting, from candidate contents acquired by the candidate acquisition means, a candidate content corresponding to a reference vector whose similarity calculated by the similarity calculation means is equal to or higher than a threshold value and presenting the selected candidate content as a recommended content to the user.

35. An information processing method comprising:

a vectorization step of, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

a matrix generation step of, when M feature vectors are produced in the vectorization step, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized;

a reference vector generation step of producing a reference vector including N elements for use as a reference in calculation of similarity, from the M columns of the second matrix produced in the matrix generation step;

a candidate acquisition step of, each time a reference vector is produced in the reference vector generation step, acquiring a candidate for a content to be recommended to a user, based on the reference vector;

a similarity calculation step of acquiring an information set including N pieces of individual information associated with a new content, assigning a weight value to each of the N pieces of individual information, produces a comparative vector whose elements are given by the weight values assigned to the N pieces of individual information, and calculates the similarity between the comparative vector and one or more reference vectors produced in the reference vector generation step; and

a presentation step of selecting, from candidate contents acquired in the candidate acquisition step, a candidate content corresponding to a reference vector whose similarity calculated by the similarity calculation means is equal to or higher than a threshold value and presenting the selected candidate content as a recommended content to the user.

36. A program executed by a computer, comprising:

a vectorization step of, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

a matrix generation step of, when M feature vectors are produced in the vectorization step, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized;

a reference vector generation step of producing a reference vector including N elements for use as a reference in calculation of similarity, from the M columns of the second matrix produced in the matrix generation step;

a candidate acquisition step of, each time a reference vector is produced in the reference vector generation step, acquiring a candidate for a content to be recommended to a user, based on the reference vector;

a similarity calculation step of acquiring an information set including N pieces of individual information associated with a new content, assigning a weight value to each of the N pieces of individual information, produces a comparative vector whose elements are given by the weight values assigned to the N pieces of individual information, and calculates the similarity between the comparative vector and one or more reference vectors produced in the reference vector generation step; and

a presentation step of selecting, from candidate contents acquired in the candidate acquisition step, a candidate content corresponding to a reference vector whose similarity calculated by the similarity calculation means is equal to or higher than a threshold value and presenting the selected candidate content as a recommended content to the user.

37. An information processing apparatus comprising:

vectorization means for, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

matrix generation means for, when M feature vectors are produced by the vectorization means, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized; and

genre setting means for calculating similarity among the M columns of the second matrix produced by the matrix generation means, classifying each of the M columns based on the calculated similarity, and setting each class obtained in the classification as a genre.

38. An information processing apparatus according to claim 37, wherein:

the genre setting means produces a classifier by learning classification for each genre by using columns belonging to the genre as training data; and

the information processing apparatus further comprises classification means for making classification by acquiring an information set including N pieces of individual information associated with a new content, assigning weight values to the respective N pieces of individual information, producing a feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information for the new content, and classifying, using the classifier produced by the genre setting means, the feature vector associated with the new content as being one of the genres set by the genre setting means.

39. An information processing apparatus according to claim 37, wherein each time the genre setting means sets one genre, the genre setting means selects an element from columns belonging to the genre and assigns a name to the genre based on the selected element.

40. An information processing method comprising:

a vectorization step of, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

a matrix generation step of, when M feature vectors are produced in the vectorization step, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized; and

a genre setting step of calculating similarity among the M columns of the second matrix produced in the matrix generation step, classifying each of the M columns based on the calculated similarity, and setting each class obtained in the classification as a genre.

41. A program executed by a computer, comprising:

a vectorization step of, each time a content is used by a user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

a matrix generation step of, when M feature vectors are produced in the vectorization step, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized; and

a genre setting step of calculating similarity among the M columns of the second matrix produced in the matrix generation step, classifying each of the M columns based on the calculated similarity, and setting each class obtained in the classification as a genre.

42. An information processing system including a server and a client used by a user, comprising:

vectorization means for, each time a content is used by the user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

matrix generation means for, when M feature vectors are produced by the vectorization means, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized;

reference vector generation means for producing a reference vector including N elements for use as a reference in calculation of similarity, from the M columns of the second matrix produced by the matrix generation means; and

similarity calculation means for calculating similarity by acquiring a new information set including N pieces of individual information, assigning weight values to the respective N pieces of individual information, producing a comparative feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information, and calculating similarity between the comparative vector and the reference vector produced by the reference vector generation means, wherein

of the vectorization means, the matrix generation means, the reference vector generation means, and the similarity calculation means, at least the matrix generation means is included in the server.

43. An information processing method for an information processing system including a server and a client used by a user, comprising:

a vectorization step of, each time a content is used by the user, acquiring an information set including N pieces of individual information associated with the content, assigning a weight value to each of the N pieces of individual information, and producing as a feature vector for the content a vector whose elements are given by the weight values assigned to the respective N pieces of individual information;

a matrix generation step of, when M feature vectors are produced in the vectorization step, producing a first matrix having N rows and M columns such that the elements thereof are given by the M feature vectors, performing an operation on the first matrix so as to convert the first matrix into a second matrix whose N×

M elements are at least partially modified such that correlations among N row elements of each of the M columns are emphasized;

a reference vector generation step of producing a reference vector including N elements for use as a reference in calculation of similarity, from the M columns of the second matrix produced in the matrix generation step; and

a similarity calculation step of calculating similarity comprising the substeps of acquiring a new information set including N pieces of individual information, assigning weight values to the respective N pieces of individual information, producing a comparative feature vector whose elements are given by the weight values assigned to the respective N pieces of individual information, and calculating similarity between the comparative vector and the reference vector produced in the reference vector generation step, wherein

of the vectorization step, the matrix generation step, the reference vector generation step, and the similarity calculation step, at least the matrix generation step is performed by the server.