U.S. patent application number 14/956449 was filed with the patent office on 2016-03-24 for recognition apparatus, method, and computer program product.
The applicant listed for this patent is Kabushiki Kaisha Toshiba. Invention is credited to Tsukasa Ike, Toshiaki Nakasu, Hidetaka Ohira, Ryuzo Okada.
Application Number | 20160086348 14/956449 |
Document ID | / |
Family ID | 47596805 |
Filed Date | 2016-03-24 |
United States Patent
Application |
20160086348 |
Kind Code |
A1 |
Nakasu; Toshiaki ; et
al. |
March 24, 2016 |
RECOGNITION APPARATUS, METHOD, AND COMPUTER PROGRAM PRODUCT
Abstract
In an embodiment, a recognition apparatus includes: an obtaining
unit configured to obtain positions of a specific part in a
coordinate system having a first axis to an n-th axis (n.gtoreq.2);
a calculating unit configured to calculate a movement vector of the
specific part; a principal axis selecting unit configured to select
a principal axis; a turning point setting unit configured to set a
position at which there is a change in the principal axis, and set
a position at which there is a change; a section setting unit
configured to set a determination target section, and set a
previous section; a determining unit configured to calculate an
evaluation value of the determination target section and an
evaluation value of the immediately previous section and, determine
which of the first axis to the n-th axis is advantageous; and a
presenting unit configured to perform the determined result.
Inventors: |
Nakasu; Toshiaki; (Tokyo,
JP) ; Ohira; Hidetaka; (Tokyo, JP) ; Ike;
Tsukasa; (Tokyo, JP) ; Okada; Ryuzo;
(Kanagawa, JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Kabushiki Kaisha Toshiba |
Tokyo |
|
JP |
|
|
Family ID: |
47596805 |
Appl. No.: |
14/956449 |
Filed: |
December 2, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14598251 |
Jan 16, 2015 |
9240047 |
|
|
14956449 |
|
|
|
|
13418890 |
Mar 13, 2012 |
8965113 |
|
|
14598251 |
|
|
|
|
Current U.S.
Class: |
382/103 |
Current CPC
Class: |
G06K 9/52 20130101; G06F
3/017 20130101; G06T 7/60 20130101; G06T 7/73 20170101; G06F 3/0485
20130101; G06F 3/04842 20130101; G06T 7/246 20170101 |
International
Class: |
G06T 7/20 20060101
G06T007/20; G06K 9/52 20060101 G06K009/52; G06T 7/00 20060101
G06T007/00; G06T 7/60 20060101 G06T007/60 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 29, 2011 |
JP |
2011-167608 |
Claims
1.-20. (canceled)
21. A recognition apparatus comprising: an obtaining unit
configured to obtain, in chronological order, positions of a
specific part of a photographic subject that are represented in a
coordinate system having a first axis to an n-th axis (n.gtoreq.2);
a calculating unit configured to calculate a movement vector of the
specific part at each of the positions obtained in chronological
order; a principal axis selecting unit configured to calculate, for
each of movement vectors that has been calculated, a first inner
product that is an inner product of the movement vector and the
first axis to an n-th inner product that is an inner product of the
movement vector and the n-th axis, and select, as a principal axis,
an axis that is used in calculating the inner product, from among
the first inner product to the n-th inner product, having the
greatest modulus; a turning point setting unit configured to set a
position at which there is a change in the principal axis as a
principal axis turning point and a direction turning point, and set
a position at which there is a change in the direction of any of
the movement vectors in the principal axis component as the
direction turning point; a section setting unit configured to set a
determination target section, in which a start point is either set
to the latest principal axis turning point when principal axis
turning points are present or set to an initial point indicating
the initially-obtained position when the principal axis turning
points are absent, and in which an end point is set to a latest
point that indicates the most-recently obtained position, set a
previous section, in which an end point is set to the start point
of the determination target section and in which a start point is
either set to a direction turning point present immediately before
the end point when the direction turning points are present before
the end point or set to the initial point when the direction
turning points are not present immediately before the end point; a
determining unit configured to calculate an evaluation value of the
determination target section and an evaluation value of the
immediately previous section and, based on the evaluation value of
the determination target section and the evaluation value of the
previous section, determine which of the first axis to the n-th
axis is advantageous for movement of the specific part in the
determination target section; and a presenting unit configured to
perform presentation according to a determination result obtained
by the determining unit, wherein the section setting unit updates
the end point of the determination target section either to the
principal axis turning point that is present immediately after the
start point of the determination target section when the principal
axis turning points are present subsequent to the start point but
before the elapse of a predetermined time period since the setting
of the start point, or to a latest point that is present after the
elapse of a predetermined time period, when no principal axis
turning point is present subsequent to the start point but before
the elapse of the predetermined time period, and sets a subsequent
section, in which a start point is set to the end point of an
updated determination target section and in which an end point is
either set to the direction turning point that is present
immediately after the start point when the direction turning points
are present subsequent to the start point or set to a latest point
that is present after the elapse of the predetermined time period
when latest points are present subsequent to the start point, and
the determining unit further calculates an evaluation value of the
subsequent section and further determines which of the first axis
to the n-th axis is advantageous for movement of the specific part
in the determination target section based on the evaluation value
of the subsequent section.
22. The apparatus according to claim 21, wherein when the
evaluation value of the determination target section is exceeding
the evaluation value of the previous section or when the previous
section is not present, when the evaluation value of the
determination target section is exceeding the evaluation value of
the subsequent section or when the subsequent section is not
present, the determining unit determines that the principal axis in
the determination target section is advantageous for movement of
the specific part in the determination target section, when the
evaluation value of the determination target section is equal to or
smaller than the evaluation value of the previous section, the
determining unit determines that the principal axis in the previous
section is advantageous for movement of the specific part in the
determination target section, and when the evaluation value of the
determination target section is equal to or smaller than the
evaluation value of the subsequent section, the determining unit
determines that the principal axis in the subsequent section is
advantageous for movement of the specific part in the determination
target section.
23. The apparatus according to claim 21, wherein the determining
unit performs sequential determination, within the predetermined
time period, of an axis advantageous for movement of the specific
part in the determination target section, and the presenting unit
further performs supplementary presentation within the
predetermined time period according to a determination result
obtained by the determining unit.
24. The apparatus according to claim 21, further comprising: a
first axis determining unit to an n-th axis determining unit
configured to respectively determine an advantageous direction
along the first axis to the n-th axis by referring to the movement
vectors that are calculated; and a determination result selecting
unit configured to select, from determination results regarding the
first axis to the n-th axis, a determination result regarding an
axis that is determined by the determining unit to be advantageous
for movement of the specific part in the determination target
section, wherein the presenting unit performs presentation
according to a selection result obtained by the determination
result selecting unit.
25. The apparatus according to claim 24, wherein each of the first
axis determining unit to the n-th axis determining unit further
calculates a progress rate of corresponding determination result
regarding the first axis to the n-th axis, when the progress rate
of a determination result of any one axis from among the first axis
to the n-th axis is within a predetermined progress rate and when
the any one axis is determined by the determining unit to be
advantageous for movement of the specific part in the determination
target section, the determination result selecting unit keeps on
selecting the determination result and the progress rate until the
progress rate represents a progress rate that finalizes the
determination result and until the any one axis is finalized as an
axis determined by the determining unit to be advantageous for
movement of the specific part in the determination target section,
and while the determination result selecting unit keeps on
selecting the determination result, the presenting unit further
performs supplementary presentation according to the selection
result.
26. A recognition apparatus comprising: an obtaining unit
configured to obtain, in chronological order, positions of a
specific part of a photographic subject that are represented in a
coordinate system having a first axis to an n-th axis (n.ltoreq.2);
a calculating unit configured to calculate a movement vector of the
specific part at each of the positions obtained in chronological
order; a principal axis selecting unit configured to calculate, for
each of the movement vectors that has been calculated, a first
inner product that is the inner product of the movement vector and
the first axis to an n-th inner product that is the inner product
of the movement vector and the n-th axis, and select, as a
principal axis, an axis that is used in calculating the inner
product, from among the first inner product to the n-th inner
product, having the greatest modulus; a turning point setting unit
configured to set a position at which there is a change in the
principal axis as a principal axis turning point and a direction
turning point, and set a position at which there is a change in the
direction of any of the movement vectors in the principal axis
component as the direction turning point; a section setting unit
configured to set a determination target section, in which a start
point is either set to the latest principal axis turning point when
principal axis turning points are present or set to an initial
point indicating the initially-obtained position when the principal
axis turning points are absent, and in which an end point is set to
the principal axis turning point present immediately after the
start point of the determination target section, when the principal
axis turning points are present subsequent to the start point but
before the elapse of a predetermined time period since the setting
of the start point, or set to a latest point present after the
elapse of a predetermined time period, when no principal axis
turning point is present subsequent to the start point but before
the elapse of the predetermined time period, and set a subsequent
section, in which a start point is set to the end point of the
determination target section and in which an end point is either
set to the direction turning point present immediately after the
start point when the direction turning points are present
subsequent to the start point or set to the latest point when the
latest point is present subsequent to the start point; a
determining unit configured to calculate an evaluation value of the
determination target section and an evaluation value of the
subsequent section and, based on the evaluation value of the
determination target section and the evaluation value of the
subsequent section, determine which of the first axis to the n-th
axis is advantageous for movement of the specific part in the
determination target section; a first axis determining unit to an
n-th axis determining unit configured to respectively determine an
advantageous direction along the first axis to the n-th axis by
referring to movement vectors that are calculated; a determination
result selecting unit configured to select, from determination
results regarding the first axis to the n-th axis, a determination
result regarding an axis that is determined by the determining unit
to be advantageous for movement of the specific part in the
determination target section, and a presenting unit configured to
perform presentation according to a selection result obtained by
the determination result selecting unit.
27. The apparatus according to claim 26, wherein each of the first
axis determining unit to the n-th axis determining unit further
calculates a progress rate of corresponding determination result
regarding the first axis to the n-th axis, when the progress rate
of a determination result of any one axis from among the first axis
to the n-th axis is within a predetermined progress rate and when
the any one axis is determined by the determining unit to be
advantageous for movement of the specific part in the determination
target section, the determination result selecting unit keeps on
selecting the determination result and the progress rate until the
progress rate represents a progress rate that finalizes the
determination result and until the any one axis is finalized as an
axis determined by the determining unit to be advantageous for
movement of the specific part in the determination target section,
and while the determination result selecting unit keeps on
selecting the determination result, the presenting unit further
performs supplementary presentation according to the selection
result.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is based upon and claims the benefit of
priority from Japanese Patent Application No. 2011-167608, filed on
Jul. 29, 2011; the entire contents of which are incorporated herein
by reference.
FIELD
[0002] Embodiments described herein relate generally to a
recognition apparatus, a recognition method, and a computer program
product.
BACKGROUND
[0003] Regarding recognition apparatuses configured to recognize
gestures, a technology is known in which, depending on the angle of
a velocity vector calculated in a section in which the velocity of
a specific part, or a specific bodily part, of a photographic
subject is equal to or greater than a threshold value, it is
determined whether the specific part performs a gesture (movement)
in the vertical direction or in the horizontal direction.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 is a configuration diagram illustrating an example of
a recognition apparatus according to a first embodiment;
[0005] FIG. 2 is a diagram illustrating an example of a gesture of
a specific part according to the first embodiment;
[0006] FIG. 3 is a diagram illustrating an example of positions of
the specific part;
[0007] FIG. 4 is a diagram illustrating an example of movement
vectors according to the first embodiment;
[0008] FIG. 5 is an explanatory diagram for explaining an example
of setting determination target sections and respective previous
sections according to the first embodiment;
[0009] FIG. 6 is a diagram illustrating another example of a
gesture performed by the specific part according to the first
embodiment;
[0010] FIG. 7 is a diagram illustrating an example of presentation
according to the first embodiment;
[0011] FIG. 8 is a flowchart for explaining an exemplary sequence
of operations performed by the recognition apparatus according to
the first embodiment;
[0012] FIG. 9 is a flowchart for explaining an exemplary sequence
of operations performed by a turning point setting unit in the
recognition apparatus according to the first embodiment;
[0013] FIG. 10 is a flowchart for explaining an exemplary sequence
of operations performed by a determining unit in the recognition
apparatus according to the first embodiment;
[0014] FIG. 11 is a configuration diagram illustrating an example
of a recognition apparatus according to a second embodiment;
[0015] FIG. 12 is a diagram illustrating another example of a
gesture performed by the specific part according to the second
embodiment;
[0016] FIG. 13 is a diagram for illustrating an example of axes
each being advantageous at a particular progress according to the
second embodiment;
[0017] FIG. 14 is a diagram for illustrating another example of
axes each being advantageous at a particular progress according to
the second embodiment;
[0018] FIG. 15 is an example of supplementary presentation
according to the second embodiment;
[0019] FIG. 16 is a diagram illustrating another example of
supplementary presentation according to the second embodiment;
[0020] FIG. 17 is a diagram illustrating still another example of
supplementary presentation according to the second embodiment;
[0021] FIG. 18 is a flowchart for explaining an exemplary sequence
of operations performed in the recognition apparatus according to
the second embodiment;
[0022] FIG. 19 is a flowchart for explaining an exemplary sequence
of operations performed by a determining unit in the recognition
apparatus according to the second embodiment;
[0023] FIG. 20 is a flowchart for explaining another exemplary
sequence of operations performed by the determining unit in the
recognition apparatus according to the second embodiment;
[0024] FIG. 21 is a configuration diagram illustrating an example
of a recognition apparatus according to a third embodiment;
[0025] FIG. 22 is a diagram of an example of setting second-axis
direction turning points according to the third embodiment;
[0026] FIG. 23 is a diagram illustrating exemplary advantageous
directions each corresponding to a progress rate according to the
third embodiment;
[0027] FIG. 24 is a diagram of feed action candidates each selected
corresponding to a progress rate according to the third
embodiment;
[0028] FIG. 25 is a flowchart illustrating an exemplary sequence of
operations performed in the recognition apparatus according to the
third embodiment;
[0029] FIG. 26 is a flowchart for explaining an exemplary sequence
of operations performed by a second axis determining unit in the
recognition apparatus according to the third embodiment;
[0030] FIG. 27 is a diagram illustrating an exemplary coordinate
system according to a fourth embodiment;
[0031] FIG. 28 is a diagram illustrating another exemplary
coordinate system according to the fourth embodiment;
[0032] FIG. 29 is a diagram illustrating a television receiver
according to a first application;
[0033] FIG. 30 is a diagram illustrating a personal computer
according to a second application; and
[0034] FIG. 31 is a diagram illustrating a medical information
terminal according to a third application.
DETAILED DESCRIPTION
[0035] According to an embodiment, a recognition apparatus
includes: an obtaining unit; a calculating unit; a principal axis
selecting unit; a turning point setting unit; a section setting
unit; a determining unit; and a presenting unit. The obtaining unit
is configured to obtain, in chronological order, positions of a
specific part of a photographic subject that are represented in a
coordinate system having a first axis to an n-th axis (n.gtoreq.2).
The calculating unit is configured to calculate a movement vector
of the specific part at each of the positions obtained in
chronological order. The principal axis selecting unit is
configured to calculate, for each of the movement vectors that has
been calculated, a first inner product that is an inner product of
the movement vector and the first axis to an n-th inner product
that is an inner product of the movement vector and the n-th axis,
and select, as a principal axis, an axis that is used in
calculating the inner product, from among the first inner product
to the n-th inner product, having the greatest modulus. The turning
point setting unit is configured to set a position at which there
is a change in the principal axis as a principal axis turning point
and a direction turning point, and set a position at which there is
a change in the direction of any of the movement vectors in the
principal axis component as the direction turning point. The
section setting unit is configured to set a determination target
section, in which a start point is either set to the latest
principal axis turning point when the principal axis turning points
are present or set to an initial point indicating the
initially-obtained position when the principal axis turning points
are absent, and in which an end point is set to a latest point that
indicates the most-recently obtained position, set a previous
section, in which an end point is set to the start point of the
determination target section and in which a start point is either
set to a direction turning point present immediately before the end
point when the direction turning points are present before the end
point or set to the initial point when the direction turning points
are not present immediately before the end point. The determining
unit is configured to calculate an evaluation value of the
determination target section and an evaluation value of the
immediately previous section and, based on the evaluation value of
the determination target section and the evaluation value of the
previous section, determine which of the first axis to the n-th
axis is advantageous for movement of the specific part in the
determination target section. The presenting unit is configured to
perform presentation according to a determination result obtained
by the determining unit.
First Embodiment
[0036] FIG. 1 is a configuration diagram illustrating an example of
a recognition apparatus 100 according to a first embodiment. As
illustrated in FIG. 1, the recognition apparatus 100 includes an
obtaining unit 102, a calculating unit 104, a principal axis
selecting unit 106, a turning point setting unit 108, a section
setting unit 110, a determining unit 112, and a presenting unit
114.
[0037] The obtaining unit 102 obtains, in chronological order, the
positions of a specific part, or a specific bodily part of a
photographic subject that are represented in a coordinate system
having a first axis and a second axis. For example, with respect to
chronological images captured using an imaging element such as an
image sensor, the obtaining unit 102 detects the position of the
specific part from each image using at least one of the techniques
such as the pattern recognizing technique, the background
differencing technique, the skin color extracting technique, and
the inter-frame differencing technique; and obtains the detected
positions of the specific part. Herein, as a position of the
specific part, it is possible to consider the center of gravity of
the detected area of the specific part.
[0038] FIG. 2 is a diagram illustrating an example of a gesture of
a specific part according to the first embodiment. FIG. 3 is a
diagram illustrating an example of positions of the specific part
that are obtained in chronological order by the obtaining unit 102.
For example, when the specific part follows a movement trajectory
as illustrated in FIG. 2; the obtaining unit 102 obtains discreet
positions P.sub.1 to P.sub.6, which are sampled from time-series
images, as the positions in chronological order of the specific
part as illustrated in FIG. 3.
[0039] In the first embodiment, the explanation is confined to an
example in which the specific part points to a hand of the
photographic subject (user). However, alternatively, the specific
part can also be the fingers of the photographic subject, the head
of the photographic subject, or a light emitting unit of a laser
point.
[0040] Herein, the positions of the specific part are represented
in the coordinate system having the first axis and the second axis.
Thus, as long as the first axis and the second axis are
discrimination targets for the gestures (movement) of the specific
part, they serve the purpose. In the first embodiment, it is an
object to discriminate the gestures in the vertical (up-down)
direction from the gestures in the horizontal (left-right)
direction. With that in mind, the first axis is treated as the axis
in the vertical direction and the second axis is treated as the
axis in the horizontal direction, and it is assumed that the first
axis and the second axis intersect with each other. However, that
is not the only possible case. For example, if the object is to
discriminate the gestures in the northwest-southeast direction from
the gestures in the northeast-southwest direction, then the first
axis can be treated as the axis in the northwest-southeast
direction and the second axis can be treated as the axis in the
northeast-southwest direction. Still alternatively, depth
information that is estimated from an imaging element or from a
distance sensor also can be considered as either one of the first
axis and the second axis.
[0041] Moreover, the positions of the specific part that are
expressed in the coordinate system having the first axis and the
second axis can either be expressed by the coordinates having those
axes as the components or be expressed by the coordinates defined
on images captured by the imaging element (expressed by pixel
positions). Furthermore, the coordinates expressing the positions
of the specific part can either be absolute coordinates or be
relative coordinates from the position of the face of the user.
Moreover, the positions of the specific part can be coordinate
values normalized from the face of the user or from the
initially-detected size of the specific part.
[0042] Meanwhile, the obtaining unit 102 can be configured not only
to obtain the positions of the specific part but also to obtain the
timings of obtaining the positions. For example, when the positions
of the specific part are expressed in x and y coordinates on
images; then, in an i-th frame, the obtaining unit 102 obtains a
position (x.sub.i, y.sub.i) as well as a timing T.sub.i. The timing
can be expressed as, for example, the elapsed time, the elapsed
clock count, or the elapsed frame count based on the activation
time of the recognition apparatus 100 or based on the start time of
obtaining positions of the obtaining unit 102. Herein, although the
explanation is given for a case of obtaining the position for each
obtained frame, it is not the only possible case.
[0043] The calculating unit 104 calculates a movement vector of the
specific part at each position obtained in chronological order. In
the first embodiment, the explanation is given for a case when each
movement vector is a relative vector with respect to the
previously-obtained position obtained by the obtaining unit 102.
However, that is not the only possible case.
[0044] FIG. 4 is a diagram illustrating an example of movement
vectors calculated by the calculating unit 104 according to the
first embodiment. For example, when the obtaining unit 102 obtains
the position P.sub.2, the calculating unit 104 calculates a
movement vector R.sub.1 that is the relative vector obtained with
respect to the previously-obtained position P.sub.1 (i.e. the
position P.sub.1 obtained in the previous frame). More
particularly, if the position P.sub.1 has coordinates (x.sub.1,
y.sub.1) and the position P.sub.2 has coordinates (x.sub.2,
y.sub.2) (herein, the second axis is assumed to be the x axis and
the first axis is assumed to be the y axis); then the calculating
unit 104 calculates a movement vector R.sub.1=(x.sub.2-x.sub.1,
y.sub.2-y.sub.1).sup.T, where T represents a transposed matrix.
[0045] Meanwhile, a movement vector need not be a relative vector
with respect to the previously-obtained position. Alternatively, a
movement vector can be a relative vector with respect to a position
obtained at some number of times earlier (obtained at some number
of frames earlier) or can be a relative vector with respect to a
direction turning point (described later in detail) present at the
previous position. Moreover, a movement vector can be vector that
has a slope calculated from the previously-obtained position (i.e.,
the position obtained in the previous frame), the
currently-obtained position (i.e., the position obtained in the
current position), and the subsequently-obtained position (i.e.,
the position obtained in the subsequent frame); and that passes
through the currently-obtained position. If the previously-obtained
position P.sub.1 has coordinates (x.sub.1, y.sub.1), the
currently-obtained position P.sub.2 has coordinates (x.sub.2,
y.sub.2), and the subsequently-obtained position P.sub.3 has
coordinates (x.sub.3, y.sub.3); then, for example, the
abovementioned slope can be considered to be the average of a slope
(y.sub.2-y.sub.1/x.sub.2-x.sub.1) of the vector between the
positions P.sub.1 and P.sub.2 and a slope
(y.sub.3-y.sub.2/x.sub.3-x.sub.2) of the vector between the
positions P.sub.2 and P.sub.3. Alternatively, a movement vector can
be a vector that has a slope calculated by estimating the movement
direction of the specific part from the direction of the blur of
the specific part on images captured by an imaging element; and
that passes through the currently-obtained position.
[0046] For each movement vector that is calculated, the principal
axis selecting unit 106 calculates a first inner product that is
the inner product of the movement vector and the first axis and
calculates a second inner product that is the inner product of the
movement vector and the second axis. Then, of the first axis and
the second axis, the principal axis selecting unit 106 selects the
axis that is used in calculating the inner product, from among the
first inner product and the second inner product, having the
greater modulus as the principal axis. That is, of the first axis
and the second axis, the principal axis selecting unit 106 selects
the axis closer to the direction of gestures of the specific
part.
[0047] For example, as illustrated in the example in FIG. 4, the
first axis and the second axis intersect at a right angle. Thus, if
the slope of a movement vector with respect to the first axis is
within the range of .+-.45.degree., that is, if the acute angle
between the first axis and a movement vector is within the range of
.+-.45.degree.; then the principal axis selecting unit 106 selects
the first axis as the principal axis. On the other hand, if the
slope of a movement vector with respect to the second axis is
within the range of .+-.45.degree., that is, if the acute angle
between the second axis and a movement vector is within the range
of .+-.45.degree.; then the principal axis selecting unit 106
selects the second axis as the principal axis. As a result, the
principal axis selecting unit 106 selects the first axis as the
principal axis of the movement vector R.sub.1 and selects the
second axis as the principal axis of each of movement vectors
R.sub.2 to R.sub.5. Hence, between the positions P.sub.1 and
P.sub.2, the first axis serves as the principal axis; and between
the positions P.sub.2 to P.sub.6, the second axis serves as the
principal axis.
[0048] Meanwhile, the principal axis selecting unit 106 can also be
configured to perform correction in the form of increasing the
inner product, from among the first inner product and the second
inner product, which was calculated using the principal axis of the
previously-calculated movement vector. That is, the principal axis
selecting unit 106 can be configured to set hysteresis for
selecting the principal axis and to perform correction in such a
way that, for example, if the first axis was selected as the
principal axis on the previous occasion, the same first axis is
selected as the current principal axis without difficulty.
[0049] For example, the principal axis selecting unit 106
calculates a principal axis inner product, which points to the
inner product of the movement vector and the axis selected as the
principal axis on the previous occasion, and calculates a
non-principal axis inner product, which points to the inner product
of the movement vector and the axis not selected as the principal
axis on the previous occasion. Then, the principal axis selecting
unit 106 applies a correction function to increment the principal
axis inner product and compares the post-correction principal axis
inner product with the pre-correction principal axis inner product.
Accordingly, the principal axis selecting unit 106 selects the axis
that is used in calculating the inner product having the greater
modulus, from among the post-correction principal axis inner
product and the principal axis inner product, as the principal
axis. Herein, for example, the correction function can be set to
.gamma. (.gamma.>1.0). Moreover, consider the case when the
first axis and the second axis intersect at a right angle. In that
case, for example, if the slope of a movement vector with respect
to the axis selected as the principal axis on the previous occasion
is within the range of .+-.50.degree., then the principal axis
selecting unit 106 can select that axis as the principal axis. On
the other hand, if the slope of a movement vector with respect to
the axis not selected as the principal axis on the previous
occasion is within the range of .+-.40.degree., then the principal
axis selecting unit 106 can select that axis as the principal
axis.
[0050] The turning point setting unit 108 sets positions at which
there is a change in the principal axis as principal axis turning
points and direction turning points, and sets positions at which
there is a change in the direction of the movement vectors in the
principal axis component as the direction turning points. For
example, as illustrated in FIG. 4, since the principal axis changes
from the first axis to the second axis at the position P.sub.2, the
turning point setting unit 108 sets the position P.sub.2 as a
principal axis turning point and a direction turning point.
Moreover, at the position P.sub.5, in the component of the second
axis serving as the principal axis, the direction of the movement
vector changes from the right-hand direction to the left-hand
direction. Thus, the turning point setting unit 108 sets the
position P.sub.5 as a direction turning point.
[0051] Meanwhile, as far as a change in the direction of any of the
movement vectors in the principal axis component is concerned, the
turning point setting unit 108 can determine that change with a
change in the sign of the inner product of the movement vector and
the principal axis. Moreover, at the positions P.sub.3 and P.sub.4,
the direction of the movement vector changes in the component of
the first axis that is not the principal axis, but not in the
component of the second axis that is the principal axis. Hence, the
turning point setting unit 108 does not set the positions P.sub.3
and P.sub.4 as direction turning points.
[0052] The section setting unit 110 sets a determination target
section, in which a start point is either set to the latest
principal axis turning point when principal axis turning points are
present or set to the initial point indicating the
initially-obtained position when principal axis turning points are
absent, and in which an end point is set to the latest point that
indicates the most-recently obtained position. Moreover, the
section setting unit 110 sets a previous section, in which an end
point is set to the start point of the corresponding determination
target section and in which a start point is either set to the
direction turning point that is present immediately before the end
point when the direction turning points are present before the end
point or set to the initial point when no direction turning point
is present before the end point. Herein, the gesture portion set in
the determination target section becomes the determination target
for determining whether that gesture is in the direction along the
first axis or in the direction along the second axis. Moreover, the
gesture portion set in the previous section is used in determining
whether or not the gesture portion set in the determination target
section is noise, which points to unnecessary information that has
no relation to the gesture (such as waving a hand) intended by the
photographic subject (user).
[0053] In the first embodiment, it is assumed that determination
target sections and the respective previous sections are set
online, that is, in real time by the section setting unit 110.
However, alternatively, it is also possible to set those sections
by performing batch processing, that is, after the gesture is
performed. Herein, for the sake of convenience, firstly, the
explanation is given for the case when the determination target
sections and the respective previous sections are set by performing
batch processing. After that, the explanation is given for the case
when those sections are set in real time.
[0054] FIG. 5 is an explanatory diagram for explaining an example
of a method of setting determination target sections and the
respective previous sections by performing batch processing. In the
example illustrated in FIG. 5, Q.sub.s represents the initial
position that is obtained by the obtaining unit 102 and that serves
as the start point; Q.sub.1 to Q.sub.4 represent principal axis
turning points and direction turning points; K.sub.1 and K.sub.2
represent direction turning points; and Q.sub.e represents the last
position that is obtained by the obtaining unit 102 and that serves
as the end point.
[0055] In the case of setting the determination target sections and
the respective previous sections by performing batch processing;
once the points Q.sub.s to Q.sub.e are obtained, the section
setting unit 110 sets determination target sections and the
respective previous sections in order and sequentially outputs the
information about those sections to the determining unit 112. The
information about a determination target section and the
corresponding previous section can contain the positions at both
ends separating that determination target section from the
corresponding previous section, or can contain the timings of
obtaining those positions, or can contain IDs of those
positions.
[0056] More particularly, firstly, the section setting unit 110
sets a section Q.sub.s to Q.sub.1 as the determination target
section in which Q.sub.s serves as the start point and Q.sub.1
serves as the end point; and sends the information about that
determination target section to the determining unit 112. In this
case, the section setting unit 110 does not set any previous
section. Subsequently, the section setting unit 110 sets a section
Q.sub.1 to Q.sub.2 as the determination target section in which
Q.sub.1 serves as the start point and Q.sub.2 serves as the end
point. Besides, the section setting unit 110 sets the section
Q.sub.s to Q.sub.1 as the corresponding previous section in which
Q.sub.s serves as the start point and Q.sub.1 serves as the end
point. Then, the section setting unit 110 sends the information
about that determination target section and the corresponding
previous section to the determining unit 112. In an identical
manner, the section setting unit 110 sets a section Q.sub.2 to
Q.sub.3 as the determination target section and sets a section
K.sub.2 to Q.sub.2 as the corresponding previous section, and
outputs that information to the determining unit 112. Similarly,
the section setting unit 110 sets a section Q.sub.3 to Q.sub.4 as
the determination target section and sets a section Q.sub.2 to
Q.sub.3 as the corresponding previous section, and outputs that
information to the determining unit 112. Moreover, the section
setting unit 110 sets a section Q.sub.4 to Q.sub.e as the
determination target section and sets a section Q.sub.3 to Q.sub.4
as the corresponding previous section, and outputs that information
to the determining unit 112.
[0057] Thus, while setting the determination target sections and
the respective previous sections by batch processing; the section
setting unit 110 sets determination target sections in each of
which the start point is either set to the initial point or set to
a principal axis turning point in a sequential manner, and in each
of which the end point is either set to a principal axis turning
point present immediately after the corresponding start point, when
the principal axis turning points are present subsequent to the
corresponding start point, or set to the last point when no
principal axis turning point is present subsequent to the
corresponding start point. Moreover, the section setting unit 110
sets previous sections in each of which the end point is set to the
start point of the subsequent determination target section, and in
each of which the start point is either set to the direction
turning point present immediately before that end point when the
direction turning points are present before the end point or set to
the initial point when no direction turning point is present before
the end point.
[0058] Explained below is the case in which the determination
target sections and the respective previous sections are set in
real time. While setting the determination target sections and the
respective previous sections in real time, every time the obtaining
unit 102 obtains a position, the section setting unit 110 sets a
determination target section and the corresponding previous
section, and outputs the information about those sections to the
determining unit 112.
[0059] The specific explanation is given with reference to FIG. 4.
Firstly, even when the obtaining unit 102 obtains the latest point
(the initial point) P.sub.1 that indicates the most-recently
obtained position, the section setting unit 110 neither sets a
determination target section nor sets a previous section.
[0060] Then, once the obtaining unit 102 obtains the latest point
P.sub.2, the section setting unit 110 sets the section P.sub.1 to
P.sub.2 as the determination target section in which the start
point is set to the initial point P.sub.1 and the end point is set
to the latest point P.sub.2, and outputs the information of that
determination target section to the determining unit 112. In this
case, the section setting unit 110 does not set any previous
section.
[0061] Subsequently, when the obtaining unit 102 obtains the latest
point P.sub.3, the position P.sub.2 becomes the principal axis
turning point. Hence, the section setting unit 110 sets the section
P.sub.2 to P.sub.3 as the determination target section in which the
start point is set to the latest principal axis turning point
P.sub.2 and the end point is set to the latest point P.sub.3.
Besides, the section setting unit 110 sets the section P.sub.1 to
P.sub.2 as the corresponding previous section in which the start
point is set to the initial point P.sub.1 and the end point is set
to the principal axis turning point P.sub.2. Then, the section
setting unit 110 sends the information about those two sections to
the determining unit 112.
[0062] Subsequently, when the obtaining unit 102 obtains the latest
point P.sub.4, the section setting unit 110 sets the section
P.sub.2 to P.sub.4 as the determination target section in which the
start point is set to the latest principal axis turning point
P.sub.2 and the end point is set to the latest point P.sub.4.
Besides, the section setting unit 110 sets the section P.sub.1 to
P.sub.2 as the corresponding previous section in which the start
point is set to the initial point P.sub.1 and the end point is set
to the principal axis turning point P.sub.2. Then, the section
setting unit 110 sends the information about those two sections to
the determining unit 112.
[0063] Subsequently, when the obtaining unit 102 obtains the latest
point P.sub.5, the section setting unit 110 sets the section
P.sub.2 to P.sub.5 as the determination target section in which the
start point is set to the latest principal axis turning point
P.sub.2 and the end point is set to the latest point P.sub.5.
Besides, the section setting unit 110 sets the section P.sub.1 to
P.sub.2 as the corresponding previous section in which the start
point is set to the initial point P.sub.1 and the end point is set
to the principal axis turning point P.sub.2. Then, the section
setting unit 110 sends the information about those two sections to
the determining unit 112.
[0064] Subsequently, when the obtaining unit 102 obtains the latest
point P.sub.6, the section setting unit 110 sets the section
P.sub.2 to P.sub.6 as the determination target section in which the
start point is set to the latest principal axis turning point
P.sub.2 and the end point is set to the latest point P.sub.6.
Besides, the section setting unit 110 sets the section P.sub.1 to
P.sub.2 as the corresponding previous section in which the start
point is set to the initial point P.sub.1 and the end point is set
to the principal axis turning point P.sub.2. Then, the section
setting unit 110 sends the information about those two sections to
the determining unit 112.
[0065] Then, the determining unit 112 calculates an evaluation
value of the determination target section and an evaluation value
of the corresponding previous section and, based on those
evaluation values, determines which of the first axis and the
second axis is advantageous for the movement of the specific part
in the determination target section. More particularly, when the
evaluation value of the determination target section exceeds the
evaluation value of the previous section, the determining unit 112
determines that the principal axis in the determination target
section is advantageous for the movement of the specific part in
the determination target section. On the other hand, when the
evaluation value of the determination target section is equal to or
smaller than the evaluation value of the previous section, the
determining unit 112 determines that the principal axis in the
previous section is advantageous for the movement of the specific
part in the determination target section.
[0066] Herein, an evaluation value points to at least one of the
following: the amount of change in the position or in the size of
the specific part (i.e., the movement distance); the rate of change
in the position or in the size of the specific part (i.e., the
movement velocity); and the rate of acceleration in the position or
in the size of the specific part (i.e., the movement acceleration).
For example, an evaluation value can be expressed as a function
having the movement distance and the movement velocity of the
specific part as illustrated in Equation (1) given below or can be
expressed as a function having the movement distance, the movement
velocity, and the movement acceleration of the specific part as
illustrated in Equation (2) given below. Still alternatively, an
evaluation value can be expressed as the movement distance of the
specific part, or the movement velocity of the specific part, or
the movement acceleration of the specific part.
E=V.times. {square root over (L)} (1)
E=V.times.L.times..alpha. (2)
[0067] where, E represents the evaluation value, V represents the
movement velocity, L represents the movement distance, and .alpha.
represents the movement acceleration.
[0068] The movement distance can be considered to be the sum of
Euclidean distances of the movement vectors in the determination
target section or can be considered to be the sum of the principal
axis components (sum of the modulus of the inner product of each
movement vector and the principal axis) of the movement vectors.
For example, in the example illustrated in FIG. 4, in the section
P.sub.2 to P.sub.5 that serves as the determination target section
at the point of time of obtaining the latest point P.sub.5, the
movement distance can be considered to be the total distance
calculated by adding the Euclidean distances of the movement
vectors R.sub.2 to R.sub.4.
[0069] The movement velocity can be considered to be the maximum
velocity among the velocities of the movement vectors in the
determination target section or can be considered to be the average
velocity of the velocities of the movement vectors that are equal
to or greater than a predetermined threshold value. Herein, if end
points P.sub.b and P.sub.c at both ends of a movement vector are
obtained at timings T.sub.b and T.sub.c, respectively; then the
velocity of that movement vector can be calculated as |(distance
between P.sub.b and P.sub.c)/(T.sub.b-T.sub.c)|. For example, in
the example illustrated in FIG. 4, the movement velocity in the
section P.sub.2 to P.sub.5 can be considered to be the maximum
velocity among the velocities of the movement vectors R.sub.2 to
R.sub.4. If the end points P.sub.2 and P.sub.3 at both ends of the
movement vector R.sub.2 are obtained at timings T.sub.2 and
T.sub.3, respectively; then the velocity of the movement vector
R.sub.2 can be calculated as |(Euclidean distance of the movement
vector R.sub.2)/(T.sub.3-T.sub.2)|.
[0070] Meanwhile, the method of calculating the movement velocity
of the specific part is not limited to the description given above.
For example, the determining unit 112 can be configured to
calculate the velocities of pixels surrounding the positions of the
specific part that are obtained by the obtaining unit 102 (i.e., to
calculate the optical flow), and to set the average or the mode of
the velocities of pixels as the movement velocity of the specific
part.
[0071] Alternatively, for example, by considering the position of
the specific part inside an image as an observation value and by
considering the position and the velocity of the specific part as
state variables, the determining unit 112 can calculate the
movement velocity of the specific part with the use of a filter
such as the Kalman filter that estimates the optimum state using a
movement model of state variables such as an observation value and
a uniform motion. At that time, as the observation value, the
determining unit 112 can further make use of the average of the
velocities of pixels surrounding the positions of the specific part
(i.e., the average of the optical flow).
[0072] The movement acceleration can be considered to be the
maximum rate of acceleration from among the rates of acceleration
of the movement vectors in the determination target section or can
be considered to be the average of the rates of acceleration of the
movement vectors that are equal to or greater than a predetermined
threshold value. Herein, if the end points P.sub.b and P.sub.c at
both ends of a movement vector are obtained at the timings T.sub.b
and T.sub.c, respectively, and if the points P.sub.a and P.sub.b at
both ends of the previous movement vector are obtained at timings
T.sub.a and T.sub.b, respectively; then the movement acceleration
of that movement vector can be calculated as |(velocity from
P.sub.b to P.sub.c)-(velocity from P.sub.a to
P.sub.c)/(T.sub.b-T.sub.c)|.
[0073] Explained below in detail with reference to FIG. 4 is the
method of determining the advantageous axis for the movement of the
specific part in a determination target section. Firstly, when the
section setting unit 110 sets the section P.sub.1 to P.sub.2 as the
determination target section, no previous section exists. Hence,
the determining unit 112 determines that the first axis serving as
the principal axis in the section P.sub.1 to P.sub.2 is
advantageous for the movement of the specific part.
[0074] Subsequently, the section setting unit 110 sets the section
P.sub.2 to P.sub.3 as the determination target section and sets the
section P.sub.1 to P.sub.2 as the corresponding previous section.
In that case, the determining unit 112 calculates the evaluation
values of the section P.sub.2 to P.sub.3 and the section P.sub.1 to
P.sub.2. Herein, it is assumed that the evaluation value of the
section P.sub.2 to P.sub.3 (the determination target section) is
equal to or smaller than the evaluation value of the section
P.sub.1 to P.sub.2 (the corresponding previous section). Thus,
although the section P.sub.2 to P.sub.3 has the second axis as the
principal axis, the determining unit 112 determines that the first
axis, which is the principal axis in the section P.sub.1 to
P.sub.2, is advantageous for the movement of the specific part.
That is, at that particular point of time, the determining unit 112
considers the movement trajectory of the section P.sub.2 to P.sub.3
as noise.
[0075] Subsequently, the section setting unit 110 sets the section
P.sub.2 to P.sub.4 as the determination target section and sets the
section P.sub.1 to P.sub.2 as the corresponding previous section.
In that case, the determining unit 112 calculates the evaluation
values of the section P.sub.2 to P.sub.4 and the section P.sub.1 to
P.sub.2. However, regarding the section P.sub.1 to P.sub.2, it is
also possible to use the evaluation value calculated on the
previous occasion. Herein, it is assumed that the evaluation value
of the section P.sub.2 to P.sub.4 (the determination target
section) exceeds the evaluation value of the section P.sub.1 to
P.sub.2 (the corresponding previous section). Thus, the determining
unit 112 determines that the second axis, which is the principal
axis in the section P.sub.2 to P.sub.4, is advantageous for the
movement of the specific part. That is, at that particular point of
time, the determining unit 112 considers the movement trajectory of
the section P.sub.2 to P.sub.4 as a gesture.
[0076] In an identical manner, when each of the section P.sub.2 to
P.sub.5 and the section P.sub.2 to P.sub.6 is set as the
determination target section, the evaluation value of the
determination target section is assumed to exceed the evaluation
value of the corresponding previous section. Hence, regarding each
of those two determination target sections, the determining unit
112 determines that the second axis serving as the principal axis
is advantageous for the movement of the specific part.
[0077] FIG. 6 is a diagram illustrating another example of a
gesture performed by the specific part according to the first
embodiment. In the example illustrated in FIG. 6, the specific part
follows a trajectory in the right-hand direction (trajectories 261
and 262) and then follows a trajectory in the upward direction
(trajectory 263). Meanwhile, in the example illustrated in FIG. 6,
the movement distance is used as the evaluation value.
[0078] During the trajectories 261 and 262, the trajectory in the
right-hand direction serves as the determination target section,
and there is no previous section. Consequently, the determining
unit 112 determines that the axis in the horizontal direction,
which is the principal axis in the determination target section, is
advantageous for the movement of the specific part.
[0079] During the trajectory 263, the trajectory in the upward
direction serves as the determination target section, and the
trajectory in the right-hand direction serves as the corresponding
previous section. Since the trajectory in the upward direction has
a shorter movement distance than the movement distance of the
trajectory in the right-hand direction, the determining unit 112
determines that the evaluation value of the trajectory in the
right-hand direction exceeds the evaluation value of the trajectory
in the upward direction. Accordingly, in that determination target
section too, the determining unit 112 determines that the axis in
the horizontal direction, which is the principal axis in the
corresponding previous section, is advantageous for the movement of
the specific part.
[0080] As a result, any gesture in the vertical direction is
considered as noise, and the horizontal direction is determined to
be advantageous as far as the entire gesture is concerned.
[0081] The presenting unit 114 performs presentation according to
the determination result of the presenting unit 114. For example,
the presenting unit 114 performs transition of the screen, which is
displayed on a display, in the direction of the axis, from among
the first axis and the second axis, which is determined by the
determining unit 112 to be advantageous for the movement of the
specific part.
[0082] FIG. 7 is a diagram illustrating an example of presentation
performed by the presenting unit 114 according to the first
embodiment. In the example illustrated in FIG. 7, the presenting
unit 114 displays an object 181 on the left side of a screen 180,
displays an object 182 in the center of the screen 180, and
displays an object 183 on the right side of the screen 180. Herein,
assume that the user performs a gesture of moving a hand 186, which
represents the specific part, in a direction 187, that is, in the
left-hand direction. Regarding that gesture, since the determining
unit 112 determines that the axis in the horizontal direction is
advantageous for the movement of the hand 186; the presenting unit
114 performs transition of the objects in the horizontal direction
(to the left side) on the screen 180. As a result, on the screen
180, the presenting unit 114 displays the object 182 on the left
side, the object 183 in the center, and an object 184 on the right
side.
[0083] Meanwhile, if the recognition apparatus 100 includes a
command issuing unit for issuing a command according to the
determination result obtained by the determining unit 112, then the
presenting unit 114 performs presentation according to the issued
command.
[0084] For example, to the command issuing unit, the determining
unit 112 outputs the ID of the axis that is determined to be
advantageous for the movement of the specific part. Herein, to the
command issuing unit, the determining unit 112 can be configured to
output the ID of the axis that is determined to be advantageous for
the movement of the specific part as well as output the information
about the determination target section such as the IDs of both end
points of the determination target section. Alternatively, only
when there is a change in the axis that is determined to be
advantageous for the movement of the specific part, the determining
unit 112 can be configured to output the ID of the new axis to the
command issuing unit.
[0085] Then, if the movement velocity of the specific part in that
axis component is equal to or greater than a threshold value, the
command issuing unit issues a command for the direction of that
axis. For example, in the example illustrated in FIG. 4, when the
section P.sub.2 to P.sub.3 is the determination target section, the
first axis becomes the advantageous axis for the movement of the
specific part in the determination target section. Therefore, if
the velocity of the movement vector R.sub.2 in the first axis
component (i.e., the modulus of the inner product of the movement
vector R.sub.2 and the first axis) is equal to or greater than a
threshold value, that is, if the velocity component of the movement
vector R.sub.2 in the upward direction is equal to or greater than
a threshold value; then the command issuing unit issues a command
for the upward direction. In contrast, when the section P.sub.2 to
P.sub.4 is the determination target section, the second axis
becomes the advantageous axis for the movement of the specific part
in the determination target section. Therefore, if the velocity
component in the right-hand direction of the greater velocity, of
the velocities in the first axis components of the movement vectors
R.sub.2 and R.sub.3, is equal to or greater than a threshold value;
then the command issuing unit issues a command for the right-hand
direction.
[0086] According to the command that has been issued, the
presenting unit performs transition of the screen being displayed
on the display.
[0087] FIG. 8 is a flowchart for explaining an exemplary sequence
of operations performed by the recognition apparatus 100 according
to the first embodiment.
[0088] Firstly, the obtaining unit 102 obtains positions of a
specific part of a photographic subject that are represented in a
coordinate system having a first axis and a second axis (Step
S101).
[0089] Then, with respect to the positions obtained by the
obtaining unit 102, the calculating unit 104 calculates relative
vectors from the respective previously-obtained positions as
movement vectors of the specific part (Step S102).
[0090] Subsequently, for each movement vector that is calculated by
the calculating unit 104, the principal axis selecting unit 106
calculates a first inner product that is the inner product of the
movement vector and the first axis and calculates a second inner
product that is the inner product of the movement vector and the
second axis. Then, of the first axis and the second axis, the
principal axis selecting unit 106 selects the axis that is used in
calculating the inner product, from among the first inner product
and the second inner product, having the greater modulus as the
principal axis (Step S103).
[0091] Subsequently, if the principal axis changes at the
previously-obtained position obtained by the obtaining unit 102,
the turning point setting unit 108 sets that position as a
principal axis turning point and a direction turning point; and if
the direction of the movement vector changes in the principal axis
component at the previously-obtained position obtained by the
obtaining unit 102, the turning point setting unit 108 sets that
position as a direction turning point (Step S104).
[0092] Then, the section setting unit 110 sets a determination
target section, in which a start point is either set to the latest
principal axis turning point when principal axis turning points are
present or set to the initially-obtained position when principal
axis turning points are absent, and in which an end point is set to
the latest point that indicates the most-recently obtained
position. Moreover, the section setting unit 110 sets a previous
section, in which an end point is set to the start point of the
determination target section and in which a start point is either
set to the direction turning point present immediately before the
end point when the direction turning points are present before the
end point or set to the initial point when no direction turning
point is present before the end point (Step S105).
[0093] Subsequently, the determining unit 112 calculates an
evaluation value of the determination target section and an
evaluation value of the corresponding previous section and, based
on those evaluation values, determines which of the first axis and
the second axis is advantageous for the movement of the specific
part in the determination target section (Step S106).
[0094] Every time the obtaining unit 102 obtains the latest
position of the specific part of the photographic subject (Step
S101), the operations from Step S102 to Step S106 are repeated, and
the presenting unit 114 performs presentation according to the
determination result obtained by the determining unit 112.
[0095] FIG. 9 is a flowchart for explaining an exemplary sequence
of operations performed by the turning point setting unit 108 in
the recognition apparatus 100 according to the first
embodiment.
[0096] Firstly, the turning point setting unit 108 obtains, from
the obtaining unit 102, a position P.sub.j of the specific part in
the latest frame as well as obtains, from the principal axis
selecting unit 106, a principal axis ID of the latest frame (Step
S120). The principal axis ID of the latest frame points to the
principal axis ID of the principal axis of a movement vector
R.sub.i, which is the relative vector with respect to the position
P.sub.j from a position P.sub.i of the specific part in the
previous frame.
[0097] Then, the turning point setting unit 108 determines whether
the position P.sub.j was obtained within two frames from the point
of time when the obtaining unit 102 started obtaining the positions
of the specific part (Step S121). If it is determined that the
position P.sub.j was obtained within two frames from the point of
time of starting to obtain the positions of the specific part (Yes
at Step S121), then the operations are ended. That is because not
enough positions are obtained to enable setting of turning
points.
[0098] On the other hand, if it is determined that the position
P.sub.j was not obtained within two frames from the point of time
of starting to obtain the positions of the specific part (No at
Step S121), then the turning point setting unit 108 obtains, from
the obtaining unit 102, the position P.sub.i of the specific part
in the previous frame and a position P.sub.h of the specific part
in the last but one frame; as well as obtains, from the principal
axis selecting unit 106, the principal axis ID of the previous
frame (Step S122). The principal axis ID of the previous frame
points to the principal axis ID of the principal axis of a movement
vector R.sub.h, which is the relative vector with respect to the
position P.sub.i from the position P.sub.h of the specific part in
the last but one frame. Meanwhile, alternatively, the principal
axis ID of the latest frame that is obtained in the previous frame
can be stored and used by the turning point setting unit 108 as the
principal axis ID of the previous frame.
[0099] Subsequently, the turning point setting unit 108 determines
whether or not the principal axis ID of the latest frame is
different than the principal axis ID of the previous frame (Step
S123).
[0100] When the principal axis ID of the latest frame is different
than the principal axis ID of the previous frame (Yes at Step
S123), the turning point setting unit 108 sets the position P.sub.i
as a principal axis turning point and a direction turning point
(Step S124). That marks the end of the operations.
[0101] On the other hand, when the principal axis ID of the latest
frame is same as the principal axis ID of the previous frame (No at
Step S123), the turning point setting unit 108 determines whether
or not the movement trajectory (movement vector) of the specific
part at the position P.sub.i is changing direction in the principal
axis component (Step S125). More particularly, the turning point
setting unit 108 determines whether or not the inner product of the
movement vector R.sub.i and the principal axis of the latest frame
has a different sign than the sign of the inner product of the
movement vector R.sub.j and the principal axis of the previous
frame. However, if the inner product of the movement vector and the
principal axis of the previous frame is equal to zero, it is
understood that there was no change in the principal axis component
at the movement vector R.sub.j. In that case, the turning point
setting unit 108 obtains a position P.sub.g of the specific part
that is obtained three times earlier; and then makes use of the
inner product between a movement vector R.sub.g, which is the
relative vector with respect to the position P.sub.h from the
position P.sub.g of the specific part in the last frame but two,
and the principal axis of the last frame but two.
[0102] When the movement trajectory (movement vector) of the
specific part at the position P.sub.i is changing direction in the
principal axis component (Yes at Step S125), that is, when both
inner products have different signs; the turning point setting unit
108 sets the position P.sub.i as a direction turning point (Step
S126). That marks the end of the operations.
[0103] On the other hand, when the movement trajectory (movement
vector) of the specific part at the position P.sub.i is not
changing direction in the principal axis component (No at Step
S125), that is, when both inner products have the same sign; the
operations are ended.
[0104] FIG. 10 is a flowchart for explaining an exemplary sequence
of operations performed by the determining unit 112 in the
recognition apparatus 100 according to the first embodiment.
[0105] Firstly, the determining unit 112 checks whether or not a
previous section is present (Step S130).
[0106] When a previous section is present (Yes at Step S130), the
determining unit 112 calculates an evaluation value E.sub.c of the
determination target section and an evaluation value E.sub.b of the
previous section, and determines whether or not the evaluation
value E.sub.c is equal to or smaller than the evaluation value
E.sub.b (Step S131).
[0107] If the evaluation value E.sub.c is equal to or smaller than
the evaluation value E.sub.b (Yes at Step S131), the determining
unit 112 determines that the principal axis in the previous section
is advantageous for the movement of the specific part in the
determination target section (Step S132). That marks the end of the
operations.
[0108] Meanwhile, if a previous section is absent (No at Step S130)
or if the evaluation value E.sub.c is not equal to or smaller than
the evaluation value E.sub.b (No at Step S131), then the
determining unit 112 determines that the principal axis in the
determination target section is advantageous for the movement of
the specific part in the determination target section (Step S133).
That marks the end of the operations.
[0109] As described above, in the first embodiment, the gesture in
a determination target section is recognized using the
corresponding previous section. Thus, according to the first
embodiment, it becomes possible to determine whether or not the
gesture in a determination target section is noise. Because of
that, even if a movement representing noise is included in a
gesture, such movement can be recognized correctly thereby enabling
prevention of a decline in the recognition accuracy.
[0110] For example, if a movement representing noise is included at
the end of a gesture, then the portion of the normal gesture
represents the previous section and the noise portion represents
the determination target section. There, by referring to the
previous section, it becomes possible to determine that the gesture
in the determination target section is noise and to make
substitution with the movement direction of the normal gesture in
the previous section. In this way, even when a movement
representing noise is included in a gesture, the noise can be
recognized in a correct manner.
Second Embodiment
[0111] In a second embodiment, the explanation is given for an
example of recognizing the gesture in a determination target
section by making use of the corresponding subsequent section. In
the following explanation, the focus is on elaborating the
differences with the first embodiment. Moreover, the constituent
elements having identical functions to the functions described in
the first embodiment are referred to by the same names and
reference numerals, and the explanation thereof is not
repeated.
[0112] FIG. 11 is a configuration diagram illustrating an example
of a recognition apparatus 200 according to the second embodiment.
In the recognition apparatus 200 according to the second
embodiment; a section setting unit 210, a determining unit 212, and
a presenting unit 214 are different as compared to the first
embodiment.
[0113] The section setting unit 210 sets a determination target
section, in which a start point is either set to the latest
principal axis turning point when principal axis turning points are
present or set to the initially-obtained position when principal
axis turning points are absent, and in which an end point is either
set to the principal axis turning point present immediately after
the start point, when the principal axis turning points are present
subsequent to the start point but before the elapse in a
predetermined time period since the setting of the start point, or
set to the latest point that indicates the most-recently obtained
position before the elapse in a predetermined time period, when no
principal axis turning point is present subsequent to the start
point but before the elapse in the predetermined time. Moreover,
the section setting unit 210 sets a subsequent section, in which a
start point is set to the end point of the determination target
section and in which an end point is either set to the direction
turning point present immediately after to the start point when the
direction turning points are present subsequent to the start point
or set to the latest point when the latest point is present
subsequent to the start point. Moreover, the gesture portion set in
the subsequent section is used in determining whether or not the
gesture portion set in the determination target section is
noise.
[0114] In the second embodiment too, it is assumed that
determination target sections and the respective subsequent
sections are set online, that is, in real time by the section
setting unit 210. However, alternatively, it is also possible to
set those sections by performing batch processing, that is, after
the gesture is performed. Herein, for the sake of convenience,
firstly, the explanation is given for the case when determination
target sections and the respective subsequent sections are set by
performing batch processing. Then, the explanation is given for the
case when those sections are set in real time.
[0115] Explained below with reference to FIG. 5 is the setting of
determination target sections and the respective subsequent
sections by performing batch processing. While setting the
determination target sections and the respective subsequent
sections by performing batch processing, after the initial position
Q.sub.s and the last position Q.sub.e are obtained, the section
setting unit 210 sets the determination target sections and the
respective subsequent sections in order, and sequentially outputs
to the determining unit 212 the information regarding the
determination target sections and the respective subsequent
sections that have been set. The information regarding a
determination target section and the corresponding subsequent
section that have been set can contain the positions at both ends
separating that determination target section from the corresponding
subsequent section, or can contain the timings of obtaining those
positions, or can contain IDs of those positions.
[0116] More particularly, firstly, the section setting unit 210
sets the section Q.sub.s to Q.sub.1 as a determination target
section in which Q.sub.s serves as the start point and Q.sub.1
serves as the end point; and sets a section Q.sub.1 to K.sub.1 as
the subsequent section in which Q.sub.1 serves as the start point
and K.sub.1 serves as the end point. Then, the section setting unit
210 sends the information about those sections to the determining
unit 212. Subsequently, the section setting unit 210 sets the
section Q.sub.1 to Q.sub.2 as a determination target section in
which Q.sub.1 serves as the start point and Q.sub.2 serves as the
end point; and sets a section Q.sub.2 to Q.sub.3 as the subsequent
section in which Q.sub.2 serves as the start point and Q.sub.3
serves as the end point. Then, the section setting unit 210 sends
the information about those sections to the determining unit 212.
In an identical manner, the section setting unit 210 sets the
section Q.sub.2 to Q.sub.3 as the determination target section,
sets a section Q.sub.3 to Q.sub.4 as the subsequent section, and
sends that information to the determining unit 212. Moreover, the
section setting unit 210 sets the section Q.sub.3 to Q.sub.4 as the
determination target section, sets a section Q.sub.4 to Q.sub.5 as
a subsequent section, and sends that information to the determining
unit 212. Lastly, the section setting unit 210 sets the section
Q.sub.4 to Q.sub.5 as the determination target section and, without
setting a subsequent section, sends that information to the
determining unit 212.
[0117] Thus, while setting the determination target sections and
the respective subsequent sections by performing batch processing;
the section setting unit 210 sets determination target sections in
each of which the start point is either set to the initial point or
set to a principal axis turning point in a sequential manner, and
in each of which the end point is either set to a principal axis
turning point present immediately after the start point, when the
principal axis turning points are present subsequent to the
corresponding start point, or set to the last point when no
principal axis turning point is present subsequent to the
corresponding start point. Moreover, the section setting unit 210
sets subsequent sections in each of which the start point is set to
the end point of the corresponding determination target section,
and in each of which the end point is either set to the direction
turning point present immediately after the corresponding start
point when the direction turning points are present subsequent to
the corresponding start point or set to the last point when no
direction turning point is present subsequent to the corresponding
start point.
[0118] Given below is the explanation regarding setting a
determination target section and the corresponding subsequent
section in real time. While setting the determination target
sections and the respective subsequent sections in real time, the
section setting unit 210 requires a maximum predetermined time
period from the start of setting a determination target section up
to the finalization of that determination target section and the
corresponding subsequent section. That is, since the start of
setting a determination target section at the timing T.sub.i, the
section setting unit 210 finalizes the determination target section
and the corresponding subsequent section before the elapse in a
predetermined time period T.sub.0a, that is, until the point of
time of T.sub.i+T.sub.0a. Thus, in the second embodiment, although
it is assumed that a determination target section and the
corresponding subsequent section are set in real time; in a precise
sense, a delay within a predetermined time period occurs before a
determination target section and the corresponding subsequent
section is set. With that, the determination performed by the
determining unit 212 (described later) also gets delayed within a
predetermined time period.
[0119] Moreover, in the second embodiment, the predetermined time
period T.sub.0a is set to 0.4 seconds. However, that is not the
only possible case. Alternatively, depending on the movement
velocity of the specific part, or the specific bodily part, in a
determination target section, the section setting unit 210 can
change the predetermined time period T.sub.0a. For example, the
predetermined time period T.sub.0a can be set to decrease with an
increase in the movement velocity.
[0120] The specific explanation is given with reference to FIG. 4.
Firstly, even after the obtaining unit 102 obtains the latest point
(the initial point) P.sub.1 that indicates the latest position, the
section setting unit 210 does neither sets a determination target
section nor sets a subsequent section.
[0121] When the obtaining unit 102 obtains the latest point P.sub.2
(assumed to be at a timing T.sub.2), the section setting unit 210
sets the section P.sub.1 to P.sub.2, in which the initial point
P.sub.1 is the start point and in which the latest point P.sub.2 is
the end point at the timing T.sub.2, as the determination target
section. At that point of time, the scope of the determination
target section is yet to be finalized. If another latest point is
obtained after the timing T.sub.2 but before the elapse in the time
period T.sub.0a, then the end point of the determination target
section may be updated. However, in the example illustrated in FIG.
4, since the position P.sub.2 serves as a principal axis turning
point present immediately after the start point, it remains to be
the end point of the determination target section.
[0122] For example, if the latest point P.sub.3 is obtained after
the timing T.sub.2 but before the elapse in the time period
T.sub.0a, then the position P.sub.2 becomes the principal axis
turning point present immediately after the start point. Hence, the
section setting unit 210 finalizes the section P.sub.1 to P.sub.2,
which has the principal axis turning point P.sub.2 as the end
point, as the determination target section. Besides, in this case,
the section setting unit 210 sets the section P.sub.2 to P.sub.3,
which has the principal axis turning point P.sub.2 as the start
point and the latest point P.sub.3 as the end point, as the
subsequent section. Then, as the information about the
determination target section when the timing T.sub.2 is the setting
start timing, the section setting unit 210 sends to the determining
unit 212 the information about the determination target section and
the corresponding subsequent section that have been set
(finalized).
[0123] Besides, for example, even in the case when the latest
points P.sub.3 to P.sub.6 are obtained after the timing T.sub.2 but
before the elapse in the time period T.sub.0a, the position P.sub.2
becomes the principal axis turning point present immediately after
the start point. Hence, the section setting unit 210 finalizes the
section P.sub.1 to P.sub.2, which has the principal axis turning
point P.sub.2 as the end point, as the determination target
section. Moreover, in this case, the section setting unit 210 sets
the section P.sub.2 to P.sub.5, which has the direction turning
point P.sub.5 that is the direction turning point present
immediately after the start point as the end point, as the
corresponding subsequent section. Then, as the information about
the determination target section when the timing T.sub.2 is the
setting start timing, the section setting unit 210 to the
determining unit 212 the information about the determination target
section and the subsequent section that have been set
(finalized).
[0124] Meanwhile, for example, if the latest point P.sub.3 is not
obtained after the timing T.sub.2 and before the elapse in the time
period T.sub.0a, then the position P.sub.2 becomes the latest
point. Hence, the section setting unit 210 finalizes the section
P.sub.1 to P.sub.2, which has the latest point P.sub.2 as the end
point, as the determination target section. In this case, the
section setting unit 210 does not set a subsequent section. Then,
as the information about the determination target section when the
timing T.sub.2 is the setting start timing, the section setting
unit 210 sends to the determining unit 212 the information about
the determination target section that has been finalized.
[0125] Subsequently, when the obtaining unit 102 obtains the latest
point P.sub.2 (assumed to be at a timing T.sub.3), the section
setting unit 210 sets the section P.sub.2 to P.sub.3, in which the
latest principal axis turning point P.sub.2 is the start point and
in which the latest point P.sub.3 is the end point at the timing
T.sub.3, as the determination target section. Herein, if the latest
point P.sub.4 is obtained after the timing T.sub.3 but before the
elapse in the time period T.sub.0a, the section setting unit 210
updates the end point to the latest point P.sub.4 because no
principal axis turning point is present subsequent to the principal
axis turning point P.sub.2. Thus, the section setting unit 210
updates the determination target section to the section P.sub.2 to
P.sub.4. In this case, the section setting unit 210 does not set a
subsequent section. Then, as the information about the
determination target section when the timing T.sub.3 is the setting
start timing, the section setting unit 210 sends to the determining
unit 212 the information about the updated determination target
section.
[0126] Until the obtaining unit 102 obtains the latest point
P.sub.6, the identical operation is performed.
[0127] Meanwhile, instead of outputting to the determining unit 212
the information about a determination target section and the
corresponding subsequent section only after those sections are
finalized, the section setting unit 210 can keep on outputting to
the determining unit 212 the information about a determination
target section and the corresponding subsequent section during the
period from the start of setting the determination target section
to the finalization of those sections.
[0128] Then, the determining unit 212 calculates an evaluation
value of the determination target section and an evaluation value
of the corresponding subsequent section and, based on those
evaluation values, determines which of the first axis and the
second axis is advantageous for the movement of the specific part
in the determination target section. More particularly, when the
evaluation value of the determination target section exceeds the
evaluation value of the corresponding subsequent section, the
determining unit 212 determines that the principal axis in the
determination target section is advantageous for the movement of
the specific part in the determination target section. On the other
hand, when the evaluation value of the determination target section
is equal to or smaller than the evaluation value of the
corresponding subsequent section, the determining unit 212
determines that the principal axis in the subsequent section is
advantageous for the movement of the specific part in the
determination target section.
[0129] For example, while determining the advantageous axis for the
movement of the specific part at a desired determination timing T,
the section setting unit 210 starts the setting of a determination
target section at the desired determination timing T and finalizes
the determination target section before the elapse in the
predetermined time period T.sub.0a from the desired determination
timing T. Then, at that point of time (i.e., at the point of time
of elapse in the predetermined time period T.sub.0a), the
determining unit 212 finalizes the axis that is advantageous for
the movement of the specific part in the determination target
section at the desired determination timing T. Herein, the desired
determination timing points to the setting start timing for setting
a determination target section by the section setting unit 210.
[0130] FIG. 12 is a diagram illustrating another example of a
gesture performed by the specific part according to the second
embodiment. In the example illustrated in FIG. 12, the specific
part follows a trajectory in the upward direction (trajectory 261)
and then follows a trajectory in the right-hand direction
(trajectories 262 and 263). Meanwhile, in the example illustrated
in FIG. 12, the movement distance is used as the evaluation
value.
[0131] During the trajectory 261 (hereinafter, referred to as
"timing T.sub.261"), the trajectory in the upward direction serves
as the determination target section. Herein, assume that, before
the elapse in the predetermined time period T.sub.0a, the
trajectory changes from the trajectory 262 to the trajectory 263.
In that case, as far as the information about a determination
target section and the corresponding subsequent section is
concerned at the time when the desired determination timing (i.e.,
the setting start timing for setting a determination target
section) is the timing T.sub.261; during the trajectory 263, that
is, at the timing obtained as (timing T.sub.261)+(timing T.sub.0a),
the concerned information indicates that the trajectory in the
upward direction still serves as the determination target section
and the trajectory in the right-hand direction serves as the
corresponding subsequent section. Since the movement distance of
the trajectory in the right-hand direction is exceeding the
movement distance of the trajectory in the upward direction, the
determining unit 212 determines that the evaluation value of the
trajectory in the right-hand direction is higher than the
evaluation value of the trajectory in the upward direction.
Accordingly, the determining unit 212 determines that the axis in
the horizontal direction, which is the principal axis in the
subsequent section, is advantageous for the movement of the
specific part in the determination target section. As a result, any
gesture in the vertical direction is considered as noise, and the
horizontal direction is determined to be advantageous as far as the
entire gesture is concerned.
[0132] Meanwhile, in the case when the section setting unit 210
keeps on outputting the information about a determination target
section and the corresponding subsequent section; the determining
unit 212 can keep on determining the axis, which is advantageous
for the movement of the specific part in the determination target
section, for a predetermined time period until the determination
target section and the subsequent section are finalized. In this
case, the determining unit 212 can also calculate the progress rate
of the predetermined time. Herein, the progress rate can be a value
obtained by interpolation to 0 of the timing T.sub.i that is the
point of time of starting the setting of a determination target
section, interpolation to 1 of the timing T.sub.i+T.sub.0a that is
the point of time of elapse in the predetermined time period
T.sub.0a, and linear interpolation of the period from the timing
T.sub.i to the timing T.sub.i+T.sub.0a. Thus, for a timing t where
T.sub.i.ltoreq.t.ltoreq.T.sub.i+T.sub.0a; the progress rate is
(t-T.sub.i)/T.sub.0a.
[0133] FIG. 13 is a diagram for illustrating an example of axes
each being advantageous at a particular progress rate determined by
the determining unit 212 according to the second embodiment. In the
example illustrated in FIG. 13, when the desired determination
timing is the timing T.sub.i, the first axis is advantageous for
the movement of the specific part in the determination target
section during the timing T.sub.i to the timing T.sub.i+20 (i.e.,
during the progress rate of 0 to 0.30) and the second axis is
advantageous for the movement of the specific part in the
determination target section during the timing T.sub.i+30 to the
timing T.sub.i+70 (i.e., during the progress rate of 0.45 to 1.00).
As a result, when the desired determination timing is the timing
T.sub.i, the second axis is finalized as the axis that is
advantageous for the movement of the specific part in the
determination target section.
[0134] Meanwhile, the progress rate can be based on the timing of
obtaining each position that is obtained by the obtaining unit 102.
Moreover, only in the case when the advantageous axis for the
movement of the specific part is determined for a particular
determination target section, the determining unit 212 can be
configured to keep on determining the advantageous axis and
calculating the progress rate of a predetermined time period.
Herein, a particular determination target section either points to
a determination target section to be subjected to the determination
of the advantageous axis immediately after an advantageous axis has
been finalized, or points to a determination target section in
which the movement vector has the movement velocity or the movement
distance equal to or greater than a threshold value.
[0135] FIG. 14 is a diagram for illustrating another example of
axes each being advantageous at a particular progress rate
determined by the determining unit 212 according to the second
embodiment. In the example illustrated in FIG. 14, sequential
determination of advantageous axes and calculation of progress
rates in predetermined time periods is performed only when the axis
advantageous to the movement of the specific part is determined for
a determination target section that is to be subjected to such
determination immediately after an advantageous axis has been
finalized.
[0136] During the timing T.sub.i to the timing T.sub.i+70 (i.e.,
during the progress rate 0 to 1.00), the condition is identical to
that illustrated in FIG. 13. Regarding the determination target
section for which the determination of advantageous axis is started
at the timing T.sub.i, the second axis is finalized as the
advantageous axis at the timing T.sub.i+70. For that reason,
regarding the determination target section for which the
determination of advantageous axis is started at the timing
T.sub.i+80, the determining unit 212 performs sequential
determination of the advantageous axis and calculates the progress
rate of a predetermined time period.
[0137] In the determination target section during the timing
T.sub.i+80 to the timing T.sub.i+150 (i.e., during the progressive
rate of 0 to 1.00), the second axis is the advantageous axis for
the movement of the specific part. As a result, the second axis is
finalized as the axis that is advantageous for the movement of the
specific part in the determination target section for which the
determination of the advantageous axis is started at the timing
T.sub.i+80.
[0138] Regarding the determination target section for which the
determination of the advantageous axis is started at the timing
T.sub.i+80, the second axis is finalized to be the advantageous
axis at the timing T.sub.i+150. Hence, regarding the determination
target section for which the determination of advantageous axis is
started at the timing T.sub.i+160, the determining unit 212
performs sequential determination of the advantageous axis and
calculates the progress rate of a predetermined time period.
[0139] When the determining unit 212 performs sequential
determination, within a predetermined time period, of the
advantageous axis for the movement of the specific part in a
determination target section, the presenting unit 214 can be
configured to further perform supplementary presentation with a
predetermined time period, that is, depending on the determination
result before the determination target section and the
corresponding subsequent section are finalized. In the second
embodiment, when the setting of a determination target section are
started at the timing T.sub.i, the advantageous axis for the
movement of the specific part in that determination target section
is finalized by the timing T.sub.i+T.sub.0a. Therefore, the
presenting unit 214 performs supplementary presentation depending
on the determination result obtained by the determining unit 212
between the timings T.sub.i and T.sub.i+T.sub.0a. After the timing
T.sub.i+T.sub.0a, by which the advantageous axis for the movement
of the specific part is finalized, the presentation performed is
identical to that described in the first embodiment.
[0140] FIG. 15 is an example of the supplementary presentation
performed by the presenting unit 214 according to the second
embodiment. For example, with reference to FIG. 13, at the timing
T.sub.i+30, the advantageous axis for the movement of the specific
part in a determination target section for which the settings are
started at the timing T.sub.i changes from the first axis to the
second axis. At that point of time, with reference to the screen
180 illustrated in FIG. 7 in which the object 181 is displayed on
the left side, the object 182 is displayed in the center, and the
object 183 is displayed on the right side; the presenting unit 214
can display a screen 280 as illustrated in FIG. 15 in which the
objects 181 to 183 are tilted in the horizontal direction (in the
left-hand direction). Then, at the timing T.sub.i+70, once the
second axis is finalized as the advantageous axis for the movement
of the specific part in a determination target section for which
such determination is started at the time timing T.sub.i; the
presenting unit 214 changes the screen 280 to the screen 180
illustrated in FIG. 7 in which the object 181 is displayed on the
left side, the object 182 is displayed in the center, and the
object 183 is displayed on the right side. As a result, it becomes
possible to notify the user about the axis that is being determined
as the advantageous axis.
[0141] FIGS. 16 and 17 are diagrams illustrating other examples of
the supplementary presentation performed by the presenting unit 214
according to the second embodiment. With reference to the screen
180 illustrated in FIG. 7 in which the object 181 is displayed on
the left side, the object 182 is displayed in the center, and the
object 183 is displayed on the right side; the presenting unit 214
can be configured to display, not the screen 280 illustrated in
FIG. 15, but a screen 281 in which the objects 181 to 183 are
rotated in horizontal direction (the left-hand direction) or a
screen 282 in which an arrow 283 is additionally displayed to
indicate the horizontal direction (the left-hand direction).
[0142] FIG. 18 is a flowchart for explaining an exemplary sequence
of operations performed in the recognition apparatus 200 according
to the second embodiment.
[0143] Herein, the operations from Steps S201 to S204 are identical
to the operations from Steps S101 to S104 in the flowchart
illustrated in FIG. 8.
[0144] Subsequently, the section setting unit 210 sets a
determination target section, in which a start point is either set
to the latest principal axis turning point when principal axis
turning points are present or set to the initially-obtained
position when principal axis turning points are absent, and in
which an end point is either set to the principal axis turning
point present immediately after the start point, when the principal
axis turning points are present subsequent to the start point but
before the elapse in a predetermined time period since the setting
of the start point, or set to the latest point that indicates the
most-recently obtained position before the elapse in a
predetermined time period, when no principal axis turning point is
present subsequent to the start point but before the elapse in the
predetermined time. Moreover, the section setting unit 210 sets a
subsequent section, in which a start point is set to the end point
of the abovementioned determination target section and in which an
end point is either set to the direction turning point present
immediately after the start point when such the direction turning
points are present subsequent to the start point or set to the
latest point when the latest point is present subsequent to the
start point (Step S205).
[0145] Then, the determining unit 212 calculates an evaluation
value of the determination target section and an evaluation value
of the corresponding subsequent section and, based on those
evaluation values, determines which of the first axis and the
second axis is advantageous for the movement of the specific part
in the determination target section (Step S206).
[0146] Every time the obtaining unit 102 obtains the latest
position of the specific part of the photographic subject (Step
S201), the operations from Step S202 to Step S206 are performed,
and the presenting unit 214 performs presentation according to the
determination result obtained by the determining unit 212.
[0147] FIG. 19 is a flowchart for explaining an exemplary sequence
of operations performed by the determining unit 212 in the
recognition apparatus 200 according to the second embodiment.
[0148] Firstly, the determining unit 212 checks whether or not a
subsequent section is present (Step S230).
[0149] When a subsequent section is present (Yes at Step S230), the
determining unit 212 calculates an evaluation value E.sub.c of the
determination target section and an evaluation value E.sub.a of the
subsequent section, and determines whether or not the evaluation
value E.sub.c is equal to or smaller than the evaluation value
E.sub.a (Step S231).
[0150] If the evaluation value E.sub.c is equal to or smaller than
the evaluation value E.sub.a (Yes at Step S231), the determining
unit 212 determines that the principal axis in the subsequent
section is advantageous for the movement of the specific part in
the determination target section (Step S232). That marks the end of
the operations.
[0151] Meanwhile, if a subsequent section is absent (No at Step
S230) or if the evaluation value E.sub.c is not equal to or smaller
than the evaluation value E.sub.a (No at Step S231), the
determining unit 212 determines that the principal axis in the
determination target section is advantageous for the movement of
the specific part in the determination target section (Step S233).
That marks the end of the operations.
[0152] In the recognition apparatus 200, there are times when a
determination target section and the corresponding subsequent
section get finalized before the elapse in a predetermined time
period since the start of setting of the determination target
section, and there are times when the advantageous axis for the
movement of the specific part in the determination target section
gets finalized before the elapse in a predetermined time period.
For example, before the elapse in a predetermined time period since
the start of setting of a determination target section, if a
principal axis turning point is present subsequent to the start
point of the determination target period and if a direction turning
point is present subsequent to the principal axis turning point;
then the determination target section and the corresponding
subsequent section get finalized before the elapse in a
predetermined time period. Moreover, before the elapse in a
predetermined time period since the start of setting of a
determination target section, if the evaluation value of the
subsequent section exceeds the evaluation value of the
determination target section, the advantageous axis for the
movement of the specific part in the determination target section
gets finalized to the principal axis of the subsequent section.
That eliminates the need of waiting for the elapse in the
predetermined time period. Thus, the determining unit 212 can be
configured to take these points into account while performing the
operations.
[0153] FIG. 20 is a flowchart for explaining another exemplary
sequence of operations performed by the determining unit 212 in the
recognition apparatus 200 according to the second embodiment.
[0154] Firstly, the determining unit 212 calculates a difference
between a current timing T and a desired determination timing
T.sub.c that is the timing of starting the setting of a
determination target section (Step S241), and determines whether or
not a predetermined time period T.sub.0a has elapsed since the
desired determination timing T.sub.e (Step S242).
[0155] If the predetermined time period T.sub.0a has not elapsed
since the desired determination timing T.sub.c (No at Step S242),
the determining unit 212 determines whether or not a subsequent
section is present (Step S243). If a subsequent section is absent
(No at Step S243), the system control returns to Step S241.
[0156] On the other hand, when a subsequent section is present (Yes
at Step S243), the determining unit 212 calculates an evaluation
value E.sub.c of the determination target section and an evaluation
value E.sub.a of the subsequent section, and determines whether or
not the evaluation value E.sub.a is exceeding the evaluation value
E.sub.c (Step S244).
[0157] If the evaluation value E.sub.a is exceeding the evaluation
value E.sub.c (Yes at Step S244), the determining unit 212
determines that the principal axis in the subsequent section is
advantageous for the movement of the specific part in the
determination target section (Step S247). That marks the end of the
operations.
[0158] In contrast, if the evaluation value E.sub.a is not greater
than the evaluation value E.sub.c (No at Step S244), the
determining unit 212 checks whether or not the subsequent section
has been finalized (Step S245). More particularly, the determining
unit 212 checks whether or not the end point of the subsequent
section is a direction turning point.
[0159] If the subsequent section has been finalized (Yes at Step
S245), that is, if the end point of the subsequent section is a
direction turning point; because of the fact that the evaluation
value E.sub.a does not exceed the evaluation value E.sub.c, the
determining unit 212 determines that the principal axis in the
determination target section is advantageous for the movement of
the specific part in the determination target section (Step S246).
That marks the end of the operations.
[0160] On the other hand, if the subsequent section has not been
finalized (No at Step S245), that is, if the end point of the
subsequent section is not a direction turning point; the system
control returns to Step S241.
[0161] Hence, when the predetermined time period T.sub.0a has
elapsed since the desired determination timing T.sub.c (Yes at Step
S242), either a subsequent section is absent or the evaluation
value E.sub.a does not exceed the evaluation value E.sub.c. In that
case, the determining unit 212 determines that the principal axis
in the determination target section is advantageous for the
movement of the specific part in the determination target section
(Step S246). That marks the end of the operations.
[0162] Meanwhile, when a subsequent section is present (Yes at Step
S243), it means that the determination target section has been
finalized. That enables the usage of the evaluation value E.sub.c
that has been already calculated. Thus, the evaluation value
E.sub.c need not be calculated again.
[0163] As described above, in the second embodiment, the gesture in
a determination target section is recognized using the
corresponding subsequent section. Thus, according to the second
embodiment, it becomes possible to determine whether or not the
gesture in a determination target section is noise. Because of
that, even if a movement representing noise is included in a
gesture, such movement can be recognized correctly thereby enabling
prevention of a decline in the recognition accuracy.
[0164] For example, if a movement representing noise is included at
the start of a gesture, then the noise portion represents the
determination target section and the portion of the normal gesture
represents the subsequent section. There, by referring to the
subsequent section, it becomes possible to determine that the
gesture in the determination target section is noise and to make
substitution with the movement direction of the normal gesture in
the subsequent section. In this way, even when a movement
representing noise is included in a gesture, the noise can be
recognized in a correct manner.
Third Embodiment
[0165] In a third embodiment, the explanation is given for an
example in which the gesture in a determination target section is
recognized by making use of the corresponding previous section as
well as the corresponding subsequent section; and in which the
direction indicated along the advantageous axis, which is
advantageous for the movement of the specific part, or the specific
bodily part, in the determination target section, is also
recognized. In the following explanation, the focus is on
elaborating the differences with the first embodiment. Moreover,
the constituent elements having identical functions to the
functions described in the first embodiment are referred to by the
same names and reference numerals, and the explanation thereof is
not repeated.
[0166] FIG. 21 is a configuration diagram illustrating an example
of a recognition apparatus 300 according to the third embodiment.
In the recognition apparatus 300 according to the third embodiment;
a section setting unit 310, a determining unit 312, and a
presenting unit 314 are different as compared to the first
embodiment. Besides, the recognition apparatus 300 further includes
a first axis determining unit 316, a second axis determining unit
318, and a determination result selecting unit 320.
[0167] The section setting unit 310 has the combined functions of
the section setting unit 110 according to the first embodiment and
the section setting unit 210 according to the second
embodiment.
[0168] That is, the section setting unit 310 sets a determination
target section, in which a start point is either set to the latest
principal axis turning point when principal axis turning points are
present or set to the initially-obtained position when principal
axis turning points are absent, and in which an end point is set to
the latest point that indicates the most-recently obtained
position. Moreover, the section setting unit 310 sets a previous
section, in which an end point is set to the start point of the
determination target section and in which a start point is either
set to the direction turning point present immediately before the
end point when the direction turning points are present before the
end point or set to the initial point when no direction turning
point is present before the end point. At this point of time, the
previous section does get finalized but the determination target
section is not yet finalized.
[0169] Then, the section setting unit 310 updates the end point
either to the principal axis turning point present immediately
after the start point, when the principal axis turning points are
present subsequent to the start point but before the elapse in a
predetermined time period since the setting of the start point, or
to the latest point that indicates the most-recently obtained
position after the elapse in a predetermined time period, when no
principal axis turning point is present subsequent to the start
point but before the elapse in the predetermined time. Moreover,
the section setting unit 310 sets a subsequent section, in which a
start point is set to the end point of the updated determination
target section and in which an end point is either set to the
direction turning point present immediately after the start point
when the direction turning points are present subsequent to the
start point or set to the latest point present after the elapse in
a predetermined time period when latest points are present
subsequent to the start point.
[0170] As described above, the section setting unit 310 has the
combined functions of the section setting unit 110 according to the
first embodiment and the section setting unit 210 according to the
second embodiment. Thus, the determining unit 312 calculates an
evaluation value of the determination target section, an evaluation
value of the corresponding previous section, and an evaluation
value of the corresponding subsequent section; and, based on those
evaluation values, determines which of the first axis and the
second axis is advantageous for the movement of the specific part
in the determination target section.
[0171] More particularly, if the evaluation value of the
determination target section is exceeding the evaluation value of
the previous section or if the previous section is absent, as well
as if the evaluation value of the determination target section is
exceeding the evaluation value of the subsequent section or if the
subsequent section is absent; the determining unit 312 determines
that the principal axis in the determination target section is
advantageous for the movement of the specific part in the
determination target section.
[0172] When the evaluation value of the determination target
section is equal to or smaller than the evaluation value of the
previous section, the determining unit 312 determines that the
principal axis in the previous section is advantageous for the
movement of the specific part in the determination target section.
As a result, without having to wait for the elapse in a
predetermined time period, the principal axis in the previous
section can be determined to be advantageous for the movement of
the specific part in the determination target section.
[0173] When the evaluation value of the determination target
section is equal to or smaller than the evaluation value of the
subsequent section, the determining unit 312 determines that the
principal axis in the subsequent section is advantageous for the
movement of the specific part in the determination target
section.
[0174] The first axis determining unit 316 and the second axis
determining unit 318 respectively determine the advantageous
direction along the first axis and the advantageous direction along
the second axis by referring to the movement vectors calculated by
the calculating unit 104. Thus, the first axis determining unit 316
determines the direction indicated along the first axis and the
second axis determining unit 318 determines the direction indicated
along the second axis. For example, if the first axis serves as the
axis in the vertical direction, then the first axis determining
unit 316 determines whether the upward direction or the downward
direction is indicated along the first axis. Similarly, if the
second axis serves as the axis in the horizontal direction, then
the second axis determining unit 318 determines whether the
left-hand direction or the right-hand direction is indicated along
the second axis. In the following explanation, indicating a
particular direction along an axis is referred to as performing
"feed action", in which the user performs an action of waving the
specific part such as a hand so as to indicate the direction of
waving.
[0175] In order to determine the direction along the second axis in
which a feed action is performed, the second axis determining unit
318 can implement the following determination method. Herein,
although the explanation is given with reference to the second axis
determining unit 318, the same explanation is also applicable to
the first axis determining unit 316.
[0176] When the velocity of the second axis component of a movement
vector (the inner product of the movement vector and the second
axis) calculated by the calculating unit 104 becomes equal to or
greater than the a threshold value, the second axis determining
unit 318 determines that a feed action is performed in the
direction of the movement vector in the second axis.
[0177] Herein, it is assumed that a position P.sub.s, which is the
start point of a movement vector R.sub.s and which is obtained by
the obtaining unit 102 at a timing T.sub.s, has coordinates
(x.sub.s, y.sub.s). Similarly, it is assumed that a position
P.sub.s+1, which is the end point of the movement vector R.sub.s
and which is obtained by the obtaining unit 102 at a timing
T.sub.s+1, has coordinates (x.sub.s+1, y.sub.s+1). In that case,
the second axis component (the x-axis component) of the movement
vector R.sub.s becomes x.sub.s+1-x.sub.s, and a second axis
component S.sub.s of the movement vector R.sub.s per unit of time
becomes S.sub.s=(x.sub.s+1-x.sub.s)/(T.sub.s+1-T.sub.s).
[0178] Moreover, a velocity V.sub.s of the second axis component of
the movement vector R.sub.s is expressed as V.sub.s=|S.sub.s|. The
second axis component of the movement vector R.sub.s has the
positive direction if the sign of S.sub.s is positive or has the
negative direction if the sign of S.sub.s is negative. Hence, when
the velocity V.sub.s becomes equal to or greater than a threshold
value, the second axis determining unit 318 determines that, if the
sign of S.sub.s is positive, a feed action is performed in the
positive direction along the second axis and determines that, if
the sign of S.sub.s is negative, a feed action is performed in the
negative direction along the second axis.
[0179] Once it is determined that a feed action has been performed,
the second axis determining unit 318 can stop performing
determination until there is a change in the direction along the
second axis component of the movement vector and can make use of
the determination result. When there is a change in the sign of
S.sub.s, the second axis determining unit 318 determines that a
change has occurred in the direction along the second axis
component of the movement vector. Meanwhile, the second axis
determining unit 318 can have zeros sandwiched in between until
there is a change in the sign of S.sub.s. For example, if the sign
of S.sub.s-2 is positive, the sign of S.sub.s-1 is 0, and the sign
of S.sub.s is negative; then, at the point of time of S.sub.s-1,
the second axis determining unit 318 does not determine that there
is a change in the direction along the second axis component.
Rather, it is at the point of time of S.sub.s that the second axis
determining unit 318 determines that there is a change in the
direction along the second axis component.
[0180] Moreover, upon determining that a feed action has been
performed, in order to ensure that that a feed action in the
opposite direction (hereinafter, referred to as "return action") is
not determined to have been performed, the second axis determining
unit 318 can be configured to ignore a return action performed
immediately after the determination that a feed action has been
performed.
[0181] Moreover, alternatively, the second axis determining unit
318 can be firstly configured to not determine that a feed action
in the opposite direction (hereinafter, referred to as "backswing
action") is performed prior to determining that a feed action has
been performed. Secondly, upon determining that a feed action has
been performed, the second axis determining unit 318 can be
configured to not determine that a feed action in the opposite
direction (hereinafter, referred to as "return action") is
performed. In that case, the second axis determining unit 318
refers to the previous section and to the subsequent section of an
operation section that serves as the determination target section,
and accordingly determines whether or not a feed action was
performed in that operation section. Herein, an operation section
points to a section in which a start point is either set to a
second-axis direction turning point when such a second-axis
direction turning point is present at which the direction along the
second axis component of the movement vector changes, or set to the
initial point indicating the initially-obtained position when a
second-axis direction turning point is absent; and in which an end
point is either set to a second-axis direction turning point
present immediately after the start point, when such a second-axis
direction turning point is present immediately after the start
point, or set to the last point that indicates the latest obtained
position, when a second-axis direction turning point is not present
immediately after the start point.
[0182] More particularly, firstly, the second axis determining unit
318 sets second-axis direction turning points for the vectors
calculated by the calculating unit 104. In the third embodiment, it
is assumed that the second axis determining unit 318 determines in
real time whether or not a feed action has been performed in an
operation section. However, that is not the only possible case.
Alternatively, that determination can also be done by performing
batch processing, that is, after the gesture has been completed.
Herein, for the sake of convenience, firstly, the explanation is
given for the case when the determination of whether or not a feed
action has been performed in an operation section is done by
performing batch processing. Then, the explanation is given for the
case when the determination is done in real time.
[0183] FIG. 22 is a diagram of an example of setting second-axis
direction turning points by the second axis determining unit 318
according to the third embodiment. In the example illustrated in
FIG. 22, "U.sub.s" indicates a start point that is the
initially-obtained position obtained by the obtaining unit 102;
"U.sub.1" and "U.sub.2" indicate second-axis direction turning
points; and "U.sub.e" indicates an end point that is the last
position obtained by the obtaining unit 102. In that case, the
second axis determining unit 318 sets the section U.sub.s to
U.sub.1, the section U.sub.1 to U.sub.2, and the section U.sub.2 to
U.sub.e as operation sections in that order; and determines whether
or not a feed action has been performed in each of those operation
sections.
[0184] More particularly, when the section U.sub.s to U.sub.1 is
set as the operation section, the second axis determining unit 318
sets the section U.sub.1 to U.sub.2 as the corresponding subsequent
section and calculates the evaluation values of the operation
section and the subsequent section. Accordingly, the second axis
determining unit 318 determines whether or not a feed action was
performed in the section U.sub.s to U.sub.1 set as the operation
section. In an identical manner, when the section U.sub.1 to
U.sub.2 is set as the operation section, the second axis
determining unit 318 sets the section U.sub.s to U.sub.1 as the
corresponding previous section and sets the section U.sub.2 to
U.sub.e as the corresponding subsequent section, and calculates the
evaluation values of the operation section, the previous section,
and the subsequent section. Accordingly, the second axis
determining unit 318 determines whether or not a feed action was
performed in the section U.sub.1 to U.sub.2 set as the operation
section. Similarly, when the section U.sub.2 to U.sub.e is set as
the operation section, the second axis determining unit 318 sets
the section U.sub.1 to U.sub.2 as the corresponding previous
section and calculates the evaluation values of the operation
section and the previous section. Accordingly, the second axis
determining unit 318 determines whether or not a feed action was
performed in the section U.sub.2 to U.sub.e set as the operation
section.
[0185] Herein, in order to determine that a feed action was
performed in an operation section, the following conditions need to
be satisfied: an evaluation value E.sub.n of that operation section
is equal to or greater than a threshold value; and the evaluation
value E.sub.n of that operation section is exceeding an evaluation
value E.sub.m of the corresponding previous section and greater
than an evaluation value E.sub.1 of the corresponding subsequent
section. The evaluation values mentioned above are identical to the
evaluation values used by the determining unit 312. However,
herein, the movement distance can be considered to be the sum of
the first axis components of the movement vectors in the operation
section (i.e., modulus of the inner products of the movement
vectors and the first axis). Moreover, the movement velocity can be
considered to be the maximum velocity among the velocities of the
first axis components of the movement vectors in the operation
section or can be considered to be the average velocity of the
velocities of the movement vectors that are equal to or greater
than a predetermined threshold value. Furthermore, the movement
acceleration can be considered to be the maximum rate of
acceleration from among the rates of acceleration of the first axis
components of the movement vectors in the operation section or can
be considered to be the average of the rates of acceleration of the
movement vectors that are equal to or greater than a predetermined
threshold value. Meanwhile, when a previous section is not present,
the evaluation value E.sub.m becomes zero; and when a subsequent
section is not present, the evaluation value E.sub.1 becomes
zero.
[0186] When the section U.sub.s to U.sub.1 is set as the operation
section, the second axis determining unit 318 does not set a
previous section but sets the section U.sub.1 to U.sub.2 as the
corresponds subsequent section. In that case, since the previous
section is absent or since the evaluation value E.sub.n exceeds the
evaluation value E.sub.1, the second axis determining unit 318
considers the operation section to be at an advantage. Moreover, if
the evaluation value E.sub.n is equal to or greater than a
threshold value, then the second axis determining unit 318
determines that a feed action was performed in the section U.sub.s
to U.sub.1 set as the operation section. Meanwhile, the second axis
determining unit 318 determines that an operation in the section
U.sub.1 to U.sub.2 is a return action in the section U.sub.s to
U.sub.1.
[0187] When the section U.sub.1 to U.sub.2 is set as the operation
section, the section U.sub.e to U.sub.1 is set as the corresponding
previous section, and the section U.sub.2 to U.sub.e is set as the
corresponding subsequent section; the evaluation value E.sub.n
becomes equal to or smaller than the evaluation value E.sub.m and
the evaluation value E.sub.n. Hence, the second axis determining
unit 318 determines that a feed action was not performed in the
section U.sub.1 to U.sub.2 set as the operation section. That is,
the second axis determining unit 318 determines that an operation
in the section U.sub.1 to U.sub.2 is a return action in the section
U.sub.s to U.sub.1.
[0188] When the section U.sub.2 to U.sub.e is set as the operation
section, the second axis determining unit 318 sets the section
U.sub.1 to U.sub.2 as the previous section and does not set a
subsequent section. In this case, since the evaluation value
E.sub.n exceeds the evaluation value E.sub.m or since the
subsequent section is not present, the second axis determining unit
318 considers the operation section to be at an advantage.
Moreover, if the evaluation value E.sub.n is equal to or greater
than a threshold value, the second axis determining unit 318
determines that a feed action was performed in the section U.sub.2
to U.sub.e set as the operation section. Meanwhile, the second axis
determining unit 318 determines that an operation in the section
U.sub.1 to U.sub.2 is a backswing action in the section U.sub.2 to
U.sub.e.
[0189] Explained above was the case when the determination of
whether or not a feed action was performed in an operation section
is done by performing batch processing. In the case of performing
such determination in real time, the task of setting a subsequent
section leads to a delay by a predetermined time period in the
determination of whether or not a feed action was performed in the
corresponding operation section. Herein, that predetermined time
period can be set to, for example, 0.6 seconds. Alternatively, the
predetermined time period can be changed depending on the movement
velocity of the specific part in the operation section. For
example, as the movement velocity goes on increasing, the
predetermined time period can be reduced. In such a case, during
the predetermined time period, that is, during the period before
the advantageous direction in the operation section is finalized;
the second axis determining unit 318 can be configured to perform
sequential determination of the advantageous direction in the
operation section and to calculate the progress rate until the
advantageous direction in the operation section is finalized.
Herein, the progress rate can be expressed in an identical manner
to the progress rate calculated by the determining unit 312.
Moreover, only when there is a change in the direction determined
to be advantageous in the operation section, the second axis
determining unit 318 can be configured to output the advantageous
direction (i.e., the determination result). Herein, although the
explanation is given with reference to the second axis determining
unit 318, the same explanation is also applicable to the first axis
determining unit 316.
[0190] FIG. 23 is a diagram illustrating exemplary advantageous
directions each corresponding to a progress rate and determined by
the second axis determining unit 318 according to the third
embodiment. During the timing T.sub.i to the timing T.sub.i+10, the
evaluation value of the operation section falls below a threshold
value or becomes equal to smaller than the corresponding previous
section. Hence, the corresponding progress rate is not calculated
and the advantageous direction in the operation section in also not
determined.
[0191] During the timing T.sub.i+20 to the timing T.sub.i+80 (i.e.,
during the progress rate of 0 to 0.90), the operation section
becomes more advantageous as compared to the corresponding previous
section, and the advantageous direction in the operation section is
leftward. At the timing T.sub.i+90 (i.e., at the progress rate of
1.00), the operation section becomes more advantageous as compared
to the corresponding previous section as well as the corresponding
subsequent section, and the advantageous direction in the operation
section is finalized to the left-hand direction. As a result, the
second axis determining unit 318 determines that a feed action in
the left-hand direction was performed in the operation section.
[0192] During the timing T.sub.i+100 to the timing T.sub.i+130,
either the operation section continues without modification even
after the feed action has been performed, or the evaluation value
of the operation section falls below a threshold value or becomes
equal to or smaller than the corresponding previous section. Hence,
the corresponding progress rate is not calculated and the
advantageous direction in the operation section in also not
determined.
[0193] During the timing T.sub.i+140 to the timing T.sub.i+150
(i.e., during the progress rate of 0 to 0.15), the operation
section becomes more advantageous as compared to the corresponding
previous section, and the advantageous direction in the operation
section is rightward.
[0194] During the timing T.sub.i+160 to the timing T.sub.i+220
(i.e., during the progress rate of 0 to 0.90), the operation
section becomes more advantageous as compared to the corresponding
previous section as well as the corresponding subsequent section,
and the advantageous direction in the operation section is
leftward. Moreover, at the timing T.sub.i+230 (i.e., at the
progress rate of 1.00), the operation section becomes more
advantageous as compared to the corresponding previous section as
well as the corresponding subsequent section, and the advantageous
direction in the operation section is finalized to the left-hand
direction. As a result, the second axis determining unit 318
determines that a feed action in the left-hand direction was
performed in the operation section.
[0195] Meanwhile, from the determination results regarding the
first axis and the second axis, the determination result selecting
unit 320 selects the determination result of the axis that is
determined by the determining unit 312 to be advantageous for the
movement of the specific part in a determination target
section.
[0196] For example, assume that the advantageous axes determined by
the determining unit 312 are as illustrated in FIG. 14, assume that
the second axis determining unit 318 determines (finalizes) that a
feed action in the left-hand direction was performed at the timing
T.sub.i, and assume that the first axis determining unit 316
determines (finalizes) that a feed action in the upward direction
was performed at the timing T.sub.i. In such a case, the
determination result selected by the determination result selecting
unit 320 is as follows.
[0197] Regarding the determination, started at the timing T.sub.i,
of the advantageous axis for the movement of the specific part in a
determination target section, the determining unit 312 finalizes
the second axis as the advantageous axis at the timing T.sub.i+70.
Therefore, at the timing T.sub.i+70, the determination result
selecting unit 320 selects the determination result at the timing
T.sub.i as obtained by the second axis determining unit 318 at the
timing T.sub.i, and assumes that a feed action in the left-hand
direction along the second axis was performed at the timing
T.sub.i.
[0198] Meanwhile, during the timing T.sub.i to the timing
T.sub.i+20, the determination result selecting unit 320 can be
configured to select the determination result obtained by the first
axis determining unit 316 obtained at the timing T.sub.i.
Similarly, during the timing T.sub.i+30 to the timing T.sub.i+60,
the determination result selecting unit 320 can be configured to
select the determination result obtained by the second axis
determining unit 318 obtained at the timing T.sub.i. That is,
during the timing T.sub.i to the timing T.sub.i+20, the feed action
candidate at the timing T.sub.i can be set to the feed action in
the upward direction along the first axis; and during the timing
T.sub.i+30 to the timing T.sub.i+60, the feed action candidate at
the timing T.sub.i can be set to the feed action in the left-hand
direction along the second axis. Meanwhile, the feed action
candidate can be a combination of the feed action and the progress
rate. Regarding the direction of a feed action at a particular
timing, it is possible to make use of the direction of the feed
action in the axis determined by the determining unit 312 to be
advantageous at that timing. Regarding the progress rate, the
progress rate obtained by the determining unit 312 can be used.
[0199] FIG. 24 is a diagram of feed action candidates each selected
corresponding to a progress rate by the determination result
selecting unit 320. During the timing T.sub.i to the timing
T.sub.i+20 (i.e., during the progress rate of 0 to 0.30), the feed
action candidate at the timing T.sub.i indicates a feed action in
the left-hand direction. During the timing T.sub.i+30 to the timing
T.sub.i+60 (i.e., during the progress rate of 0.45 to 0.90), the
feed action candidate at the timing T.sub.i indicates a feed action
in the upward direction. At the timing T.sub.i+70 (i.e., at the
progress rate of 1.00), the feed action candidate at the timing
T.sub.i is finalized to a feed action in the upward direction.
[0200] In the case when each of the first axis determining unit 316
and the second axis determining unit 318 performs sequential
determination of the axis advantageous in an operation section
while calculating the progress rates; the determination result
selecting unit 320 can be configured to select, from the
determination results of the first axis and the second axis, the
determination result and the progress rate for the advantageous
axis for the movement of the bodily part in a determination target
section determined by the determining unit 312.
[0201] Herein, the explanation is given for an example in which, as
the determination result regarding the axis determined by the
determining unit 312 to be advantageous for the movement of the
specific part in a determination target section, the determination
result selecting unit 320 selects the determination result
regarding the second axis and the corresponding progress rate.
However, the same explanation is also applicable to the case of
selecting the determination result regarding the first axis and the
corresponding progress rate.
[0202] Once the second axis determining unit 318 starts calculating
the progress rate, the determining unit 312 also starts calculating
the progress rate. That is, once the second axis determining unit
318 starts calculating the progress rate at the timing T.sub.i, the
determining unit 312 also starts calculating the progress rate at
the timing T.sub.i. For a certain period of time, the calculation
of progress rates is carried on.
[0203] Herein, in order to simplify the explanation, during the
progress rate of 0.00 to 0.99 calculated by the determining unit
312, if the determining unit 312 has determined that the second
axis is the advantageous axis for the movement of the specific part
in a determination target section, it is assumed that the
determining unit 312 outputs a notification that the second axis is
the advantageous axis candidate. Moreover, at the progress rate of
1.00 calculated by the determining unit 312, if the determining
unit 312 has determined that the second axis is the advantageous
axis for the movement of the specific part in a determination
target section, it is assumed that the determining unit 312 outputs
a notification that the second axis is finalized as the
advantageous axis. In a case other than the abovementioned cases,
that is, during the progress rate of 0.00 to 1.00 calculated by the
determining unit 312, if the determining unit 312 has determined
that the first axis is the advantageous axis for the movement of
the specific part in a determination target section, it is assumed
that the determining unit 312 does not perform any output.
[0204] During a predetermined progress rate (for example, 0.00 to
0.25) calculated by the second axis determining unit 318, once the
determining unit 312 outputs that the second axis is the
advantageous axis candidate or that the second axis is finalized as
the advantageous axis; the determination result selecting unit 320
starts, at that point of time as the start timing, the selection of
the determination result of the second axis determining unit 318
and the corresponding progress rate. In any other case, the
determination result selecting unit 320 does not select the
determination result of the second axis determining unit 318 and
the corresponding progress rate.
[0205] Then, if the determining unit 312 stops performing the
output during the period from the start timing of starting the
selection until the timing at which the progress rate calculated by
the second axis determining unit 318 becomes 1.00, then the
determination result selecting unit 320 stops the selection at the
timing at which the determining unit 312 has stopped performing the
output.
[0206] Moreover, assume that, during the period from the start
timing of starting the selection until the timing at which the
progress rate calculated by the second axis determining unit 318
becomes 1.00, the determining unit 312 keeps on performing the
output and outputs a notification that the second axis is finalized
as the advantageous axis. In that case, irrespective of the
determination result obtained subsequently by the second axis
determining unit 318, the determination result selecting unit 320
keeps on selecting the progress rates calculated by the second axis
determining unit 318. Then, at the point of time when the progress
rate becomes 1.00, the second axis determining unit 318 finalizes
that a feed action has been performed in the direction indicated by
the determination result of the second axis determining unit
318.
[0207] Furthermore, assume that, even when the period from the
start timing of starting the selection reaches the timing at which
the progress rate calculated by the second axis determining unit
318 becomes 1.00, the determining unit 312 is still outputting a
notification that the second axis is the advantageous axis
candidate. In that case, until the determining unit 312 outputs a
notification that the second axis is finalized as the advantageous
axis, the determination result selecting unit 320 keeps on
selecting the progress rate of 0.99 as the progress rate calculated
by the second axis determining unit 318. Once the determining unit
312 outputs a notification that the second axis is finalized as the
advantageous axis, the determination result selecting unit 320
selects the progress rate of 1.00 and finalizes that a feed action
has been performed in the direction indicated by the second axis
determining unit 318. However, while the determination result
selecting unit 320 keeps on selecting the progress rate of 0.99 as
the progress rate calculated by the second axis determining unit
318; if the determining unit 312 stops performing the output, then
the determination result selecting unit 320 stops the selection at
the timing at which the determining unit 312 has stopped performing
the output.
[0208] Herein, the presenting unit 314 performs presentation
according to the selection result obtained by the determination
result selecting unit 320. In the case when, from the determination
results regarding the first axis and the second axis, the
determination result selecting unit 320 selects the determination
result of the advantageous axis for the movement of the specific
part in a determination target section determined by the
determining unit 312 and selects the corresponding progress rate;
the presenting unit 314 can perform supplementary presentation, as
explained in the second embodiment, until the selection result is
finalized, that is, while the selection is underway. Once the
selection result is finalized, the subsequent presentation is
identical to that explained in the first embodiment.
[0209] FIG. 25 is a flowchart illustrating an exemplary sequence of
operations performed in the recognition apparatus 300 according to
the third embodiment.
[0210] Herein, the operations from Steps S301 to S304 are identical
to the operations from Steps S101 to S104 in the flowchart
illustrated in FIG. 8.
[0211] Subsequently, the section setting unit 310 sets a
determination target section, in which a start point is either set
to the latest principal axis turning point when principal axis
turning points are present or set to the initially-obtained
position when principal axis turning points are absent, and in
which an end point is set to the latest point that indicates the
most-recently obtained position. Moreover, the section setting unit
310 sets a previous section, in which an end point is set to the
start point of the determination target section and in which a
start point is either set to the direction turning point present
immediately before the end point when the direction turning points
are present before the end point or set to the initial point when
no direction turning point is present before the end point. Then,
the section setting unit 310 updates the end point either to the
principal axis turning point present immediately after the start
point, when the principal axis turning points are present
subsequent to the start point but before the elapse in a
predetermined time period since the setting of the start point, or
to the latest point that indicates the most-recently obtained
position after the elapse in a predetermined time period, when no
principal axis turning point is present subsequent to the start
point but before the elapse in the predetermined time. Furthermore,
the section setting unit 310 sets a subsequent section, in which a
start point is set to the end point of the updated determination
target section and in which an end point is either set to the
direction turning point present immediately after the start point
when the direction turning points are present subsequent to the
start point or set to the latest point present after the elapse in
a predetermined time period when latest points are present
subsequent to the start point (Step S305).
[0212] Then, the determining unit 312 calculates an evaluation
value of the determination target section, an evaluation value of
the corresponding previous section, and an evaluation value of the
corresponding subsequent section; and, based on those evaluation
values, determines which of the first axis and the second axis is
advantageous for the movement of the specific part in the
determination target section (Step S306).
[0213] The first axis determining unit 316 refers to the movement
vectors calculated by the calculating unit 104 and accordingly
determines the advantageous direction along the first axis (Step
S307).
[0214] Similarly, the second axis determining unit 318 refers to
the movement vectors calculated by the calculating unit 104 and
accordingly determines the advantageous direction along the second
axis (Step S308).
[0215] Then, from the determination results regarding the first
axis and the second axis, the determination result selecting unit
320 selects the determination result of the axis that is determined
by the determining unit 312 to be advantageous for the movement of
the specific part in a determination target section (Step
S310).
[0216] Every time the obtaining unit 102 obtains the latest
position of the specific part of the photographic subject (Step
S301), the operations from Step S302 to Step S310 are repeated, and
the presenting unit 314 performs presentation according to the
determination result of the determination result selecting unit
320.
[0217] Explained below is an exemplary sequence of operations
performed by the determining unit 312. The sequence of operations
performed by the determining unit 312 is a combination of the
flowchart illustrated in FIG. 10 and the flowchart illustrated in
FIG. 20. More particularly, in the flowchart illustrated in FIG.
10, when the evaluation value E.sub.c is not equal to or smaller
than the evaluation value E.sub.b (No at Step S131), the system
control proceeds not to Step S133 but to Step S241 illustrated in
the flowchart in FIG. 20.
[0218] FIG. 26 is a flowchart for explaining an exemplary sequence
of operations performed by the second axis determining unit 318 in
the recognition apparatus 300 according to the third embodiment.
Herein, although not explained, the same sequence of operations is
also performed by the first axis determining unit 316. Moreover,
the explanation with reference to FIG. 26 is given for a case when
the determination of whether or not a feed action was performed in
an operation section is done in real time.
[0219] Firstly, as a determination target section, the second axis
determining unit 318 sets an operation section in which a start
point is either set to a second-axis direction turning point when
such a second-axis direction turning point is present at which the
direction along the second axis component of the movement vector
changes, or set to the initial point indicating the
initially-obtained position when a second-axis direction turning
point is absent; and in which an end point is set to the latest
point that indicates the most-recently obtained position. Then, the
second axis determining unit 318 calculates an evaluation value of
the operation section and checks whether or not the evaluation is
equal to or greater than a threshold value (Step S351).
[0220] If the evaluation value of the operation section is equal to
or greater than a threshold value (Yes at Step S351), then the
second axis determining unit 318 checks whether or not the
operation section is more advantageous as compared to the previous
condition (Step S352). More particularly, the second axis
determining unit 318 sets a previous section in which an end point
is set to the start point of the operation section and in which a
start point is either set to a second-axis direction turning point
present immediately before the end point, when the second-axis
direction turning points are present before the end point, or set
to the initial point when a second-axis direction turning point is
not present before the end point. Then, the second axis determining
unit 318 calculates an evaluation value of the previous section and
checks whether or not the evaluation value of the operation section
is exceeding the evaluation value of the previous section.
[0221] When the operation section is more advantageous as compared
to the corresponding previous condition (Yes at Step S352), that
is, when the evaluation value of the operation section is exceeding
the evaluation value of the corresponding previous section; the
second axis determining unit 318 checks whether the operation
section is more advantageous as compared to the subsequent
condition (Step S353). More particularly, the second axis
determining unit 318 updates the end point either to the
second-axis direction turning point present immediately after the
start point of the operation section, when such a second-axis
direction turning point is present immediately after the start
point of the operation section before the elapse in a predetermined
time period since the setting of the operation section, or to the
latest point that indicates the most-recently obtained position
after the elapse in a predetermined time period, when no
second-axis direction turning point is present immediately after
the start point of the operation section before the elapse in the
predetermined time. Moreover, the second axis determining unit 318
sets a subsequent section, in which a start point is set to the end
point of the updated operation section and in which an end point is
either set to the second-axis direction turning point subsequent to
the start point when such a second-axis direction turning point is
present subsequent to the start point or set to the latest point
present after the elapse in a predetermined time period when latest
points are present subsequent to the start point. Then, the second
axis determining unit 318 calculates an evaluation value of the
subsequent section and checks whether or not the evaluation value
of the operation section is exceeding the evaluation value of the
subsequent section.
[0222] When the operation section is more advantageous as compared
to the corresponding subsequent condition (Yes at Step S353), that
is, when the evaluation value of the operation section is exceeding
the evaluation value of the corresponding subsequent section; the
second axis determining unit 318 determines that a feed action in
the direction of the movement vectors in the second axis was
performed in the operation section (Step S354).
[0223] On the other hand, when the evaluation value of the
operation section is smaller than the threshold value (No at Step
S351) or when the operation section is not advantageous as compared
to the previous condition (No at Step S352) or when the operation
section is not advantageous as compared to the subsequent condition
(No at Step S353); the second axis determining unit 318 determines
that the action performed in the operation section was not a feed
action (Step S355).
[0224] As described above, in the third embodiment, the gesture in
a determination target section is recognized using the
corresponding previous section and the corresponding subsequent
section. Thus, according to the third embodiment, it becomes
possible to determine whether or not the gesture in a determination
target section is noise. Because of that, even if a movement
representing noise is included in a gesture, such movement can be
recognized correctly thereby enabling prevention of a decline in
the recognition accuracy. For example, even if a movement
representing noise is included at the start and at the end of a
gesture, such movement can be recognized correctly. Moreover,
according to the third embodiment, it is also possible to recognize
the direction indicated along each axis.
Fourth Embodiment
[0225] In the first to third embodiments described above, the
explanation is given for a case in which two axes are included in
the coordinate system representing the positions of a specific part
of a photographic subject. That is, there are two axes that are
considered to be the determination targets for the gestures
performed by the specific part. In a fourth embodiment, the
explanation is given for a case in which three or more axes are
included in the coordinate system representing the positions of a
specific part of a photographic subject. Herein, it is assumed that
the axes intersect with each other at a single point.
[0226] FIG. 27 is a diagram illustrating an exemplary coordinate
system according to the fourth embodiment. In the coordinate system
illustrated in the example in FIG. 27, a first axis to a third axis
are set corresponding to mutually independent directions of x, y,
and z, respectively that define a three-dimensional space. Hence,
determination of gestures can be performed in the up-down
direction, the left-right direction, and the front-back
direction.
[0227] FIG. 28 is a diagram illustrating another exemplary
coordinate system according to the fourth embodiment. In the
coordinate system illustrated in FIG. 28, it is possible to set the
axes corresponding to three different directions on the same plane
(for example, three axes rotated by 120.degree. from each other on
the same plane).
[0228] The principal axis selecting unit 106 calculates, for each
movement vector that is calculated, a first inner product that is
the inner product of the movement vector and the first axis, a
second inner product that is the inner product of the movement
vector and the second axis, and a third inner product that is the
inner product of the movement vector and the third axis. Then, of
the first to third axes, the principal axis selecting unit 106
selects the axis that is used in calculating the inner product
having the greatest modulus, from among the first inner product to
the third inner product, as the principal axis.
[0229] Consider the case when more than one inner product has the
greatest modulus. In that case, if the axes that are used in
calculating the inner products having the greatest modulus include
the previous principal axis, then the principal axis selecting unit
106 selects the previous principal axis as the current principal
axis. On the other hand, if the axes that are used in calculating
the inner products having the greatest modulus do not include the
previous principal axis, then the principal axis selecting unit 106
selects, as the principal axis, an arbitrary axis from among the
axes that are used in calculating the inner products having the
greatest modulus.
[0230] Application
[0231] Explained below are exemplary applications of the
recognition apparatus according to the embodiments described above.
The applications explained below are only exemplary, and the
recognition apparatus can also be implemented in other devices.
[0232] First Application
[0233] FIG. 29 is a diagram illustrating a television receiver 400
according to a first application. In the television receiver 400
illustrated in FIG. 29, the recognition apparatus according to any
one of the embodiments described above is implemented, and the
channel operations are performed via gestures.
[0234] The television receiver 400 has a camera 402 installed
therein for capturing images of the viewer. As the specific part,
the television receiver 400 detects and follows a hand 404 of the
viewer in order to recognize feed actions. It is illustrated that a
television screen 406 is currently displaying a channel 4. When the
viewer performs a feed action of swiftly moving the hand 404
leftward, the channel can be changed to a channel 3. Similarly,
when the viewer performs a feed action of swiftly moving the hand
404 rightward, the channel can be changed to a channel 5.
[0235] Second Application
[0236] FIG. 30 is a diagram illustrating a personal computer 500
according to a second application. In the personal computer 500
illustrated in FIG. 30, the recognition apparatus according to any
one of the embodiments described above is implemented. Herein, the
web contents or the text displayed in a window 508 on a screen 506
can be scrolled via gestures of a hand 504.
[0237] The personal computer 500 has a camera 502 installed therein
for capturing images of the user. As the specific part, the
personal computer 500 detects and follows the hand 504 of the user
in order to recognize feed actions. Then, in the direction of the
feed action that has been detected, the personal computer 500
issues a scroll command for scrolling in that direction. At that
time, the amount of scrolling, which is displayed on a scroll bar
510, can be set to a predetermined constant amount. Alternatively,
the amount of scrolling can be changed depending on the speed of
the feed action. For example, faster the feed action, greater can
be the amount of scrolling.
[0238] As a result, the user can scroll the contents, which are
displayed in the window 508, in the direction of the feed action
and thus browse the web contents or the text.
Third Embodiment
[0239] FIG. 31 is a diagram illustrating a medical information
terminal 600 according to a third application. In the medical
information terminal 600 illustrated in FIG. 31, the recognition
apparatus according to any one of the embodiments described above
is implemented. The medical information terminal 600 is used in
providing information to a doctor who is performing a surgery.
[0240] Since the doctor who is performing a surgery has to keep a
hand 604 clean, it is not possible to use the hand 604 to directly
operate the medical information terminal 600, which provides
information such as the monitoring information or the laboratory
results of the patient.
[0241] For that reason, by implementing the recognition apparatus
according to any one of the embodiments described above in the
medical information terminal 600, it becomes possible for the
doctor to operate the medical information terminal 600 in a
contactless manner via gestures of the hand 604.
[0242] The medical information terminal 600 has a camera 602
installed therein for capturing images of the doctor. As the
specific part, the medical information terminal 600 detects and
follows the hand 604 of the doctor in order to recognize feed
actions. Then, in the direction of the feed action that has been
detected, the medical information terminal 600 issues a scroll
command for scrolling in that direction or issues a change command
for changing the display information.
MODIFICATION EXAMPLE
[0243] The invention is not limited to the embodiments described
above and it is possible to implement various modifications within
the scope of the invention.
[0244] For example, the first embodiment and the second embodiment
can be combined together. In that case, the section setting unit,
the determining unit, and the presenting unit perform operations
identical to the operations described in the third embodiment.
Moreover, in the third embodiment, although the section setting
unit 310 sets previous sections as well as subsequent sections with
respect to determination target sections, the section setting unit
310 can alternatively be configured to set either previous sections
or subsequent sections as described in the first embodiment or in
the second embodiment.
[0245] As described above, the television receiver 400, the
personal computer 500, and the medical information terminal 600 are
explained as applications of the recognition apparatus. Besides,
the recognition apparatus can also be implemented in other devices
such as game consoles. That is, the recognition apparatus can also
be implemented in devices that enable the user to perform feed
actions regarding device control menus or feed actions regarding
displayed contents with the use of gesture interfaces provided for
easy device operations without having to hold or wear remote
controllers, sensors, markers, or the like.
[0246] Meanwhile, the recognition apparatus according to each
embodiment and each modification example described above has the
hardware configuration of a commonly-used computer that includes a
control device such as a central processing unit (CPU), a memory
device such as a read only memory (ROM) or a random access memory
(RAM), an external memory device such as a hard disk drive (HDD) or
a solid state drive (SSD), a display device such as a display, an
input device such as a mouse or a keyboard, and a communication
device such as a communication interface (I/F).
[0247] Meanwhile, a recognition program, which is executed in the
recognition apparatus according to each embodiment and each
modification example described above, can be provided in the form
of an installable or executable file on a computer-readable
recording medium such as a compact disk read only memory (CD-ROM),
a compact disk recordable (CD-R), a digital versatile disk (DVD),
or a flexible disk (FD).
[0248] Alternatively, the recognition program, which is executed in
the recognition apparatus according to each embodiment and each
modification example described above, can be saved on a computer
connected to a network and downloaded from that computer via the
network. Still alternatively, the recognition program, which is
executed in the recognition apparatus according to each embodiment
and each modification example described above, can be made
available for distribution through the network such as the
Internet. Still alternatively, the recognition program, which is
executed in the recognition apparatus according to each embodiment
and each modification example described above, can be stored in
advance in a ROM or the like.
[0249] Herein, the recognition program, which is executed in the
recognition apparatus according to each embodiment and each
modification example described above, contains modules for
implementing the functions of the abovementioned constituent
elements in a computer. Regarding the actual hardware, a CPU
retrieves the recognition program from, for example, an HDD and
runs it so that the recognition program is loaded in a RAM. As a
result, the functions of the abovementioned constituent elements
are implemented in the RAM.
[0250] In this way, according to an aspect of the invention, it
becomes possible to prevent a decline in the recognition
accuracy.
[0251] While certain embodiments have been described, these
embodiments have been presented by way of example only, and are not
intended to limit the scope of the inventions. Indeed, the novel
embodiments described herein may be embodied in a variety of other
forms; furthermore, various omissions, substitutions and changes in
the form of the embodiments described herein may be made without
departing from the spirit of the inventions. The accompanying
claims and their equivalents are intended to cover such forms or
modifications as would fall within the scope and spirit of the
inventions.
* * * * *