U.S. patent application number 13/080989 was filed with the patent office on 2012-04-19 for storage medium having stored thereon game program, image processing apparatus, image processing system, and image processing method.
This patent application is currently assigned to NINTENDO CO., LTD.. Invention is credited to Toshiaki SUZUKI.
Application Number | 20120094773 13/080989 |
Document ID | / |
Family ID | 45934619 |
Filed Date | 2012-04-19 |
United States Patent
Application |
20120094773 |
Kind Code |
A1 |
SUZUKI; Toshiaki |
April 19, 2012 |
STORAGE MEDIUM HAVING STORED THEREON GAME PROGRAM, IMAGE PROCESSING
APPARATUS, IMAGE PROCESSING SYSTEM, AND IMAGE PROCESSING METHOD
Abstract
A face image is acquired during a predetermined game or before a
start of the predetermined game, and a first character object is
created, the first character object including the face image. Then,
in the predetermined game, a game related to the first character
object is advanced in accordance with an operation of a player. At
least when a success in the game related to the first character
object has been determined, the face image is saved in a storage
area in an accumulating manner.
Inventors: |
SUZUKI; Toshiaki; (Kyoto,
JP) |
Assignee: |
NINTENDO CO., LTD.
Kyoto
JP
|
Family ID: |
45934619 |
Appl. No.: |
13/080989 |
Filed: |
April 6, 2011 |
Current U.S.
Class: |
463/43 |
Current CPC
Class: |
A63F 2300/695 20130101;
A63F 2300/8076 20130101; A63F 13/655 20140902; A63F 13/79 20140902;
A63F 13/533 20140902; A63F 2300/6653 20130101; A63F 13/5255
20140902; A63F 2300/66 20130101; A63F 2300/105 20130101; A63F
2300/1093 20130101; A63F 13/211 20140902; A63F 13/26 20140902; A63F
13/213 20140902; A63F 2300/61 20130101; A63F 2300/69 20130101; A63F
2300/5553 20130101; A63F 13/837 20140902; A63F 13/46 20140902 |
Class at
Publication: |
463/43 |
International
Class: |
A63F 9/24 20060101
A63F009/24 |
Foreign Application Data
Date |
Code |
Application Number |
Oct 15, 2010 |
JP |
2010-232869 |
Dec 28, 2010 |
JP |
2010-293443 |
Claims
1. A computer-readable storage medium having stored thereon a game
program to be executed by a computer of a game apparatus that
displays an image on a display device, the game program causing the
computer to execute: an image acquisition step of acquiring a face
image and temporarily storing the acquired face image in a first
storage area, during a predetermined game or before a start of the
predetermined game; a step of creating a first character object,
the first character object being a character object including the
face image stored in the first storage area; a first game
processing step of, in the predetermined game, advancing a game
related to the first character object in accordance with an
operation of a player; a determination step of determining a
success in the game related to the first character object; and a
step of, at least when a success in the game has been determined in
the determination step, saving the face image stored in the first
storage area, in a second storage area in an accumulating
manner.
2. The computer-readable storage medium having stored thereon the
game program according to claim 1, wherein in the image acquisition
step, the face image is acquired and temporarily stored in the
first storage area before the start of the predetermined game.
3. The computer-readable storage medium having stored thereon the
game program according to claim 1, further causing the computer to
execute: a step of creating a second character object, the second
character object being a character object including a face image
selected automatically or by the player from among the face images
saved in the second storage area, wherein in the first game
processing step, in the predetermined game, a game related to the
second character object is additionally advanced in accordance with
an operation of the player.
4. The computer-readable storage medium having stored thereon the
game program according to claim 1, further causing the computer to
execute: a step of creating a second character object, the second
character object being a character object including a face image
selected automatically or by the player from among the face images
saved in the second storage area; and a second game processing step
of advancing a game related to the second character object in
accordance with an operation of the player.
5. The computer-readable storage medium having stored thereon the
game program according to claim 2, wherein the game apparatus is
capable of acquiring an image from a capturing device, and in the
image acquisition step, the face image is acquired from the
capturing device before the start of the predetermined game.
6. The computer-readable storage medium having stored thereon the
game program according to claim 5, wherein the game apparatus is
capable of acquiring an image from a first capturing device that
captures a front direction of a display surface of the display
device, and an image from a second capturing device that captures a
direction of a back surface of the display surface of the display
device, the first capturing device and the second capturing device
serving as the capturing device, and the image acquisition step
includes: a step of acquiring a face image captured by the first
capturing device in preference to acquiring a face image captured
by the second capturing device; and a step of, after the face image
from the first capturing device has been saved in the second
storage area, permitting the face image captured by the second
capturing device to be acquired.
7. The computer-readable storage medium having stored thereon the
game program according to claim 1, further causing the computer to
execute: a step of specifying attributes of the face images saved
in the second storage area; and a step of prompting the player to
acquire a face image corresponding to an attribute different from
the attributes specified from the face images saved in the second
storage area.
8. The computer-readable storage medium having stored thereon the
game program according to claim 3, wherein the first game
processing step includes: a step of advancing the game related to
the first character object by attacking the character objects in
accordance with an operation of the player, and in the first game
processing step, an attack on the first character object is a valid
attack for succeeding in the game related to the first character
object, and an attack on the second character object is an invalid
attack for succeeding in the game related to the first character
object.
9. The computer-readable storage medium having stored thereon the
game program according to claim 4, further causing the computer to
execute: a step of creating a third character object, the third
character object being a character object including a face image
different from the face image included in the second character
object, wherein the second game processing step includes: a step of
advancing the game related to the second character obj ect by
attacking the character objects in accordance with an operation of
the player, and in the second game processing step, an attack on
the second character object is a valid attack for succeeding in the
game related to the second character object, and an attack on the
third character object is an invalid attack for succeeding the game
related to the second character object.
10. The computer-readable storage medium having stored thereon the
game program according to claim 3, further causing the computer to
execute: a step of creating a third character object, the third
character object being a character object including the face image
stored in the first storage area and being smaller in dimensions
than the first character object; and a step of creating a fourth
character object, the fourth character object being a character
object including a face image different from the face image stored
in the first storage area and being smaller in dimensions than the
first character object, wherein the first game processing step
includes: a step of advancing the game related to the first
character object by attacking the character objects in accordance
with an operation of the player; a step of, when the fourth
character object has been attacked, advancing deformation of the
face image included in the first character object; and a step of,
when the third character object has been attacked, reversing the
deformation such that the face image included in the first
character object approaches the original face image stored in the
first storage area.
11. The computer-readable storage medium having stored thereon the
game program according to claim 4, further causing the computer to
execute: a step of creating a third character object, the third
character object being a character object including the same face
image as the face image included in the second character object and
being smaller in dimensions than the second character object; and a
step of creating a fourth character object, the fourth character
object being a character object including a face image different
from the face image included in the second character object and
being smaller in dimensions than the second character object,
wherein the second game processing step includes: a step of
advancing the game related to the second character obj ect by
attacking the character objects in accordance with an operation of
the player; a step of, when the fourth character object has been
attacked, advancing deformation of the face image included in the
second character object; and a step of, when the third character
object has been attacked, reversing the deformation such that the
face image included in the second character object approaches the
original face image saved in the second storage area.
12. The computer-readable storage medium having stored thereon the
game program according to claim 1, wherein in the step of creating
the first character object, a character object including a face
image obtained by deforming the face image stored in the first
storage area is created as the first character object, and the
first game processing step includes: a step of, when the game
related to the first character object has been successful,
restoring the deformed face image to the original face image stored
in the first storage area.
13. The computer-readable storage medium having stored thereon the
game program according to claim 4, wherein in the step of creating
the second character object, a character object including a face
image obtained by deforming the face image saved in the second
storage area is created as the second character object, and the
second game processing step includes: a step of, when the game
related to the second character object has been successful,
restoring the deformed face image to the original face image saved
in the second storage area.
14. The computer-readable storage medium having stored thereon the
game program according to claim 1, wherein in the image acquisition
step, the face image is acquired and temporarily stored in the
first storage area during the predetermined game, and in the first
game processing step, in accordance with the creation of the first
character object based on the acquisition of the face image during
the predetermined game, the first character object is caused to
appear in the predetermined game, and the game related to the first
character object is advanced.
15. The computer-readable storage medium having stored thereon the
game program according to claim 14, further causing the computer to
execute: a captured image acquisition step of acquiring a captured
image captured by a real camera; a display image generation step of
generating a display image in which a virtual character object that
appears in the predetermined game is placed so as to have, as a
background, the captured image acquired in the captured image
acquisition step; and a display control step of displaying on the
display device the display image generated in the display image
generation step, wherein in the image acquisition step, during the
predetermined game, at least one face image is extracted from the
captured image displayed on the display device, and is temporarily
stored in the first storage area.
16. The computer-readable storage medium having stored thereon the
game program according to claim 15, wherein in the display image
generation step, the display image is generated by placing the
first character object such that, when displayed on the display
device, the first character object overlaps a position of the face
image in the captured image, the face image extracted in the image
acquisition step.
17. The computer-readable storage medium having stored thereon the
game program according to claim 16, wherein in the captured image
acquisition step, captured images of a real world captured in real
time by the real camera are repeatedly acquired, in the display
image generation step, the captured images repeatedly acquired in
the captured image acquisition step are sequentially set as the
background, in the image acquisition step, face images
corresponding to the already extracted face image are repeatedly
acquired from the captured images sequentially set as the
background, in the step of creating the first character object, the
first character object is repeatedly created so as to include the
face images repeatedly acquired in the image acquisition step, and
in the display image generation step, the display image is
generated by placing the repeatedly created first character object
such that, when displayed on the display device, the repeatedly
created first character object overlaps positions of the face
images in the respective captured images, the face images
repeatedly acquired in the image acquisition step.
18. The computer-readable storage medium having stored thereon the
game program according to claim 1, wherein the game apparatus is
capable of using image data stored in storage means for storing
data not temporarily, and in the image acquisition step, before the
start of the predetermined game, at least one face image is
extracted from the image data stored in the storage means, and is
temporarily stored in the first storage area.
19. An image processing apparatus connectable to a display device,
the image processing apparatus comprising: image acquisition means
for acquiring a face image and temporarily storing the acquired
face image in a first storage area, during a predetermined game or
before a start of the predetermined game; means for creating a
character object, the character object including the face image
stored in the first storage area; game processing means for, in the
predetermined game, advancing a game related to the character
object in accordance with an operation of a player by displaying a
game related to the character object on the display device;
determination means for determining a success in the game related
to the character object; and means for, at least when a success in
the game has been determined by the determination means, saving the
face image stored in the first storage area, in a second storage
area in an accumulating manner.
20. An image processing apparatus comprising: a display device;
image acquisition means for acquiring a face image and temporarily
storing the acquired face image in a first storage area, during a
predetermined game or before a start of the predetermined game;
means for creating a character object, the character object
including the face image stored in the first storage area; game
processing means for, in the predetermined game, advancing a game
related to the character object in accordance with an operation of
a player by displaying the game related to the character object on
the display device; determination means for determining a success
in the game related to the character object; and means for, at
least when a success in the game has been determined by the
determination means, saving the face image stored in the first
storage area, in a second storage area in an accumulating
manner.
21. An image processing system that displays information including
an image on a display device, the image processing system
comprising: image acquisition means for acquiring a face image and
temporarily storing the acquired face image in a first storage
area, during a predetermined game or before a start of the
predetermined game; means for creating a character object, the
character object including the face image stored in the first
storage area; game processing means for, in the predetermined game,
advancing a game related to the character object in accordance with
an operation of a player by displaying the game related to the
character object on the display device; determination means for
determining a success in the game related to the character object;
and means for, at least when a success in the game has been
determined by the determination means, saving the face image stored
in the first storage area, in a second storage area in an
accumulating manner.
22. An image processing method performed by a computer that
displays an image on a display device, the computer executing: an
image acquisition step of acquiring a face image and temporarily
storing the acquired face image in a first storage area, during a
predetermined game or before a start of the predetermined game; a
step of creating a character object, the character object including
the face image stored in the first storage area; a game processing
step of, in the predetermined game, advancing a game related to the
character object in accordance with an operation of a player; a
determination step of determining a success in the game related to
the character object; and a step of, at least when a success in the
game has been determined in the determination step, saving the face
image stored in the first storage area, in a second storage area in
an accumulating manner.
Description
CROSS REFERENCE TO RELATED APPLICATION
[0001] The disclosures of Japanese Patent Application No.
2010-232869, filed on Oct. 15, 2010, and Japanese Patent
Application No. 2010-293443, filed on Dec. 28, 2010, are
incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] The present invention relates to a storage medium having
stored thereon a game program, an image processing apparatus, an
image processing system, and an image processing method.
[0004] 2. Description of the Background Art
[0005] Conventionally, as disclosed in, for example, Japanese
Laid-Open Patent Publication No. 2006-72669, a proposal is made for
an apparatus that displays an image obtained by combining an image
of the real world with an image of a virtual world. Further, as
disclosed in, for example, Japanese Laid-Open Patent Publication
No. 2010-142592, a proposal is also made for an information
processing technique, such as a game using a user image, which is
information obtained in the real world. In this technique, in the
progression of the game, at a time when a user image included in
image data (e.g., an image of a human face area) satisfies
conditions defined in advance, the image data is captured.
[0006] An image of a face area (hereinafter a "face image") is an
image of the most characteristic part of a living thing, and
therefore is very useful as information for reflecting the real
world on a virtual world. The conventional techniques, however, do
not make sufficient use of the feature of a face image in which it
is possible to reflect a situation in the real world on a virtual
world.
SUMMARY OF THE INVENTION
[0007] It is an object of the present invention to assist a user in
the acquisition, the collection, and the like of face images, and
also to make it possible to represent a virtual world on which the
real world is reflected by the face images.
[0008] To achieve the above object, the present invention may
employ, for example, the following configurations. It is understood
that when the description of the scope of the appended claims is
interpreted, the scope should be interpreted only by the
description of the scope of the appended claims. If the description
of the scope of the appended claims contradicts the description of
these columns, the description of the scope of the appended claims
has priority.
[0009] A configuration example of a computer-readable storage
medium having stored thereon a game program according to the
present invention is executed by a computer of a game apparatus
that displays an image on a display device. The game program
causing the computer to execute an image acquisition step, a step
of creating a first character object, a first game processing step,
a determination step, and a step of saving in a second storage area
in an accumulating manner. The image acquisition step acquires a
face image and temporarily stores the acquired face image in a
first storage area, during a predetermined game or before a start
of the predetermined game. The step of creating a first character
object creates a first character object, the first character object
being a character object including the face image stored in the
first storage area. The first game processing step, in the
predetermined game, advances a game related to the first character
object in accordance with an operation of a player. The
determination step determines a success in the game related to the
first character object. The step of saving in a second storage area
in an accumulating manner, at least when a success in the game has
been determined in the determination step, saves the face image
stored in the first storage area, in a second storage area in an
accumulating manner.
[0010] Based on the above, until a success in the game is
determined, the player cannot save in the second storage area the
face image acquired during the game or before the start of the game
and temporarily stored in the first storage area, and therefore
enjoys a sense of tension. On the other hand, the face image may be
saved in the second storage area when a success in the game has
been determined, whereby, for example, it is possible to utilize
the acquired face image even after the game has ended in the first
game processing step. This causes the player to tackle the game of
the first game processing step very enthusiastically and with their
concentration.
[0011] In addition, in the image acquisition step, before the start
of the predetermined game, the face image may be acquired and
temporarily stored in the first storage area.
[0012] Based on the above, it is possible to execute the game of
the first game processing step such that the face image acquired
before the start of the predetermined game serves as a target to be
saved in the second storage area.
[0013] In addition, the game program may further cause the computer
to execute a step of creating a second character object. The step
of creating a second character object creates a second character
object, the second character object being a character object
including a face image selected automatically or by the player from
among the face images saved in the second storage area. In this
case, in the first game processing step, in the predetermined game,
a game related to the second character object may be additionally
advanced in accordance with an operation of the player.
[0014] Based on the above, it is possible to utilize in a game the
face image saved in the second storage area. Accordingly, it is
possible to advance a game related to a character object including
the face image acquired in the current predetermined game and
related to a character object including a previously stored face
image. This makes it possible to make various representations in
the game.
[0015] In addition, the game program may further cause the computer
to execute a step of creating a second character object and a
second game processing step. The step of creating a second
character object creates a second character object, the second
character object being a character object including a face image
selected automatically or by the player from among the face images
saved in the second storage area. The second game processing step
advances a game related to the second character object in
accordance with an operation of the player.
[0016] Based on the above, the player can create the second
character object including a face image selected from among the
face images saved in the second storage area, and execute the game
of the second game processing step. That is, the player can enjoy
the game of the second game processing step by utilizing the face
images stored by succeeding in the game of the first game
processing step. In this case, the character object that appears in
the game includes a face image selected automatically or by an
operation of the player, and therefore, the player can simply
introduce a mental picture of the real world into a virtual
world.
[0017] In addition, the game apparatus may be capable of acquiring
an image from a capturing device. In this case, in the image
acquisition step, the face image may be acquired from the capturing
device before the start of the predetermined game.
[0018] Based on the above, it is possible to cause a character
object including a face image captured by the capturing device, to
appear in a game, and save the face image captured by the capturing
device, in an accumulating manner.
[0019] In addition, the game apparatus may be capable of acquiring
an image from a first capturing device that captures a front
direction of a display surface of the display device, and an image
from a second capturing device that captures a direction of a back
surface of the display surface of the display device, the first
capturing device and the second capturing device serving as the
capturing device. In this case, the image acquisition step may
include: a step of acquiring a face image captured by the first
capturing device in preference to acquiring a face image captured
by the second capturing device; and a step of, after the face image
from the first capturing device has been saved in the second
storage area, permitting the face image captured by the second
capturing device to be acquired.
[0020] Based on the above, the acquisition of a face image using
the first capturing device is preferentially made, the first
capturing device capturing the front direction of the display
surface of the display device. This increases the possibility that
a face image of the player of the game apparatus or the like who
views the display surface of the display device is preferentially
acquired. This increases the possibility of restricting the
acquisition of an image with the second capturing device in the
state where the player of the game apparatus or the like is not
specified, the second capturing device capturing the direction of
the display surface of the display device.
[0021] In addition, the game program may further cause the computer
to execute a step of specifying attributes of the face images and a
step of prompting the player to acquire a face image. The step of
specifying attributes of the face images specifies attributes of
the face images saved in the second storage area. The step of
prompting the player to acquire a face image prompts the player to
acquire a face image corresponding to an attribute different from
the attributes specified from the face images saved in the second
storage area.
[0022] Based on the above, it is possible to reduce the imbalance
among the attributes of the face images saved in an accumulating
manner, and thereby assist the player who wishes to save face
images having various attributes in an accumulating manner.
[0023] In addition, the first game processing step may include a
step of advancing the game related to the first character object by
attacking the character objects in accordance with an operation of
the player. In this case, in the first game processing step, an
attack on the first character object may be a valid attack for
succeeding in the game related to the first character object, and
an attack on the second character object may be an invalid attack
for succeeding in the game related to the first character
object.
[0024] Based on the above, the player needs to control an attack on
the second character object in the game of the first game
processing step, and therefore selects and attacks the first
character object. Thus, the player needs to correctly recognize the
first character object, and requires concentration.
[0025] In addition, the game program may further cause the computer
to execute a step of creating a third character object. The step of
creating a third character object creates a third character object,
the third character object being a character object including a
face image different from the face image included in the second
character object. In this case, the second game processing step may
include a step of advancing the game related to the second
character object by attacking the character objects in accordance
with an operation of the player. In the second game processing
step, an attack on the second character object may be a valid
attack for succeeding in the game related to the second character
object, and an attack on the third character object may be an
invalid attack for succeeding the game related to the second
character object.
[0026] Based on the above, the player needs to control an attack on
the third character object in the game of the second game
processing step, and therefore, selects and attacks the second
character object. Thus, the player needs to correctly recognize the
second character object, and requires concentration.
[0027] In addition, the game program may further cause the computer
to execute a step of creating a third character object and a step
of creating a fourth character object. The step of creating a third
character object creates a third character object, the third
character object being a character object including the face image
stored in the first storage area and being smaller in dimensions
than the first character object. The step of creating a fourth
character object creates a fourth character object, the fourth
character object being a character object including a face image
different from the face image stored in the first storage area and
being smaller in dimensions than the first character object. In
this case, the first game processing step may include: a step of
advancing the game related to the first character object by
attacking the character objects in accordance with an operation of
the player; a step of, when the fourth character object has been
attacked, advancing deformation of the face image included in the
first character object; and a step of, when the third character
object has been attacked, reversing the deformation such that the
face image included in the first character object approaches the
original face image stored in the first storage area.
[0028] Based on the above, the player needs to correctly recognize
the first character object, the third character object, and the
fourth character object, and requires concentration. Particularly
when the acquired face image is a face image of a person in an
intimate relationship with the player, the player can tackle the
game of the first game processing step increasingly
enthusiastically.
[0029] In addition, the game program may further cause the computer
to execute a step of creating a third character object and a step
of creating a fourth character object. The step of creating a third
character object creates a third character object, the third
character object being a character object including the same face
image as the face image included in the second character object and
being smaller in dimensions than the second character object. The
step of creating a fourth character object creates a fourth
character object, the fourth character object being a character
object including a face image different from the face image
included in the second character object and being smaller in
dimensions than the second character object. In this case, the
second game processing step may include: a step of advancing the
game related to the second character object by attacking the
character objects in accordance with an operation of the player; a
step of, when the fourth character object has been attacked,
advancing deformation of the face image included in the second
character object; and a step of, when the third character object
has been attacked, reversing the deformation such that the face
image included in the second character object approaches the
original face image saved in the second storage area.
[0030] Based on the above, the player needs to correctly recognize
the second character object, the third character object, and the
fourth character object, and requires concentration.
[0031] In addition, in the step of creating the first character
object, a character object including a face image obtained by
deforming the face image stored in the first storage area may be
created as the first character object. In this case, the first game
processing step may include a step of, when the game related to the
first character object has been successful, restoring the deformed
face image to the original face image stored in the first storage
area.
[0032] Based on the above, for example, when the acquired face
image is a face image of a person in an intimate relationship with
the player, the player can tackle the game of the first game
processing step increasingly enthusiastically in order to restore
the deformed face image to the original face image.
[0033] In addition, in the step of creating the second character
object, a character object including a face image obtained by
deforming the face image saved in the second storage area may be
created as the second character object. In this case, the second
game processing step may include a step of, when the game related
to the second character object has been successful, restoring the
deformed face image to the original face image saved in the second
storage area.
[0034] Based on the above, for example, when the acquired face
image is a face image of a person in an intimate relationship with
the player, the player can tackle the game of the second game
processing step increasingly enthusiastically in order to restore
the deformed face image to the original face image.
[0035] In addition, in the image acquisition step, the face image
may be acquired and temporarily stored in the first storage area
during the predetermined game. In this case, in the first game
processing step, in accordance with the creation of the first
character object based on the acquisition of the face image during
the predetermined game, the first character object may be caused to
appear in the predetermined game, and the game related to the first
character object may be advanced.
[0036] Based on the above, it is possible to execute the game of
the first game processing step such that the face image acquired
during the predetermined game serves as a target to be saved in the
second storage area.
[0037] In addition, the game program may further cause the computer
to execute a captured image acquisition step, a display image
generation step, and a display control step. The captured image
acquisition step acquires a captured image captured by a real
camera. The display image generation step generates a display image
in which a virtual character object that appears in the
predetermined game is placed so as to have, as a background, the
captured image acquired in the captured image acquisition step. The
display control step displays on the display device the display
image generated in the display image generation step. In this case,
in the image acquisition step, during the predetermined game, at
least one face image may be extracted from the captured image
displayed on the display device, and may be temporarily stored in
the first storage area.
[0038] Based on the above, a face image included in a captured
image of the real world displayed as a background appears as a
character object. This makes it possible to save the face image in
an accumulating manner by a success in a game related to the
character object.
[0039] In addition, in the display image generation step, the
display image may be generated by placing the first character
object such that, when displayed on the display device, the first
character object overlaps a position of the face image in the
captured image, the face image extracted in the image acquisition
step.
[0040] Based on the above, it is possible to represent the first
character object as if appearing from the captured image, and
display an image as if the first character object is present in a
real space captured by the real camera.
[0041] In addition, in the captured image acquisition step,
captured images of a real world captured in real time by the real
camera may be repeatedly acquired. In the display image generation
step, the captured images repeatedly acquired in the captured image
acquisition step may be sequentially set as the background. In the
image acquisition step, face images corresponding to the already
extracted face image may be repeatedly acquired from the captured
images sequentially set as the background. In the step of creating
the first character object, the first character object may be
repeatedly created so as to include the face images repeatedly
acquired in the image acquisition step. In the display image
generation step, the display image may be generated by placing the
repeatedly created first character object such that, when displayed
on the display device, the repeatedly created first character
object overlaps positions of the face images in the respective
captured images, the face images repeatedly acquired in the image
acquisition step.
[0042] Based on the above, even when the capturing position and the
capturing direction of the real camera have changed, it is possible
to place the first character object in accordance with the changes,
and even when the position and the expression of a person who is a
subject from which a face image has been acquired have changed, it
is also possible to reflect the changes on the first character
object. This makes it possible to display the first character
object as if present in a real space represented by a captured
image captured in real time.
[0043] In addition, the game apparatus may be capable of using
image data stored in storage means for storing data not
temporarily. In this case, in the image acquisition step, before
the start of the predetermined game, at least one face image may be
extracted from the image data stored in the storage means, and may
be temporarily stored in the first storage area.
[0044] Based on the above, a face image is acquired from image data
stored in advance in the game apparatus. This makes it possible
that a face image acquired in advance by another application or the
like (e.g., a face image included in an image photographed by a
camera capturing application, or included in an image received from
another device by a communication application) serves as an
acquisition target.
[0045] In addition, other configuration examples of the present
invention may be carried out in the form of an image processing
apparatus and an image processing system that include means for
executing the above steps, and may be carried out in the form of an
image processing method including operations performed in the above
steps.
[0046] Based on a configuration example of the present invention,
it is possible to assist the user in the acquisition, the
collection, and the like of face images, and also to represent a
virtual world on which the real world is reflected by the face
images.
[0047] These and other objects, features, aspects and advantages of
the present invention will become more apparent from the following
detailed description of the present invention when taken in
conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0048] FIG. 1 is a front view showing an example of a game
apparatus 10 in an open state;
[0049] FIG. 2 is a side view showing an example of the game
apparatus 10 in the open state;
[0050] FIG. 3A is a left side view showing an example of the game
apparatus 10 in a closed state;
[0051] FIG. 3B is a front view showing an example of the game
apparatus 10 in the closed state;
[0052] FIG. 3C is a right side view showing an example of the game
apparatus 10 in the closed state;
[0053] FIG. 3D is a rear view showing an example of the game
apparatus 10 in the closed state;
[0054] FIG. 4 is a diagram showing an example of a user holding the
game apparatus 10 with both hands;
[0055] FIG. 5 is a diagram showing an example of a user holding the
game apparatus 10 with one hand;
[0056] FIG. 6 is a block diagram showing an example of the internal
configuration of the game apparatus 10;
[0057] FIG. 7 is an example of face images displayed as a list;
[0058] FIG. 8 is another example of face images displayed as a
list;
[0059] FIG. 9 is an example of a screen for attaching a face image
to an enemy object;
[0060] FIG. 10 is a diagram illustrating a face image selection
screen;
[0061] FIG. 11 is a diagram showing an example of various data
stored in a main memory in accordance with the execution of an
image processing program according to the present invention by the
game apparatus of FIG. 1;
[0062] FIG. 12 is an example of the data structure of face image
management information;
[0063] FIG. 13 is an example of an aggregate table where already
acquired face images are classified by attribute;
[0064] FIG. 14 is a flow chart showing an example of the operation
of the game apparatus 10 according to a first embodiment;
[0065] FIG. 15 is a flow chart showing an example of a detailed
process of a face image acquisition process 1;
[0066] FIG. 16 is a flow chart showing an example of a detailed
process of a face image acquisition process 2;
[0067] FIG. 17 is a flow chart showing an example of a detailed
process of a list display process;
[0068] FIG. 18 is a flow chart showing an example of a detailed
process of a cast determination process;
[0069] FIG. 19A is a flow chart showing an example of a detailed
process of a face image management assistance process 1;
[0070] FIG. 19B is a flow chart showing an example of a detailed
process of a face image management assistance process 2;
[0071] FIG. 19C is a flow chart showing an example of a detailed
process of a face image management assistance process 3;
[0072] FIG. 20A is a diagram showing an overview of a virtual
space, which is an example of the image processing program;
[0073] FIG. 20B is a diagram showing the relationship between a
screen model and an .alpha.-texture;
[0074] FIG. 21 is a diagram showing an example of the virtual
space;
[0075] FIG. 22 is a diagram showing a virtual three-dimensional
space (game world) defined in a game program, which is an example
of the image processing program;
[0076] FIG. 23 is an example of process steps of examples of the
forms of display performed on an upper LCD of a game apparatus,
which is an example of the apparatus that executes the image
processing program;
[0077] FIG. 24 is an example of process steps of examples of the
forms of display performed on the upper LCD of the game apparatus,
which is an example of the apparatus that executes the image
processing program;
[0078] FIG. 25 is an example of process steps of examples of the
forms of display performed on the upper LCD of the game apparatus,
which is an example of the apparatus that executes the image
processing program;
[0079] FIG. 26 is an example of process steps of examples of the
forms of display performed on the upper LCD of the game apparatus,
which is an example of the apparatus that executes the image
processing program;
[0080] FIG. 27A is a diagram showing an example of silhouette
models of a shadow object as viewed from above;
[0081] FIG. 27B is a diagram showing an example of silhouette
models of the shadow object;
[0082] FIG. 28 is a diagram showing an example of the
non-transparencies of objects;
[0083] FIG. 29 is a flow chart showing an example of the operation
of image processing performed by the game apparatus executing the
image processing program;
[0084] FIG. 30 is a subroutine flow chart showing an example of a
detailed operation of an enemy-object-related process;
[0085] FIG. 31 is a subroutine flow chart showing an example of a
detailed operation of a bullet-object-related process;
[0086] FIG. 32A is a subroutine flow chart showing an example of a
detailed operation of a display image updating process (a first
drawing method) of the image processing program;
[0087] FIG. 32B is a subroutine flow chart showing an example of a
detailed operation of a display image updating process (a second
drawing method) of the image processing program according to the
present invention;
[0088] FIG. 33 is a diagram illustrating an example of a rendering
process in the first drawing method;
[0089] FIG. 34 is a diagram illustrating the positional
relationships between objects of FIG. 33;
[0090] FIG. 35 is a diagram illustrating an example of a process of
rendering a camera image;
[0091] FIG. 36 is a diagram illustrating an example of a coordinate
system used when the camera image is rendered;
[0092] FIG. 37 is a diagram illustrating an example of a process of
rendering a virtual space;
[0093] FIG. 38 is a diagram illustrating the positional
relationship between object of FIG. 37;
[0094] FIG. 39 is a diagram illustrating an example of a coordinate
system of a boundary surface, used when the virtual space is
rendered;
[0095] FIG. 40 is a diagram showing an example of a display image
generated by the image processing program;
[0096] FIG. 41 is a diagram showing an example of a screen of a
game apparatus according to a second embodiment;
[0097] FIG. 42 is a flow chart showing an example of the operation
of the game apparatus according to the second embodiment;
[0098] FIG. 43 is a diagram showing an example of a screen of a
game apparatus according to a third embodiment;
[0099] FIG. 44 is a flow chart showing an example of the operation
of the game apparatus according to the third embodiment;
[0100] FIG. 45 is a diagram showing an example of a screen
according to a fourth embodiment;
[0101] FIG. 46 is a flow chart showing an example of the operation
of a game apparatus according to the fourth embodiment;
[0102] FIG. 47 is a diagram showing an example of a screen
displayed on an upper LCD of a game apparatus according to a fifth
embodiment;
[0103] FIG. 48 is a diagram showing an example of the screen
displayed on the upper LCD of the game apparatus according to the
fifth embodiment;
[0104] FIG. 49 is a subroutine flow chart showing an example of a
detailed operation of a during-game face image acquisition process
performed by executing the image processing program according to
the fifth embodiment;
[0105] FIG. 50 is a subroutine flow chart showing an example of a
detailed operation of a yet-to-appear process performed in step 202
of FIG. 49; and
[0106] FIG. 51 is a subroutine flow chart showing an example of a
detailed operation of an already-appeared process performed in step
208 of FIG. 49.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
First Embodiment
[0107] A description is given of a specific example of an image
processing apparatus that executes an image processing program
according to an embodiment of the present invention. The following
embodiment, however, is merely illustrative, and the present
invention is not limited to the configuration of the following
embodiment.
[0108] It should be noted that in the following embodiment, data
processed by a computer is illustrated using graphs and natural
language. More specifically, however, the data is specified by
computer-recognizable pseudo-language, commands, parameters,
machine language, arrays, and the like. The present invention does
not limit the method of representing the data.
[0109] <Configuration Example of Hardware>
[0110] First, with reference to the drawings, a description is
given of a hand-held game apparatus 10 as an example of the image
processing apparatus that executes the image processing program
according to the present embodiment. The image processing apparatus
according to the present invention, however, is not limited to a
game apparatus. The image processing apparatus according to the
present invention may be a given computer system, such as a
general-purpose computer.
[0111] It should be noted that the image processing program
according to the present embodiment is a game program. The image
processing program according to the present invention, however, is
not limited to a game program. The image processing program
according to the present invention can be applied by being executed
by a given computer system. Further, the processes of the present
embodiment may be subjected to distributed processing by a
plurality of networked devices, or may be performed by a network
system where, after main processes are performed by a server, the
process results are distributed to terminals, or may be performed
by a so-called cloud network.
[0112] FIGS. 1, 2, 3A, 3B, 3C, and 3D are each a plan view showing
an example of the appearance of the game apparatus 10. The game
apparatus 10 shown in FIGS. 1 through 3D includes a capturing
section (camera), and therefore is capable of capturing an image
with the capturing section, displaying the captured image on a
screen, and storing data of the captured image. Further, the game
apparatus 10 is capable of executing a game program stored in an
exchangeable memory card, or a game program received from a server
or another game apparatus via a network. The game apparatus 10 is
also capable of displaying on the screen an image generated by
computer graphics processing, such as an image captured by a
virtual camera set in a virtual space. It should be noted that in
the present specification, the act of obtaining image data with the
camera is described as "capturing", and the act of storing the
image data of the captured image is described as
"photographing".
[0113] The game apparatus 10 shown in FIGS. 1 through 3D includes a
lower housing 11 and an upper housing 21. The lower housing 11 and
the upper housing 21 are joined together by a hinge structure so as
to be openable and closable in a folding manner (foldable). That
is, the upper housing 21 is attached to the lower housing 11 so as
to be rotatable (pivotable) relative to the lower housing 11. Thus,
the game apparatus 10 has the following two forms: a closed state
where the upper housing 21 is in firm contact with the lower
housing 11 (FIGS. 3A and 3C); and a state where the upper housing
21 has rotated relative to the lower housing 11 such that the state
of firm contact is released (an open state). The rotation of the
upper housing 21 is allowed to the position where, as shown in FIG.
2, the upper housing 21 and the lower housing 11 are approximately
parallel to each other in the open state (see FIG. 2).
[0114] FIG. 1 is a front view showing an example of the game
apparatus 10 being open (in the open state). A planar shape of each
of the lower housing 11 and the upper housing 21 is a
wider-than-high rectangular plate-like shape having a longitudinal
direction (horizontal direction (left-right direction): an
x-direction in FIG. 1) and a transverse direction ((up-down
direction): a y-direction in FIG. 1). The lower housing 11 and the
upper housing 21 are joined together at the longitudinal upper
outer edge of the lower housing 11 and the longitudinal lower outer
edge of the upper housing 21 by a hinge structure so as to be
rotatable relative to each other. Normally, a user uses the game
apparatus 10 in the open state. The user stores away the game
apparatus 10 in the closed state. Further, the upper housing 21 can
maintain the state of being stationary at a desired angle formed
between the lower housing 11 and the upper housing 21 due, for
example, to a frictional force generated at the connecting part
between the lower housing 11 and the upper housing 21. That is, the
game apparatus 10 can maintain the upper housing 21 stationary at a
desired angle with respect to the lower housing 11. Generally, in
view of the visibility of a screen provided in the upper housing
21, the upper housing 21 is open at a right angle or an obtuse
angle with the lower housing 11. Hereinafter, in the closed state
of the game apparatus 10, the respective opposing surfaces of the
upper housing 21 and the lower housing 11 are referred to as "inner
surfaces" or "main surfaces". Further, the surfaces opposite to the
respective inner surfaces (main surfaces) of the upper housing 21
and the lower housing 11 are referred to as "outer surfaces".
[0115] Projections 11A are provided at the upper long side portion
of the lower housing 11, each projection 11A projecting
perpendicularly (in a z-direction in FIG. 1) to an inner surface
(main surface) 11B of the lower housing 11. A projection (bearing)
21A is provided at the lower long side portion of the upper housing
21, the projection 21A projecting perpendicularly to the lower side
surface of the upper housing 21 from the lower side surface of the
upper housing 21. Within the projections 11A, 21A and 11A, for
example, a rotating shaft (not shown) is accommodated so as to
extend in the x-direction from one of the projections 11A through
the projection 21A to the other projection 11A. The upper housing
21 is freely rotatable about the rotating shaft, relative to the
lower housing 11. Thus, the lower housing 11 and the upper housing
21 are connected together in a foldable manner.
[0116] The inner surface 11B of the lower housing 11 shown in FIG.
1 includes a lower liquid crystal display (LCD) 12, a touch panel
13, operation buttons 14A through 14L, an analog stick 15, a first
LED 16A, and a microphone hole 18.
[0117] The lower LCD 12 is accommodated in the lower housing 11. A
planar shape of the lower LCD 12 is a wider-than-high rectangle,
and is placed such that the long side direction of the lower LCD 12
coincides with the longitudinal direction of the lower housing 11
(the x-direction in FIG. 1). The lower LCD 12 is provided in the
center of the inner surface (main surface) of the lower housing 11.
The screen of the lower LCD 12 is exposed through an opening of the
inner surface of the lower housing 11. The game apparatus 10 is in
the closed state when not used, so that the screen of the lower LCD
12 is prevented from being soiled or damaged. As an example, the
number of pixels of the lower LCD 12 is 320 dots.times.240 dots
(horizontal.times.vertical). Unlike an upper LCD 22 described
later, the lower LCD 12 is a display device that displays an image
in a planar manner (not in a stereoscopically visible manner). It
should be noted that although an LCD is used as a display device in
the first embodiment, any other display device may be used, such as
a display device using electroluminescence (EL). Further, a display
device having a desired resolution may be used as the lower LCD
12.
[0118] The touch panel 13 is one of input devices of the game
apparatus 10. The touch panel 13 is mounted so as to cover the
screen of the lower LCD 12. In the first embodiment, the touch
panel 13 may be, but is not limited to, a resistive touch panel.
The touch panel may also be a touch panel of any pressure type,
such as an electrostatic capacitance type. In the first embodiment,
the touch panel 13 has the same resolution (detection accuracy) as
that of the lower LCD 12. The resolutions of the touch panel 13 and
the lower LCD 12, however, may not necessarily need to be the
same.
[0119] The operation buttons 14A through 14L are each an input
device for providing a predetermined input. Among the operation
buttons 14A through 14L, the cross button 14A (direction input
button 14A), the button 14B, the button 14C, the button 14D, the
button 14E, the power button 14F, the select button 14J, the home
button 14K, and the start button 14L are provided on the inner
surface (main surface) of the lower housing 11.
[0120] The cross button 14A is cross-shaped, and includes buttons
for indicating at least up, down, left, and right directions,
respectively. The cross button 14A is provided in an lower area of
the area to the left of the lower LCD 12. The cross button 14A is
placed so as to be operated by the thumb of a left hand holding the
lower housing 11.
[0121] The button 14B, the button 14C, the button 14D, and the
button 14E are placed in a cross formation in an upper portion of
the area to the right of the lower LCD 12. The button 14B, the
button 14C, the button 14D, and the button 14E, are placed where
the thumb of a right hand holding the lower housing 11 is naturally
placed. The power button 14F is placed in a lower portion of the
area to the right of the lower LCD 12.
[0122] The select button 14J, the home button 14K, and the start
button 14L are provided in a lower area of the lower LCD 12.
[0123] The buttons 14A through 14E, the select button 14J, the home
button 14K, and the start button 14L are appropriately assigned
functions, respectively, in accordance with the program executed by
the game apparatus 10. The cross button 14A is used for, for
example, a selection operation and a moving operation of a
character during a game. The operation buttons 14B through 14E are
used for, for example, a determination operation or a cancellation
operation. The power button 14F is used to power on/off the game
apparatus 10.
[0124] The analog stick 15 is a device for indicating a direction.
The analog stick 15 is provided to an upper portion of the area to
the left of the lower LCD 12 of the inner surface (main surface) of
the lower housing 11. That is, the analog stick 15 is provided
above the cross button 14A. The analog stick 15 is placed so as to
be operated by the thumb of a left hand holding the lower housing
11. The provision of the analog stick 15 in the upper area places
the analog stick 15 at the position where the thumb of the left
hand of the user holding the lower housing 11 is naturally placed.
The cross button 14A is placed at the position where the thumb of
the left hand holding the lower housing 11 is moved slightly
downward. This enables the user to operate the analog stick 15 and
the cross button 14A by moving up and down the thumb of the left
hand holding the lower housing 11. The key top of the analog stick
15 is configured to slide parallel to the inner surface of the
lower housing 11. The analog stick 15 functions in accordance with
the program executed by the game apparatus 10. When, for example,
the game apparatus 10 executes a game where a predetermined object
appears in a three-dimensional virtual space, the analog stick 15
functions as an input device for moving the predetermined object in
the three-dimensional virtual space. In this case, the
predetermined object is moved in the direction in which the key top
of the analog stick 15 has slid. It should be noted that the analog
stick 15 may be a component capable of providing an analog input by
being tilted by a predetermined amount in any one of up, down,
right, left, and diagonal directions.
[0125] It should be noted that the four buttons, namely the button
14B, the button 14C, the button 14D, and the button 14E, and the
analog stick 15 are placed symmetrically to each other with respect
to the lower LCD 12. This also enables, for example, a left-handed
person to provide a direction indication input using these four
buttons, namely the button 14B, the button 14C, the button 14D, and
the button 14E, depending on the game program.
[0126] The first LED 16A (FIG. 1) notifies the user of the on/off
state of the power supply of the game apparatus 10. The first LED
16A is provided on the right of an end portion shared by the inner
surface (main surface) of the lower housing 11 and the lower side
surface of the lower housing 11. This enables the user to view
whether or not the first LED 16A is lit on, regardless of the
open/closed state of the game apparatus 10.
[0127] The microphone hole 18 is a hole for a microphone built into
the game apparatus 10 as a sound input device. The built-in
microphone detects a sound from outside the game apparatus 10
through the microphone hole 18. The microphone and the microphone
hole 18 are provided below the power button 14F on the inner
surface (main surface) of the lower housing 11.
[0128] The upper side surface of the lower housing 11 includes an
opening 17 (a dashed line shown in FIGS. 1 and 3D) for a stylus 28.
The opening 17 can accommodate the stylus 28 that is used to
perform an operation on the touch panel 13. It should be note that,
normally, an input is provided to the touch panel 13 using the
stylus 28. The touch panel 13, however, can be operated not only by
the stylus 28 but also by a finger of the user.
[0129] The upper side surface of the lower housing 11 includes an
insertion slot 11D (a dashed line shown in FIGS. 1 and 3D), into
which an external memory 45 having a game program stored thereon is
to be inserted. Within the insertion slot 11D, a connector (not
shown) is provided for electrically connecting the game apparatus
10 and the external memory 45 in a detachable manner. The
connection of the external memory 45 to the game apparatus 10
causes a processor included in internal circuitry to execute a
predetermined game program. It should be noted that the connector
and the insertion slot 11D may be provided on another side surface
(e.g., the right side surface) of the lower housing 11.
[0130] The inner surface 21B of the upper housing 21 shown in FIG.
1 includes loudspeaker holes 21E, an upper LCD 22, an inner
capturing section 24, a 3D adjustment switch 25, and a 3D indicator
26 are provided. The inner capturing section 24 is an example of a
first capturing device.
[0131] The upper LCD 22 is a display device capable of displaying a
stereoscopically visible image. The upper LCD 22 is capable of
displaying a left-eye image and a right-eye image, using
substantially the same display area. Specifically, the upper LCD 22
is a display device using a method in which the left-eye image and
the right-eye image are displayed alternately in the horizontal
direction in predetermined units (e.g., in every other line). It
should be noted that the upper LCD 22 may be a display device using
a method in which the left-eye image and the right-eye image are
displayed alternately for a predetermined time. Further, the upper
LCD 22 is a display device capable of displaying an image
stereoscopically visible with the naked eye. In this case, a
lenticular type display device or a parallax barrier type display
device is used so that the left-eye image and the right-eye image
that are displayed alternately in the horizontal direction can be
viewed separately with the left eye and the right eye,
respectively. In the first embodiment, the upper LCD 22 is a
parallax-barrier-type display device. The upper LCD 22 displays an
image stereoscopically visible with the naked eye (a stereoscopic
image), using the right-eye image and the left-eye image. That is,
the upper LCD 22 allows the user to view the left-eye image with
their left eye, and the right-eye image with their right eye, using
the parallax barrier. This makes it possible to display a
stereoscopic image giving the user a stereoscopic effect
(stereoscopically visible image). Furthermore, the upper LCD 22 is
capable of disabling the parallax barrier. When disabling the
parallax barrier, the upper LCD 22 is capable of displaying an
image in a planar manner (the upper LCD 22 is capable of displaying
a planar view image, as opposed to the stereoscopically visible
image described above. This is a display mode in which the same
displayed image can be viewed with both the left and right eyes).
Thus, the upper LCD 22 is a display device capable of switching
between: the stereoscopic display mode for displaying a
stereoscopically visible image; and the planar display mode for
displaying an image in a planar manner (displaying a planar view
image). The switching of the display modes is performed by the 3D
adjustment switch 25 described later.
[0132] The upper LCD 22 is accommodated in the upper housing 21. A
planar shape of the upper LCD 22 is a wider-than-high rectangle,
and is placed at the center of the upper housing 21 such that the
long side direction of the upper LCD 22 coincides with the long
side direction of the upper housing 21. As an example, the area of
the screen of the upper LCD 22 is set greater than that of the
lower LCD 12. Specifically, the screen of the upper LCD 22 is set
horizontally longer than the screen of the lower LCD 12. That is,
the proportion of the width in the aspect ratio of the screen of
the upper LCD 22 is set greater than that of the lower LCD 12. The
screen of the upper LCD 22 is provided on the inner surface (main
surface) 21B of the upper housing 21, and is exposed through an
opening of the inner surface of the upper housing 21. Further, the
inner surface of the upper housing 21 is covered by a transparent
screen cover 27. The screen cover 27 protects the screen of the
upper LCD 22, and integrates the upper LCD 22 and the inner surface
of the upper housing 21, and thereby provides unity. As an example,
the number of pixels of the upper LCD 22 is 800 dots.times.240 dots
(horizontal.times.vertical). It should be noted that an LCD is used
as the upper LCD 22 in the first embodiment. The upper LCD 22,
however, is not limited to this, and a display device using EL or
the like may be used. Furthermore, a display device having any
resolution may be used as the upper LCD 22.
[0133] The loudspeaker holes 21E are holes through which sounds
from loudspeakers 44 that serve as a sound output device of the
game apparatus 10 are output. The loudspeakers holes 21E are placed
symmetrically with respect to the upper LCD. Sounds from the
loudspeakers 44 described later are output through the loudspeaker
holes 21E.
[0134] The inner capturing section 24 functions as a capturing
section having an imaging direction that is the same as the inward
normal direction of the inner surface 21B of the upper housing 21.
The inner capturing section 24 includes an imaging device having a
predetermined resolution, and a lens. The lens may have a zoom
mechanism.
[0135] The inner capturing section 24 is placed: on the inner
surface 21B of the upper housing 21; above the upper edge of the
screen of the upper LCD 22; and in the center of the upper housing
21 in the left-right direction (on the line dividing the upper
housing 21 (the screen of the upper LCD 22) into two equal left and
right portions). Such a placement of the inner capturing section 24
makes it possible that when the user views the upper LCD 22 from
the front thereof, the inner capturing section 24 captures the
user's face from the front thereof. A left outer capturing section
23a and a right outer capturing section 23b will be described
later.
[0136] The 3D adjustment switch 25 is a slide switch, and is used
to switch the display modes of the upper LCD 22 as described above.
The 3D adjustment switch 25 is also used to adjust the stereoscopic
effect of a stereoscopically visible image (stereoscopic image)
displayed on the upper LCD 22. The 3D adjustment switch 25 is
provided at an end portion shared by the inner surface and the
right side surface of the upper housing 21, so as to be visible to
the user, regardless of the open/closed state of the game apparatus
10. The 3D adjustment switch 25 includes a slider that is slidable
to any position in a predetermined direction (e.g., the up-down
direction), and the display mode of the upper LCD 22 is set in
accordance with the position of the slider.
[0137] When, for example, the slider of the 3D adjustment switch 25
is placed at the lowermost position, the upper LCD 22 is set to the
planar display mode, and a planar image is displayed on the screen
of the upper LCD 22. It should be noted that the same image may be
used as the left-eye image and the right-eye image, while the upper
LCD 22 remains set to the stereoscopic display mode, and thereby
performs planar display. On the other hand, when the slider is
placed above the lowermost position, the upper LCD 22 is set to the
stereoscopic display mode. In this case, a stereoscopically visible
image is displayed on the screen of the upper LCD 22. When the
slider is placed above the lowermost position, the visibility of
the stereoscopic image is adjusted in accordance with the position
of the slider. Specifically, the amount of deviation in the
horizontal direction between the position of the right-eye image
and the position of the left-eye image is adjusted in accordance
with the position of the slider.
[0138] The 3D indicator 26 indicates whether or not the upper LCD
22 is in the stereoscopic display mode. For example, the 3D
indicator 26 is an LED, and is lit on when the stereoscopic display
mode of the upper LCD 22 is enabled. The 3D indicator 26 is placed
on the inner surface 21B of the upper housing 21 near the screen of
the upper LCD 22. Accordingly, when the user views the screen of
the upper LCD 22 from the front thereof, the user can easily view
the 3D indicator 26. This enables the user to easily recognize the
display mode of the upper LCD 22 even when viewing the screen of
the upper LCD 22.
[0139] FIG. 2 is a right side view showing an example of the game
apparatus 10 in the open state. The right side surface of the lower
housing 11 includes a second LED 16B, a wireless switch 19, and the
R button 14H. The second LED 16B notifies the user of the
establishment state of the wireless communication of the game
apparatus 10. The game apparatus 10 is capable of wirelessly
communicating with other devices, and the second LED 16B is lit on
when wireless communication is established between the game
apparatus 10 and other devices. The game apparatus 10 has the
function of establishing connection with a wireless LAN by, for
example, a method based on the IEEE 802.11.b/g standard. The
wireless switch 19 enables/disables the function of the wireless
communication. The R button 14H will be described later.
[0140] FIG. 3A is a left side view showing an example of the game
apparatus 10 being closed (in the closed state). The left side
surface of the lower housing 11 shown in FIG. 3A includes an
openable and closable cover section 11C, the L button 14H, and the
sound volume button 141. The sound volume button 141 is used to
adjust the sound volume of the loudspeakers of the game apparatus
10.
[0141] Within the cover section 11C, a connector (not shown) is
provided for electrically connecting the game apparatus 10 and a
data storage external memory 46 (see FIG. 1). The data storage
external memory 46 is detachably attached to the connector. The
data storage external memory 46 is used to, for example, store
(save) data of an image captured by the game apparatus 10. It
should be noted that the connector and the cover section 11C may be
provided on the right side surface of the lower housing 11. The L
button 14H will be described later.
[0142] FIG. 3B is a front view showing an example of the game
apparatus 10 in the closed state. The outer surface of the upper
housing 21 shown in FIG. 3B includes a left outer capturing section
23a, a right outer capturing section 23b, and a third LED 29.
[0143] The left outer capturing section 23a and the right outer
capturing section 23b each includes an imaging device (e.g., a CCD
image sensor or a CMOS image sensor) having a predetermined common
resolution, and a lens. The lens may have a zoom mechanism. The
imaging directions of the left outer capturing section 23a and the
right outer capturing section 23b (the optical axis of the camera)
are each the same as the outward normal direction of the outer
surface 21D. That is, the imaging direction of the left outer
capturing section 23a and the imaging direction of the right outer
capturing section 23b are parallel to each other. Hereinafter, the
left outer capturing section 23a and the right outer capturing
section 23b are collectively referred to as an "outer capturing
section 23". The outer capturing section 23 is an example of a
second capturing device.
[0144] The left outer capturing section 23a and the right outer
capturing section 23b included in the outer capturing section 23
are placed along the horizontal direction of the screen of the
upper LCD 22. That is, the left outer capturing section 23a and the
right outer capturing section 23b are placed such that a straight
line connecting between the left outer capturing section 23a and
the right outer capturing section 23b is placed along the
horizontal direction of the screen of the upper LCD 22. When the
user has pivoted the upper housing 21 at a predetermined angle
(e.g., 90.degree.) relative to the lower housing 11, and views the
screen of the upper LCD 22 from the front thereof, the left outer
capturing section 23a is placed on the left side of the user
viewing the screen, and the right outer capturing section 23b is
placed on the right side of the user (see FIG. 1). The distance
between the left outer capturing section 23a and the right outer
capturing section 23b is set to correspond to the distance between
both eyes of a person, and may be set, for example, in the range
from 30 mm to 70 mm. It should be noted, however, that the distance
between the left outer capturing section 23a and the right outer
capturing section 23b is not limited to this range. It should be
noted that in the first embodiment, the left outer capturing
section 23a and the right outer capturing section 23b are fixed to
the housing 21, and therefore, the imaging directions cannot be
changed.
[0145] The left outer capturing section 23a and the right outer
capturing section 23b are placed symmetrically with respect to the
line dividing the upper LCD 22 (the upper housing 21) into two
equal left and right portions. Further, the left outer capturing
section 23a and the right outer capturing section 23b are placed in
the upper portion of the upper housing 21 and in the back of the
portion above the upper edge of the screen of the upper LCD 22, in
the state where the upper housing 21 is in the open state (see FIG.
1). That is, the left outer capturing section 23a and the right
outer capturing section 23b are placed on the outer surface of the
upper housing 21, and, if the upper LCD 22 is projected onto the
outer surface of the upper housing 21, is placed above the upper
edge of the screen of the projected upper LCD 22.
[0146] Thus, the left outer capturing section 23a and the right
outer capturing section 23b of the outer capturing section 23 are
placed symmetrically with respect to the center line of the upper
LCD 22 extending in the transverse direction. This makes it
possible that when the user views the upper LCD 22 from the front
thereof, the imaging directions of the outer capturing section 23
coincide with the directions of the respective lines of sight of
the user's right and left eyes. Further, the outer capturing
section 23 is placed in the back of the portion above the upper
edge of the screen of the upper LCD 22, and therefore, the outer
capturing section 23 and the upper LCD 22 do not interfere with
each other inside the upper housing 21. Further, when the inner
capturing section 24 provided on the inner surface of the upper
housing 21 as shown by a dashed line in FIG. 3B is projected onto
the outer surface of the upper housing 21, the left outer capturing
section 23a and the right outer capturing section 23b are placed
symmetrically with respect to the projected inner capturing section
24. This makes it possible to reduce the upper housing 21 in
thickness as compared to the case where the outer capturing section
23 is placed in the back of the screen of the upper LCD 22, or the
case where the outer capturing section 23 is placed in the back of
the inner capturing section 24.
[0147] The left outer capturing section 23a and the right outer
capturing section 23b can be used as a stereo camera, depending on
the program executed by the game apparatus 10. Alternatively,
either one of the two outer capturing sections (the left outer
capturing section 23a and the right outer capturing section 23b)
may be used solely, so that the outer capturing section 23 can also
be used as a non-stereo camera, depending on the program. When a
program is executed for causing the left outer capturing section
23a and the right outer capturing section 23b to function as a
stereo camera, the left outer capturing section 23a captures a
left-eye image, which is to be viewed with the user's left eye, and
the right outer capturing section 23b captures a right-eye image,
which is to be viewed with the user's right eye. Yet alternatively,
depending on the program, images captured by the two outer
capturing sections (the left outer capturing section 23a and the
right outer capturing section 23b) may be combined together, or may
be used to compensate for each other, so that imaging can be
performed with an extended imaging range. Yet alternatively, a
left-eye image and a right-eye image that have a parallax may be
generated from a single image captured using one of the outer
capturing sections 23a and 23b, and a pseudo-stereo image as if
captured by two cameras can be generated. To generate the
pseudo-stereo image, it is possible to appropriately set the
distance between virtual cameras.
[0148] The third LED 29 is lit on when the outer capturing section
23 is operating, and informs that the outer capturing section 23 is
operating. The third LED 29 is provided near the outer capturing
section 23 on the outer surface of the upper housing 21.
[0149] FIG. 3C is a right side view showing an example of the game
apparatus 10 in the closed state. FIG. 3D is a rear view showing an
example of the game apparatus 10 in the closed state.
[0150] The L button 14G and the R button 14H are provided on the
upper side surface of the lower housing 11 shown in FIG. 3D. The L
button 14G is provided at the left end portion of the upper side
surface of the lower housing 11, and the R button 14H is provided
at the right end portion of the upper side surface of the lower
housing 11. The L button 14G and the R button 14H are appropriately
assigned functions, respectively, in accordance with the program
executed by the game apparatus 10. For example, the L button 14G
and the R button 14H function as shutter buttons (capturing
instruction buttons) of the capturing sections described above.
[0151] It should be noted that although not shown in the figures, a
rechargeable battery that serves as the power supply of the game
apparatus 10 is accommodated in the lower housing 11, and the
battery can be charged through a terminal provided on the side
surface (e.g., the upper side surface) of the lower housing 11.
[0152] FIGS. 4 and 5 each show an example of the state of the use
of the game apparatus 10. FIG. 4 is a diagram showing an example of
a user holding the game apparatus 10 with both hands.
[0153] In the example shown in FIG. 4, the user holds the side
surfaces and the outer surface (the surface opposite to the inner
surface) of the lower housing 11 with both palms, middle fingers,
ring fingers, and little fingers, such that the lower LCD 12 and
the upper LCD 22 face the user. Such holding enables the user to
perform operations on the operation buttons 14A through 14E and the
analog stick 15 with their thumbs, and to perform operations on the
L button 14G and the R button 14H with their index fingers, while
holding the lower housing 11.
[0154] FIG. 5 is a diagram showing an example of a user holding the
game apparatus 10 with one hand. In the example shown in FIG. 5,
when providing an input on the touch panel 13, the user releases
one of the hands having held the lower housing 11 therefrom, and
holds the lower housing 11 only with the other hand. This makes it
possible to provide an input to the touch panel 13 with the one
hand.
[0155] FIG. 6 is a block diagram showing an example of the internal
configuration of the game apparatus 10. The game apparatus 10
includes, as well as the components described above, electronic
components, such as an information processing section 31, a main
memory 32, an external memory interface (external memory I/F) 33, a
data storage external memory I/F 34, a data storage internal memory
35, a wireless communication module 36, a local communication
module 37, a real-time clock (RTC) 38, an acceleration sensor 39,
an angular velocity sensor 40, a power circuit 41, and an interface
circuit (I/F circuit) 42. These electronic components are mounted
on electronic circuit boards, and are accommodated in the lower
housing 11 (or may be accommodated in the upper housing 21).
[0156] The information processing section 31 is information
processing means including a central processing unit (CPU) 311 that
executes a predetermined program, a graphics processing unit (GPU)
312 that performs image processing, and the like. In the first
embodiment, a predetermined program is stored in a memory (e.g.,
the external memory 45 connected to the external memory I/F 33, or
the data storage internal memory 35) included in the game apparatus
10. The CPU 311 of the information processing section 31 executes
the predetermined program, and thereby performs the image
processing described later or game processing. It should be noted
that the program executed by the CPU 311 of the information
processing section 31 may be acquired from another device by
communication with said another device. The information processing
section 31 further includes a video RAM (VRAM) 313. The GPU 312 of
the information processing section 31 generates an image in
accordance with an instruction from the CPU 311 of the information
processing section 31, and draws the image in the VRAM 313. The GPU
312 of the information processing section 31 outputs the image
drawn in the VRAM 313 to the upper LCD 22 and/or the lower LCD 12,
and the image is displayed on the upper LCD 22 and/or the lower LCD
12.
[0157] To the information processing section 31, the main memory
32, the external memory I/F 33, the data storage external memory
I/F 34, and the data storage internal memory 35 are connected. The
external memory I/F 33 is an interface for establishing a
detachable connection with the external memory 45. The data storage
external memory I/F 34 is an interface for establishing a
detachable connection with the data storage external memory 46.
[0158] The main memory 32 is volatile storage means used as a work
area or a buffer area of the information processing section 31 (the
CPU 311). That is, the main memory 32 temporarily stores various
types of data used for image processing or game processing, and
also temporarily stores a program acquired from outside (the
external memory 45, another device, or the like) the game apparatus
10. In the first embodiment, the main memory 32 is, for example, a
pseudo SRAM (PSRAM).
[0159] The external memory 45 is nonvolatile storage means for
storing the program executed by the information processing section
31. The external memory 45 is composed of, for example, a read-only
semiconductor memory. When the external memory 45 is connected to
the external memory I/F 33, the information processing section 31
can load a program stored in the external memory 45. In accordance
with the execution of the program loaded by the information
processing section 31, a predetermined process is performed. The
data storage external memory 46 is composed of a readable/writable
non-volatile memory (e.g., a NAND flash memory), and is used to
store predetermined data. For example, the data storage external
memory 46 stores images captured by the outer capturing section 23
and/or images captured by another device. When the data storage
external memory 46 is connected to the data storage external memory
I/F 34, the information processing section 31 loads an image stored
in the data storage external memory 46, and the image can be
displayed on the upper LCD 22 and/or the lower LCD 12.
[0160] The data storage internal memory 35 is composed of a
readable/writable non-volatile memory (e.g., a NAND flash memory),
and is used to store predetermined data. For example, the data
storage internal memory 35 stores data and/or programs downloaded
by wireless communication through the wireless communication module
36.
[0161] The wireless communication module 36 has the function of
establishing connection with a wireless LAN by, for example, a
method based on the IEEE 802.11.b/g standard. Further, the local
communication module 37 has the function of wirelessly
communicating with another game apparatus of the same type by a
predetermined communication method (e.g., infrared communication).
The wireless communication module 36 and the local communication
module 37 are connected to the information processing section 31.
The information processing section 31 is capable of transmitting
and receiving data to and from another device via the Internet,
using the wireless communication module 36, and is capable of
transmitting and receiving data to and from another game apparatus
of the same type, using the local communication module 37.
[0162] The acceleration sensor 39 is connected to the information
processing section 31. The acceleration sensor 39 detects the
magnitudes of accelerations (linear accelerations) in the
directions of straight lines along three axial (x, y, and z axes in
the present embodiment) directions, respectively. The acceleration
sensor 39 is provided, for example, within the lower housing 11. As
shown in FIG. 1, the long side direction of the lower housing 11 is
defined as an x-axis direction; the short side direction of the
lower housing 11 is defined as a y-axis direction; and the
direction perpendicular to the inner surface (main surface) of the
lower housing 11 is defined as a z-axis direction. The acceleration
sensor 39 thus detects the magnitudes of the linear accelerations
produced in the respective axial directions. It should be noted
that the acceleration sensor 39 is, for example, an electrostatic
capacitance type acceleration sensor, but may be an acceleration
sensor of another type. Further, the acceleration sensor 39 may be
an acceleration sensor for detecting an acceleration in one axial
direction, or accelerations in two axial directions. The
information processing section 31 receives data indicating the
accelerations detected by the acceleration sensor 39 (acceleration
data), and calculates the orientation and the motion of the game
apparatus 10.
[0163] The angular velocity sensor 40 is connected to the
information processing section 31. The angular velocity sensor 40
detects angular velocities generated about three axes (x, y, and z
axes in the present embodiment) of the game apparatus 10,
respectively, and outputs data indicating the detected angular
velocities (angular velocity data) to the information processing
section 31. The angular velocity sensor 40 is provided, for
example, within the lower housing 11. The information processing
section 31 receives the angular velocity data output from the
angular velocity sensor 40, and calculates the orientation and the
motion of the game apparatus 10.
[0164] The RTC 38 and the power circuit 41 are connected to the
information processing section 31. The RTC 38 counts time, and
outputs the counted time to the information processing section 31.
The information processing section 31 calculates the current time
(date) based on the time counted by the RTC 38. The power circuit
41 controls the power from the power supply (the rechargeable
battery accommodated in the lower housing 11, which is described
above) of the game apparatus 10, and supplies power to each
component of the game apparatus 10.
[0165] The I/F circuit 42 is connected to the information
processing section 31. A microphone 43, a loudspeaker 44, and the
touch panel 13 are connected to the I/F circuit 42. Specifically,
the loudspeaker 44 is connected to the I/F circuit 42 through an
amplifier not shown in the figures. The microphone 43 detects a
sound from the user, and outputs a sound signal to the I/F circuit
42. The amplifier amplifies the sound signal from the I/F circuit
42, and outputs the sound from the loudspeaker 44. The I/F circuit
42 includes: a sound control circuit that controls the microphone
43 and the loudspeaker 44 (amplifier); and a touch panel control
circuit that controls the touch panel 13. For example, the sound
control circuit performs A/D conversion and D/A conversion on the
sound signal, and converts the sound signal to sound data in a
predetermined format. The touch panel control circuit generates
touch position data in a predetermined format, based on a signal
from the touch panel 13, and outputs the touch position data to the
information processing section 31. The touch position data
indicates the coordinates of the position (touch position), on the
input surface of the touch panel 13, at which an input has been
provided. It should be noted that the touch panel control circuit
reads a signal from the touch panel 13, and generates the touch
position data, once in a predetermined time. The information
processing section 31 acquires the touch position data, and thereby
recognizes the touch position, at which the input has been provided
on the touch panel 13.
[0166] An operation button 14 includes the operation buttons 14A
through 14L described above, and is connected to the information
processing section 31. Operation data is output from the operation
button 14 to the information processing section 31, the operation
data indicating the states of inputs provided to the respective
operation buttons 14A through 141 (indicating whether or not the
operation buttons 14A through 141 have been pressed). The
information processing section 31 acquires the operation data from
the operation button 14, and thereby performs processes in
accordance with the inputs provided to the operation button 14.
[0167] The lower LCD 12 and the upper LCD 22 are connected to the
information processing section 31. The lower LCD 12 and the upper
LCD 22 each display an image in accordance with an instruction from
the information processing section 31 (the GPU 312). In the first
embodiment, the information processing section 31 causes the lower
LCD 12 to display an image for a hand-drawn image input operation,
and causes the upper LCD 22 to display an image acquired from
either one of the outer capturing section 23 and the inner
capturing section 24. That is, for example, the information
processing section 31 causes the upper LCD 22 to display a
stereoscopic image (stereoscopically visible image) using a
right-eye image and a left-eye image that are captured by the inner
capturing section 24, or causes the upper LCD 22 to display a
planar image using one of a right-eye image and a left-eye image
that are captured by the outer capturing section 23.
[0168] Specifically, the information processing section 31 is
connected to an LCD controller (not shown) of the upper LCD 22, and
causes the LCD controller to set the parallax barrier to on/off.
When the parallax barrier is on in the upper LCD 22, a right-eye
image and a left-eye image that are stored in the VRAM 313 of the
information processing section 31 (that are captured by the outer
capturing section 23) are output to the upper LCD 22. More
specifically, the LCD controller repeatedly alternates the reading
of pixel data of the right-eye image for one line in the vertical
direction, and the reading of pixel data of the left-eye image for
one line in the vertical direction, and thereby reads the right-eye
image and the left-eye image from the VRAM 313. Thus, the right-eye
image and the left-eye image are each divided into strip images,
each of which has one line of pixels placed in the vertical
direction, and an image including the divided left-eye strip images
and the divided right-eye strip images alternately placed is
displayed on the screen of the upper LCD 22. The user views the
images through the parallax barrier of the upper LCD 22, whereby
the right-eye image is viewed with the user's right eye, and the
left-eye image is viewed with the user's left eye. This causes the
stereoscopically visible image to be displayed on the screen of the
upper LCD 22.
[0169] The outer capturing section 23 and the inner capturing
section 24 are connected to the information processing section 31.
The outer capturing section 23 and the inner capturing section 24
each capture an image in accordance with an instruction from the
information processing section 31, and output data of the captured
image to the information processing section 31. In the first
embodiment, the information processing section 31 gives either one
of the outer capturing section 23 and the inner capturing section
24 an instruction to capture an image, and the capturing section
that has received the instruction captures an image, and transmits
data of the captured image to the information processing section
31. Specifically, the user selects the capturing section to be
used, through an operation using the touch panel 13 and the
operation button 14. The information processing section 31 (the CPU
311) detects that an capturing section has been selected, and the
information processing section 31 gives the selected one of the
outer capturing section 23 and the inner capturing section 24 an
instruction to capture an image.
[0170] When started by an instruction from the information
processing section 31 (CPU 311), the outer capturing section 23 and
the inner capturing section 24 perform capturing at, for example, a
speed of 60 images per second. The captured images captured by the
outer capturing section 23 and the inner capturing section 24 are
sequentially transmitted to the information processing section 31,
and displayed on the upper LCD 22 or the lower LCD 12 by the
information processing section 31 (GPU 312). When output to the
information processing section 31, the captured images are stored
in the VRAM 313, are output to the upper LCD 22 or the lower LCD
12, and are deleted at predetermined times. Thus, images are
captured at, for example, a speed of 60 images per second, and the
captured images are displayed, whereby the game apparatus 10 can
display views in the imaging ranges of the outer capturing section
23 and the inner capturing section 24, on the upper LCD 22 of the
lower LCD 12 in real time.
[0171] The 3D adjustment switch 25 is connected to the information
processing section 31. The 3D adjustment switch 25 transmits to the
information processing section 31 an electrical signal in
accordance with the position of the slider.
[0172] The 3D indicator 26 is connected to the information
processing section 31. The information processing section 31
controls whether or not the 3D indicator 26 is to be lit on. When,
for example, the upper LCD 22 is in the stereoscopic display mode,
the information processing section 31 lights on the 3D indicator
26.
[0173] <Descriptions of Functions>
[0174] Next, a description is given of an overview of an example of
game processing performed by the game apparatus 10. The game
apparatus 10 provides the function of collecting face images by
acquiring and saving, for example, face images of people through
the inner capturing section 24, the outer capturing section 23, or
the like in accordance with an operation of a user (hereinafter
also referred to as a "player"). To collect face images, the user
executes a game (first game) using an acquired face image, and when
the result of the game has been successful, the user can save the
acquired image. It should be noted that the user can acquire a face
image, which is a target to be saved, from: an image captured by
the inner capturing section 24, the outer capturing section 23, or
the like before executing the first game; an image acquired by an
application different from the first game before executing the
first game; an image captured by the inner capturing section 24,
the outer capturing section 23, or the like during the execution of
the first game; or the like. As described later, the game apparatus
10 saves the face image acquired before the first game or during
the first game, in a saved data storage area Do (see FIG. 11),
which is accessible during the execution of the game processing.
Then, the user repeats a similar operation on the game apparatus
10, and thereby can collect a plurality of face images, add the
face images to the saved data storage area Do, and accumulate the
face images. The saved data storage area Do is an area accessible
to the game apparatus 10 that is executing the game. Accordingly,
the acquired face image is saved in the saved data storage area Do,
and thereby is available in the subsequent processes. Further, the
game apparatus 10 reads data accumulated in the saved data storage
area Do, and thereby displays a list of face images collected as a
result of the first game. Then, the game apparatus 10 executes a
game (second game) using a face image selected by the user, or a
face image automatically selected by the game apparatus 10, from
among the displayed list of face images.
[0175] The games executed by the game apparatus 10 (the first game
and the second game) are, for example, each a game where the user
makes an attack on enemy objects EO by aiming at them, and destroys
the enemy objects EO. In the first embodiment, for example, a face
image acquired by the user and yet to be saved in the saved data
storage area Do, or a face image acquired by the user and already
saved in the saved data storage area Do, is mapped as a texture
onto a character object, such as an enemy object EO.
[0176] First, at the stage of the execution of the first game
according to the first embodiment, the user can execute the first
game by acquiring a desired face image through an capturing
section, such as a camera. Then, when having succeeded in the first
game, the user can save the acquired face image in the saved data
storage area Do in an accumulating manner, cause a list of the face
images to be displayed, and use the face images in the second game.
Here, "in an accumulating manner" means that when the user has
acquired a new face image and further succeeded in the first game,
the new face image is added.
[0177] At the stage of the execution of the second game, the user
can select a desired face image from among the collected face
images, and create an enemy object EO. Then, the user can execute a
game using the enemy object EO created using the desired face
image, for example, a game where the user destroys the created
enemy object EO. Without an operation of the user, however, the
game apparatus 10 may automatically select a face image, for
example, randomly from the saved data storage area Do, and may
create an enemy object EO or the like. It should be noted that also
at the stage of the execution of the first game, a character object
may be created using a face image already collected in the saved
data storage area Do, and may be caused to appear in the game
together with a character object created using a face image yet to
be saved in the saved data storage area Do. It should be noted that
hereinafter, when a plurality of enemy objects EO are distinguished
from one another, the enemy objects EO are referred to as, for
example, "enemy objects EO1, EO2 . . . ." On the other hand, when
the enemy objects EO1, EO2, and the like are collectively referred
to, or a plurality of enemy objects do not need to be distinguished
from one another, the enemy objects EO are referred to as "enemy
objects EO".
[0178] Next, with reference to FIGS. 7 through 10, a description is
given of examples of display of the game apparatus 10 according to
the first embodiment. FIG. 7 is an example of face images displayed
as a list on the upper LCD 22 of the game apparatus 10. As
described above, when the user has succeeded in the first game, the
user can save acquired face images, and cause a list of the face
images to be displayed as shown in FIG. 7. The face images
displayed as a list are each obtained by texture-mapping image data
acquired by, for example, the inner capturing section 24, the left
outer capturing section 23a, or the right outer capturing section
23b, onto a three-dimensional model of a human facial surface. The
image data is attached as a texture to, for example, the surface of
a three-dimensional model formed by combining a plurality of
polygons. In the game executed by the game apparatus 10, however,
the face images are not limited to those obtained by performing
texture-mapping on three-dimensional models. For example, the game
apparatus 10 may display, as a face image, image data held in a
simple two-dimensional pixel array. Alternatively, as shown in FIG.
7, among the face images displayed as a list, at least one face
image may be held in a simple two-dimensional pixel array, and the
other face images may be obtained by performing texture-mapping on
three-dimensional models.
[0179] In FIG. 7, a face image G1 is surrounded by a heavy line L1.
The heavy line L1 indicates that the face image G1 is in the state
of being selected. "The state of being selected" means the state of
being selected as a processing target by, for example, the user
operating the operation buttons 14 or the like. The state of being
selected is also referred to as "the state of being focused on".
For example, each time the user presses the operation buttons 14,
the face image in the state of being selected is switched from left
to right, or top to bottom. For example, when the user has pressed
the right direction of the cross button 14A in the state of FIG. 7,
the face image in the state of being selected transfers to the
right, such as from the face image G1 to a face image G2 and from
the face image G2 to the face image G3. "The face image in the
state of being selected transfers" means that the face image
surrounded by the heavy line L1 is switched on the screen of the
upper LCD 22.
[0180] It should be noted that in FIG. 7, a horizontal row of face
images is referred to as a "tier". In the case where a face image
G4 at the right end of one of the tiers is in the state of being
selected, if the user further presses the right direction of the
cross button 14A, the face image in the state of being selected is
switched to a face image G5 at the left end of the next lower tier.
Conversely, for example, in the case where the face image G5 at the
left end of the lower tier is in the state of being selected, if
the user presses the left direction of the cross button 14A, the
face image in the state of being selected transfers upward and to
the left, such as from the face image G5 to the face image G4 and
from the face image G4 to the face image G3.
[0181] The switching of the state of being selected, however, is
not limited to the pressing of the left and right directions of the
cross button 14A, and the state of being selected may be switched
by pressing the up and down directions. Further, the switching of
the state of being selected is not limited to the pressing of the
cross button 14A, and the image in the state of being selected may
be switched by pressing other operation buttons 14, such as the
operation button 14B (A button). Alternatively, the face image in
the state of being selected may be switched by performing an
operation on the touch panel 13 of the lower LCD 12. For example,
the game apparatus 10 displays in advance on the lower LCD 12 a
list of the face images similar to the list of the face images
displayed on the upper LCD 22. Then, the game apparatus 10 may
detect an operation on the touch panel 13, and thereby detect which
face image has entered the state of being selected. Then, the game
apparatus 10 may display the face image having entered the state of
being selected, e.g., the face image G1, by surrounding the face
image G1 by the heavy line L1.
[0182] As described above, the user can browse a list of currently
already acquired face images, using the screen shown in FIG. 7
displayed on the upper LCD 22. Further, the user can cause a
desired face image, from among the list of currently already
acquired images, to enter the state of being selected. Then, for
example, the user can fix the state of being selected by selecting
a predetermined determination button, such as a determination
button displayed on the lower LCD 12, using the touch panel 13, the
operation buttons 14, or the like. Furthermore, for example, the
user may press the button 14C (also referred to as a "B button"),
whereby the screen of the list of the face images on the upper LCD
22 is closed, and a screen that waits for a menu selection (not
shown) is displayed.
[0183] FIG. 8 is another example of the face images displayed as a
list. In the example of FIG. 8, the face image G2 is in the state
of being selected, and is surrounded by a heavy line L2. In the
example of FIG. 8, when the face image G2 is in the state of being
selected, face images related to the face image G2 are reacting.
For example, a heart mark is displayed near a face image G0, and
the face image G0 is giving a look with one eye closed to the face
image G2 in the state of being selected. Further, for example, the
face image G3 and a face image G7 are also turning their faces
toward and giving looks to the face image G2 in the state of being
selected.
[0184] The reactions of the face images related to the face image
in the state of being selected, however, are not limited to actions
such as: turning its face; giving a look with one eye closed while
a heart mark is displayed near the face image; and turning its face
and giving a look. For example, a related face image may show
reactions such as smiling and producing a voice. Conversely, a face
image unrelated to the face image in the state of being selected
may change its expression from a smiling expression to a straight
expression. Alternatively, a face image unrelated to the face image
in the state of being selected may turn in the direction opposite
to the direction of the face image in the state of being
selected.
[0185] Here, a related face image may be defined at the stage of,
for example, the acquisition of a face image. For example, groups
for classifying face images may be set in advance, and when a face
image is acquired, the user may input the group to which the face
image to be acquired belongs. Alternatively, for example, a group
of face images may be defined in accordance with the progression of
the game, and the face images may be classified. For example, when
the face image G1 has been newly acquired using the face image G0
during the progression of the game, it may be determined that the
face image G0 and the face image G1 are face images that are
related to each other and belong to the same group.
[0186] As shown in FIG. 8, when a face image, e.g., the face image
G2, has been caused to enter the state of being selected through
the operation buttons 14 or the like, the face images G0, G3, G7,
and the like react. This makes it possible to represent a sense of
affinity and the like between the acquired face images. That is, it
is possible to not only simply classify collected face images into
groups, but also represent the affinities between a plurality of
face images by the expressions and the reactions of the face
images. This makes it possible to represent in a virtual world of
the game apparatus 10 the associations, the intimacies, and the
like between people in the real world.
[0187] FIGS. 9 and 10 are examples of screens for attaching a face
image to an enemy object EO, which is one of game characters. FIG.
9 is an example of an enemy object EO's head selection screen. For
example, when the user has performed an operation on the touch
panel 13 to select a menu "select boss" from menus displayed on the
lower LCD 12 of the game apparatus 10, a list of the head shapes of
enemy objects EO prepared in the game apparatus 10 as shown in FIG.
9 is displayed. Such an operation, however, is not limited to an
operation on the touch panel 13, and the list of the head shapes as
shown in FIG. 9 may be displayed by, for example, an operation on
the operation buttons 14 or the like.
[0188] In FIG. 9, three types of head shapes, namely head shapes
H1, H2, and H3, are displayed so as to be selected. For example,
the head shape H1 includes: a facial surface portion H12 formed of
a three-dimensional model; and a peripheral portion H13 surrounding
the facial surface portion H12. An enemy object EO to appear in the
game is formed as shown in FIGS. 7 and 8 by texture-mapping a face
image onto the facial surface portion H12, which is a
three-dimensional model.
[0189] The peripheral portion H13 may have a shape suggesting the
feature of the enemy object EO to appear in the game. For example,
if the peripheral portion H13 has a shape representing a helmet, it
is possible to represent an aggressive mental picture of the enemy
object EO. In FIG. 9, three types of head shapes, namely the head
shapes H1, H2, and H3, are illustrated on the list of the head
shapes, which has two rows and four columns, and undefined marks H0
are displayed in addition to these head shapes. Thus, the head
shapes H1 through H3 are merely illustrative, and the types of head
shapes are not limited to three types. For example, a new head
shape may be added from a medium of an upgraded game program, a
website where parts in the game are provided on the Internet, or
the like.
[0190] In FIG. 9, a label LB with "boss" indicates that the head
shape H1 has been caused by the user to enter the state of being
selected. The label LB can be moved by, for example, the cross
button 14A. The user performs an operation on the cross button 14A
or the like to move the label LB, and thereby can cause, for
example, the head shape H2 or H3 to enter the state of being
selected. Then, after either head shape has been caused to enter
the state of being selected, the user may press the operation
button 14B (A button) or the like to determine the state of being
selected. The determination of the state of being selected places
the label LB, and fixes the selection of the head shape in the
state of being selected, e.g., the head shape H1 in the example of
FIG. 9. It should be noted that the user may press the operation
button 14C (B button) when the screen shown in FIG. 9 is displayed
on the upper LCD 22, whereby the enemy object EO's head selection
screen is closed, and display returns to the previous operation
screen.
[0191] FIG. 10 is a diagram illustrating a face image selection
screen. For example, when the user has pressed the operation button
14B (A button) or the like to determine the head shape in the state
of being selected in the state where the enemy object EO's head
selection screen shown in FIG. 9 is displayed on the upper LCD 22,
the screen shown in FIG. 10 is displayed. The screen shown in FIG.
10 is similar to those of FIGS. 7 and 8, but is different from
those of FIGS. 7 and 8 in that the peripheral portion H13 of the
head shape selected in FIG. 9 is added to the face image in the
state of being selected.
[0192] On the screen shown in FIG. 10, the user can operate the
cross button 14A or the like to switch the face image in the state
of being selected. That is, the user can switch the face image in
the state of being selected, such as from the face image G0 to G1
and from the face image G1 to G2. Then, display is performed such
that the face image caused to enter the state of being selected is
texture-mapped onto the facial surface portion of the head shape.
For example, in the example of FIG. 10, the face image G2 is
texture-mapped onto the facial surface portion H12 of the head
shape H1 selected on the screen shown in FIG. 9, and is displayed
in combination with the peripheral portion H13.
[0193] Such a combination of the peripheral portion H13 and the
face image G2 is displayed, whereby an enemy object EO is
temporarily created. The user imagines a mental picture of the
enemy to confront in the game, by the temporarily displayed enemy
object EO. The user can operate the cross button 14A or the like to
switch the face image in the state of being selected, and thereby
can switch the face image of the enemy object EO. That is, the user
can switch the faces of the enemy objects EO, one after another,
and thereby can create an enemy object EO that fits a mental
picture of the enemy to fight with in the game.
[0194] That is, in the game apparatus 10, for example, the face
images collected by succeeding in the first game and accumulated in
the saved data storage area Do as described above are used in the
subsequent second game. That is, the game apparatus 10 performs
game processing using enemy objects EO created using the collected
face images. For example, in accordance with an operation of the
user, the game apparatus 10 performs a process termed a "cast
determination process" before the execution of the game, and
generates enemy objects EO that fit mental pictures formed by the
user, or the game apparatus 10 automatically generates enemy
objects EO before the execution of the game. "Automatically" means
that for example, the game apparatus 10 can generate enemy objects
EO by selecting a required number of face images, i.e., generate
enemy objects EO to appear in the game, randomly from among the
collected face images. Further, for example, in accordance with the
history of the game processing performed by the user in the past,
the game apparatus 10 may create enemy objects EO by selecting face
images expected to be desired next by the user, based on the
properties, the taste, and the like of the user. In accordance with
the execution history of the game that has been executed by the
user up to the current time, the game apparatus 10 may select a
face image to be used next, based on face images, together with the
attributes of the subjects of the face images, such as age, gender,
friendship (family, friends, and relationships in work, school, and
community), or, if a subject is a living thing such as a pet, the
ownership relationship of the subject. Further, for example, the
game apparatus 10 may select a face image to be used next, based on
the performances of the user in the game executed in the past.
[0195] The game apparatus 10 performs game processing (the second
game) using the enemy objects EO created by the specification made
by such operations of the user, or created by the processing of the
game apparatus 10. It should be noted that a character object that
appears in the game, which is described using the term "enemy
object EO" as an example, is not limited to an object having an
adversarial relationship with the user, and may be a friend
character object. Further, the present invention is not limited to
a game where there are relationships such as enemies and friends,
and may be a game where a player object representing the user
themselves appears. Alternatively, the present invention may be,
for example, a game where an object termed an "agent" appears, the
object assisting the user in executing the game.
[0196] The game apparatus 10 executes a game where various
character objects, such as the enemy objects EO described above,
appear. To the character objects that appear in the game, the face
images collected by the user succeeding in the first game are
attached by texture mapping or the like. Accordingly, in the game
executed by the game apparatus 10, the character objects including
the face images collected by the user themselves appear. Thus,
using images of portions representing the features of people and
living things, such as face images, the user can execute a game
where the real-world relationships with the people represented by
the face images or with the living things of the face images are
reflected on the various character objects. For example, it is
possible to execute a game including emotions, such as affection,
friendliness, favorable impression, and hatred.
[0197] It should be noted that also in the face image selection
screen shown in FIG. 10, similarly to that of FIG. 8, when a face
image has entered the state of being selected, other face images
related to the face image in the state of being selected may show
reactions. For example, in the example of FIG. 10, the face image
G2 is in the state of being selected in combination with the
peripheral portion 13 of the head shape H1, and a related face
image, e.g., the face image G4, is smiling with its face turned
toward the face image G2 and giving a look to the face image G2.
Further, the face image G5 is giving an envious look to the face
image G2 with its face turned upward. Furthermore, face images G8
and G9 are also giving looks to the face image G2. In FIG. 10, in
contrast, the face images other than the face images G4, G5, G8,
and G9 are not showing any reactions to the fact that the face
image G2 has entered the state of being selected. Such differences
in reaction make it possible to represent the relationships of
affinity between a plurality of face images. As described above,
when a face image has a specific relationship with the user, for
example, when the face image and the user belong to a group of a
plurality of friends, the game apparatus 10 can perform drawing by
introducing intimacy relationships between people in the real
world, into a virtual world represented by the game apparatus
10.
[0198] <Example of Various Data>
[0199] FIG. 11 is a diagram showing an example of various data
stored in the main memory 32 by executing the image processing
program.
[0200] It should be noted that programs for performing the
processing of the game apparatus 10 are included in a memory built
into the game apparatus 10 (e.g., the data storage internal memory
35), or included in the external memory 45 or the data storage
external memory 46, and the programs are: loaded from the built-in
memory, or loaded from the external memory 45 through the external
memory I/F 33 or from the data storage external memory 46 through
the data storage external memory I/F 34, into the main memory 32
when the game apparatus 10 is turned on; and executed by the CPU
311.
[0201] Referring to FIG. 11, the main memory 32 stores the programs
loaded from the built-in memory, the external memory 45, or the
data storage external memory 46, and temporary data generated in
the image processing. Referring to FIG. 11, in a data storage area
of the main memory 32, the following are stored: operation data Da;
real camera image data Db; real world image data Dc; boundary
surface data Dd; back wall image data De; enemy object data Df;
bullet object data Dg; score data Dh; motion data Di; virtual
camera data Dj; rendered image data Dk; display image data Dl;
aiming cursor image data Dm; management data Dn; saved data storage
area Do; and the like. Further, in a program storage area of the
main memory 32, a group of various programs Pa are stored that
configure the image processing program.
[0202] <<Operation Data Da>>
[0203] The operation data Da indicates operation information of an
operation of the user on the game apparatus 10. The operation data
Da includes controller data Da1 and angular velocity data Da2. The
controller data Da indicates that the user has operated a
controller, such as the operation buttons 14 or the analog stick
15, of the game apparatus 10. The angular velocity data Da2
indicates the angular velocities detected by the angular velocity
sensor 40. For example, the angular velocity data Da2 includes
x-axis angular velocity data indicating an angular velocity about
the x-axis, y-axis angular velocity data indicating an angular
velocity about the y-axis, and z-axis angular velocity data
indicating an angular velocity about the z-axis, the angular
velocities detected by the angular velocity sensor 40. For example,
the operation data from the operation buttons 14 or the analog
stick 15 and the angular velocity data from the angular velocity
sensor 40 are acquired per unit of time in which the game apparatus
10 performs processing (e.g., 1/60 seconds), and are stored in the
controller data Da1 and the angular velocity data. Da2,
respectively, in accordance with the acquisition, to thereby be
updated.
[0204] It should be noted that game processing (e.g., the processes
performed in FIG. 20A and thereafter) will be described later using
an example where the controller data Da1 and the angular velocity
data Da2 are each updated every one-frame period, which corresponds
to the processing cycle. Alternatively, the controller data Da1 and
the angular velocity data Da2 may be updated in another processing
cycle. For example, the controller data Da1 may be updated in each
cycle of detecting the operation of the user on a controller, such
as the operation buttons 14 of the analog stick 15, and the updated
controller data Da1 may be used in each processing cycle. In this
case, the cycles of updating the controller data Da1 and the
angular velocity data Da2 differ from the processing cycle.
[0205] <<Real Camera Image Data Db>>
[0206] The real camera image data Db indicates a real camera image
captured by either one of the outer capturing section 23 and the
inner capturing section 24. In the following descriptions of
processing, in the step of acquiring a real camera image, the real
camera image data Db is updated using a real camera image captured
by either one of the outer capturing section 23 and the inner
capturing section 24. It should be noted that the cycle of updating
the real camera image data Db using the real camera image captured
by the outer capturing section 23 or the inner capturing section 24
may be the same as the unit of time of the processing of the game
apparatus 10 (e.g., 1/60 seconds), or may be shorter than this unit
of time. When the cycle of updating the real camera image data Db
is shorter than the cycle of the processing of the game apparatus
10, the real camera image data Db may be updated as necessary,
independently of the processing described later. In this case, in
the step described later of acquiring a real camera image, the
process may be performed invariably using the most recent real
camera image indicated by the real camera image data Db.
Hereinafter, in the present embodiment, the real camera image data
Db is data indicating a real camera image captured by the outer
capturing section 23 (e.g., the left outer capturing section
23a).
[0207] <<Real World Image Data Dc>>
[0208] In the game processing described later, e.g., the process of
the execution of the game in step 18 of FIG. 14, more specifically,
in the processes shown in FIG. 20A and thereafter, a boundary
surface 3 is introduced that is obtained by texture-mapping a real
camera image captured by a real camera of the game apparatus 10
(the outer capturing section 23 or the inner capturing section 24).
The real world image data Dc is data for generating a real world
image that seems to be present on the boundary surface 3, using the
real camera image captured by the real camera of the game apparatus
10 (the outer capturing section 23 or the inner capturing section
24). In a first drawing method described later, for example, the
real world image data Dc includes texture data of the real camera
image for attaching the real world image to the boundary surface (a
screen object in the display range of a virtual camera). Further,
in a second drawing method described later, for example, the real
world image data Dc includes: data of a planar polygon for
generating the real world image; texture data of the real camera
image to be mapped onto the planar polygon; and data indicating the
position of the planar polygon in a virtual space (the position
from a real world drawing camera described later).
[0209] <<Boundary Surface Data Dd>>
[0210] The boundary surface data Dd is data for, in combination
with the real world image data Dc described above, generating the
real world image that seems to be present on the boundary surface
3. In the first drawing method, for example, the boundary surface
data Dd is data concerning the screen object, and includes: opening
determination data (corresponding to data of an .alpha.-texture
described later) indicating the state (e.g., the presence or
absence of an opening) of each point included in the boundary
surface 3; data indicating the placement position of the boundary
surface 3 in the virtual space (the coordinates of the boundary
surface 3 in the virtual space); and the like. Further, in the
second drawing method, for example, the boundary surface data Dd is
data for representing an opening in a planar polygon of the real
world image, and includes: opening determination data
(corresponding to data of an .alpha.-texture described later)
indicating the state (e.g., the presence or absence of an opening)
of each point included in the boundary surface 3; data indicating
the placement position of the boundary surface 3 in the virtual
space (the coordinates of the boundary surface 3 in the virtual
space); and the like. The data indicating the placement position of
the boundary surface 3 in the virtual space is, for example,
conditional equations for a spherical surface (relational
expressions for defining a spherical surface in the virtual space),
and indicates the existence range of the boundary surface 3 in the
virtual space.
[0211] The opening determination data indicating the state of being
open is, for example, two-dimensional (e.g., a rectangular shape
having 2048 pixels.times.384 pixels) texture data in which the
alpha value (non-transparency) of each point can be set. The alpha
value is a value of from "0" to "1", with "0" being minimum and "1"
being maximum. The alpha value indicates transparent by "0", and
indicates non-transparent by "1". The opening determination data
can indicate that a position where "0" is stored in the opening
determination data is in the state of being open, and a position
where "1" is stored is not in the state of being open. The alpha
value can be set in, for example, an image of a game world
generated in the game apparatus 10, or a pixel block unit including
a pixel or a plurality of pixels in the upper LCD 22. In the
present embodiment, "predetermined values over 0 but less than 1
(0.2 in the present embodiment)" are stored in an unopen area. This
data is not used when applied to the real world image. When applied
to the real world image, alpha values of "0.2" stored in the
opening determination data are handled as "1". It should be noted
that an alpha values of "0.2" is used to draw a shadow ES of each
of the enemy objects EO described above. The setting of the alpha
value and the range of the alpha value, however, does not limit the
image processing program according to the present invention.
[0212] In the image processing program according to the present
embodiment, in the first drawing method, it is possible to generate
the real world image having an opening by multiplying: the opening
determination data corresponding to an area of the range of the
visual space of the virtual camera; by color information (pixel
values) of a texture of the real world image to be attached to the
boundary surface 3. Further, in the second drawing method, it is
possible to generate the real world image having an opening by
multiplying: the opening determination data corresponding to an
area of the range of the visual space of a virtual world drawing
camera; by color information (pixel values) of the real world image
(specifically, rendered image data of the real camera image
rendered with a parallel projection described later using the real
world image data Dc). This is because when alpha values of "0"
stored at the position of the opening are multiplied by the color
information of the real world image at the position, the values of
the color information of the real world image are "0" (the state of
being completely transparent).
[0213] It should be noted that in the first drawing method, as
described later, an image to be displayed on the upper LCD 22 is
generated by rendering a virtual space image in which virtual
objects are placed so as to include an object of the real world
image to which the opening determination data is applied.
[0214] In addition, in the second drawing method, specifically, as
described later, the virtual space image is rendered, taking into
account the opening determination data. That is, the priority of
each virtual object relative to the boundary surface (the priority
relative to the real world image) is determined based on the
opening determination data, and the virtual space image is
generated by rendering each virtual object. Then, an image to be
displayed on the upper LCD 22 is generated by combining the real
world image with the virtual space image generated as described
above.
[0215] In addition, in the image processing program according to
the present embodiment, the shape of the boundary surface 3 is a
spherical surface (see FIGS. 20A and 20B). Then, in the present
embodiment, the shape of the opening determination data may be
defined as rectangular. The opening determination data of this
rectangular shape is mapped onto a central portion of the spherical
surface as shown in FIGS. 20A and 20B, whereby it is possible to
cause the points of the opening determination data to correspond to
the points of the boundary surface.
[0216] It should be noted that in the present embodiment, the
opening determination data is only data corresponding to the
central portion of the spherical surface shown in FIG. 20A.
Accordingly, the opening determination data may not be present
depending on the orientation of the virtual camera (the virtual
world drawing camera in the second drawing method). When the
opening determination data is not present as described above, the
real world image is drawn as it is. That is, the real world image
is drawn on the condition that .alpha.-values of "1" are set.
[0217] The image processing for an opening created in the boundary
surface 3 will be described later.
[0218] <<Back Wall Image Data De>>
[0219] The back wall image data De is data concerning a back wall
BW, which is present in a second space 2. For example, the back
wall image data De includes: image data for generating an image of
the back wall BW; data indicating the position of a polygon model
defining the back wall BW in the virtual space; and the like.
[0220] The polygon model defining the back wall BW is typically a
model that has a radius greater than that of the sphere shown in
FIG. 20A, about a vertical axis extending through the position of
the virtual camera (the virtual world drawing camera in the second
drawing method), and has the same shape as that of the central
portion of the sphere shown in FIG. 20A. That is, the model
defining the back wall BW includes the boundary surface 3. Further,
the polygon model may be a planar polygon placed behind the
position of an opening to be formed in the boundary surface 3.
Furthermore, each time an opening is formed in the boundary surface
3, a planar polygon defining the projection surface of the opening
may be placed in the second space 2.
[0221] Image data (texture) to be attached to the polygon model of
the back wall BW may be given data. This image data represents
another space (second space 2) existing behind the real world
image, and therefore, the image data is preferably an image
representing unreality, such as an image representing outer space,
the sky, or an area in water, because it is possible to cause the
player a strange feeling as if an unreal space exists behind real
space. For example, when the user is playing the game according to
the present embodiment in a room, it is possible to give the user a
feeling as if an unreal space exists outside the room.
Alternatively, a texture of the back wall may represent landscapes
that are not normally seen, such as a desert and a wilderness. As
described above, the selection of a texture of the back wall BW
allows the player to notice a desired mental picture in another
world hidden behind a real image represented as a background of the
game world.
[0222] In addition, for example, if the image data is an image that
can use repeated representations, such as an image of outer space,
it is possible to reduce the data size of the image data (texture).
Further, if the image data is such an image, it is possible to draw
an image of the back wall BW without specifying the position where
the back wall BW is to be drawn in the virtual space. This is
because if an image can use repeated representations, the image is
drawn without depending on the position (the repeated pattern can
be represented on the entire polygon model).
[0223] It should be noted that in the present embodiment, the
priority of drawing described later is defined by alpha values, and
therefore, it is assumed that an alpha value is defined for the
image data. In the present embodiment, it is assumed that an alpha
value of "1" is defined for the image data.
[0224] <<Enemy Object Data Df>
[0225] The enemy object data Df is data concerning an enemy object
EO, and includes substance data Df1, silhouette data Df2, and
opening shape data Df3.
[0226] The substance data Df1 is data for drawing the substance of
the enemy object EO, and includes, for example, a polygon model
defining a three-dimensional shape of the substance of the enemy
object EO, and texture data to be mapped onto the polygon model.
The texture data may be, for example, a photograph of the face of
the user or the like captured by each capturing section of the game
apparatus 10. It should be noted that in the present embodiment,
the priority of drawing described later is defined by alpha values,
and therefore, it is assumed that an alpha value is defined for the
texture data. In the present embodiment, it is assumed that an
alpha value of "1" is defined for the texture data.
[0227] The silhouette data Df2 is data for semi-transparently
drawing in the real world image the shadow of the enemy object EO
present in the second space 2, and includes a polygon model and
texture data to be attached to the polygon model. For example, this
silhouette model includes eight planar polygons, and is placed at
the same position as that of the enemy object EO present in the
second space 2. The silhouette model to which a texture is attached
is drawn, for example, semi-transparently, in the real world image
as viewed from the virtual world drawing camera, whereby it is
possible to represent the shadow of the enemy object EO present in
the second space 2. Further, the texture data of the silhouette
data Df2 may be, for example, images of the enemy object EO as
viewed from all directions as shown in FIGS. 27A and 27B (e.g.,
eight planar polygons). Furthermore, these images may each be an
image obtained by simplifying the silhouette model of the enemy
object EO. It should be noted that in the present embodiment, the
priority of drawing described later is defined by alpha values, and
therefore, it is assumed that an alpha value is defined for the
texture data to be attached to the silhouette model. In the present
embodiment, it is assumed that an alpha value of "1" is defined for
the texture data in the shadow image portion, and an alpha value of
"0" is defined in the portion where there is no shadow image (the
peripheral portion).
[0228] The opening shape data Df3 is data concerning the shape of
an opening generated in the boundary surface 3 when the enemy
object EO moves between a first space 1 and the second space 2. In
the present embodiment, the opening shape data Df3 is data for
setting alpha values of "0" at the position in the opening
determination data corresponding to the position in the boundary
surface 3 where the opening is generated. For example, the opening
shape data Df3 is texture data that corresponds to the shape of the
opening to be generated and has alpha values of "0". It should be
noted that in the present embodiment alpha values of "0" are set in
the opening determination data for the shape indicated by the
opening shape data Df3, the shape formed around the portion
corresponding to the position through which the enemy object EO has
passed in the boundary surface 3. The image processing performed
when the enemy object EO generates an opening in the boundary
surface 3 will be described later.
[0229] <<Bullet Object Data Dg>>
[0230] The bullet object data Dg is data concerning a bullet object
BO, which is fired in accordance with an attack operation of the
player. For example, the bullet object data Dg includes: a polygon
model and bullet image (texture) data for drawing the bullet object
BO; data indicating the placement direction and the placement
position of the bullet object BO; and data indicating the moving
velocity and the moving direction (e.g., a moving velocity vector)
of the bullet object BO. It should be noted that in the present
embodiment, the priority of drawing described later is defined by
alpha values, and therefore, it is assumed that an alpha value is
defined for the bullet image data. In the present embodiment, an
alpha value of "1" is defined for the bullet image data.
[0231] <<Score Data Dh>>
[0232] The score data Dh indicates the score of a game where the
enemy object EO appears. For example, as described above, points
are added to the score of the game when the user has vanquished the
enemy object BO by an attack operation, and points are deducted
from the score of the game when the enemy object EO has reached the
position of the user (i.e., the placement position of the virtual
camera in the virtual space).
[0233] <<Motion Data Di>>
[0234] The motion data Di indicates the motion of the game
apparatus 10 in real space. As an example, the motion of the game
apparatus 10 is calculated by the angular velocities detected by
the angular velocity sensor 40.
[0235] <<Virtual Camera Data Dj>>
[0236] The virtual camera data Dj is data concerning a virtual
camera set in the virtual space. In the first drawing method, for
example, the virtual camera data Dj includes data indicating the
placement direction and the placement position of a virtual camera
in the virtual space. Further, in the second drawing method, for
example, the virtual camera data Dj includes: data indicating the
placement direction and the placement position of a real world
drawing camera in the virtual space; and data indicating the
placement direction and the placement position of a virtual world
drawing camera in the virtual space. Then, for example, the data
indicating the placement direction and the placement position of
the virtual camera in the virtual space in the first drawing
method, and the data indicating the placement direction and the
placement position of the virtual world drawing camera in the
virtual space in the second drawing method change in accordance
with the motion of the game apparatus 10 (angular velocities)
indicated by the motion data Di. Further, the virtual camera data
Dj includes angle-of-view (drawing range) data of the virtual
camera. With this, in accordance with changes in the positions and
the orientations of the virtual camera in the first drawing method
and the virtual world drawing camera in the second drawing method,
a drawing range (drawing position) in the boundary surface 3
changes.
[0237] <<Rendered Image Data Dk>>
[0238] The rendered image data Dk is data concerning an image
rendered by processing described later.
[0239] In the first drawing method, the real world image is
rendered as an object in the virtual space, and therefore, the
rendered image data Dk includes rendered image data of the virtual
space. The rendered image data of the virtual space is data
indicating a virtual world image obtained by rendering with a
perspective projection from the virtual camera the virtual space
where the enemy object EO, the bullet object BO, the boundary
surface 3 (screen object) to which the real world image is applied
as a texture, and the back wall BW are placed.
[0240] On the other hand, in the second drawing method, the real
world image and the virtual world image are rendered by virtual
cameras different from each other, and therefore, the rendered
image data Dk includes rendered image data of the real camera image
and rendered image data of the virtual space. The rendered image
data of the real camera image indicates the real world image
obtained by rendering with a parallel projection from the real
world image drawing camera a planar polygon on which a texture of
the real camera image is mapped. The rendered image data of the
virtual space indicates the virtual world image obtained by
rendering with a perspective projection from the virtual world
drawing camera the virtual space where the enemy object EO, the
bullet object BO, the boundary surface 3, and the back wall BW are
placed.
[0241] <<Display Image Data Dl>>
[0242] The display image data Dl indicates a display image to be
displayed on the upper LCD 22. In the first drawing method, for
example, a display image to be displayed on the upper LCD 22 is
generated by a process of rendering the virtual space. Further, in
the second drawing method, for example, a display image to be
displayed on the upper LCD 22 is generated by combining the
rendered image data of the camera image with the rendered image
data of the virtual space by a method described later.
[0243] <<Aiming Cursor Image Data Dm>>
[0244] The aiming cursor image data Dm is image data of an aiming
cursor AL that is displayed on the upper LCD 22. The image data may
be given data.
[0245] It should be noted that in the present embodiment, the data
concerning each object (the boundary surface data Dd, the back wall
image data De, the substance data Df1, the silhouette data Df2, and
the bullet image data) includes information about the priority, the
information defining the priority of drawing. In the present
embodiment, the information about the priority uses alpha values.
The relationship between the alpha values and the image processing
will be described later.
[0246] In addition, in the present embodiment, the data concerning
each object used for drawing includes data indicating whether or
not a depth determination is to be made between the object and
another. As described above, the data is set such that a depth
determination is valid between each pair of: the enemy object EO;
the bullet object BO; a semi-transparent enemy object; an effect
object; and the screen object (boundary surface 3). Further, the
data is set such that a depth determination is valid "between the
shadow planar polygon (silhouette data Df2) and the enemy object EO
(substance data Df1)", "between the shadow planar polygon
(silhouette data Df2) and the bullet object BO", "between the
shadow planar polygon (silhouette data Df2) and the
semi-transparent enemy object", and "between the shadow planar
polygon (silhouette data Df2) and the effect object". Furthermore,
the data is set such that a depth determination is invalid between
the shadow planar polygon (silhouette data Df2) and the screen
object (boundary surface data Dd).
[0247] <<Management Data Dn>>
[0248] The management data Dn is data for managing: data to be
processed by the game apparatus 10, such as collected face images;
data accumulated by the game apparatus 10; and the like. The
management data Dn includes face image management information Dn1,
a face image attribute aggregate table Dn2, and the like. The face
image management information Dn1 stores: the destination for
storing the data of each face image (e.g., the address in the main
memory 32 or the like); the source of acquiring the face image
(e.g., the inner capturing section 24 or the outer capturing
section 23); the attributes of the face image (e.g., the gender,
the age, and the like of the subject of the face image);
information of other face images related to the face image; and the
like. Further, the face image attribute aggregate table Dn2 stores
by attribute the numbers of collections of the face images
currently already collected by the user. For example, when the
subjects of the collected face images are classified by gender,
age, and the like, the collection achievement value of each
category is stored. Examples of the data structures of the face
image management information Dn1 and the face image attribute
aggregate table Dn2 will be described later.
[0249] <<Saved Data Storage Area Do>>
[0250] The saved data storage area Do is an area where, when the
information processing section 31 executes the image processing
program such as a game program, data to be processed by the
information processing section 31, the resulting data of the
process of the information processing section 31, and the like are
saved. As an example, in the present embodiment, data of a face
image acquired by the game apparatus 10 through the inner capturing
section 24, the outer capturing section 23, the wireless
communication module 36, the local communication module 37, and the
like is saved. In the present embodiment, for example, the
information processing section 31 executes the first game in the
state where a face image acquired by the game apparatus 10 is
temporarily stored in the main memory 32. Then, when it is
determined that the user has succeeded in the first game in
accordance with an operation of the user, the information
processing section 31 saves in the saved data storage area Do the
face image temporarily stored in the main memory 32. The face image
saved in the saved data storage area Do is available in the
subsequent game processing or the like.
[0251] The structure of the saved data storage area Do is not
particularly limited. For example, the saved data storage area Do
may be placed in the same physical address space as that of a
regular memory, so as to be accessible to the information
processing section 31. Further, for example, the saved data storage
area Do may allow in advance the information processing section 31
to secure (or allocate) a predetermined block unit or a
predetermined page unit at a necessary time. Furthermore, for
example, the saved data storage area Do may have a structure where
connections are made by management information, such as points
connecting blocks, as in the file system of a computer.
[0252] In addition, the saved data storage area Do may, for
example, secure an individual area for each program executed by the
game apparatus 10. Accordingly, when a game program has been loaded
into the main memory 32, the information processing section 31 may
access the saved data storage area Do (input and output data) based
on management information or the like of the game program.
[0253] In addition, the saved data storage area Do of a program may
be accessible to the information processing section 31 that is
executing another program. With this, data processed in the program
may be delivered to said another program. For example, the
information processing section 31 that is executing the second game
may create a character object by reading data of a face image saved
in the saved data storage area Do as a result of the execution of
the first game described later. It should be noted that the saved
data storage area Do is an example of a second storage area.
[0254] <Structures of Various Data>
[0255] With reference to FIGS. 12 and 13, descriptions are given of
examples of data structures for managing face images in the game
apparatus 10.
[0256] FIG. 12 is an example of the data structure of the face
image management information Dn1 for managing face images saved in
the game apparatus 10. The game apparatus 10 stores data of saved
face images in the face image management information Dn1, and
thereby can display a list of the face images on the screen of the
upper LCD 22 in the form of, for example, FIGS. 7 and 8. The face
image management information Dn1 is, for example, created as
information in which a record is prepared for each face image. The
face image management information Dn1 is, for example, saved in the
data storage internal memory 35 or the data storage external memory
46. In FIG. 12, the elements of a record are illustrated by a
record 1. Further, in FIG. 12, details of a record 2 and thereafter
are not shown. Furthermore, although not shown in the figures, the
information processing section 31 may, for example, save the total
number of records of the face image management information Dn1,
i.e., the total number of acquired face images, in the data storage
internal memory 35, the data storage external memory 46, or the
like.
[0257] In the example of FIG. 12, the face image management
information Dn1 includes, for example, face image identification
information, the address of face image data, the source of
acquiring the face image, the estimation of gender, the estimation
of age, and pieces of related face image information 1 through N.
FIG. 12, however, is an example of the face image management
information Dn1, and this does not mean that face image management
information is limited to the elements shown in FIG. 12.
[0258] The face image identification information is information
uniquely identifying the saved face image. The face image
identification information may be, for example, a serial
number.
[0259] The address of face image data is, for example, the address
where data of the face image is stored in the data storage internal
memory 35 or the data storage external memory 46. However, for
example, when the data of the face image is stored in a storage
medium in which a file system is constructed by an OS (operating
system), a path name, a file name, and the like in the file system
may be set as the address of face image data.
[0260] The source of acquiring the face image is, for example,
information identifying the capturing device that has acquired the
face image. As the source of acquiring the face image, for example,
information identifying the inner capturing section 24, the left
outer capturing section 23a, or the right outer capturing section
23b is set. However, when both the left outer capturing section 23a
and the right outer capturing section 23b have been used to acquire
the face image, information indicating both capturing sections is
set. Further, for example, when the face image has been acquired by
a capturing device other than the inner capturing section 24, the
left outer capturing section 23a, and the right outer capturing
section 23b, e.g., by a capturing device provided outside the game
apparatus 10, information indicating such a state (e.g., "other")
is set. "When the face image has been acquired by a capturing
device provided outside the game apparatus 10" is, for example, the
case where an image captured by another game apparatus 10 similar
to the game apparatus 10 has been acquired through the external
memory interface 33, the wireless communication module 36, the
local communication module 37, or the like. Furthermore, examples
of such a case also include the cases: where an image obtained by a
camera not included in the game apparatus 10 has been acquired;
where an image obtained by a scanner has been acquired; and where
an image such as a video image obtained from a video device has
been acquired, each image obtained through the external memory
interface 33, the wireless communication module 36, or the
like.
[0261] The estimation of gender is information indicating whether
the face image is male or female. The estimation of gender may be,
for example, made by a process shown in another embodiment
described later. The estimation of age is information indicating
the age of a person represented by the face image. The estimation
of age may be, for example, made by a process shown in another
embodiment described later.
[0262] Each of the pieces of related image identification
information 1 through N is information indicating another face
image related to the face image. For example, as the pieces of
related image identification information 1 through N, pieces of
face image identification information of up to N related other face
images may be set. The related other face images may be, for
example, specified by an operation of the user through a GUI. For
example, when a face image has been newly acquired, the information
processing section 31 may detect, in the state where the user has
operated the operation buttons 14 or the like to cause one or more
face images related to the acquired face image to enter the state
of being selected, an operation on the GUI of giving an instruction
to set related images. Alternatively, the acquired face image may
be classified by categories prepared by the game apparatus 10, such
as themselves, friends, colleagues, and strangers. Then, face
images belonging to the same category may be linked together using
the pieces of related image identification information 1 through N.
However, when face images are classified by the categories prepared
by the game apparatus 10, an element "classification of face
images" may be simply prepared, instead of the preparation of the
entry of the pieces of related face image identification
information 1 through N, so that themselves, friends, colleagues,
strangers, and the like may be set. Further, in FIG. 12, a fixed
number is used for the pieces of related image identification
information 1 through N. Alternatively, the number N may be a
variable number. In this case, the number N that is already set may
be held in the face image management information Dn1. Furthermore,
for example, in the face image management information Dn1, the
records of face images related to each other may be connected
together by chains of pointers.
[0263] FIG. 13 shows the data structure of the face image attribute
aggregate table Dn2. The face image attribute aggregate table Dn2
is a table where already acquired face images are classified by
attribute, and the numbers of the classified images are aggregated.
Hereinafter, the face image attribute aggregate table Dn2 will also
be referred to simply as an "aggregate table". The information
processing section 31 saves the aggregate table shown in FIG. 13
in, for example, the data storage internal memory 35 or the data
storage external memory 46. In the example of FIG. 13, the
aggregate table stores the number of acquired face images in each
row defined by performing classification with the combination of
gender (male or female) and an age bracket (under 10, 10's, 20's,
30's, 40's, 50's, 60's, or 70 or over). That is, each row of the
table shown in FIG. 13 includes elements such as gender, age, and
the number of acquired face images. In the numbers of acquired face
images, already acquired face images are classified, and the
numbers of the classified acquired face images are aggregated. The
categories and the attributes of face images, however, are not
limited to genders or age brackets that are shown in FIG. 13.
[0264] <Example of Process Flow>
[0265] With reference to FIGS. 14 through 19, a description is
given of an example of the operation of the image processing
program executed by the information processing section 31 of the
game apparatus 10. First, when the power (the power button 14F) to
the game apparatus 10 has been turned on, the CPU 311 executes a
boot program (not shown). This causes the programs stored in the
built-in memory, the external memory 45, or the data storage
external memory 46, to be expanded in the main memory 32 into the
form of being executable by the CPU 311. Here, "the form of being
executable by the CPU 311" means, for example, the form where
machine instructions for the CPU 311 are written in a predetermined
order and placed at appropriate addresses in the main memory 32, so
as to be readable by a control section that processes the machine
instructions for the CPU 311. The expansion into the form of being
executable is also referred to simply as "loading". It should be
noted that in FIGS. 14 through 19, processes not directly related
to the first embodiment are not described.
[0266] FIG. 14 is a flow chart showing an example of the operation
of the information processing section 31. After a series of
processes performed after the turning on of the power, when the
information processing section 31 has detected an operation of the
user, for example, an operation through the touch panel 13 or the
operation buttons 14 on a graphical user interface (hereinafter a
"GUI") displayed on the lower LCD 12, such as a graphics object,
e.g., a menu or an icon, the information processing section 31
performs the process of FIG. 14. Hereinafter, an operation of the
user on the GUI through the touch panel 13 or the operation buttons
14 is referred to simply as an "operation on the GUI". In the
example of FIG. 14, the information processing section 31 waits for
an operation of the user (step 8). Hereinafter, "steps" are
abbreviated as "S" in the drawings.
[0267] Next, when having detected an operation of the user, the
information processing section 31 performs the processes of step 9
and thereafter. For example, when the operation of the user on the
GUI is an instruction to "acquire a face image with the inner
capturing section 24" ("Yes" in step 9), the information processing
section 31 performs a face image acquisition process 1 (step 10).
Here, the instruction "to acquire a face image with the inner
capturing section 24" is, for example, an instruction for
acquisition using the inner capturing section 24, in accordance
with an operation of the user on the GUI or the like. Subsequently,
the information processing section 31 proceeds to step 19. The face
image acquisition process 1 will be described later with reference
to FIG. 15. On the other hand, when the operation of the user on
the GUI is not an instruction "to acquire a face image with the
inner capturing section 24" ("No" in step 9), the information
processing section 31 proceeds to step 11.
[0268] Next, for example, when the operation of the user on the GUI
is an instruction to "acquire a face image with the outer capturing
section 23" ("Yes" in step 11), the information processing section
31 performs a face image acquisition process 2 (step 12).
Subsequently, the information processing section 31 proceeds to
step 19. Here, the instruction "to acquire a face image with the
outer capturing section 23" is, for example, an instruction for
acquisition using the outer capturing section 23 by an operation of
the user on the GUI or the like. The face image acquisition process
2 will be described later with reference to FIG. 16. On the other
hand, when the operation of the user on the GUI is not an
instruction "to acquire a face image with the outer capturing
section 23" ("No" in step 11), the information processing section
31 proceeds to step 13.
[0269] Next, when the operation of the user on the GUI is an
instruction to display a list of collected face images ("Yes" in
step 13), the information processing section 31 performs a list
display process (step 14). Subsequently, the information processing
section 31 proceeds to step 19. The list display process will be
described later with reference to FIG. 17. On the other hand, when
the operation of the user on the GUI is not an instruction to
display a list of collected face images ("No" in step 13), the
information processing section 31 proceeds to step 15.
[0270] When the operation of the user on the GUI is an instruction
to determine a cast ("Yes" in step 15), the information processing
section 31 performs a cast determination process (step 16).
Subsequently, the information processing section 31 proceeds to
step 19. The cast determination process will be described later
with reference to FIG. 18. On the other hand, when the operation of
the user on the GUI is not an instruction to determine a cast ("No"
in step 15), the information processing section 31 proceeds to step
17.
[0271] When the operation of the user is an instruction to execute
a game ("Yes" in step 17), the information processing section 31
executes the game (step 18). The process of step 18 is an example
of a second game processing step. The game apparatus 10 performs
the game processing of the game where various character objects,
such as enemy objects EO, created in the cast determination process
in step 16 appear. The type of the game is not limited. For
example, the game executed in step 18 may be a game where the user
fights with enemy objects EO created in the cast determination
process. In this case, for example, the user fights with enemy
objects EO having face images collected in the face image
acquisition process 1 in step 10 and in the face image acquisition
process 2 in step 12, and displayed in the list display process in
step 14. Further, for example, this game may be an adventure game
where a player object representing the user moves forward by
overcoming various hurdles, obstacles, and the like. Alternatively,
examples of the game may include: a war simulation where historical
characters appear; a management simulation where a player object
appear; and a driving simulation of a vehicle or the like, where a
player object appears. Yet alternatively, the game may be a novel
game modeled on the original of a novel, where character objects
appear. Yet alternatively, the game may be one termed a
role-playing game (RPG) where the user controls a main character
and characters that appear in a story, to play their roles. Yet
alternatively, the game may be one where the user simply has some
training with the assistance of an agent that appears.
[0272] To the character objects that appear in such game
processing, face images collected by the user having succeeded in
the first game in step 10 are attached by texture mapping or the
like. Accordingly, in the game executed in step 18, the character
objects including the face images collected by the user themselves
appear. Thus, using images of portions representing people and
living things, such as face images, the user can execute a game
where the real-world relationships with the people (or the living
things) of the face images are reflected on the various character
objects. For example, it is possible to perform game processing
including emotions, such as affection, friendliness, favorable
impression, and hatred.
[0273] On the other hand, when the operation of the user is not an
instruction to execute a game ("No" in step 17), the information
processing section 31 proceeds to step 19.
[0274] Then, the information processing section 31 determines
whether or not the process is to be ended. When having detected
through the GUI an instruction to end the process, the information
processing section 31 ends the process of FIG. 14. On the other
hand, when having detected through the GUI an instruction not to
end the process (e.g., an instruction to retry the process), the
information processing section 31 performs a face image management
assistance process 1 (step 1A). The face image management
assistance process 1 is, for example, a process of, based on
already acquired face images, providing the user with information
about the attributes and the like of a face image to be acquired
next, so as to assist the user in acquiring a face image. A
detailed process of the face image management assistance process 1
will be described later with reference to FIG. 19A. Subsequently,
the information processing section 31 returns to step 8.
[0275] FIG. 15 is a flow chart showing an example of a detailed
process of the face image acquisition process 1 (step 10 of FIG.
14). In this process, the information processing section 31 first
performs a face image management assistance process 2 (step 100).
The face image management assistance process 2 is, for example, a
process of, based on already acquired face images, providing the
user with information about the attributes and the like of a face
image to be acquired next, so as to assist the user in acquiring a
face image. A detailed process of the face image management
assistance process 2 will be described later with reference to FIG.
19B.
[0276] Next, the information processing section 31 performs a face
image acquisition process (step 101). The CPU 311 of the
information processing section 31 performs the process of step 101
as an example of image acquisition means.
[0277] The information processing section 31 obtains images
captured by, for example, the inner capturing section 24, the left
outer capturing section 23a, and/or the right outer capturing
section 23b in predetermined cycles, and displays the obtained
images on the upper LCD 22. In this case, the display cycle may be
the same as the unit of time of the processing of the game
apparatus 10 (e.g., 1/60 seconds), or may be shorter than this unit
of time. Immediately after the power to the game apparatus 10 has
been turned on and the image processing program has been loaded, or
in an initial state immediately after the process of FIG. 14 has
been started, the information processing section 31 displays, for
example, an image from the inner capturing section 24 on the upper
LCD 22. It should be noted that in the lower LCD 12, for example,
an capturing section selection GUI is prepared so as to select at
least one of the inner capturing section 24, the left outer
capturing section 23a, and the right outer capturing section 23b
(including the case where both the left outer capturing section 23a
and the right outer capturing section 23b are used). In the process
of FIG. 15, it is assumed that the user can operate the capturing
section selection GUI to freely switch capturing sections to be
used. Hereinafter, the inner capturing section 24, the left outer
capturing section 23a, and/or the right outer capturing section 23b
that are, due to the initial state or the operation on the
capturing section selection GUI, used for capturing are referred to
simply as an "capturing section".
[0278] For example, when the inner capturing section 24 is used, if
the user turns their face toward the inner surface 21B of the upper
housing 21 in the state where the upper housing 21 is open, the
user's face is displayed on the upper LCD 22. Then, when the user
has pressed, for example, the R button 14H (or the L button 14G),
the information processing section 31 acquires, as data, an image
from the inner capturing section 24 that is displayed on the upper
LCD 22, and temporarily stores the acquired data in the main memory
32. At this time, the data of the image is only present in the main
memory 32, and is not saved in the saved data storage area Do
described later. The data present in the main memory 32 is only
used in the game in step 106 described later, and as will be
described later, is discarded when the game has not been successful
and has been ended. The main memory 32 is an example of a first
data storage area.
[0279] It should be noted that in the processing according to the
present embodiment, the face image acquired in step 101 is
texture-mapped onto the facial surface portion or the like of an
enemy object EO, and the game is executed. Accordingly, in the
process of step 101, it is preferable that the face image should be
acquired by clipping particularly the face portion from the image
acquired from the capturing section. In the present embodiment, for
example, it is assumed that the following processing is performed.
(1) The information processing section 31 detects the contour of
the face in the acquired image. The contour of the face is
estimated from the distance between the eyes, and the positional
relationships between the eyes and the mouth. That is, the
information processing section 31 recognizes the boundary line
between the contour of the face and the background, based on the
arrangement of the eyes and the mouth, using the dimensions of a
standard face. The boundary line can be acquired by combining, for
example, differential processing (contour enhancement) and average
processing (smoothing calculation), which are normal image
processing. It should be noted that the method of detecting the
contour of the face may be another known method. (2) The
information processing section 31 fits the obtained face image with
the dimensions of the facial surface portion of the head shape of
the enemy object EO by enlarging or reducing the obtained face
image. This process enables the game apparatus 10 to even acquire
face images varying to some extent in dimensions and attach the
acquired face images to enemy objects EO.
[0280] In the game apparatus 10 according to the present
embodiment, however, the process of acquiring a face image is not
limited to the procedure described above. For example, when a face
image is acquired, a face image having target dimensions may be
acquired from the capturing section, instead of the acquisition of
an image from a given distance and in given dimensions. For
example, a face image may be acquired on the condition that a
distance from a subject is established such that the distance
between the eyes of the face image obtained from the subject
approximates a predetermined number of pixels. For example, the
information processing section 31 may derive the distance from the
subject. Alternatively, on the condition that a distance from a
subject is established, the information processing section 31 may,
for example, lead a person who is the subject, or the user who is
the capturer, to adjust the angle of the subject's face with
respect to the direction of the optical axis of the capturing
section. Further, instead of the user pressing, for example, the R
button 14H (or the L button 14G) to save the image, when it is
determined that the adjustment of the distance from the subject and
the adjustment of the angle of the face with respect to the
direction of the optical axis of the capturing section are
completed, the information processing section 31 may save the
image. For example, the information processing section 31 may
display marks representing target positions for positioning the
eyes and the mouth, in superposition with the face image of the
subject on the upper LCD 22. Then, when the positions of the eyes
and the mouth of the subject that have been acquired from the
capturing section have fallen within predetermined tolerance ranges
from the marks of the target positions corresponding to the eyes
and the mouth, the information processing section 31 may save the
image in a memory.
[0281] It should be noted that when the face image is acquired in
step 101, the information processing section 31 updates the number
of acquired face images in the corresponding row of the face image
attribute aggregate table Dn2 shown in FIG. 13. "The corresponding
row" means, for example, the row of the attributes corresponding to
the gender and the age that are estimated in step 1002 of FIG. 19B
described later.
[0282] Next, the information processing section 31 displays the
image acquired in the process of step 101 on, for example, the
upper LCD 22 (step 102).
[0283] Next, the information processing section 31 performs a
process of selecting an enemy object EO (step 103). Here, the
information processing section 31 prompts the user to select the
head shape of an enemy object EO. For example, the information
processing section 31 may display the list of head shapes as shown
in FIG. 9, and may receive the selection of the user through the
GUI. Then, the information processing section 31 sets the acquired
face image as a texture of the enemy object EO (step 104), and
generates the enemy object EO (step 105). The enemy object
generated in step 105 is an example of a first character object.
The information processing section 31 performs the process of step
105 as an example of means for creating a character object.
[0284] Then, the information processing section 31 executes a game
using the generated enemy object EO (step 106). The CPU 311 of the
information processing section 31 performs the process of step 106
as an example of first game processing means. Here, the type of the
game is not limited. The game is, for example, a game simulating a
battle with the enemy object EO. Alternatively, the game may be,
for example, a game where the user competes with the enemy object
EO in score. Then, after the execution of the game, the information
processing section 31 determines whether or not the user has
succeeded in the game (step 107). The information processing
section 31 performs the process of step 107 as an example of means
for determining a success or a failure. A "success" is, for
example, the case where the user has defeated the enemy object EO
in the game where the user fights with the enemy object EO.
Alternatively, a "success" is, for example, the case where the user
has scored more points than the enemy object EO in the game where
the user competes with the enemy object EO in score. Yet
alternatively, a "success" may be, for example, the case where the
user has reached a goal in a game where the user overcomes
obstacles and the like set by the enemy object EO.
[0285] It should be noted that in the game executed in step 106, as
well as a character object including the face image acquired in
step 101, a character object using a face image already collected
in the past may be caused to appear. For example, when a face image
already collected in the past is attached to an enemy object EO or
a friend object and appears, the user can play a game on which
human relationships in the real world and the like are
reflected.
[0286] When the user has succeeded in the game, the information
processing section 31 saves, in the saved data storage area Do of
the game, data of the face image present in the main memory 32 that
has been acquired in step 101 described above, in addition to data
of face images that have been saved up to the current time (step
109). The CPU 311 of the information processing section 31 performs
the process of step 109 as an example of means for saving. The
saved data storage area Do of the game is a storage area where the
information processing section 31 that executes the game can
perform writing and reading, the storage area constructed in, for
example, the main memory 32, the data storage internal memory 35,
or the data storage external memory 46. Data of a new face image is
stored in the saved data storage area Do of the game, whereby the
information processing section 31 that executes the game can
display on the screen of the upper LCD 22 the data of the new face
image by adding the data to, for example, the list of the face
images described with reference to FIGS. 7 and 8. As described
above, based on the process of FIG. 15, the user executes the game
(first game) in order to save the face image acquired in step 101
in the saved data storage area Do of the game. In the game, for
example, a character object using a face image that has been saved
in the saved data storage area Do by the user up to the current
time is caused to appear, whereby the user who executes the game
with the game apparatus 10 can collect a new face image, and add
the new face image to the saved data storage area Do, while
reflecting human relationships in the real world and the like.
[0287] At this time, to manage the face image newly saved in the
saved data storage area Do of the game, the information processing
section 31 generates the face image management information Dn1
described with reference to FIG. 12, and saves the face image
management information Dn1 in the data storage internal memory 35
or the data storage external memory 46. That is, the information
processing section 31 newly generates face image identification
information, and sets the face image identification information as
a record of the face image management information Dn1. Further, the
information processing section 31 sets the address and the like of
the face image newly saved in the saved data storage area Do of the
game, as the address of face image data. Furthermore, the
information processing section 31 sets the source of acquiring the
face image, the estimation of gender, the estimation of age, pieces
of related face image identification information 1 through N, and
the like.
[0288] In addition, the information processing section 31 may
estimate the attributes of the face image added to the saved data
storage area Do, to thereby update the aggregate result of the face
image attribute aggregate table Dn2 described with reference to
FIG. 13. That is, the information processing section 31 may newly
estimate the gender, the age, and the like of the face image added
to the saved data storage area Do, and may reflect the estimations
on the aggregate result of the face image attribute aggregate table
Dn2.
[0289] In addition, the information processing section 31 may
permit the user to, for example, copy or modify the data stored in
the saved data storage area Do of the game, or transfer the data
through the wireless communication module 36. Then, the information
processing section 31 may, for example, save, copy, modify, or
transfer the face image stored in the saved data storage area Do in
accordance with an operation of the user through the GUI, or with
an operation of the user through the operation buttons 14.
[0290] On the other hand, when the user has not succeeded in the
game, the information processing section 31 inquires of the user as
to whether or not to retry the game (step 108). For example, the
information processing section 31 displays on the upper LCD 22 a
message indicating an inquiry about whether or not to retry the
game, and receives the selection of the user in accordance with an
operation on the GUI provided on the lower LCD 12 (e.g., a positive
icon, a negative icon, or a menu) through the touch panel 13, an
operation through the operation buttons 14, or the like. When the
user has given an instruction to retry the game, the information
processing section 31 returns to step 106. On the other hand, when
the user has not given an instruction to retry the game, the
information processing section 31 discards the face image acquired
in step 101 (step 110), and ends the process. It should be noted
that when the game has not been successful, the information
processing section 31 may discard the face image acquired in step
101, without waiting for an instruction to retry the game in step
108.
[0291] With reference to FIG. 16, a description is given below of
an example of a detailed process of the face image acquisition
process 2 (step 12 of FIG. 14). In this process, a description is
given of an example of the process of, in a face image acquisition
process, leading the user to acquire a face image with the inner
capturing section 24 prior to the two capturing sections of the
outer capturing section 23 (the left outer capturing section 23a
and the right outer capturing section 23b). The reason why the game
apparatus 10 causes the user to first acquire a face image with the
inner capturing section 24 is that the acquisition of a face image
with the inner capturing section 24 clarifies, for example, the
owner and the like of the game apparatus 10, so as to increase the
possibility of restricting the use of the game apparatus 10 by
another person.
[0292] With reference to FIG. 16, a description is given of an
example of the operation of the image processing program executed
by the information processing section 31 of the game apparatus 10.
In this process, the information processing section 31 first
determines whether or not a face image has already been acquired by
the inner capturing section 24 (step 121). The determination of
whether or not a face image has already been acquired by the inner
capturing section 24 may be made, for example, with reference to
the face image management information Dn1 shown in FIG. 12 and
based on whether or not there is a record where the inner capturing
section 24 is set as the source of acquiring the face image.
Alternatively, for example, when the game apparatus 10 has the
function of registering in the game apparatus 10 the face image of
the owner of the game apparatus 10 acquired by the inner capturing
section 24, the determination may be made based on whether or not
the face image of the owner has already been registered.
[0293] When a face image has not already been acquired by the inner
capturing section 24 ("No" in step 121), the information processing
section 31 prompts the user to first perform a face image
acquisition process with the inner capturing section 24 (step 124),
and ends the process of this subroutine. More specifically, for
example, the information processing section 31 displays on the
upper LCD 22 a message indicating "In the game apparatus 10, if a
face image has not already been acquired by the inner capturing
section 24, a face image acquisition process cannot be performed
with the outer capturing section 23". Alternatively, the
information processing section 31 may request the user to first
register the face image of the owner.
[0294] On the other hand, when a face image has already been
acquired by the inner capturing section 24 ("Yes" in step 121), the
information processing section 31 performs a face image management
assistance process 3 (step 122). The face image management
assistance process 3 will be described later with reference to FIG.
19C. Then, the information processing section 31 performs a face
image acquisition process with the outer capturing section 23 (step
123). For example, when the outer capturing section 23 is used, if
the user directs the outer surface 21D of the upper housing 21 to
another person's face in the state where the upper housing 21 is
open, said another person's face is displayed on the upper LCD 22.
Then, when the user has pressed, for example, the R button 14H (or
the L button 14G), the information processing section 31 acquires,
as data, an image from the outer capturing section 23 that is
displayed on the upper LCD 22, and temporarily stores the acquired
data in the main memory 32. At this time, the data of the image is
only present in the main memory 32, and is not saved in the saved
data storage area Do. The data present in the main memory 32 is
only used in the game in step 129 described later, and as will be
described later, is discarded when the game has not been successful
and has been ended.
[0295] It should be noted that in the processing according to the
present embodiment, the face image acquired in step 123 can also be
texture-mapped onto the facial surface portion or the like of an
enemy object EO, and the game can be executed. Accordingly, in the
process of step 123, it is preferable that the face image should be
acquired by clipping particularly the face portion from the image
acquired from the capturing section, by a process similar to that
of step 101 described above. Further, also when a face image is
acquired in step 123, the information processing section 31 updates
the number of acquired face images in the corresponding row of the
face image attribute aggregate table Dn2 shown in FIG. 13. "The
corresponding row" means, for example, the row of the attributes
corresponding to the gender and the age that are estimated in step
1202 of FIG. 19C described later.
[0296] Next, the information processing section 31 displays the
image acquired in the process of step 123 on, for example, the
upper LCD 22 (step 125).
[0297] Next, the information processing section 31 performs a
process of selecting an enemy object EO (step 126). Here, the
information processing section 31 prompts the user to select the
head shape of an enemy object EO. For example, the information
processing section 31 may display the list of head shapes as shown
in FIG. 9, and may receive the selection of the user through the
GUI. Then, the information processing section 31 sets the acquired
face image as a texture of the enemy object EO (step 127), and
generates the enemy object EO (step 128). The enemy object
generated in step 128 is also an example of the first character
object. The information processing section 31 performs the process
of step 128 as an example of the means for creating a character
object.
[0298] Then, the information processing section 31 executes a game
using the generated enemy object EO (step 129). The CPU 311 of the
information processing section 31 performs the process of step 129
as an example of the first game processing means. The game executed
in step 129 is similar to that of step 106. That is, the type of
the game executed in step 129 varies, and possible examples of the
game may include: a game simulating a battle with the enemy object
EO; and a game where the user competes with the enemy object EO in
score. Then, after the execution of the game, the information
processing section 31 determines whether or not the user has
succeeded in the game (step 130). The information processing
section 31 performs the process of step 130 as an example of the
means for determining a success or a failure. A "success" is, for
example, the case where the user has defeated the enemy object EO
in the game where the user fights with the enemy object EO.
Alternatively, a "success" is, for example, the case where the user
has scored more points than the enemy object EO in the game where
the user competes with the enemy object EO in score. Yet
alternatively, a "success" may be, for example, the case where the
user has reached a goal in a game where the user overcomes
obstacles and the like set by the enemy object EO.
[0299] It should be noted that in the game executed in step 129, as
well as a character object including the face image acquired in
step 123, a character object using a face image already collected
in the past may be caused to appear. For example, when a face image
already collected in the past is attached to an enemy object EO or
a friend object and appears, the user can play a game on which
human relationships in the real world and the like are
reflected.
[0300] When the user has succeeded in the game, the information
processing section 31 saves, in the saved data storage area Do of
the game, data of the face image present in the main memory 32 that
has been acquired in step 123 described above, in addition to data
of face images that have been saved up to the current time (step
132), and ends the process of the subroutine. The CPU 311 of the
information processing section 31 performs the process of step 132
as an example of the means for saving. Data of a new face image is
stored in the saved data storage area Do of the game, whereby the
information processing section 31 that executes the game can
display on the screen of the upper LCD 22 the data of the new face
image by adding the data to, for example, the list of the face
images described with reference to FIGS. 7 and 8. As described
above, based on the process of FIG. 16, the user executes the game
(first game) in order to save the face image acquired in step 123
in the saved data storage area Do of the game. In the game, for
example, a character object using a face image that has been saved
in the saved data storage area Do by the user up to the current
time is caused to appear, whereby the user who executes the game
with the game apparatus 10 can collect a new face image, and add
the new face image to the saved data storage area Do, while
reflecting human relationships in the real world and the like.
[0301] At this time, as in the face image acquisition process 1 in
step 10, to manage the face image newly saved in the saved data
storage area Do of the game, the information processing section 31
generates the face image management information Dn1 described with
reference to FIG. 12, and saves the face image management
information Dn1 in the data storage internal memory 35 or the data
storage external memory 46. That is, the information processing
section 31 newly generates face image identification information,
and sets the face image identification information as a record of
the face image management information Dn1. Further, the information
processing section 31 sets the address and the like of the face
image newly saved in the saved data storage area Do of the game, as
the address of face image data. Furthermore, the information
processing section 31 sets the source of acquiring the face image,
the estimation of gender, the estimation of age, pieces of related
face image identification information 1 through N, and the like. In
addition, the information processing section 31 may estimate the
attributes of the face image added to the saved data storage area
Do, to thereby update the aggregate result of the face image
attribute aggregate table Dn2 described with reference to FIG. 13.
That is, the information processing section 31 may newly estimate
the gender, the age, and the like of the face image added to the
saved data storage area Do, and may reflect the estimations on the
aggregate result of the face image attribute aggregate table Dn2.
In addition, the information processing section 31 may permit the
user to, for example, copy or modify the data stored in the saved
data storage area Do of the game, or transfer the data through the
wireless communication module 36. Then, the information processing
section 31 may, for example, save, copy, modify, or transfer the
face image stored in the saved data storage area Do in accordance
with an operation of the user through the GUI, or with an operation
of the user through the operation buttons 14.
[0302] On the other hand, when the user has not succeeded in the
game, the information processing section 31 inquires of the user as
to whether or not to retry the game (step 131). For example, the
information processing section 31 displays on the upper LCD 22 a
message indicating an inquiry about whether or not to retry the
game, and receives the selection of the user in accordance with an
operation on the GUI provided on the lower LCD 12 (e.g., a positive
icon, a negative icon, or a menu) through the touch panel 13, an
operation through the operation buttons 14, or the like. When the
user has given an instruction to retry the game, the information
processing section 31 returns to step 129. On the other hand, when
the user has not given an instruction to retry the game, the
information processing section 31 discards the face image acquired
in step 123 (step 133), and ends the process of the subroutine. It
should be noted that when the game has not been successful, the
information processing section 31 may discard the face image
acquired in step 123, without waiting for an instruction to retry
the game in step 131.
[0303] FIG. 17 is a flow chart showing an example of a detailed
process of the list display process (step 14 of FIG. 14). In this
process, the information processing section 31 first reads already
registered face images from the saved data storage area Do of the
data storage internal memory 35 or the data storage external memory
46, and displays the already registered face images on the upper
LCD 22 (step 140). More specifically, the information processing
section 31 acquires the addresses of face image data of the face
images from the face image management information Dn1 saved in the
saved data storage area Do. Then, the information processing
section 31 may read the face images from the addresses in the data
storage internal memory 35, the data storage external memory 46, or
the like, and may display the face images on the upper LCD 22.
[0304] Next, the information processing section 31 waits for an
operation of the user (step 141). Then, in accordance with an
operation of the user, the information processing section 31
determines whether or not a face image is in the state of being
selected (step 142). The determination of whether or not a face
image is in the state of being selected is made based on, when the
list of the face images are displayed on the upper LCD 22 as shown
in FIGS. 7 and 8, the state of the operation after the user has
pressed the operation buttons 14 or the like, or the state of the
operation through the GUI. It should be noted that the face image
in the state of being selected is displayed by being surrounded by,
for example, the heavy line L1, or the heavy line L2, as shown in
FIGS. 7 and 8.
[0305] Then, when any one of the face images is in the state of
being selected, the information processing section 31 searches for
face images related to the face image in the state of being
selected, using the face image management information Dn1 (see FIG.
12) (step 143).
[0306] Then, the information processing section 31 performs a
process of causing the found face images to react, such as causing
the found face images to give looks to the face image in the state
of being selected (step 144). The process of causing the found face
images to react can be performed by, for example, the following
procedure. For example, the following are prepared in advance: a
plurality of patterns of eyes, in which the orientation of eyes are
directed to another face image as shown in FIG. 8; and a plurality
of patterns of a face, in which the orientation of a face is
directed to another face image as shown in FIG. 8. Then, based on
the relationships between the positions of the found face images
and the position of the face image in the state of being selected,
the corresponding patterns of the orientations of eyes and the
orientations of faces are selected. Then, the corresponding
patterns of the orientations of eyes and the orientations of faces
of the face images may be displayed so as to switch the patterns of
the orientations of eyes and the orientations of faces of the
already displayed face images. That is, images of eyes determined
based on the relationships between the positions of the found face
images and the position of the face image in the state of being
selected may replace the eye portions of the original face images.
To change the orientations of the faces, display may be performed
by switching the entire face images. Alternatively, for example,
patterns of eyes may be prepared in advance, in which the
orientation of eyes are changed at predetermined angles, e.g., at
units of 15 degrees in a 360 degree direction. Then, based on the
positional relationships between the face image in the state of
being selected and the found face images, angles may be determined,
and the patterns of eyes at the angles closest to the determined
angles may be selected.
[0307] In addition, concerning the orientation of a face, patterns
of a face image are prepared in which, on the assumption that the
case of being directed in the normal direction of the screen is 0
degrees, the orientation is changed in the left-right directions at
angles, e.g., 30 degrees, 60 degrees, and 90 degrees. Further,
patterns are also prepared in which the orientation is changed in
the up-down directions at, for example, approximately 30 degrees.
Further, patterns may be prepared in which, for a face image whose
orientation has been changed in the left-right direction at an
angle of 90 degrees, the orientation is further changed in the
up-down direction, i.e., diagonally upward (e.g., 15 degrees
upward, 30 degrees upward, and 45 degrees upward) and diagonally
downward (e.g., 15 degrees downward, 30 degrees downward, and 45
degrees downward). Then, based on the positional relationships
between the face image in the state of being selected and the found
face images, angles may be determined, and the angles of faces
closest to the corresponding angles may be selected. Further, to
emphasize intimacy, an expression such as an animation of a
three-dimensional model closing one eye may be displayed. Further,
a heart mark and the like may be prepared in advance, and displayed
near the face images related to the face image in the state of
being selected.
[0308] In the determination in step 142, when a face image is not
in the state of being selected, the information processing section
31 performs another process (step 145). Said another process
includes, for example, an operation on another GUI provided on the
lower LCD 12, and a process on operation buttons 14 other than the
operation buttons 14 used for the selection of face images (buttons
14a, 14b, 14c, and the like). Subsequently, the information
processing section 31 determines whether or not the process is to
be ended (step 146). For example, when having detected that the
button 14c (B button) has been pressed while the screen shown in
FIG. 8 is displayed, the information processing section 31
determines that an instruction has been given to "return", and ends
the process of FIG. 17. When the process is not to be ended, the
information processing section 31 returns to step 140.
[0309] FIG. 18 is a flow chart showing an example of a detailed
process of the cast determination process (step 16 of FIG. 14). In
this process, the information processing section 31 first displays
a list of the head shapes of enemy objects EO (step 160). It should
be noted that here, the description is given, taking enemy objects
EO as an example; however, also when a character object other than
the enemy objects EO is generated, a process similar to the
following process is performed. The information processing section
31 stores the head shapes of the enemy objects EO in the data
storage internal memory 35 in advance, for example, before the
shipment of the game apparatus 10, or at the installation or the
upgrading of the image processing program. The information
processing section 31 reads the head shapes of the enemy objects EO
currently stored in the data storage internal memory 35, and
displays the head shapes of the enemy objects EO in the arrangement
as shown in FIG. 9.
[0310] Next, the information processing section 31 detects a
selection operation of the user through the GUI, the operation
buttons 14, or the like, and receives the selection of the head
shape of an enemy object EO (step 161). When the selection of the
head shape of an enemy object EO has been ended, subsequently, the
information processing section 31 displays a list of face images
(step 162). Then, the information processing section 31 detects a
selection operation of the user through the GUI, the operation
buttons 14, or the like, and receives the selection of a face image
(step 163). It should be noted that in the example of the process
of FIG. 18, the information processing section 31 determines a face
image by detecting the selection operation in step 163. Instead of
such a process, however, the information processing section 31 may
automatically determine a face image. For example, the information
processing section 31 may select a face image randomly from among
the face images accumulated in the saved data storage area Do.
Alternatively, the information processing section 31 may save in
advance the history of the game of the user using the game
apparatus 10, in the main memory 32, the external memory 45, the
data storage external memory 46, the data storage internal memory
35, or the like, and may select a face image in accordance with the
properties, the taste, and the like of the user that are estimated
from the history of the user. For example, in accordance with the
frequencies of the user selecting face images in the past, the
information processing section 31 may determine a face image to be
selected next.
[0311] Then, the information processing section 31 sets the
selected face image as a texture of the enemy object EO (step 164).
Then, the information processing section 31 generates the enemy
object EO by texture-mapping the selected face image onto the
facial surface portion of the enemy object EO (step 165). The enemy
object generated in step 165 is an example of a second character
object. Then, the information processing section 31 displays the
generated enemy object EO on the screen of the upper LCD 22 in the
form of, for example, the enemy object EO shown in FIG. 10.
[0312] In addition, the information processing section 31 performs
a process of causing related face images to react (step 166). This
process is similar to the processes of steps 143 and 144 of FIG.
17. Further, in accordance with an operation of the user on the
GUI, the information processing section 31 determines whether or
not the generated enemy object EO is to be fixed (step 167). When
the enemy object EO is not to be fixed, the information processing
section 31 returns to step 162, and receives the selection of a
face image. However, when the enemy object EO is not to be fixed,
the information processing section 31 may return to step 160, and
may receive the selection of the head shape of an enemy object EO.
On the other hand, when the enemy object EO is to be fixed, the
information processing section 31 ends the process. The information
processing section 31 performs the game processing shown in step 18
of FIG. 14, using the fixed enemy object EO. Although not shown in
FIG. 18, however, a menu of the GUI or the like may be prepared so
as to end the process of FIG. 18 without fixing the enemy object
EO.
[0313] FIG. 19A is a flow chart showing an example of a detailed
process of the face image management assistance process 1 (step 1A
of FIG. 14). In this process, the information processing section 31
reads the attributes of already acquired face images from the face
image attribute aggregate table Dn2 (step 1A1). Then, the
information processing section 31 searches the read face image
attribute aggregate table Dn2 for an unacquired attribute or an
attribute including a small number of acquired face images. An
"unacquired attribute" means, for example, an attribute whose
number of acquired face images is 0 in the table shown in FIG. 13.
Further, "an attribute including a small number of acquired face
images" means, for example, an attribute whose number of acquired
face images is, when sorting is performed using the numbers of
acquired face images in the table shown in FIG. 13 as sorting keys,
included in predetermined ranks from the bottom.
[0314] Next, the information processing section 31 performs a
process of prompting the user to acquire a face image corresponding
to an unacquired attribute (step 1A2). For example, the information
processing section 31 may display on the lower LCD 12 or the upper
LCD 22 a message combining the attribute "male", the attribute
"10's", with the phrase "the number of acquired images is 0", based
on the table shown in FIG. 13. Further, for example, the
information processing section 31 may display a message combining
the attribute "male", the attribute "10's", with the phrase "the
number of acquired images is small". The number of the combinations
of the attributes to be displayed (a row in the table shown in FIG.
13), however, is not limited to one, and two or more combinations
may be displayed. Then, for example, when having detected that the
user has pressed the operation button 14B (A button), the
information processing section 31 may end the process of FIG. 19A.
Subsequently, the information processing section 31 returns to step
8 of FIG. 14.
[0315] It should be noted that here, the description is given,
taking the face image management assistance process 1 shown in FIG.
19A, as an example of a detailed process performed at the time of
the determination of whether the game is to be ended in FIG. 14
(step 1A). The face image management assistance process 1, however,
is not limited to the time of the determination of whether the game
is to be ended after the execution of the game (step 1A). For
example, the information processing section 31 may perform the face
image management assistance process 1 in order to prompt the user
to acquire a face image during the list display process (step 14),
the cast determination process (step 16), the execution of the game
(step 18), or the like.
[0316] FIG. 19B is a flow chart showing an example of a detailed
process of the face image management assistance process 2 (step 100
of FIG. 15). In this process, first, with reference to the number
of acquired images, the information processing section 31
determines the presence or absence of an acquired image (step
1000). The information processing section 31 may store the number
of acquired images as, for example, the number of records in the
face image management information Dn1 in the main memory 32, the
external memory 45, the data storage external memory 46, the data
storage internal memory 35, or the like.
[0317] Then, when an acquired image is not present ("No" in the
determination in step 1000), the information processing section 31
ends the process. On the other hand, when an acquired image is
present ("Yes" in the determination in step 1000), the information
processing section 31 proceeds to step 1001. Then, the information
processing section 31 receives a request to acquire the face image
(step 1001). The information processing section 31 recognizes the
request to acquire the face image, for example, when having
received an acquisition instruction through the L button 14G or the
R button 14H in the state where the face image is displayed on the
upper LCD 22 through the inner capturing section 24 or the outer
capturing section 23.
[0318] Then, the information processing section 31 estimates the
attributes, e.g., the gender and the age, of the face image
acquired through the inner capturing section 24 or the outer
capturing section 23 and displayed on the upper LCD 22 (step 1002).
For example, the gender can be estimated from the size of the
skeleton including the cheekbones and the mandible that are
included in the face image, and the dimensions of the face. That
is, the information processing section 31 calculates the relative
dimensions of the contour of the face relative to the distance
between the eyes and the distances between the eyes and the mouth
(e.g., the width of the face, and the distance between the eyes and
the chin). Then, when the relative dimensions are close to
statistically obtained male average values, it may be determined
that the face image is male. Further, for example, when the
relative dimensions are close to statistically obtained female
average values, it may be determined that the face image is
female.
[0319] In addition, the information processing section 31 may
store, in advance, feature information by gender and by age bracket
(e.g., under 10, 10's, 20's, 30's, 40's, 50's, 60's, or 70 or
over), such as the average positions of parts of faces and the
number of wrinkles in the portions of faces. Then, the information
processing section may calculate the feature information of the
face image, for example, acquired through the outer capturing
section 23 and displayed on the upper LCD 22, and may estimate the
age bracket closest to the calculated feature information. The
above descriptions of the specification of the gender and the age,
however, is illustrative, and the determination of the gender and
the age is not limited to the above process. In the process of step
1002, it is possible to apply various gender determination
techniques and various age specification techniques that are
conventionally proposed.
[0320] Next, the information processing section 31 prompts the user
to acquire a face image having an unacquired attribute (step 1003).
For example, the information processing section 31 may display on
the upper LCD 22 a message prompting the user to acquire a face
image having an unacquired attribute. This process is similar to
that of FIG. 19A. Then, the information processing section 31
determines whether or not the user has performed an operation of
switching acquisition target face images (step 1004). For example,
when the features, e.g., the distance between the eyes and the
distances between the eyes and the mouth, of the face image
included in the image acquired through the inner capturing section
24 or the outer capturing section 23, have changed, the information
processing section 31 determines that acquisition target face
images have been switched. Further, for example, when an
acquisition instruction through the L button 14G or the R button
14H has been simply canceled, and an acquisition instruction has
been given again through the L button 14G or the R button 14H, the
information processing section 31 may determine that acquisition
target face images have been switched. Then, when acquisition
target face images have been switched, the information processing
section 31 returns to step 1002.
[0321] On the other hand, when acquisition target face images have
not been switched, the information processing section 31 ends the
process as it is. Here, "when acquisition target face images have
not been switched" is, for example, the case where the user has
ended the face image management assistance process 2 through the
GUI, the operation button 14C (B button), or the like.
Alternatively, for example, when the state where acquisition target
face images are not switched has continued for a predetermined
time, the information processing section 31 may determine that
acquisition target face images have not been switched. In this
case, the information processing section 31 proceeds to step 101 of
FIG. 15, and performs the face image acquisition process. It should
be noted that as has already been described in the process of step
101 of FIG. 15, the information processing section 31 updates the
number of acquired face images in the corresponding row of the face
image attribute aggregate table Dn2 shown in FIG. 13.
[0322] In addition, in the determination process of step 1004,
"when acquisition target face images have not been switched" is,
for example, the case where the amount of change in the distance
between the eyes and the amounts of change in the distances between
the eyes and the mouth are within tolerances. Alternatively, for
example, when an acquisition instruction from the L button 14G or
the R button 14H has not been simply canceled, but has continued
for a predetermined time, the information processing section 31 may
determine that the acquisition instruction has not been
canceled.
[0323] FIG. 19C is a flow chart showing an example of a detailed
process of the face image management assistance process 3 (step 122
of FIG. 16). The process of FIG. 19C (steps 1201 through 1204) is
similar to steps 1001 through 1004 in the process of FIG. 19B, and
therefore is not described.
[0324] Based on the processes of FIGS. 19A through 19C, the
information processing section 31 leads the user to preferentially
acquire a face image corresponding to an unacquired attribute. Such
a process makes it possible to assist a face image collection
process of a user who wishes to acquire face images having as
balanced attributes as possible.
[0325] It should be noted that in the present embodiment, an
example of the process is shown where age brackets are classified
as under 10, 10's, 20's, 30's, 40's, 50's, 60's, or 70 or over. The
present invention, however, is not limited to such classification
of age brackets. For example, the age brackets may be further
classified into smaller categories. Alternatively, age brackets may
be roughly classified, such as children, adults, and the
elderly.
[0326] In the present embodiment, when having received an
acquisition instruction through the L button 14G or the R button
14H, the information processing section 31 recognizes a request to
acquire a face image. Instead of such a process, however, for
example, as has already been described in the present embodiment,
to acquire from the capturing section a face image having target
dimensions, in a process of deriving the distance between the game
apparatus 10 and the face, the angle between the optical axis of
the capturing section and the face, and the like, the information
processing section 31 may estimate the attributes, e.g., the gender
and the age, of the face image. That is, when, for example, the
information processing section 31 acquires a face image in real
time or in each frame cycle (e.g., 1/60 seconds) for such a
deriving process, the information processing section 31 may specify
the attributes of the face image from the acquired face image.
[0327] <Detailed Example of Game Processing>
[0328] Here, with reference to FIGS. 20A through 40, a description
is given of an example of the game processing of the game where
character objects using face images appear. The following
processing is performed in, for example, step 18 of FIG. 14, using
enemy objects EO including face images accumulated in the saved
data storage area Do.
[0329] First, in the present embodiment, a description is given of
an overview of a game that can be played by the player executing
the game program with the game apparatus 10. The game according to
the present embodiment is a so-called shooting game where the
player, as a main character of the game, shoots down enemy
characters that appear in a virtual three-dimensional space
prepared as a game world. For example, the virtual
three-dimensional space forming the game world (a virtual space
(also referred to as a "game space")) is displayed on a display
screen of the game apparatus 10 (e.g., the upper LCD 22) from the
player's point of view (a so-called first-person point of view). As
a matter of course, display may be performed from a third-person
point of view. When the player has shot down an enemy character,
points are added to the score. In contrast, when an enemy character
has collided with the player (specifically, when the enemy
character has reached within a certain distance from the position
of the virtual camera), points are deducted from the score.
[0330] In addition, in the game according to the present
embodiment, display is performed by combining an image of the real
world acquired by the capturing section included in the game
apparatus 10 (hereinafter referred to as a "real world image"),
with a virtual world image representing the virtual space.
Specifically, the virtual space is divided into an area closer to
the virtual camera (hereinafter referred to as a "front area") and
an area further from the virtual camera (hereinafter referred to as
a "back area"). Then, an image representing a virtual object
present in the front area is displayed in front of the real world
image, and the virtual object present in the back area is displayed
behind the real world image. More specifically, as will be
described later, combination is made such that the virtual object
present in the front area is given preference over the real world
image, and the real world image is given preference over the
virtual object present in the back area.
[0331] The method of combining the real world image with the
virtual world image is not limited. For example, the real world
image may be rendered with the virtual object by a common virtual
camera such that real world image may be present as an object in
the same virtual space as the virtual object (more specifically,
for example, by being attached as a texture of a virtual
object).
[0332] In addition, in another example, a first rendered image may
be obtained by rendering the real world image from a first virtual
camera (hereinafter referred to as a "real world drawing camera"),
and a second rendered image may be obtained by rendering the
virtual object from a second virtual camera (hereinafter referred
to as a "virtual world drawing camera"). Then, the first rendered
image may be combined with the second rendered image such that the
virtual object present in the front area is give preference over
the real world image, and the real world image is given preference
over the virtual object present in the back area.
[0333] In the first method, typically, the object to which the real
world image is applied as a texture (hereinafter referred to as a
"screen object") may be placed at a position, which is the boundary
between the front area and the back area, and may be drawn together
with the virtual object, such as an enemy object, as viewed from
the common virtual camera. In this case, typically, the object to
which the real world image is attached is an object having a
surface which has a certain distance from the virtual camera and
whose normal line coincides with the direction of the line of sight
of the virtual camera, and the real world image may be attached to
this surface (hereinafter referred to as a "boundary surface") as a
texture.
[0334] In addition, in the second method, the second rendered image
is obtained by rendering the virtual object while making a depth
determination (determination by Z-buffering) based on the boundary
surface between the front area and the back area (hereinafter
referred to simply as a "boundary surface"), and the first rendered
image is obtained by performing rendering by attaching the real
world image as a texture to a surface which has a certain distance
from the virtual camera and whose normal line coincides with the
direction of the line of sight of the virtual camera. Then, when
the second rendered image is combined with the first rendered image
such that the second rendered image is given preference over the
first rendered image, a combined image is generated, in which the
real world image seems to be present on the boundary surface.
[0335] In either method, the relationships between the distance
from, and the angle of view of, the virtual camera and the size of
the object of the real world image (the size in the direction of
the line of sight) are set such that the real world image includes
the range of the field of view of the virtual camera.
[0336] It should be noted that hereinafter, the first method is
referred to as a "first drawing method", and the second method is
referred to as a "second drawing method".
[0337] In addition, when predetermined event occurrence conditions
in the game have been satisfied, a part of the real world image is
opened, and display is performed such that the virtual space in the
back area can be viewed through the opening. Further, an enemy
character object is present in the front area, and when
predetermined conditions have been satisfied, a special enemy
character (a so-called "boss character") appears in the back area.
This stage is completed by shooting down the boss character.
Several stages are prepared, and the game is completed by
completing all the stages. In contrast, when predetermined game
over conditions have been satisfied, the game is over.
[0338] In a typical example of the first drawing method described
above, for the opening in the real world image, data indicating the
position of the opening may be set on the boundary surface of the
screen object. More specifically, the non-transparency of a texture
to be applied to the boundary surface (a so-called .alpha.-texture)
may indicate open or unopen. Further, in the second drawing method,
data indicating the position of the opening may be set on the
boundary surface.
[0339] In addition, in the present embodiment, the open/unopen
state is set in the real world image. Alternatively, another image
processing may be performed on the real world image. For example,
given image processing can be performed by common technical
knowledge of those skilled in the art, such as attaching dirt to
the real world image, or pixelizating the real world image. Also in
these examples, data may be set that indicates the position where
image processing is performed on the boundary surface.
[0340] <Game World>
[0341] As described above, in the game according to the present
embodiment, a game screen is displayed that represents the virtual
space having such an improved sense of depth that the existence of
the virtual space (back area) is felt also behind the real image.
It should be noted that the real world image may be a regular image
captured by a monocular camera, or may be a stereo image captured
by a compound eye camera.
[0342] In the game according to the present embodiment, an image
captured by the outer capturing section 23 is used as the real
world image. That is, a real world image in the periphery of the
player captured by the outer capturing section 23 (a real-world
moving image acquired in real time) is used during the game play.
Accordingly, when the user (the player of the game) holding the
game apparatus 10 has changed the imaging range of the outer
capturing section 23 by changing the orientation of the game
apparatus 10 in the left-right direction or the up-down direction
during the game play, the real world image displayed on the upper
LCD 22 also changes so as to follow the change in the imaging
range.
[0343] Here, the change in the orientation of the game apparatus 10
during the game play is made roughly in accordance with: (1) the
player's intention; or (2) the intention (scenario) of the game.
When the player has intentionally changed the orientation of the
game apparatus 10 during play, the real world image captured by the
outer capturing section 23 changes. This makes it possible to
intentionally change the real world image displayed on the upper
LCD 22.
[0344] In addition, the angular velocity sensor 40 of the game
apparatus 10 detects the change in the orientation of the game
apparatus 10, and the orientation of the virtual camera is changed
in accordance with the detected change. More specifically, the
current orientation of the virtual camera is changed in the
direction of the change in the orientation of the outer capturing
section 23. Further, the current orientation of the virtual camera
is changed by the amount of change (angle) in the orientation of
the outer capturing section 23. That is, when the orientation of
the game apparatus 10 is changed, the real world image changes, and
the displayed range of the virtual space changes. That is, a change
in the orientation of the game apparatus 10 changes the real world
image in conjunction with the virtual world image. This makes it
possible to display a combined image as if the real world is
associated with the virtual world. It should be noted that in the
present embodiment, the position of the virtual camera is not
changed. Alternatively, the position of the virtual camera may be
changed by detecting the movement of the game apparatus 10.
[0345] It should be noted that in the second drawing method, such a
process of changing the orientation of the virtual camera is
applied to the virtual world drawing camera, but is not applied to
the real world drawing camera.
[0346] In addition, when an object is displayed at a local
position, such as the end of the screen (e.g., the right end or the
left end) during the game play, the player naturally intends to
attempt to capture the object at the center of the screen, and
therefore moves the game apparatus 10 (outer capturing section 23).
As a result, the real world image displayed on the screen changes.
Such a change in the orientation of the game apparatus 10 (a change
in the real world image) can be naturally made by the user, by
performing programming such that an object displayed in accordance
with the scenario of the game is intentionally displayed at the end
of the screen.
[0347] <Details of Virtual Space>
[0348] (Drawing of Real World Image)
[0349] The real world image captured by the outer capturing section
23 is combined with the virtual space such that the real world
image seems to be present at the boundary position between the
front area and the back area of the virtual space. FIG. 20A shows
an example of the virtual space according to the present
embodiment. FIG. 20B shows the relationship between a screen model
and an .alpha.-texture according to the present embodiment. In the
first drawing method, to display the real world image, a screen
object may be formed by, as shown in FIG. 20A, setting a spherical
model (the screen model described above) having its center at the
position of the virtual camera in the virtual space, and attaching
the real world image to the inner surface of the sphere. More
specifically, the real world image is attached as a texture to the
screen model, in the entire portion of the viewing volume of the
virtual camera. The remaining portion of the screen model is set to
transparent, and therefore is not viewed on the screen. In this
example, the boundary surface is a spherical surface, that is, as
shown in FIG. 20A, the area closer to the virtual camera than the
surface of the sphere is the front area (corresponding to a "second
area" according to the present invention), and the area further
from the virtual camera than the surface of the sphere is the back
area (corresponding to a "first area" according to the present
invention).
[0350] In the second drawing method, to display the real world
image, a planar polygon to which a texture of the real world image
is attached is placed in the virtual space. In the virtual space,
the relative position of the planar polygon relative to the real
world drawing camera is always fixed. That is, the planar polygon
is placed so as to have a certain distance from the real world
drawing camera, and is placed such that the normal direction of the
planar polygon coincides with the point of view (optical axis) of
the real world drawing camera.
[0351] In addition, the planar polygon is set to include the range
of the field of view of the real world drawing camera.
Specifically, the size of the planar polygon and the distance of
the planar polygon from the virtual camera are set such that the
planar polygon can include the range of the field of view of the
virtual camera. The real world image is attached to the entire
surface of the planar polygon on its virtual camera side. Thus,
when the planar polygon to which the real world image is attached
is drawn from the virtual camera, display is performed such that
the real world image corresponds to the entire area of an image
generated by the virtual camera.
[0352] It should be noted that as shown in FIG. 21, the boundary
surface may be cylindrical. FIG. 21 shows another example of the
virtual space according to the present embodiment. In this case, in
the virtual space, a virtual cylindrical peripheral surface
(boundary surface) is placed, whose center axis is a vertical axis
extending through the position of the virtual camera (in the
present embodiment, it is assumed that a Y-axis of the virtual
space corresponds to the vertical direction, and an X-axis and a
Z-axis correspond to the horizontal directions). As described
above, however, the cylindrical peripheral surface is not an object
to be viewed, but is an object used for an opening process. The
outer peripheral surface of the cylinder divides the virtual space
into the first space where the virtual camera is placed
(corresponding to the "second area" according to the present
invention), and the second space existing around the first space
(corresponding to the "first area" according to the present
invention).
[0353] (Process of Opening Real World. Image)
[0354] Further, in the game according to the present embodiment, an
opening is provided in the real world image so that the player
recognizes the existence of the back area behind the real world
image. More clearly, the portion of the opening included in the
real world image is displayed in a transparent or semi-transparent
manner, and is combined with the world behind this portion. With
this, in the game, the occurrence of a predetermined event triggers
the opening (removal) of a part of the real world image, and an
image representing another virtual space existing behind the real
world image (back area) is displayed through the opening.
[0355] In the present embodiment, the boundary surface is a
spherical surface, and such a process of displaying the back area
by providing an opening in the real world image is achieved in the
first drawing method by a texture attached to the inner surface of
the spherical screen object described above, as shown in FIGS. 20A
and 20B. Hereinafter, this texture is referred to as a "screen
.alpha.-texture" (opening determination data described later). In
the present embodiment, the screen .alpha.-texture is attached to a
portion that completes a circuit by 360 degrees around the virtual
camera at least in a certain direction. More specifically, as shown
in FIG. 20B, the screen .alpha.-texture is attached to a central
portion of the sphere, i.e., a portion that completes a circuit by
360 degrees around the position of the virtual camera in a
direction parallel to the XY-plane and has a predetermined width in
the Y-direction (hereinafter referred to as an
".alpha.-texture-applied portion"). The process as described above
can simplify data included in the screen .alpha.-texture.
Specifically, in the present embodiment, the screen .alpha.-texture
has a rectangular shape. The attachment of the .alpha.-texture to
the portion shown in FIG. 20A causes pieces of information of the
dots on the screen .alpha.-texture to correspond to sets of
coordinates of the .alpha.-texture-applied portion in the screen
object.
[0356] As described above, the screen object to which the real
world image is attached and on which the .alpha.-texture is set is
drawn from the virtual camera, and therefore, drawing is performed
such that the real world image having an opening is present on the
boundary surface (the inner surface of the sphere). In the
.alpha.-texture, the portion corresponding to the real world image
is calculated by drawing from the virtual camera.
[0357] Also in the second drawing method, data indicating the
position of an opening is set on the boundary surface of the
virtual space (here, the inner surface of the sphere). Typically,
data is set that indicates the presence or absence of an opening at
each point of the boundary surface. More specifically, a spherical
object similar to the above is placed in the virtual world where a
virtual object is present, and a similar .alpha.-texture is set on
the spherical object. Then, when the real world image is rendered,
rendering is performed by applying to the planar polygon described
above an .alpha.-texture corresponding to the portion drawn by the
virtual world drawing camera, the corresponding .alpha.-texture
included in the .alpha.-texture set on the spherical object.
Alternatively, after a process is performed of making the opening
transparent in the real world image using the .alpha.-texture
corresponding to this portion, rendering is performed by the real
world drawing camera such that the real world image after this
process is attached to the planar polygon described above. It
should be noted that this spherical object is an object used only
to calculate an opening, but is an object not drawn when the
virtual world is drawn.
[0358] It should be noted that in the present embodiment, data
indicating an opening is data having information of each point of
the boundary surface. Alternatively, the data may be information
defining the position of an opening in the boundary surface by a
calculation formula.
[0359] In the second space, a polygon (object) is placed, to which
a background image (texture) of the second space included in the
field of view of the virtual camera through an opening is to be
attached. The background of the second space is occasionally
referred to as a "back wall".
[0360] In the first space, objects are placed so as to represent
enemy characters and various characters representing bullets for
shooting down the enemy characters. Also in the second space,
predetermined objects (e.g., some of the enemy characters) are
placed. The objects placed in the virtual space move in the virtual
space in accordance with logic (algorithm) programmed in
advance.
[0361] In addition, some of the enemy characters can move between
the first space and the second space through an opening formed in
the boundary surface, or can move between the first space and the
second space by forming an opening in the boundary surface
themselves. A particular event for forming an opening in the game
is, for example, an event where an enemy character collides with
the boundary surface (a collision event). Alternatively, the event
is where in the progression of the game scenario, the boundary
surface is destroyed based on predetermined timing, and an enemy
character present in the second space enters the first space (an
enemy character appearance event). Yet alternatively, an opening
may be automatically formed in accordance with the passage of time.
Yet alternatively, an opening may be repaired in accordance with a
predetermined game operation of the player. For example, the player
may reduce (repair) a formed opening by hitting the opening with a
bullet.
[0362] FIG. 22 shows a virtual three-dimensional space (game world)
defined in the game program, which is an example of the image
processing program according to the embodiment. It should be noted
that as described above, in the present embodiment, the boundary
surface is spherical; however, in FIG. 22, the boundary surface is
shown as cylindrical for convenience. As described above, in the
game according to the present embodiment, display is performed on
the upper LCD 22 of the game apparatus 10 such that the virtual
world image representing the virtual three-dimensional space and
the real world image are combined together.
[0363] In addition, as shown in FIG. 22, the virtual space in the
game according to the present embodiment is divided into the first
space 1 and the second space 2 by the boundary surface 3 formed of
the spherical surface having its central axis extending through the
position of the virtual camera.
[0364] On the boundary surface 3, a camera image CI, which is a
real world image captured by a real camera built into the game
apparatus 10 (FIG. 23), is combined with the virtual world image as
if the camera image CI is present at a position on the boundary
surface 3, by the processes of steps 81 and 82 described later in
the first drawing method, or by the processes of steps 83 through
step 85 described later in the second drawing method.
[0365] In the present embodiment, the real world image is a planar
view image. The virtual world image is also a planar view image.
That is, a planar view image is displayed on the upper LCD 22. The
real world image, however, may be a stereoscopically visible image.
The present embodiment is not limited by the type of the real world
image. It should be noted that in the present embodiment, the
camera image CI may be a still image, or may be a real-time real
world image (moving image). In the game according to the present
embodiment program, the camera image CI is a real-time real world
image. Further, the camera image CI, which is a real world image,
is not limited by the type of the camera. For example, the camera
image CI may be an image obtained by a camera that can be
externally connected to the game apparatus 10. Furthermore, in the
present embodiment, the camera image CI may be an image acquired
from the outer capturing section 23 (compound eye camera) and/or
the inner capturing section 24 (monocular camera). In the game
according to the present embodiment program, the camera image CI is
an image acquired using one of the left outer capturing section 23a
and the right outer capturing section 23b of the outer capturing
section 23 as a monocular camera.
[0366] As described above, the first space 1 is a space closer when
viewed from the virtual camera than the boundary surface 3, and is
also a space surrounded by the boundary surface 3. Further, the
second space 2 is a space behind the boundary surface 3 as viewed
from the virtual camera. Although not shown in FIGS. 20A and 21, a
back wall BW surrounding the boundary surface 3 is present. That
is, the second space 2 is a space present between the boundary
surface 3 and the back wall BW. To the back wall BW, a given image
is attached. For example, to the back wall BW, an image
representing outer space prepared in advance is attached, and
display is performed such that the second space 2, which is outer
space, exists behind the first space 1. That is, the first space 1,
the boundary surface 3, the second space 2, and the back wall BW
are placed in the order from the closer area to the further area,
as viewed from the virtual camera.
[0367] As described above, however, the image processing program
according to the present invention is not limited to a game
program, and these settings and rules do not limit the image
processing program according to the present invention. It should be
noted that as shown in FIG. 22, enemy objects EO can move in the
virtual three-dimensional space, and can move between the first
space 1 and the second space 2 through the boundary surface 3
described above. When an enemy object EO moves between the first
space 1 and the second space 2 in the boundary surface 3 by passing
through an area captured by the virtual camera, representation is
made such that on an image displayed on the upper LCD 22, the enemy
object EO moves from the further area to the closer area, or moves
from the closer area to the further area, by passing through the
real world image.
[0368] On the screen, display is performed such that the enemy
object EO moves between the first space 1 and the second space 2,
using an opening (hole) produced in the real world image due to the
game scenario or an event. FIGS. 22 and 24 show the state where an
enemy object EO moves between the first space 1 and the second
space 2 by forming an opening in the boundary surface 3 or passing
through an opening already present in the boundary surface 3.
[0369] It should be noted that in the image processing program
according to the present embodiment, objects present in the first
space 1 or the second space 2 have three types: enemy objects EO, a
bullet object BO, and a back wall BW. The image processing program
according to the present invention is not limited to the types of
the objects. In the image processing program according to the
present embodiment, objects are virtual physical bodies present in
the virtual space (the first space 1 and the second space 2). For
example, in the image processing program according to the present
embodiment, given objects, such as obstacle objects, may be
present.
[0370] <Examples of Forms of Display>
[0371] FIGS. 23 through 26 show examples of the game screen
displayed on the upper LCD 22. Descriptions are given below of
examples of the forms of display shown in the respective
figures.
[0372] First, a description is given of an aiming cursor AL, which
is displayed commonly in FIGS. 23 through 26. In FIGS. 23 through
26, the aiming cursor AL for a bullet object BO is displayed
commonly on the upper LCD 22, the bullet object BO fired in
accordance with an attack operation using the game apparatus 10
(e.g., pressing the button 14B (A button)). In the game according
to the present embodiment program, the aiming cursor AL is set so
as to be directed in a predetermined direction in accordance with
the program executed by the game apparatus 10.
[0373] For example, the aiming cursor AL is set so as to be fixed
in the direction of the line of sight of the virtual camera, i.e.,
at the center of the screen of the upper LCD 22. In this case, as
described above, in the present embodiment, the direction of the
line of sight of the virtual camera (the virtual camera in the
first drawing method or the virtual world drawing camera in the
second drawing method) is changed in accordance with the imaging
direction of the outer capturing section 23. Thus, the player can
change the direction of the aiming cursor AL in the virtual space
by changing the orientation of the game apparatus 10. Then, the
player performs an attack operation by, for example, pressing the
button 14B (A button) of the game apparatus 10 with the thumb of
the right hand holding the lower housing 11. With this, the player
fires the bullet object BO by the attack operation, to thereby
vanquish an enemy object EO and repair an opening present in the
boundary surface 3, in the game according to the present
embodiment.
[0374] Next, descriptions are given separately of FIGS. 23 through
26.
[0375] In FIG. 23, an enemy object EO present in the first space 1
and a camera image CI captured by the real camera built into the
game apparatus 10 are displayed on the upper LCD 22. The enemy
object EO is arbitrarily set.
[0376] The enemy object EO is, for example, an object obtained by
using, as a texture, an image (e.g., a photograph of a person's
face) stored in the data storage external memory 46 or the like of
the game apparatus 10, and attaching the image to a
three-dimensional polygon model of a predetermined shape (a polygon
model representing a three-dimensional shape of a human head) by a
predetermined method.
[0377] Further, in the present embodiment, the camera image CI
displayed on the upper LCD 22 is, as described above, a real-time
real world image captured by the real camera built into the game
apparatus 10. Alternatively, for example, the camera image CI may
be an image (e.g., a photograph of a landscape) stored in the data
storage external memory 46 or the like of the game apparatus
10.
[0378] In the state where the camera image CI is displayed on the
upper LCD 22, the enemy object EO can arbitrarily move. For
example, the enemy object EO present in the first space 1 can move
to the second space 2. FIG. 24 shows an example of the state where
the enemy object EO present in the first space 1 moves from the
first space 1 to the second space 2. In the example shown in FIG.
24, the enemy object EO present in the first space 1 moves to the
second space 2 by forming an opening in the boundary surface 3. The
enemy object EO having moved to the second space 2 is displayed as
a shadow (silhouette model) ES at a position in an unopen area in
the boundary surface 3, as viewed from the virtual camera. Further,
the second space 2 is viewed through the opening in the boundary
surface 3. That is, when an opening is present in the boundary
surface 3 in the field of view of the virtual camera, a part of an
image of the second space 2 is displayed through the opening, on
the upper LCD 22. The image of the second space 2 is specifically
objects present in the second space 2, such as the enemy object EO
and a back wall BW that are present in the second space 2. The
shadow ES represents the shadow of the enemy object EO. FIG. 27A
shows silhouette models of the shadow of the enemy object EO as
viewed from above. Further, FIG. 27B is an example of silhouette
models of the shadow of the enemy object EO. As shown in FIGS. 27A
and 27B, in the present embodiment, for the enemy object EO,
silhouette models are set to correspond to a plurality of
orientations. Specifically, the silhouette models are, for example,
eight planar polygons shown in FIG. 27A. The silhouette models
(eight planar polygons) are placed at the same position as that of
the enemy object EO, which is a substance model. Further, the
planar polygons have sizes included in the substance model (do not
protrude beyond the substance model). Further, to each planar
polygon, a texture is attached that is obtained by drawing a shadow
image of the enemy object EO as viewed in the normal direction of
the surface of the planar polygon. When the enemy object EO is
present behind an unopen area in the boundary surface 3, the shadow
ES is displayed by drawing the corresponding silhouette model.
[0379] It should be noted that all the eight planar polygons are
rendered. When the enemy object EO is present behind an unopen area
in the boundary surface 3, the substance model of the enemy object
EO is hidden by the boundary surface (screen object) 3 based on a
depth determination, and therefore is not drawn. It is set,
however, such that a silhouette model is not subjected to a depth
determination with the boundary surface (screen object) 3, and
therefore, even when the enemy object EO (and its silhouette model)
is present behind an unopen area in the boundary surface 3, the
silhouette model is drawn, and the shadow is displayed as shown in
FIGS. 24 and 25. However, when the enemy object EO is present in
front of the boundary surface 3, or is present behind an open area
in the boundary surface 3, the silhouette model is present in the
back of the substance model, and therefore, the substance model of
the enemy object EO is drawn. Accordingly, the silhouette model is
not drawn, and therefore, the shadow is not displayed. This is
because the silhouette model is set to be included in the substance
model of the enemy object EO.
[0380] On the upper LCD 22, display is performed such that images
are combined together in the following preference order.
[0381] (1) An image of an object present in the first space 1; (2)
in an unopen area in the real world image, a combined image of a
shadow image of an object present in the second space 2 and the
real world image (e.g., a semi-transparent shadow image is combined
with the real world image); and (3) in an open area in the real
world image, an image (substance image) of an object present in the
second space 2 is preferentially combined, and a back wall image is
combined in the back of the image. Depending on the state of the
movement of the enemy object EO present in the second space 2,
however, there may be a scene where the enemy object EO is present
across an open area and an unopen area. That is, there may be a
scene where the enemy object EO is present on the edge of an
opening, as viewed from the virtual camera. FIG. 25 shows such a
state where the enemy object EO present in the second space 2 has
moved to the edge of an opening set in the boundary surface 3. As
shown in FIG. 25, the enemy object EO present in the second space 2
is displayed on the upper LCD 22 such that: an image of the enemy
object EO is displayed as it is in the region of the second space 2
that can be viewed through the opening, as viewed from the virtual
camera; and the shadow ES is displayed in the region of the second
space 2 that cannot be viewed through the opening, as viewed from
the virtual camera.
[0382] More specifically, as shown in FIG. 28, in each object, data
of a non-transparency (alpha value) is set. FIG. 28 shows an
example of the non-transparencies (alpha values) set for objects in
the present embodiment. For a texture of the substance model of an
enemy object, a non-transparency of 1 is set in the entire model.
Further, for a texture of each of the silhouette models (planar
polygons) of the enemy object, a non-transparency of 1 is set in
the entire shadow image. A bullet model is set in a similar manner.
Among enemy objects, for a texture of each of a semi-transparent
object model and an effect model, for example, a non-transparency
of 0.6 is set in the entire model. For a screen object (the
spherical screen model shown in FIG. 20A), a non-transparency of
0.2 is set as a material, and a non-transparency of 1 or 0 is set
on each point of the .alpha.-texture, which is a texture of the
screen object. "1" indicates an unopen portion, and "0" indicates
an open portion. That is, for the screen object, two types of
settings: that of the material and that of the texture, are made as
non-transparency values.
[0383] In addition, a depth determinations is valid between each
pair of: an enemy object; a bullet object; a semi-transparent enemy
object, an effect object, and the screen object. A depth
determination is valid "between the shadow planar polygon and the
enemy object", "between the shadow planar polygon and the bullet
object", "between the shadow planar polygon and the
semi-transparent enemy object", and "between the shadow planar
polygon and the effect object". A depth determination is invalid
between the shadow planar polygon and the screen object.
[0384] When a depth determination is valid, rendering is performed
in accordance with a normal perspective projection. A hidden
surface is removed in accordance with the depth direction from the
virtual camera. When a depth determination is invalid, an object is
rendered even if a target object is present in an area closer to
the virtual camera than that of the object.
[0385] Then, in the present embodiment, when rendering is
performed, it is possible to set a formula for rendering on the
object-by-object basis. Specifically, the formulas are set as
follows.
[0386] The substance of the enemy object, the bullet object, the
semi-transparent enemy object, and the effect object are drawn by
the following formula.
"color of object.times.non-transparency of object+color of
background.times.(1-non-transparency of object)"
[0387] The screen object is drawn by the following formula.
"color of object (color of real world image).times.non-transparency
of texture of object+color of background.times.(1-non-transparency
of texture of object)"
[0388] The silhouette model of the enemy object is drawn by the
following formula.
"color of object.times.(1-non-transparency of material of
background)+color of background.times.non-transparency of material
of background)"
[0389] It should be noted that when the enemy object is drawn, the
background of the enemy object is the screen object (boundary
surface 3), and therefore, in the above formula, "non-transparency
of material of background" is "non-transparency of material of
screen object (boundary surface 3)".
[0390] Based on the various settings as described above, when the
enemy object is present behind an unopen portion in the boundary
surface, not the substance but the shadow of the enemy object is
displayed. When the enemy object is present in front of the
boundary surface, or when the enemy object is present in an opening
in the boundary surface, not the shadow but the substance of the
enemy object is displayed.
[0391] In addition, in the game according to the present
embodiment, an opening present in the boundary surface 3 can be
repaired by hitting it with the bullet object BO. FIG. 26 shows the
state where an opening present in the boundary surface 3 is closed
by hitting it with the bullet object BO. As shown in FIG. 26, when,
for example, the bullet object BO has collided with an unopen area
present in the boundary surface 3, data of the unopen state is set
for a boundary surface present in a certain range from the
collision point. With this, when an opening is present in a certain
range from a collision point, the opening is closed. It should be
noted that in the present embodiment, the bullet object BO having
collided with the opening disappears (thus, the bullet object BO
has disappeared in FIG. 26). Further, when having collided with an
opening in the boundary surface 3, the bullet object BO moves to
the second space by passing through the opening.
[0392] It should be noted that, as described above, on the upper
LCD 22, the real-time real world image captured by the real camera
built into the game apparatus 10 is displayed as an image such that
the real-time real world image seems to be present on the boundary
surface 3. A change in the direction of the game apparatus 10 in
real space also changes the imaging range captured by the game
apparatus 10, and therefore also changes the camera image CI
displayed on the upper LCD 22. In this case, the game apparatus 10
changes the position and the direction of the virtual camera (the
virtual world drawing camera in the second drawing method) in the
virtual space in accordance with the motion of the game apparatus
10 in real space. With this, the enemy object EO displayed as if
placed in real space and an opening present in the boundary surface
3 are displayed as if placed at the same positions in real space
even when the direction of the game apparatus 10 has changed in
real space. For example, it is assumed that the imaging direction
of the real camera of the game apparatus 10 is turned left. In this
case, the display position of the enemy object EO displayed on the
upper LCD 22 and the opening present in the boundary surface 3 move
in the direction opposite to the turn in the imaging direction of
the real camera (in the right direction), that is, the direction of
the virtual camera (the virtual world drawing camera in the second
drawing method) in the virtual space, where the enemy object EO and
the opening present in the boundary surface 3 are placed, moves to
the left as does that of the real camera. Thus, even when a change
in the direction of the game apparatus 10 also changes the imaging
range of the real camera, the enemy object EO and the opening
present in the boundary surface 3 are displayed on the upper LCD 22
as if placed in a real space represented by the camera image
CI.
[0393] <<Examples of Operations of Image
Processing>>
[0394] Next, with reference to FIGS. 29 through 31 and FIGS. 32A
and 32B, descriptions are given of examples of specific processing
operations performed by the image processing program according to
the present embodiment executed by the game apparatus 10. FIG. 29
is a flow chart showing an example of the operation of image
processing performed by the game apparatus 10 executing the image
processing program. FIG. 30 is a subroutine flow chart showing an
example of a detailed operation of an enemy-object-related process
performed in step 53 of FIG. 29. FIG. 31 is a subroutine flow chart
showing an example of a detailed operation of a
bullet-object-related process performed in step 54 of FIG. 29.
FIGS. 32A and 32B are each a subroutine flow chart showing an
example of a detailed operation of a display image updating process
(the first drawing method and the second drawing method) performed
in step 57 of FIG. 29.
[0395] <<Example of Image Processing>>
[0396] With reference to FIG. 29, a description is given of the
operation of the information processing section 31. First, when the
power (the power button 14F) of the game apparatus 10 has been
turned on, the CPU 311 executes a boot program (not shown). This
causes the programs stored in the built-in memory, the external
memory 45, or the data storage external memory 46, to be loaded
into the main memory 32. In accordance with the execution of the
loaded programs by the information processing section 31 (the CPU
311), the steps shown in FIG. 29 are performed. It should be noted
that in FIGS. 29 through 32A, processes not directly related to the
present invention and peripheral processes are not described.
[0397] Referring to FIG. 29, the information processing section 31
performs the initialization of the image processing (step 51), and
proceeds to the subsequent step. For example, the information
processing section 31 sets the initial position and the initial
direction of the virtual camera for generating a virtual world
image (an image of the virtual space) in the virtual camera data
Dj, and sets the coordinate axes (e.g., X, Y, and Z axes) of the
virtual space where the virtual camera is placed. Subsequently, the
information processing section 31 acquires various data from each
component of the game apparatus 10 (step 52), and proceeds to the
subsequent step 53. For example, the information processing section
31 updates the real camera image data Db using a camera image
captured by the currently selected capturing section (the outer
capturing section 23 in the present embodiment). For example, the
information processing section 31 acquires data indicating that the
operation button 14 or the analog stick 15 has been operated, to
thereby update the controller data Da1. Further, the information
processing section 31 acquires angular velocity data indicating the
angular velocities detected by the angular velocity sensor 40, to
thereby update the angular velocity data Da2.
[0398] Next, the information processing section 31 performs an
enemy-object-related process (step 53), and proceeds to the
subsequent step 54. With reference to FIG. 30, the
enemy-object-related process is described below.
[0399] Referring to FIG. 30, the information processing section 31
determines whether or not conditions for the appearance of an enemy
object EO have been satisfied (step 61). For example, the
conditions for the appearance of an enemy object EO may be: that
the enemy object EO appears at predetermined time intervals; that
in accordance with the disappearance of the enemy object EO from
the virtual world, a new enemy object EO appears; or that the enemy
object EO appears at a random time. It should be noted that the
conditions for the appearance of an enemy object EO is, for
example, set by the group of various programs Pa stored in the main
memory 32.
[0400] Then, when the conditions for the appearance for an enemy
object EO have been satisfied, the information processing section
31 proceeds to the subsequent step 62. On the other hand, when the
conditions for the appearance of an enemy object EO have not been
satisfied, the information processing section 31 proceeds to the
subsequent step 63.
[0401] In step 62, the information processing section 31 generates
and initializes the enemy object data Df corresponding to the enemy
object EO that has satisfied the conditions for the appearance, and
proceeds to the subsequent step 63. For example, the information
processing section 31 acquires the substance data Df1, the
silhouette data Df2, the opening shape data Df3, and data of
polygons corresponding to the enemy object EO, using the group of
various programs Pa stored in the main memory 32. The information
processing section 31 generates the enemy object data Df including
the above items of data. Further, for example, the information
processing section 31 initializes: data indicating the placement
direction and the placement position of the polygons corresponding
to the enemy object EO in the virtual space; and data indicating
the moving velocity and the moving direction of the enemy object EO
in the virtual space, the data included in the generated enemy
object data Df. The initialization is made by a known method.
[0402] Next, the information processing section 31 moves the enemy
object EO placed in the virtual space (step 63), and proceeds to
the subsequent step 64. As an example, the information processing
section 31 updates data indicating the placement position of the
enemy object EO, the data included in the enemy object data Df,
based on the data indicating the moving velocity and the moving
direction of the enemy object EO in the virtual space, the data
included in the enemy object data Df. At this time, the information
processing section 31 updates the data indicating the placement
direction of the enemy object EO, the data included in the enemy
object data Df, based on the data indicating the moving direction.
After the update, the information processing section 31 may update
the data indicating the moving velocity and the moving direction of
the enemy object EO in the virtual space, the data included in the
enemy object data Df. The update of the data indicating the moving
velocity and the moving direction allows the enemy object EO to
move in the virtual space at a given velocity in a given
direction.
[0403] Next, the information processing section 31 determines
whether or not the enemy object EO has reached a certain distance
from the position of the virtual camera (the virtual camera in the
first drawing method or the virtual world drawing camera in the
second drawing method) (step 64). For example, the information
processing section 31 compares the data indicating the placement
position of the enemy object EO, the data included in the enemy
object data Df, with data indicating the placement position of the
virtual camera (the virtual camera in the first drawing method or
the virtual world drawing camera in the second drawing method), the
data included in the virtual camera data Dj. Then, when the two
items of data have satisfied predetermined conditions (e.g., the
distance between the placement position of the enemy object EO and
the placement position of the virtual camera has fallen below a
predetermined value), the information processing section 31
determines that the enemy object EO has reached the certain
distance from the position of the virtual camera, and when the two
items of data have not satisfied the predetermined conditions, the
information processing section 31 determines that the enemy object
EO has not reached the certain distance from the position of the
virtual camera. It should be noted that hereinafter, when the term
"virtual camera" is simply used without distinguishing between the
first drawing method and the second drawing method, the "virtual
camera" refers to the virtual camera in the first drawing method or
the virtual world drawing camera in the second drawing method. When
it is determined that the enemy object EO has reached the certain
distance from the position of the virtual camera, the information
processing section 31 proceeds to the subsequent step 65. On the
other hand, when it is determined that the enemy object EO has not
reached the certain distance from the position of the virtual
camera, the information processing section 31 proceeds to step
66.
[0404] In step 65, the information processing section 31 performs a
point deduction process, and proceeds to the subsequent step 66.
For example, the information processing section 31 deducts a
predetermined value from the score of the game indicated by the
score data Dh, to thereby update the score data Dh using the score
after the deduction. It should be noted that in the point deduction
process, the information processing section 31 may perform a
process of causing the enemy object EO having reached the certain
distance from the position of the virtual camera, to disappear from
the virtual space (e.g., initializing the enemy object data Df
concerning the enemy object EO having reached the certain distance
from the position of the virtual camera, such that the enemy object
EO is not present in the virtual space). Further, the predetermined
value in the point deduction process may be a given value, and for
example, may be set by the group of various programs Pa stored in
the main memory 32.
[0405] In step 66, the information processing section 31 determines
whether or not the enemy object EO is to pass through the boundary
surface 3 (the enemy object EO is to move between the first space 1
and the second space 2). For example, the information processing
section 31 compares the data indicating the placement position of
the enemy object EO, the data included in the enemy object data Df,
with the data indicating the placement position of the boundary
surface 3, the data included in the boundary surface data Dd. Then,
when the two items of data have satisfied predetermined conditions,
the information processing section 31 determines that the enemy
object EO is to pass through the boundary surface 3. When the two
items of data have not satisfied the predetermined conditions, the
information processing section 31 determines that the enemy object
EO is not to pass through the boundary surface 3. It should be
noted that the predetermined conditions are, for example, that the
coordinates (placement position) of the enemy object EO in the
virtual space satisfy conditional equations for the spherical
surface of the boundary surface 3. As described above, the data
indicating the placement position of the boundary surface 3 in the
virtual space indicates the existence range of the boundary surface
3 in the virtual space, and is, for example, conditional equations
for the spherical surface (the shape of the boundary surface 3
according to the present embodiment). When the placement position
of the enemy object EO satisfies the conditional equations, the
enemy object EO is present on the boundary surface 3 in the virtual
space. In the present embodiment, for example, in such a case, it
is determined that the enemy object EO is to pass through the
boundary surface 3.
[0406] When it is determined that the enemy object EO is to pass
through the boundary surface 3, the information processing section
31 proceeds to the subsequent step 67. On the other hand, when it
is determined that the enemy object EO is not to pass through the
boundary surface 3, the information processing section 31 ends the
process of this subroutine.
[0407] In step 67, the information processing section 31 performs a
process of updating the opening determination data included in the
boundary surface data Dd, and ends the process of the subroutine.
This process is a process for registering, in the boundary surface
data Dd, information of an opening produced in the boundary surface
3 by the enemy object EO passing through the boundary surface 3.
For example, in the first drawing method and the second drawing
method, the information processing section 31 multiplies: the alpha
values of the opening determination data of an area having its
center at a position corresponding to the position where the enemy
object EO passes through the boundary surface 3 in the virtual
space, the opening determination data included in the boundary
surface data Dd; by the alpha values of the opening shape data Df3.
The opening shape data Df3 is texture data in which alpha values of
"0" are stored and which has its center at the placement position
of the enemy object EO. Accordingly, based on the multiplication,
the alpha values of the opening determination data of the area
where the opening is generated so as to have its center at the
placement position of the enemy object EO (the coordinates of the
position where the enemy object EO passes through the boundary
surface 3) are "0". That is, the information processing section 31
can update the state of the boundary surface (specifically, the
opening determination data) without determining whether or not an
opening is already present in the boundary surface 3. It should be
noted that it may be determined whether or not an opening is
already present at the position of the collision between the enemy
object and the boundary surface. Then, when an opening is not
present, an effect may be displayed such that a real world image
corresponding to the collision position flies as fragments.
[0408] In addition, in the updating process of the opening
determination data, the information processing section 31 may
perform a process of staging the generation of the opening (e.g.,
causing a wall to collapse at the position where the opening is
generated). In this case, the information processing section 31
needs to determine whether or not the position where the enemy
object EO passes through the boundary surface 3 (the range where
the opening is to be generated) has already been open. The
information processing section 31 can determine whether or not the
range where the opening is to be generated has already been open,
by, for example, multiplying: data obtained by inverting the alpha
values of the opening shape data Df3 from "0" to "1"; by the alpha
values of the opening determination data multiplied as described
above. That is, when the entire range where the opening is to be
generated has already been open, the alpha values of the opening
determination data are "0". Thus, the multiplication results are
"0". On the other hand, when even a part of the range where the
opening is to be generated is not open, there is a part where the
alpha values of the opening determination data are not "0". Thus,
the multiplication results are other than "0".
[0409] It should be noted that the opening shape data Df3 of the
enemy object EO is texture data in which alpha values of "0" are
stored so as to correspond to the shape of the enemy object EO. The
information processing section 31 may convert the alpha values of
the texture data into "1", based on a predetermined event. When the
above process is performed after the conversion, the alpha values
of the opening shape data Df3 are "1". Thus, the alpha values of
the opening determination data are not changed. In this case, the
enemy object EO passes through the boundary surface 3 without
forming an opening. That is, this makes it possible to stage the
enemy object EO as if it slips through the boundary surface 3 (see
FIG. 22). It should be noted that the predetermined event, for
example, may be time intervals defined by random numbers or
predetermined intervals, or may be the satisfaction of
predetermined conditions in the game. These events may be, for
example, set by the group of various programs Pa stored in the main
memory 32.
[0410] Referring back to FIG. 29, after the enemy-object-related
process in step 53, the information processing section 31 performs
a bullet-object-related process (step 54), and proceeds to the
subsequent step 55. With reference to FIG. 31, the
bullet-object-related process is described below.
[0411] Referring to FIG. 31, the information processing section 31
moves a bullet object BO in the virtual space in accordance with a
moving velocity vector that is set (step 71), and proceeds to the
subsequent step 72. For example, the information processing section
31 updates data indicating the placement direction and the
placement position of the bullet object BO, based on data
indicating the moving velocity vector, the data included in the
bullet object data Dg. At this time, the information processing
section 31 may update the data indicating the moving velocity
vector, by a known method. Further, for example, depending on the
type of the bullet object BO, the information processing section 31
may change the method of updating the data indicating the moving
velocity vector. For example, when the bullet object BO is a ball,
the information processing section 31 may update the data
indicating the moving velocity vector, taking into account the
effect of a gravity in the vertical direction in the virtual
space.
[0412] Next, the information processing section 31 determines
whether or not the user of the game apparatus 10 has performed a
firing operation (step 72). For example, with reference to the
controller data Da1, the information processing section 31
determines whether or not the user has performed a predetermined
firing operation (e.g., pressing the button 14B (A button)). When
the firing operation has been performed, the information processing
section 31 proceeds to the subsequent step 73. On the other hand,
when the firing operation has not been performed, the information
processing section 31 proceeds to the subsequent step 74.
[0413] In step 73, in accordance with the firing operation, the
information processing section 31 places the bullet object BO at
the position of the virtual camera in the virtual space, sets the
moving velocity vector of the bullet object BO, and proceeds to the
subsequent step 74. For example, the information processing section
31 generates the bullet object data Dg corresponding to the firing
operation. Then, for example, the information processing section 31
stores the data indicating the placement position and the placement
direction (the direction of the line of sight) of the virtual
camera, the data included in the virtual camera data Dj, in the
data indicating the placement position and the placement direction
of the bullet object BO, the data included in the generated bullet
object data Dg. Further, for example, the information processing
section 31 stores a given value in the data indicating the moving
velocity vector, the data included in the generated bullet object
data Dg. The value to be stored in the data indicating the moving
velocity vector may be set by the group of various programs Pa
stored in the main memory 32.
[0414] In step 74, the information processing section 31 determines
whether or not the enemy object EO and the bullet object BO have
made contact with each other in the virtual space. For example, by
comparing the data indicating the placement position of the enemy
object EO, the data included in the enemy object data Df, with the
data indicating the placement position of the bullet object BO, the
data included in the bullet object data Dg, the information
processing section 31 determines whether or not the enemy object EO
and the bullet object BO have made contact with each other in the
virtual space. For example, when the data indicating the placement
position of the enemy object EO and the data indicating the
placement position of the bullet object BO have satisfied
predetermined conditions, the information processing section 31
determines that the enemy object EO and the bullet object BO have
made contact with each other. If not, the information processing
section 31 determines that the enemy object EO and the bullet
object BO have not made contact with each other. It should be noted
that the predetermined conditions are, for example, that the
distance between the placement position of the enemy object EO and
the placement position of the bullet object BO falls below a
predetermined value. The predetermined value may be, for example, a
value based on the size of the enemy object EO.
[0415] When it is determined that the enemy object EO and the
bullet object BO have made contact with each other, the information
processing section 31 proceeds to the subsequent step 75. On the
other hand, when it is determined that the enemy object EO and the
bullet object BO have not made contact with each other, the
information processing section 31 proceeds to the subsequent step
76.
[0416] In step 75, the information processing section 31 performs a
point addition process, and proceeds to the subsequent step 76. For
example, in the point addition process, the information processing
section 31 adds predetermined points to the score of the game
indicated by the score data Dh, to thereby update the score data Dh
using the score after the addition. Further, in the point addition
process, the information processing section 31 performs a process
of causing both objects having made contact with each other based
on the determination in step 84 described above (i.e., the enemy
object EO and the bullet object BO), to disappear from the virtual
space (e.g., initializing the enemy object data Df concerning the
enemy object EO having made contact with the bullet object BO, and
the bullet object data Dg concerning the bullet object BO having
made contact with the enemy object EO, such that the enemy object
EO and the bullet object BO are not present in the virtual space).
It should be noted that the predetermined points in the point
addition process may be a given value, and may be, for example, set
by the group of various programs Pa stored in the main memory
32.
[0417] In step 76, the information processing section 31 determines
whether or not the bullet object BO has made contact with an unopen
area in the boundary surface 3. For example, using the placement
position of the bullet object BO included in the bullet object data
Dg and the opening determination data, the information processing
section 31 determines whether or not the bullet object BO has made
contact with an unopen area in the boundary surface 3.
[0418] For example, the information processing section 31
determines whether or not the data indicating the placement
position of the bullet object BO, the data included in the bullet
object data Dg, satisfies conditional equations for the spherical
surface of the boundary surface 3, as in the process of the enemy
object EO. Then, when the data indicating the placement position of
the bullet object BO does not satisfy the conditional equations for
the spherical surface, the information processing section 31
determines that the bullet object BO has not made contact with the
boundary surface 3. On the other hand, when the data indicating the
placement position of the bullet object BO satisfies the
conditional equations for the spherical surface of the boundary
surface 3, the bullet object BO is present on the boundary surface
3 in the virtual space. At this time, the information processing
section 31, for example, acquires the alpha values of the opening
determination data of a predetermined area having its center at a
position corresponding to the position where the bullet object BO
is present on the boundary surface 3. The predetermined area is a
predetermined area having its center at the contact point of the
bullet object BO and the boundary surface 3. Then, when the alpha
values of the opening determination data corresponding to at least
a part of the predetermined area are alpha values of "1", which
correspond to an unopen area, the information processing section 31
determines that the bullet object BO has made contact with an
unopen area in the boundary surface 3.
[0419] When it is determined that the bullet object BO has made
contact with an unopen area in the boundary surface 3, the
information processing section 31 proceeds to the subsequent step
77. On the other hand, when it is determined that the bullet object
BO has not made contact with an unopen area in the boundary surface
3, the information processing section 31 proceeds to the subsequent
step 78.
[0420] In step 77, the information processing section 31 performs a
process of updating the opening determination data, and proceeds to
the subsequent step 78. For example, in the updating process, the
information processing section 31 updates, in the boundary surface
3, the alpha values of the opening determination data of the
predetermined area having its center at the position corresponding
to the placement position of the bullet object BO that has made
contact with the unopen area in the boundary surface 3 based on the
determination, to alpha values of "1", which correspond to an
unopen area. When the bullet object BO has made contact with the
unopen area by this updating process, all the alpha values of the
opening determination data in a predetermined area having its
center at the contact point are updated to "1". Accordingly, when
there is a part where the alpha values of the opening determination
data are set to "0" in the predetermined area having its center at
the contact point, the alpha values of the opening determination
data of this part are also updated to "1". That is, when the bullet
object BO has made contact with the edge of an opening provided in
the boundary surface 3, the opening included in a predetermined
area having its center at the position of the contact is repaired
to the state of being unopen. Further, in the updating process, the
information processing section 31 performs a process of causing the
bullet object BO having made contact based on the determination in
step 76, to disappear from the virtual space (e.g., initializing
the bullet object data Dg concerning the bullet object BO having
made contact with the unopen area in the boundary surface 3, such
that the bullet object BO is not present in the virtual space). It
should be noted that the predetermined area used in the updating
process may be a given area, and may be, for example, set by the
group of various programs Pa stored in the main memory 32.
[0421] In step 78, the information processing section 31 determines
whether or not the bullet object BO has reached a predetermined
position in the virtual space. The predetermined position may be,
for example, the position where a back wall BW is present in the
virtual space. In this case, for example, the information
processing section 31 determines whether or not the data indicating
the placement position of the bullet object BO, the data included
in the bullet object data Dg, indicates that the bullet object BO
has collided with the back wall BW.
[0422] Then, when the bullet object BO has reached the
predetermined position, the information processing section 31
proceeds to the subsequent step 77. On the other hand, when the
bullet object BO has not reached the predetermined position, the
information processing section 31 ends the process of this
subroutine.
[0423] In step 77, the information processing section 31 performs a
process of causing the bullet object BO having reached the
predetermined position based on the determination in step 76
described above, to disappear from the virtual space, and ends the
process of the subroutine. For example, the information processing
section 31 performs a process of causing the bullet object BO
having reached the predetermined position based on the
determination in step 76 described above, to disappear from the
virtual space (e.g., initializing the bullet object data Dg
concerning the bullet object BO such that the bullet object BO is
not present in the virtual space).
[0424] Referring back to FIG. 29, after the bullet-object-related
process in step 54 described above, the information processing
section 31 calculates the motion of the game apparatus 10 (step
55), and proceeds to the subsequent step 56. As an example, the
information processing section 31 calculates the motion of the game
apparatus 10 (e.g., a change in the imaging direction of the real
camera provided in the game apparatus 10) using the angular
velocities indicated by the angular velocity data Da2, to thereby
update the motion data Di using the calculated motion.
Specifically, when the user has changed in real space the imaging
direction of the real camera provided in the game apparatus 10, the
orientation of the entire game apparatus 10 also changes, and
therefore, angular velocities corresponding to the change are
generated in the game apparatus 10. Then, the angular velocity
sensor 40 detects the angular velocities generated in the game
apparatus 10, whereby data indicating the angular velocities is
stored in the angular velocity data Da2. Thus, using the angular
velocities indicated by the angular velocity data Da2, the
information processing section 31 can calculate the direction and
the amount (angle) that have changed in the imaging direction of
the real camera provided in the game apparatus 10, as the motion of
the game apparatus 10.
[0425] Next, in accordance with the motion of the game apparatus
10, the information processing section 31 changes the position of
the virtual camera in the virtual space (step 56), and proceeds to
the subsequent step 57. For example, using the motion data Di, the
information processing section 31 imparts the same changes as those
in the imaging direction of the real camera of the game apparatus
10 in real space, to the virtual camera in the virtual space, to
thereby update the virtual camera data Dj using the position and
the direction of the virtual camera after the changes. As an
example, if the imaging direction of the real camera of the game
apparatus 10 in real space has turned left by A.degree., the
direction of the virtual camera in the virtual space also turns
left by A.degree.. With this, the enemy object EO and the bullet
object BO displayed as if placed in real space are displayed as if
placed at the same positions in real space even when the direction
and the position of the game apparatus 10 have changed in real
space.
[0426] Next, the information processing section 31 performs a
process of updating the display image (step 57), and proceeds to
the subsequent step 58. With reference to FIGS. 32A and 32B, the
display image updating process is described below. FIG. 32A is the
display image updating process in the first drawing method.
Further, FIG. 32B is the display image updating process in the
second drawing method.
[0427] First, a description is given of the display image updating
process in the first drawing method.
[0428] Referring to FIG. 32A, the information processing section 31
performs a process of attaching the real camera image acquired in
step 52 to the screen object (boundary surface 3) included in the
viewing volume of the virtual camera (step 81), and proceeds to the
subsequent step 82. For example, the information processing section
31 updates the texture data of the real camera image included in
the real world image data Dc, using the real camera image data Db
updated in step 52. Then, the information processing section 31
obtains the point where the direction of the line of sight of the
virtual camera overlaps the boundary surface 3, using the data
indicating the placement direction and the placement position of
the virtual camera in the virtual space, the data included in the
virtual camera data Dj. The information processing section 31
attaches the texture data of the real camera image included in the
real world image data Dc, such that the obtained point is the
center, to thereby update the boundary surface data Dd. At this
time, the information processing section 31 acquires the opening
determination data set for the area to which the texture data is
attached, such that the opening determination data corresponds to
the area corresponding to all the pixels of the texture data. Then,
the information processing section 31 applies to the texture data
the alpha values ("0" or "0.2") set in the acquired opening
determination data. Specifically, the information processing
section 31 multiplies: color information of all the pixels of the
texture data of the real camera image to be attached; by the alpha
values at the corresponding positions of the opening determination
data. By this process, an opening is represented in the real world
image as described above. It should be noted that in the
multiplication, an alpha value of "0.2" (an unopen area) stored in
the opening determination data is handled as an alpha value of "1"
set as the material described above. Further, in the present
embodiment, the texture data of the real camera image to be
attached to the boundary surface 3 is image data of an area that is
wider than the field of view of a virtual camera C0.
[0429] Next, the information processing section 31 generates a
display image by a process of rendering the virtual space (step
82), and ends the process of this subroutine. For example, the
information processing section 31 generates an image obtained by
rendering the virtual space where the boundary surface 3 (screen
object), the enemy object EO, the bullet object BO, and the back
wall BW are placed, to thereby update the rendered image data of
the virtual space using the generated image, the rendered image
data included in the rendered image data Dk. Further, the
information processing section 31 updates the display image data Dl
using the rendered image data of the virtual space. With reference
to FIGS. 33 and 34, an example of the rendering process is
described below.
[0430] FIG. 33 shows an example of the placement of the enemy
object EO, the bullet object BO, the boundary surface 3 (the screen
object in which the opening determination data is set), and the
back wall BW in the virtual space. Further, FIG. 34 shows the
positional relationships between the objects on the assumption that
the virtual camera C0 in FIG. 33 is directed in the direction of
(X, Y, Z)=(0, 0, 1) from the origin. As described above, the enemy
object EO, the bullet object BO, the boundary surface 3, and the
back wall BW are each placed in accordance with the data indicating
the placement position included in the corresponding one of the
enemy object data Df, the bullet object data Dg, the boundary
surface data Dd, and the back wall image data De. Further, in the
virtual space, the virtual camera C0 for rendering the virtual
space is placed in accordance with the data indicating the
placement direction and the placement position, the data included
in the virtual camera data Dj.
[0431] As shown in FIG. 33 (or FIG. 34), the information processing
section 31 renders with a perspective projection from the virtual
camera C0 the enemy object EO, the bullet object BO, and the back
wall BW that are placed in the virtual space, so as to include the
boundary surface 3. At this time, the information processing
section 31 takes into account the information about the priority of
drawing. In a normal perspective projection, an object present in
the second space 2 is not drawn due to the presence of the boundary
surface 3. In the game according to the present embodiment, an
opening is provided in the boundary surface 3 (real world image),
so that a part of the second space 2 can be viewed through the
opening. Further, the shadow of the object present in the second
space 2 is drawn in combination with the real world image. This
makes it possible to give the user a feeling as if the virtual
world further exists beyond the real world image. Specifically, the
information processing section 31 performs the rendering process
using the information about the priority of drawing. It should be
noted that in the image processing program according to the present
embodiment, alpha values are used as an example of the priority of
drawing.
[0432] In the perspective projection described above, the object
present in the second space 2 (the enemy object EO or the back wall
BW in the present embodiment) is present behind the boundary
surface 3. Here, the boundary surface 3 is the screen object to
which the texture data of the real camera image is applied in the
direction of the field of view (the range of the field of view) of
the virtual camera C0 in step 81 described above. Further, as
described above, to the texture data of the real camera image, the
opening determination data corresponding to each position is
applied. Accordingly, in the range of the field of view of the
virtual camera C0, the real world image to which the opening
determination data is applied is present.
[0433] It should be noted that in the present embodiment, for
example, in an area having the opening determination data in which
alpha values of "0" are stored (an open area), the information
processing section 31 draws (renders) images of a virtual object
and the back wall BW that are present in the second space 2, in an
area that can be viewed through the open area. Further, in an area
having the opening determination data in which alpha values of
"0.2", which correspond to an unopen area, are stored (an area
handled as an area where alpha values of "1" are stored as an
unopen area), the information processing section 31 does not draw
the virtual object and the back wall BW that are present in the
second space 2. That is, in the image to be displayed, the real
world image attached in step 81 described above is drawn in the
portion corresponding to this area.
[0434] Therefore, in an area having the opening determination data
in which "0" is stored as viewed from the virtual camera C0,
rendering is performed such that image data included in the
substance data Df1 or the back wall image data De is drawn. Then,
on the upper LCD 22, images of the virtual object and the back wall
BW are displayed in the portion corresponding to this area.
[0435] In addition, in an area having the opening determination
data in which alpha values of "0.2", which indicate an unopen area,
are stored as viewed from the virtual camera C0 (an area handled as
an area where alpha values of "1" are stored as an unopen area),
the virtual object and the back wall BW that are present in the
second space 2 are not drawn. That is, in the image to be displayed
on the upper LCD 22, the real world image is drawn in the portion
corresponding to this area. For the shadow ES (silhouette model) of
the enemy object EO present in the second space 2 described above,
however, a depth determination is set to invalid between the
boundary surface 3 and the shadow ES. Accordingly, alpha values of
"1" of the silhouette model are greater than alpha values of "0.2"
of the boundary surface 3, and therefore, the shadow ES is drawn in
an area where alpha values of "1", which indicate an unopen area,
are stored (an area having the opening determination data in which
alpha values of "0.2" are stored). With this, an image of the
shadow ES is drawn on the real world image.
[0436] In addition, when the enemy object EO is present in the
first space 1 such that the shadow ES (silhouette model) of the
enemy object EO has a size included in the substance model and is
placed in such a manner, and such that a depth determination is set
to valid between the substance model of the enemy object EO and the
silhouette model of the shadow ES, the silhouette model is hidden
by the substance model, and therefore is not drawn.
[0437] It should be noted that in the present embodiment, as shown
in FIG. 20A, the shape of the boundary surface 3 is a central
portion of a spherical surface, and therefore, the opening
determination data may not be present depending on the direction of
the line of sight of the virtual camera C0. In this case, the above
process is performed on the assumption that the opening
determination data is present in which alpha values of "0.2" are
stored in a simulated manner. That is, an area where the opening
determination data is not present is handled as an area where alpha
values of "1", which indicate an unopen area, are stored.
[0438] In addition, the silhouette data Df2 included in the enemy
object data Df corresponding to the enemy object EO according to
the present embodiment is set such that the normal directions of a
plurality of planar polygons correspond to radiation directions as
viewed from the enemy object EO, and to each planar polygon, a
texture of the silhouette image of the enemy object EO as viewed
from the corresponding direction is applied. Accordingly, in the
image processing program according to the present embodiment, the
shadow of the enemy object EO in the virtual space image is
represented as an image on which the orientation of the enemy
object EO in the second space 2 is reflected.
[0439] In addition, the information processing section 31 performs
the rendering process such that the image data included in the
aiming cursor image data Dm is preferentially drawn at the center
of the field of view of the virtual camera C0 (the center of the
image to be rendered).
[0440] By the above process, the information processing section 31
renders with a perspective projection the enemy object EO, the
bullet object BO, and the back wall BW that are placed in the
virtual space, and generates a virtual world image as viewed from
the virtual camera C0 (an image including the aiming cursor AL), to
thereby update the rendered image data of the virtual space (step
82). Then, the information processing section 31 updates the
display image data Dl, using the updated rendered image data of the
virtual space.
[0441] Next, a description is given of the display image updating
process in the second drawing method.
[0442] In FIG. 32B, the information processing section 31 performs
a process of rendering the real camera image acquired in step 52
described above (step 83), and proceeds to the subsequent step 84.
For example, the information processing section 31 updates the
texture data of the real camera image included in the real world
image data Dc using the real camera image Db updated in step 52
described above. The information processing section 31 generates an
image obtained by rendering the real camera image using the updated
real world image data Dc, to thereby update the rendered image data
of the real camera image using the generated image, the rendered
image data included in the rendered image data Dk. With reference
to FIGS. 35 and 36, a description is given below of an example of
the rendering process of the real camera image.
[0443] In the present embodiment, as shown in FIG. 35, the
information processing section 31 sets, as a texture, a real camera
image obtained from the real camera of the game apparatus 10, and
generates a planar polygon on which the texture is mapped. Then,
the information processing section 31 generates, as a real world
image, an image obtained by rendering the planar polygon with a
parallel projection from a real world image drawing camera C1.
Here, a description is given of an example of the method of
generating a real world image in the case where the entire real
camera image obtained from the real camera of the game apparatus 10
is displayed on the entire display screen of the upper LCD 22. It
should be noted that in the present embodiment, the combined image
according to the present embodiment (the combined image of a real
world image and a virtual world image) is displayed on the entire
display screen of the upper LCD 22. Alternatively, the combined
image may be displayed in a part of the display screen of the upper
LCD 22. In this case, the entire real camera image is displayed in
the entire combined image.
[0444] First, a planar polygon is considered, on which a texture
having i pixels is mapped in 1 unit of a coordinate system of the
virtual space where the planar polygon is placed. In this case, a
texture having i pixels.times.i pixels is mapped onto an area of 1
unit.times.1 unit of the coordinate system. Here, it is assumed
that the display screen of the upper LCD 22 has horizontal W
dots.times.vertical H dots, and the entire texture of the real
camera image corresponds to the entire display screen having W
dots.times.H dots. That is, it is assumed that the size of the
texture data of the camera image is horizontal W
pixels.times.vertical H pixels.
[0445] In this case, the planar polygon only needs to be placed
such that 1 dot.times.1 dot on the display screen corresponds to a
texture of 1 pixel.times.1 pixel in the real camera image, and the
above coordinate system only needs to be defined as shown in FIG.
36. That is, an XY coordinate system of the virtual space where the
planar polygon is placed is set such that the width of the planar
polygon, on the entire main surface of which the texture of the
camera image is mapped, corresponds to W/i units of the coordinate
system, and the height of the planar polygon corresponds to H/i
units of the coordinate system. The planar polygon is placed such
that when the center of the main surface of the planar polygon, on
which the texture is mapped, coincides with the origin of the XY
coordinate system of the virtual space, the horizontal direction of
the planar polygon corresponds to the X-axis direction (the right
direction is the X-axis positive direction), and the vertical
direction of the planar polygon corresponds to the Y-axis direction
(the up direction is the Y-axis positive direction). In this case,
in the main surface of the planar polygon, on which the texture is
mapped: the top right corner position is placed at (X, Y)=(W/2i,
H/2i); the bottom right corner position is placed at (X, Y)=(W/2i,
-H/2i); the top left corner position is placed at (X, Y)=(-W/2i,
H/2i); and the bottom left corner position is placed at (X,
Y)=(-W/2i, -H/2i).
[0446] With the arrangement as described above, an area of 1
unit.times.1 unit in the above coordinate system corresponds to an
area of i pixels.times.i pixels in the texture, and therefore, an
area of horizontal (W/i).times.vertical (H/i) in the planar polygon
corresponds to the size of W pixels.times.H pixels in the
texture.
[0447] As described above, the planar polygon placed in the
coordinate system of the virtual space is rendered with a parallel
projection such that 1 pixel in the real camera image (texture)
corresponds to 1 dot on the display screen. Thus, a real world
image is generated that corresponds to the camera image obtained
from the real camera of the game apparatus 10.
[0448] It should be noted that as described above, the texture data
of the real camera image included in the real world image data Dc
is updated by the real camera image data Db. There is, however, a
case where a size of horizontal A.times.vertical B of an image in
the real camera image data Db does not coincide with a size of
horizontal W.times.vertical H of the texture data. In this case,
the information processing section 31 updates the texture data by a
given method. For example, the information processing section 31
may update the texture data, using an image obtained by enlarging
or reducing the sizes of horizontal A and vertical B of the image
in the real camera image data Db so as to coincide with an image
having a size of W.times.H (an image of the texture data).
Alternatively, for example, it is assumed that the sizes of
horizontal A and vertical B of the image in the real camera image
data Db are greater than the sizes of horizontal W and vertical H
of the texture data, respectively. In this case, for example, the
information processing section 31 may update the texture data by
clipping an image having a size of W.times.H (an image of the
texture data) from a predetermined position in the image in the
real camera image data Db. Yet alternatively, for example, it is
assumed that at least one of the sizes of horizontal A and vertical
B of the image in the real camera image data Db is smaller than the
sizes of horizontal W and vertical H in the texture data. In this
case, for example, the information processing section 31 may update
the texture data by enlarging the image in the real camera image
data Db so as to excess the size of the texture data, and
subsequently clipping an image having a size of W.times.H (an image
of the texture data) from a predetermined position in the enlarged
image.
[0449] In addition, in the present embodiment, the
horizontal.times.vertical size of the display screen of the upper
LCD 22 coincides with the horizontal.times.vertical size of the
texture data in the real camera image; however, these sizes do not
need to coincide with each other. In this case, the size of the
display screen of the upper LCD 22 and the size of the real world
image do not coincide with each other. The information processing
section 31 may change the size of the real world image by a known
method when the real world image is displayed on the display screen
of the upper LCD 22.
[0450] Next, as shown in FIG. 32B, the information processing
section 31 performs a process of rendering the virtual space (step
84), and proceeds to the subsequent step 85. For example, the
information processing section 31 generates, taking the opening
determination data into account, an image obtained by rendering the
virtual space where the enemy object EO, the bullet object BO, and
the back wall BW are placed, to thereby update the rendered image
data of the virtual space using the generated image, the rendered
image data included in the rendered image data Dk. With reference
to FIGS. 37 through 39, an example of the rendering process is
described below.
[0451] FIG. 37 shows an example of the placement of the enemy
object EO, the bullet object BO, the boundary surface 3 (opening
determination data), and the back wall BW in the virtual space.
Further, FIG. 38 shows the positional relationships between the
objects on the assumption that the virtual camera (virtual world
drawing camera) in FIG. 37 is directed in the direction of (X, Y,
Z)=(0, 0, -1) from the origin. As described above, the enemy object
EO, the bullet object BO, the boundary surface 3, and the back wall
BW are each placed in accordance with the data indicating the
placement position included in the corresponding one of the enemy
object data Df, the bullet object data Dg, the boundary surface
data Dd, and the back wall image data De. Further, in the virtual
space, a virtual world drawing camera C2 for rendering the virtual
space is placed in accordance with the data indicating the
placement direction and the placement position, the data included
in the virtual camera data Dj.
[0452] Here, first, a description is given of the position of the
boundary surface 3 (opening determination data). As described
above, in the image processing program according to the present
embodiment, a real image in which an opening is provided is
generated by multiplying the opening determination data by color
information of the real world image (the rendered image data of the
real camera image). Accordingly, for example, 1 horizontal
coordinate unit.times.1 vertical coordinate unit in the rendered
image data of the real camera image (see the positional
relationships in the planar polygon in FIGS. 35 and 36) corresponds
to 1 horizontal coordinate unit.times.1 vertical coordinate unit of
the boundary surface 3 (specifically, the opening determination
data) in the virtual space. That is, it is assumed that when the
boundary surface 3 is viewed from the virtual world drawing camera
C2 shown in FIG. 37 or 38 with a perspective projection, the range
of the boundary surface 3 in the field of view of the virtual world
drawing camera C2 corresponds to the horizontal.times.vertical size
of the rendered image data of the real camera image.
[0453] FIG. 39 shows an example of the positional relationship
between the virtual world drawing camera C2 and the boundary
surface 3. The case is considered where the boundary surface 3 is
subjected to a perspective projection from the virtual world
drawing camera C2 directed in the direction of (X, Y, Z)=(0, 0, -1)
from the origin. In this case, if the boundary surface 3 is placed
at the position of Z=Z0 shown in FIG. 39, 1 horizontal coordinate
unit.times.1 vertical coordinate unit in the opening determination
data corresponds to 1 horizontal coordinate unit.times.1 vertical
coordinate unit in the rendered image data of the real camera
image. Here, the position of Z=Z0 is the position where, when the
angle of view in the Y-axis direction of the virtual world drawing
camera C2 that performs a perspective projection on boundary
surface 3 is .theta., the length between the fixation point of the
virtual world drawing camera C2 and the display range in the Y-axis
positive direction is H/2i. It should be noted that as described
above, "H" is the number of vertical dots on the display screen of
the upper LCD 22, and "i" is the number of pixels in the texture to
be mapped onto 1 unit of the coordinate system of the virtual
space. Then, if the distance between the center of the virtual
world drawing camera C2 and the position of Z=Z0 is D (D>0), the
following formula is obtained.
tan .theta.=(H/2i)/D=H/2Di
[0454] Thus, when a virtual world image is generated by performing
a perspective projection on the enemy object EO and the like
described later, taking the boundary surface 3 into account, the
settings of the virtual world drawing camera C2 for generating the
virtual world image are "the angle of view .theta. in the Y-axis
direction=tan.sup.-1 (H/2Di), and the aspect ratio=W:H". Then, the
boundary surface 3 (specifically, the opening determination data
indicating the state of the boundary surface 3) is placed at the
view coordinates of Z=Z0 from the virtual world drawing camera C2.
With this, the range of the boundary surface 3 in the field of view
of the virtual world drawing camera C2 has a size of W.times.H.
[0455] Next, the rendering process of the virtual space is
described. The information processing section 31 generates an image
obtained by rendering the virtual space such that the boundary
surface 3 is present at the position described above. The
information processing section 31 performs the rendering process
taking into account the combination of the real world image to be
made later. An example of the rendering process is specifically
described below.
[0456] The information processing section 31 renders with a
perspective projection from the virtual world drawing camera C2 the
enemy object EO, the bullet object BO, and the back wall BW that
are placed in the virtual space, such that the boundary surface 3
is present as shown in FIG. 37 (or FIG. 38). At this time, the
information processing section 31 takes into account the
information about the priority of drawing. In a normal perspective
projection, rendering is performed such that an object present
closer when viewed from the virtual camera in the virtual space is
preferentially drawn. Accordingly, in the normal perspective
projection, an object present in the second space 2 is not drawn
due to the presence of the boundary surface 3. In the game
according to the present embodiment, an opening is provided in the
boundary surface 3 (real world image), so that a part of the second
space 2 can be viewed through the opening. Further, the shadow of
the object present in the second space 2 is drawn in combination
with the real world image. This makes it possible to give the user
a feeling as if the virtual world further exists beyond the real
world image. Specifically, the information processing section 31
performs the rendering process using the information about the
priority of drawing. It should be noted that in the image
processing program according to the present embodiment, alpha
values are used as an example of the information about the priority
of drawing.
[0457] In the perspective projection described above, the object
present in the second space 2 (the enemy object EO or the back wall
BW in the present embodiment) is present behind the boundary
surface 3. Here, the opening determination data is set in the
boundary surface 3. As described above, the opening determination
data is texture data of a rectangle in which alpha values are
stored, and sets of coordinates in the texture data correspond to
positions on the boundary surface in the virtual space. Thus, the
information processing section 31 can specify an area of the
opening determination data in the range of the field of view of the
virtual world drawing camera C2, the area corresponding to the
object present in the second space 2.
[0458] It should be noted that in the present embodiment, for
example, in an area having the opening determination data in which
alpha values of "0" are stored (an open area), the information
processing section 31 draws (renders) images of a virtual object
and the back wall that are present in the second space 2, in an
area that can be viewed through the open area. Further, in an area
having the opening determination data in which alpha values of
"0.2", which correspond to an unopen area, are stored (an area
handled as an area where alpha values of "1" are stored as an
unopen area), the information processing section 31 does not draw
the virtual object and the back wall that are present in the second
space 2. That is, in the image to be displayed, a real world image
is drawn in the portion corresponding to this area by a combination
process in step 85 described later.
[0459] Therefore, in an area having the opening determination data
in which "0" is stored as viewed from the virtual world drawing
camera C2, rendering is performed such that image data included in
the substance data Df1 or the back wall image data De is drawn.
Then, on the upper LCD 22, images of the virtual object and the
back wall are displayed in the portion corresponding to this area
by the combination process in step S85 described later.
[0460] In addition, in an area having the opening determination
data in which alpha values of "0.2", which indicate an unopen area,
are stored as viewed from the virtual world drawing camera C2 (an
area handled as an area where alpha values of "1" are stored as an
unopen area), the virtual object and the back wall that are present
in the second space 2 are not drawn. That is, in the image to be
displayed on the upper LCD 22, a real world image is drawn in the
portion corresponding to this area by the combination process in
step 85 described later. For the shadow ES (silhouette model) of
the enemy object EO described above, however, a depth determination
is set to invalid between the shadow ES and the boundary surface 3.
Accordingly, alpha values of "1" of the silhouette model are
greater than alpha values of "0.2" of the boundary surface 3, and
therefore, the shadow ES is drawn in an area where alpha values of
"1", which indicate an unopen area, are stored. With this, the
shadow ES of the enemy object EO is drawn on the real world image.
Further, when the enemy object EO is present in the first space 1
such that the silhouette model of the enemy object EO has a size
included in the substance model and is placed in such a manner, and
such that a depth determination is set to valid between the
substance model of the enemy object EO and the silhouette model,
the silhouette model is hidden by the substance model, and
therefore is not drawn.
[0461] It should be noted that in the present embodiment, as shown
in FIG. 20A, the shape of the boundary surface 3 is a central
portion of a spherical surface, and therefore, the opening
determination data may not be present depending on the direction of
the field of view of the virtual world drawing camera C2. In this
case, the above process is performed on the assumption that the
opening determination data is present in which alpha values of
"0.2" are stored in a simulated manner. That is, an area where the
opening determination data is not present is handled as an area
where alpha values of "1", which indicate an unopen area, are
stored.
[0462] It should be noted that the silhouette data Df2 included in
the enemy object data Df corresponding to the enemy object EO
according to the present embodiment is set such that the normal
directions of a plurality of planar polygons correspond to
radiation directions as viewed from the enemy object, and to each
planar polygon, a texture of the silhouette image of the enemy
object as viewed from the corresponding direction is applied.
Accordingly, in the image processing program according to the
present embodiment, the shadow ES of the enemy object EO in the
virtual space image is represented as an image on which the
orientation of the enemy object in the second space 2 is
reflected.
[0463] By the above process, the information processing section 31
renders with a perspective projection the enemy object EO, the
bullet object BO, and the back wall BW that are placed in the
virtual space, and generates a virtual world image as viewed from
the virtual world drawing camera C2, to thereby update the rendered
image data of the virtual space (step 84 of FIG. 32B). It should be
noted that the image generated by this process is an image obtained
by excluding the real world image from the display image shown in
FIG. 40.
[0464] Next, the information processing section 31 generates a
display image obtained by combining the real world image with the
virtual space image (step 85), and ends the process of this
subroutine.
[0465] For example, the information processing section 31 generates
a combined image of the real world image and the virtual space
image by combining the rendered image data of the real camera image
with the rendered image of the virtual space such that the rendered
image of the virtual space is given preference. Then, the
information processing section 31 generates a display image by
preferentially combining the image data included in the aiming
cursor image data at the center of the combined image (the center
of the field of view of the virtual world drawing camera C2) (FIG.
40). FIG. 40 shows an example of the display image generated by the
first drawing method or the second drawing method. It should be
noted that when a virtual space image is not stored in the rendered
image data of the virtual space, the information processing section
31 may store the real world image stored in the rendered image data
of the camera image as it is in the display image data Dl.
[0466] As described above, the updating process of the display
image (subroutine) is completed by the first drawing method or the
second drawing method.
[0467] Referring back to FIG. 29, after the updating process of the
display image in step 57, the information processing section 31
displays the display image on the upper LCD 22 (step 60), and
proceeds to the subsequent step. For example, the CPU 311 of the
information processing section 31 stores the display image data Dl
updated in the step 57 described above (the display image) in the
VRAM 313. Then, the GPU 312 of the information processing section
31 outputs the display image drawn in the VRAM 313 to the upper LCD
22, whereby the display image is displayed on the upper LCD 22.
[0468] Next, the information processing section 31 determines
whether or not the game is to be ended (step 59). Conditions for
ending the game may be, for example: that the predetermined
conditions described above (the game is completed or the game is
over) have been satisfied; or that the user has performed an
operation for ending the game. When the game is not to be ended,
the information processing section 31 proceeds to step 52 described
above, and repeats the same process. On the other hand, when the
game is to be ended, the information processing section 31 ends the
process of the flow chart.
[0469] <Operations and Effects of Image Processing According to
First Embodiment>
[0470] As described above, in the image processing program
according to the present embodiment, as shown in the processes of
FIGS. 15 and 16, the face image acquired in the face image
acquisition process is not released for the user until the user
succeeds in the game (first game). That is, the user cannot save
the acquired face image in the saved data storage area Do of the
game until the user succeeds in the game. Further, the user cannot,
for example, copy, modify, or transfer the acquired face image. On
the other hand, when the user stops retrying the game in the state
where the user has failed in the game, the acquired face image is
discarded, and the process ends. Further, depending on the manner
of the process of the first game, for example, the execution mode
or the specification, when the result of the first game is a
failure, the face image is immediately discarded, and the process
ends. Accordingly, until the acquired face image is handed over,
that is, until the face image is saved in the saved data storage
area Do, the user is fixated on the game, and pursues a success
with enthusiasm. That is, based on the image processing program
according to the present embodiment, the user can tackle the game
very seriously.
[0471] In addition, when the operation of the user on the GUI at
the start of the game is an instruction to "acquire a face image
with the inner capturing section 24" ("Yes" in step 9 of FIG. 14),
and the face image acquisition process 1 (step 10 of FIG. 14) is
performed, the following effects are also expected. That is, when a
face image is acquired by performing capturing with the inner
capturing section 24 at the start of the game (typically, before
the first game is started), face images different from capture to
capture are obtained. Thus, for example, as compared to the case of
selecting a specific image stored in another device and using the
selected image in the game, a desire for a success in the game is
increased. Further, a similar effect is expected also in the case
of performing the face image acquisition process 2 by the outer
capturing section 23 (step 12 of FIG. 14). This is because also
when a face image is acquired by performing capturing with the
outer capturing section 23 at the start of the game (typically,
before the first game is started), face images different from
capture to capture are obtained.
[0472] In addition, based on the image processing program according
to the present embodiment, when the user has succeeded in the first
game, the user can collect, in the saved data storage area Do,
various face images, such as a face image of the user themselves,
face images of people around the user, a face image included in an
image obtained by a video device, and a face image of a living
thing owned by the user. The game apparatus 10 can display the
collected face images, for example, on the screen as shown in FIGS.
7 and 8. Then, the game apparatus 10 represents the state where,
for example, on the screen as shown in FIG. 8, face images related
to the face image in the state of being selected show reactions.
Examples of the reactions include: giving a look to the face image
in the state of being selected with one eye closed; and turning its
face to the face image. Accordingly, in a virtual reality world
including the face images collected in the game apparatus 10, the
game apparatus 10 can represent relationships based on human
relationships, intimacies, and the like in the real world. As a
result, it is possible to cause the user having collected the face
images, a sense of affinity for the virtual reality world including
the face images, a familiarity with the collected face images,
emotions similar to those toward people or living things in the
real world, and the like.
[0473] In addition, based on the image processing program according
to the present embodiment, it is possible to generate an enemy
object EO by texture-mapping a face image selected from among the
collected face images onto the facial surface portion of the enemy
object EO, and execute the game. The user can freely determine a
cast by attaching a face image selected from among the collected
face images to the enemy object EO that appears in the game.
Accordingly, during the execution of the game, the user can enhance
the possibility of becoming increasingly enthusiastic about the
game, by an effect obtained from the face of the enemy object
EO.
[0474] In addition, based on the image processing program according
to the present embodiment, in the case where an enemy object EO is
generated, when a face image has entered the state of being
selected in order to be attached to the enemy object EO, face
images related to the face image in the state of being selected
show reactions. Accordingly, it is possible to cause the user who
determines the cast of the enemy object EO, a sense of affinity for
the virtual reality world, a familiarity with the face images
displayed as a list, and emotions similar to those toward people in
the real world, and the like.
[0475] It should be noted that in the first embodiment, an enemy
object EO is generated by attaching a face image to the enemy
object EO. Such a process, however, is not limited to the
generation of an enemy object EO, and can also be applied to the
generation of character objects in general that appear in the game.
For example, a face image acquired by the user may be attached to
an agent who guides an operation on the game apparatus 10 or the
progression of the game. Alternatively, a face image acquired by
the user may be attached to characters that appear in the game
apparatus 10, such as: a character object representing the user
themselves; a character object that appears in the game in a
friendly relationship with the user; a character object
representing the owner of the game apparatus; and the like.
[0476] In the above descriptions, a person's face is assumed to be
a face image; however, the present invention is not limited to a
face image of a person, and can also be applied to a face image of
an animal. For example, face images may be collected by performing
the face image acquisition process described in the first
embodiment, in order to acquire face images of various animals,
such as mammals, e.g., dogs, cats, and horses, birds, fish,
reptiles, amphibians, and insects. For example, with the game
apparatus 10, it is possible to represent the relationships between
people and animals in the real world, such that, as shown in the
relationships between the people on the screen shown in FIG. 8, a
little bird chirps at, a dog barks at, and a cat gives a look to,
the person in the state of being selected. On the other hand, in
the relationship between pet and master, the game apparatus 10 can
reflect emotions, consciousness, and real-world relationships on
the virtual world such that when a face image of the pet has
entered the state of being selected, face images corresponding to
the master and their family smile at, or give looks to, the pet.
Then, it is possible to execute the game while making the user
conscious of real-world relationships, by attaching collected faces
to enemy objects EO and other character objects by the cast
determination process.
[0477] It should be noted that the relationship between pet and
master, the relationships between the master and their family, and
the like may be defined by the face image management information
Dn1 through a UIF (user interface), so that reference can be made
to these relationships for the relationships between face images.
It may be set such that emotions such as love and hate, and good
and bad emotions toward a pet of a loved person and a pet of a
hated person can be defined. Alternatively, for example, setting
may be stored in the face image management information Dn1 such
that an animal whose face image has succeeded in being saved in the
saved data storage area Do of the game when the result of the game
executed with a face image of the master has been successful is in
an intimate relationship with the master. With the game apparatus
10, the user can execute a game in which a character object is
generated and on which consciousness in the real world is
reflected, based on the various face images collected as described
above.
[0478] In addition, in the image processing program according to
the present embodiment, as shown in FIG. 16, the user is led to
acquire a face image by performing capturing with the inner
capturing section 24 prior to the two capturing sections of the
outer capturing section 23 (the left outer capturing section 23a
and the right outer capturing section 23b). The inner capturing
section 24 is used mainly to capture the user who operates the game
apparatus 10, and therefore is difficult to be used to capture a
person other than the user. Further, the inner capturing section 24
is used mainly to capture the user who operates the game apparatus
10, and therefore is suitable to capture the owner. Accordingly,
this process has an effect in prohibiting the use of the outer
capturing section 23 in the state where neither the user nor the
owner of the game apparatus 10 has a face image saved in the game
apparatus 10. This makes it possible to increase the possibility
of, for example, prohibiting a third person from capturing an
image, using the game apparatus 10 whose owner is not specified or
the game apparatus 10 whose user is not specified.
[0479] In addition, in the image processing program according to
the present embodiment, as shown in FIGS. 19A through 19C, the
information processing section 31 leads the user to preferentially
capture a face image corresponding to an unacquired attribute. Such
a process makes it possible to assist a face image collection
process of a user who wishes to acquire face images having as
balanced attributes as possible.
[0480] In addition, in the image processing program according to
the present embodiment, display is performed such that a real world
image obtained from a real camera and a virtual space image
including an object present behind the real world image are
combined.
[0481] Therefore, in the image processing program according to the
present embodiment, it is possible to generate an image capable of
attracting the user's interest, by performing drawing so as to
represent unreality in a background in which a real world image is
used.
[0482] In addition, when an object is present behind the real world
image in a combined image to be displayed (e.g., the enemy object
EO present in the second space 2), a substance image of the object
is displayed in the real world image (boundary surface 3), in an
area where an opening is present. Further, a shadow image of the
object is displayed in the real world image, in an area where an
opening is not present (see FIG. 24). Furthermore, the substance
image and the shadow image are each an image corresponding to the
orientation based on the placement direction or the moving
direction of the object in the virtual space.
[0483] Therefore, in the image processing program according to the
present embodiment, it is possible to generate an image in which
the user can recognize the activities, such as the number and the
moving directions, of objects present behind the real world
image.
[0484] In addition, in the image processing program according to
the present embodiment, an image of an unreal space, such as an
image of outer space, can be used as image data of the back wall
BW. The image of the unreal space can be viewed through an opening
in the real world image. The opening is specified at a position in
the virtual space. Then, the orientation of the real camera and the
orientation of the virtual camera are associated together.
[0485] Therefore, in the image processing program according to the
present embodiment, it is possible to provide an opening at a
position corresponding to the orientation of the real camera, and
represent the opening at the same position in the real world image.
That is, in the image processing program according to the present
embodiment, even when the orientation of the real camera has
changed, the opening is represented at the same position in real
space. This makes it possible to generate an image that can be
recognized by the user as if real space is linked with the unreal
space.
[0486] In addition, the real world image in which an opening is
represented is generated by the multiplication of the real world
image obtained from the real camera and alpha values.
[0487] Therefore, in the image processing program according to the
present embodiment, it is possible to represent and generate an
opening by a simplified method.
[0488] In addition, an opening in the real world image that is
generated by the enemy object EO passing through the boundary
surface 3 is generated by multiplying: the opening shape data Df3
included in the enemy object data Df; by the opening determination
data corresponding to a predetermined position.
[0489] Therefore, in the image processing program according to the
present embodiment, it is possible to set an opening corresponding
to the shape of a character having collided, by a simplified
method.
[0490] In addition, in the image processing program according to
the present embodiment, it is possible to draw a shadow image by
comparing alpha values. Further, it is possible to switch between
the on/off states of the drawing of a shadow image by changing
alpha values set in the silhouette data Df2.
[0491] Therefore, in the image processing program according to the
present embodiment, it is possible to leave the drawing of a shadow
image to the GPU, and also switch between the display and hiding of
a shadow by a simplified operation.
Second Embodiment
[0492] With reference to FIGS. 41 and 42, a description is given
below of an image processing apparatus that executes an image
processing program according to a second embodiment of the present
invention. In the first embodiment, the first game is executed in
the face image acquisition process 1 (step 10 of FIG. 14) and the
face image acquisition process 2 (step 12 of FIG. 14). When the
user has succeeded in the first game, the game apparatus 10 permits
the image acquired in the face image acquisition process 1 and the
face image acquisition process 2 to be stored in the saved data
storage area Do. Then, if the user succeeds in the first game, the
user can sequentially add face images acquired by a similar process
to the saved data storage area Do. Then, based on the face images
collected in the saved data storage area Do, the game apparatus 10
creates character objects, such as enemy objects EO, for example,
in accordance with an operation of the user or automatically. Then,
the game apparatus 10 causes the character objects created based on
the face images collected by the user, to appear for the user in
the first game (step 106 of FIG. 15 and step 129 of FIG. 16), the
second game (step 18 of FIG. 14), and the like, and provides the
user with a virtual world on which human relationships and the like
in the real world are reflected. Accordingly, in such a virtual
world on which the real world is reflected, the user can enjoy
executing, for example, the game as shown in FIGS. 20A through 26.
In the present embodiment, a description is given of an example of
another game processing performed in such a virtual world on which
the real world is reflected. Similarly to the first embodiment, the
game according to the present embodiment is also provided to the
user by the information processing section 31 of the game apparatus
10 executing the image processing program expanded in the main
memory 32. Further, similarly to the game according to the first
embodiment, the game according to the present embodiment may be,
for example, executed as the first game (step 106 of FIG. 15 and
step 129 of FIG. 16) for the face image acquisition process 1 in
step 10 and the face image acquisition process 2 in step 12, the
processes shown in FIG. 14. Furthermore, for example, similarly to
the second game (step 18 of FIG. 14), the game according to the
present embodiment may be executed on the assumption that face
images are collected and accumulated in the saved data storage area
Do of the game.
[0493] FIG. 41 is an example of a screen displayed on the upper LCD
22 of the game apparatus 10 according to the present embodiment.
The procedure of the creation of this screen is similar to that of
the first embodiment. That is, the case is assumed where, for
example, the user holds the lower housing 11 with both hands as
shown in FIG. 4, such that the lower housing 11 and the upper
housing 21 of the game apparatus 10 are in the open state. At this
time, the user can view the display screen of the upper LCD 22.
Further, in this state, the outer capturing section 23 can, for
example, capture space ahead in the line of sight of the user.
During the execution of the game according to the present
embodiment, the game apparatus 10 displays in the background of the
screen an image captured by the outer capturing section 23. More
specifically, the information processing section 31 texture-maps,
on a frame-by-frame basis, an image captured by the outer capturing
section 23 onto the background portion of the screen of the game.
When the user has changed the orientation of the game apparatus 10
while holding it in their hand, an image acquired through the outer
capturing section 23 in the direction of the line of sight of the
outer capturing section 23 after the change is displayed in the
background of the game. That is, in the background of the screen
shown in FIG. 41, an image acquired from the direction in which the
user has directed the outer capturing section 23 of the game
apparatus 10 is embedded.
[0494] In addition, on the background of the screen, for example,
an enemy object EO1 is displayed, the enemy object EO1 created in
accordance with the procedure described with reference to the
example of the screen shown in FIG. 10 and the cast determination
process in FIG. 18. Display is performed such that a face image
selected in the cast determination process in FIG. 18 is
texture-mapped on the facial surface portion of the enemy object
EO1. It should be noted that the facial surface portion of the
enemy object EO1 does not necessarily need to be formed by texture
mapping. Alternatively, the enemy object EO1 may be displayed by
simply combining the peripheral portion H13 of the head shape of
the enemy object EO shown in FIG. 9 with the face image. Thus,
hereinafter, an expression is used, such as "a face image is
attached to the facial surface portion of an enemy object".
[0495] In addition, in FIG. 41, around the enemy object EO1, enemy
objects EO2 through EO7, which are smaller than the enemy object
EO1, are displayed. As described above, on the screen shown in FIG.
41, seven enemy objects EO in total are displayed, namely the enemy
objects EO1 through EO7. In the process of the game apparatus 10
according to the present embodiment, however, the number of enemy
objects EO is not limited to seven. It should be noted that as has
already been described in the first embodiment, when the enemy
objects EO1 through EO7 do not need to be distinguished from one
another, the enemy objects EO are referred to as "enemy objects
EO".
[0496] In addition, to any one or more of the enemy objects EO1
through EO7, e.g., to the enemy object EO6, the same face image as
that of the enemy object EO1 is attached. On the other hand, to the
other enemy objects EO2 through EO5, face images different from
that of the enemy object EO1 are attached.
[0497] In addition, on the screen shown in FIG. 41, an aiming
cursor AL for attacking the enemy object EO1 and the like is
displayed. The positional relationships and the relative movement
relationships between the aiming cursor AL and the background, and
between the aiming cursor AL and the enemy objects EO1 and the
like, are similar to those described in the first embodiment (FIGS.
23 through 26).
[0498] That is, the enemy object EO1 freely moves around on the
screen shown in FIG. 41. More specifically, the enemy object EO1
freely moves around in a virtual space having an image captured by
the outer capturing section 23 as its background. Accordingly, it
seems to the user viewing the upper LCD 22 as if the enemy object
EO1 freely moves in the space where the user themselves is placed.
Further, the enemy objects EO2 through EO7 are placed around the
enemy object EO1.
[0499] Therefore, when the user has changed the orientation of the
game apparatus 10 relative to the enemy objects EO1 through EO7
that freely move around in the virtual space, the user can point
the aiming cursor AL displayed on the screen at the enemy objects
EO1 through EO7. When the user has pressed the operation button 14B
(A button) corresponding to a trigger button in the state where the
aiming cursor AL is pointed at the enemy objects EO1 through EO7,
the user can fire a bullet at the enemy objects EO1 through
EO7.
[0500] In the game according to the present embodiment, however, an
attack on, among the enemy objects EO1 through EO7, those other
than one having the same face image as that of the enemy object EO1
is not a valid attack. For example, when the enemy object EO1 or an
enemy object having the same face image as that of the enemy object
EO1 has been attacked by the user, the user scores points, or the
enemy objects lose points. Further, when the enemy objects EO2
through EO7, each of which is smaller in dimensions than the enemy
object EO1, have been attacked by the user, the user scores more
points. Alternatively, when the enemy objects EO2 through EO7 have
been attacked, the enemy objects lose more points than when the
enemy object EO1 has been attacked. An attack on, among the enemy
objects EO2 through EO7, those having face images different from
that of the enemy object EO1, however, is an invalid attack. That
is, the user is obliged to attack an enemy object having the same
face image as that of the enemy object EO1. Hereinafter, an enemy
object having a face image different from that of the enemy object
EO1 is referred to as a "misidentification object". It should be
noted that in FIG. 41, the enemy objects EO2 through EO7 have head
shapes of the same type. Among the enemy objects EO2 through EO7,
however, any of the enemy objects EO2 through EO5, EO7, and the
like, which are misidentification objects, may have head shapes of
different types. It should be noted that as has already been
described, when the enemy objects EO1 through EO7 and the like do
not need to be distinguished from one another, the enemy objects EO
are referred to simply as "enemy objects EO".
[0501] With reference to FIG. 42, a description is given of an
example of the operation of the image processing program executed
by the information processing section 31 of the game apparatus 10.
FIG. 42 is a flow chart showing an example of the operation of the
information processing section 31. In this process, the information
processing section 31, for example, receives the selection of a
face image, and generates enemy objects (step 30). The process of
step 30 is, for example, similar to the cast determination process
in FIG. 18.
[0502] Next, the information processing section 31 generates
misidentification objects (step 31). The misidentification objects
may be generated by, for example, attaching face images other than
the face image of the enemy objects EO specified in step 30, to the
facial surface portion of the head shape of the enemy objects EO.
The specification of the face images of the misidentification
objects is not limited. For example, the face images of the
misidentification objects may be selected from among face images
already acquired by the user, as shown in FIGS. 7 and 8.
Alternatively, for example, the face images of the
misidentification objects may be stored in advance in the data
storage internal memory 35 before the shipment of the game
apparatus 10. Yet alternatively, the face images of the
misidentification objects may be stored in the data storage
internal memory 35 simultaneously at the installation or the
upgrading of the image processing program. Further, for example,
face images obtained by deforming the face image of the enemy
objects EO, for example, face images obtained by switching parts of
the face, such as eyes, nose, and mouth, with those of another face
image, may be used for the misidentification objects.
[0503] Next, the information processing section 31 starts the game
of the enemy objects EO and the misidentification objects (step
32). Then, the information processing section 31 determines whether
or not the user has made an attack (step 33). The attack of the
user is detected by a trigger input, for example, the pressing of
the operation buttons 14B in the state where the aiming cursor AL
shown in FIG. 41 is pointed at the enemy objects EO. When the user
has made an attack, the information processing section 31
determines whether or not the attack has been made on an
appropriate enemy object EO (step 35). When an appropriate enemy
object EO has been attacked, the information processing section 31
destroys the enemy object EO, and adds points to the score of the
user (step 36). On the other hand, in the determination in step 35,
when an attack on a misidentification object, which is not the
appropriate enemy objects EO, has been detected, the information
processing section 31 performs nothing on the assumption that the
attack is invalid. Further, in the determination in step 33, when
the user has not made an attack, the information processing section
31 performs another process (step 34). Said another process is, for
example, a process specific to each game. Examples of said another
process include: a process of propagating the enemy object EO6 and
the misidentification objects EO2 through EO5 in FIG. 41; and a
process of switching the position of the enemy object EO6 and the
positions of the misidentification objects EO2 through EO7 and the
like in FIG. 41.
[0504] Then, the information processing section 31 determines
whether or not the game is to be ended (step 37). The game is
ended, for example, when the user has destroyed all the propagating
enemy objects EO, or when the score of the user has exceeded a
reference value. Alternatively, the game is ended, for example,
when the enemy objects EO have propagated so as to exceed a
predetermined limit, or when the points lost by the user have
exceeded a predetermined limit. When the game is not to be ended,
the information processing section 31 returns to step 33.
[0505] As described above, based on the image processing program
according to the present embodiment, it is possible to create enemy
objects EO using collected face images, and execute the game. This
enables the user to execute a game in a virtual reality world,
based on face images of people existing in the real world.
[0506] In addition, based on the game apparatus 10 according to the
present embodiment, the game is executed by confusing the user in
combination of appropriate enemy objects EO and misidentification
objects. Accordingly, the user needs to correctly recognize the
face images of the enemy objects EO. As a result, the user requires
a capacity to distinguish the enemy objects EO and concentration.
Thus, the game apparatus 10 according to the present embodiment
makes it possible to cause the user a sense of tension when the
game is executed, or to stimulate the user's brain while the user
recognizes the face images.
[0507] In the second embodiment, for example, as in the cast
determination process in step 16 and the process of the execution
of the game in step 18 of FIG. 14, the game is executed by creating
enemy objects EO based on face images already stored in the saved
data storage area Do of the game. As the process of step 30 of FIG.
42, however, for example, the processes of steps 100 through 105 of
FIG. 15 may be performed. That is, the game according to the
present embodiment may be executed in the state where a face image
has been acquired in the face image acquisition process, but has
yet to be stored in the saved data storage area Do of the game.
Then, as in steps 107 through 110 of FIG. 15, when the game has
been successful, the acquired face image may be stored in the saved
data storage area Do of the game. With such a configuration, as in
the first embodiment, the user tackles the game increasingly
enthusiastically in order to save the acquired face image.
Third Embodiment
[0508] With reference to FIGS. 43 and 44, a description is given
below of an image processing apparatus that executes an image
processing program according to a third embodiment of the present
invention. Similarly to the game according to the first embodiment,
the game according to the present embodiment may be, for example,
executed as the first game (step 106 of FIG. 15 and step 129 of
FIG. 16) for the face image acquisition process 1 in step 10 and
the face image acquisition process 2 in step 12, the processes
shown in FIG. 14. Further, for example, similarly to the second
game (step 18 of FIG. 14), the game according to the present
embodiment may be executed on the assumption that face images are
collected and accumulated in the saved data storage area Do of the
game.
[0509] That is, as in the case described in the second embodiment,
the information processing section 31 of the game apparatus 10
executes the game according to the present embodiment as an example
of the processing of the cast determination process in the first
embodiment (step 16 of FIG. 14) and the second game (the game
executed in step 18 of FIG. 14). Further, the information
processing section 31 of the game apparatus 10 can execute the game
according to the present embodiment also as the first game
according to the first embodiment (the game executed in step 106 of
FIG. 15 and step 129 of FIG. 16).
[0510] In addition, in the second embodiment, a description is
given of an example of the game processing of the game where a face
image is acquired, and enemy objects EO including the acquired face
image and misidentification objects are used. Then, in the second
embodiment, it is determined that an attack on a misidentification
object is an invalid attack.
[0511] In the present embodiment, a description is given of a game
where, when an attack on a misidentification object has been
detected, a part of the face image of an enemy object EO is
replaced with a part of another face image, instead of the game
according to the second embodiment. For example, the enemy object
EO is formed by combining the peripheral portion of the enemy
object EO (see H13 in FIG. 9) with a face image of the user. A face
image of a person close to the user may be used instead of the face
image of the user. In this case, the face image of the user (or the
face image of the close person) may represent the state of being
constrained by the enemy object EO. Then, when the user has
defeated the enemy object EO in the game, representation may be
made such that the face image of the user (or the face image of the
close person) is released from the enemy object EO. On the other
hand, when the user has continued to fail in the game, the face
image of the user (or the face image of the close person)
constrained by the enemy object EO gradually deforms. Then, when
the limit of the deformation has exceeded a certain limit, the game
may be ended.
[0512] FIG. 43 is an example of a screen displayed on the upper LCD
22 according to the present embodiment. In the example of FIG. 43,
an enemy object EO1 (an example of a first character object) is
displayed. Further, around the enemy object EO1, an enemy object
EO11 (an example of a second character object) and
misidentification objects EO12 through EO16 (examples of a third
character object), which are smaller in dimensions than the enemy
object EO1, are displayed. To the enemy object EO11, the same face
image as that of the enemy object EO1 is attached. On the other
hand, the configuration of the misidentification objects EO12
through EO16 is similar to the second embodiment, and face images
different from that of the enemy object EO1 are attached to the
misidentification objects EO12 through EO16. It should be noted
that the configuration of FIG. 43 is illustrative, and the number
of enemy objects being smaller in dimensions than the enemy object
EO1, namely the enemy object EO11, is not limited to one.
[0513] In the game according to the present embodiment, for
example, it is easy to point the aiming cursor AL at the enemy
object EO1, which is larger in dimensions, and therefore, even when
the user has attacked the enemy object EO1 and a bullet has hit the
enemy object EO1, the points scored by the user or the damage
inflicted on the enemy object EO1 are small. Further, it is
difficult to point the aiming cursor AL at the enemy object EO11,
which is smaller in dimensions, and therefore, when the user has
attacked the enemy object EO11 and a bullet has hit the enemy
object EO11, the points scored by the user or the damage inflicted
on the enemy object EO11 are greater than those in the case of the
enemy object EO1.
[0514] In addition, in the present embodiment, when the
misidentification objects EO12 through EO16 have been attacked by
the user, a part of the face image attached to the enemy object EO1
is replaced with that of another face image. For example, in the
case of FIG. 43, in the face image attached to the enemy object
EO1, an eyebrow and an eye are replaced with an eyebrow and an eye
of the face image EO13. As described above, when the user is
attempting to attack the smaller enemy object EO11, the
misidentification objects EO12 through EO16 lead to the deformation
of the enemy object EO1 by confusing the user. It should be noted
that in FIG. 43, the misidentification objects EO12 through EO16
have head shapes of the same type. Any of the misidentification
objects EO12 through EO16, however, may have head shapes of
different types.
[0515] With reference to FIG. 44, a description is given of an
example of the operation of the image processing program executed
by the information processing section 31 of the game apparatus 10.
FIG. 44 is a flow chart showing an example of the operation of the
information processing section 31. In this process, the processes
of steps 40 through 42 are similar to the processes of steps 30
through 32 of FIG. 42, and therefore are not described.
[0516] Then, when having detected an attack on the enemy objects EO
(step 43), the information processing section 31 reduces the
deformation of the face image, and brings the face image of the
enemy object EO1 closer to the face image that is originally
attached (step 44). In this case, when the enemy object EO11 shown
in FIG. 43, which is smaller in dimensions, has been attacked by
the user, the user may score more points than an attack on the
enemy object EO1, which is larger in dimensions. When the enemy
object EO11, which is smaller in dimensions, has been attacked by
the user, the degree of the reduction of the deformation of the
face image may be greater than when the enemy object EO1, which is
larger in dimensions, has been attacked by the user.
[0517] On the other hand, when having detected an attack on the
misidentification objects EO12 through EO16 and the like (step 45),
the information processing section 31 advances the switching of
parts of the face image attached to the enemy object EO1. That is,
the information processing section 31 additionally deforms the face
image (step 46). Further, when having detected a state other than
an attack on the enemy objects EO and an attack on the
misidentification objects, the information processing section 31
performs another process (step 47). Said another process is similar
to that in the case of step 34 of FIG. 42. For example, the
information processing section 31 propagates the enemy objects
EO.
[0518] Then, the information processing section 31 determines
whether or not the game is to be ended (step 48). It is determined
that the game is to be ended, for example, when the deformation of
the face image of the enemy object EO has exceeded a reference
limit. Alternatively, it is determined that the game is to be
ended, for example, when the user has destroyed the enemy objects
EO and scored points of a predetermined limit. When the game is not
to be ended, the information processing section 31 returns to step
43.
[0519] As described above, based on the game apparatus 10 according
to the present embodiment, when the user has succeeded in attacking
the enemy objects EO, the deformed face image is restored. Further,
when the misidentification objects have been attacked by the user,
the deformation of the face image further is advanced. Accordingly,
the user needs to tackle the game with their concentration, and
this increases a sense of tension during the execution of the game,
and therefore makes it possible to train concentration. Further,
based on the game apparatus 10 according to the present embodiment,
a face image of the user or a face image of a person close to the
user is deformed. This makes it possible to increase the
possibility that the user becomes enthusiastic about a game in a
virtual reality world on which the real world is reflected.
[0520] In the third embodiment, for example, as in the process of
step 30 of FIG. 42, the cast determination process in step 16, and
the process of the execution of the game in step 18 of FIG. 14, the
description is given, assuming the case where the game is executed
by creating enemy objects EO based on face images already stored in
the saved data storage area Do of the game. As the process of step
40 of FIG. 44, however, for example, the processes of steps 100
through 105 of FIG. 15 may be performed. That is, the game
according to the third embodiment may be executed in the state
where a face image has been acquired in the face image acquisition
process, but has yet to be stored in the saved data storage area Do
of the game. Then, as in steps 107 through 110 of FIG. 15, when the
game has been successful, the acquired face image may be stored in
the saved data storage area Do of the game. With such a
configuration, as in the first embodiment, the user tackles the
game increasingly enthusiastically.
Fourth Embodiment
[0521] With reference to FIGS. 45 and 46, a description is given
below of an image processing apparatus that executes an image
processing program according to a fourth embodiment of the present
invention. Similarly to the game according to the first embodiment,
the game according to the present embodiment may be, for example,
executed as the first game (step 106 of FIG. 15 and step 129 of
FIG. 16) for the face image acquisition process 1 in step 10 and
the face image acquisition process 2 in step 12, the processes
shown in FIG. 14. Further, for example, similarly to the second
game (step 18 of FIG. 14), the game according to the present
embodiment may be executed on the assumption that face images are
collected and accumulated in the saved data storage area Do of the
game.
[0522] That is, as in the case described in the second embodiment,
the information processing section 31 of the game apparatus 10 can
execute the game according to the present embodiment as an example
of the processing of the cast determination process in the first
embodiment (step 16 of FIG. 14) and the second game (the game
executed in step 18 of FIG. 14). Further, the information
processing section 31 of the game apparatus 10 can execute the game
according to the present embodiment also as the first game
according to the first embodiment (the game executed in step 106 of
FIG. 15 and step 129 of FIG. 16).
[0523] In addition, in the third embodiment, a description is given
of an example of the game processing of the game where a face image
is acquired, and enemy objects EO including the acquired face image
and misidentification objects are used. Further, in the third
embodiment, when an attack on a misidentification object has been
detected, a part of the face image of one of the enemy objects EO
is replaced with a part of another face image.
[0524] In the present embodiment, a description is given of a
process where at the start of the game, a part of the face included
in an enemy object EO is already replaced with a part of another
face image, and when the user has won the game, the part of the
face included in the enemy object EO returns to that of the
original face image, instead of the game according to the second
embodiment and the game according to the third embodiment.
[0525] FIG. 45 is an example of a screen displayed on the upper LCD
22 according to the present embodiment. On the left of the screen,
a list of face images that can be attached to enemy objects EO (a
character column) is displayed as characters. The screen of the
game apparatus 10, however, does not need to include the list of
face images. Further, for example, when the information processing
section 31 has detected, through the GUI, that the user has
performed an operation of requesting the display of a list, the
list of face images may be displayed. Furthermore, the display
position of the list of face images is not limited to the left of
the screen as shown in FIG. 45. In the character column, face
images, such as face images PS1 through PS5, are displayed.
[0526] In addition, on the screen shown in FIG. 45, an enemy object
EO20 and enemy objects EO21 through EO25 are displayed. The enemy
object EO20, the enemy objects EO21 through EO25, and the like are,
for example, enemy objects EO created on the screen shown in FIGS.
7 through 9 in the first embodiment, or in the selection operation
as shown in the cast determination process in FIG. 18. In the
present embodiment, however, the enemy object EO20 is drawn larger
at the center of the screen, and the enemy object EO21 through EO25
and the like are drawn around the enemy object EO20. Further, for
example, to the enemy object EO20, the face image PS1 has been
originally attached. Still further, for example, to the enemy
object EO22, the face image PS2 has been originally attached.
Furthermore, to the enemy object EO25, the face image PS5 has been
originally attached.
[0527] Before the start of the game, however, parts of the faces
are switched between the enemy object EO20 and the enemy objects
EO21 through EO25. For example, noses are switched between the
enemy object EO20 and the enemy object EO22. Further, for example,
left eyebrows and left eyes are switched between the enemy object
EO20 and the enemy object EO25. The switching of parts of the faces
may be, for example, performed on a polygon-by-polygon basis when
the face images are texture-mapped, the polygons forming
three-dimensional models onto which the face images are
texture-mapped.
[0528] Such switching of parts of the faces may be performed by,
for example, randomly changing the number of parts to be switched
and target parts to be switched. Further, for example, the number
of parts to be switched may be determined in accordance with a
success or a failure in, and the score of, the game that has
already been executed or another game. For example, when the
performance, or the degree of achievement, of the user has been
excellent in the game that has already been executed, the number of
parts to be switched is decreased. When the performance, or the
degree of achievement, of the user has been poor, the number of
parts to be switched is increased. Alternatively, the game may be
divided into levels, and the number of parts to be switched may be
changed in accordance with the level of the game. For example, the
number of parts to be switched is decreased at an introductory
level, whereas the number of parts to be switched is increased at
an advanced level.
[0529] In addition, a face image of the user may be acquired by the
inner capturing section 24, face recognition may be performed, and
the number of parts to be switched may be determined in accordance
with the expression obtained from the recognition. For example, a
determination may be made on: the case where the face image is
smiling; the case where the face image is surprised; the case where
the face image is sad; and the case where the face image is almost
expressionless. Then, the number of parts to be switched may be
determined in accordance with the determination result. The
expression of the face may be determined in accordance with: the
dimensions of the eyes; the area of the mouth; the shape of the
mouth; the positions of the contours of the cheeks relative to
reference points, such as the centers of the eyes, the center of
the mouth, and the nose; and the like. For example, an
expressionless face image of the user may be registered in advance,
and the expression of the user's face may be estimated from the
difference values between: values obtained when a face image of the
user has been newly acquired, such as the dimensions of the eyes,
the area of the mouth, the shape of the mouth, the positions of the
contours of the cheeks from the reference points, and the like; and
values obtained from the face image registered in advance. It
should be noted that such a method of estimating the expression of
the face is not limited to the above procedure, and various
procedures can be used.
[0530] In the present embodiment, in the game apparatus 10, the
user executes a game where the user fights with the enemy objects
EO, parts of whose faces are switched as shown in FIG. 45, by
attacking the enemy objects EO. The manner of making an attack is
similar to those of the procedures described in the first through
third embodiments. Then, when the user has won battles with the
enemy objects EO, the switched parts may return to the original
face images.
[0531] With reference to FIG. 46, a description is given of an
example of the operation of the image processing program executed
by the information processing section 31 of the game apparatus 10.
In this process, the process of step 90 is similar to those of step
30 of FIG. 42 and step 40 of FIG. 44. In the present embodiment,
subsequently, the information processing section 31 of the game
apparatus 10 switched parts of the faces (step 91). Then, the
information processing section 31 executes the game (step 92).
Then, the information processing section 31 determines whether or
not the game has been successful (step 93). When the game has been
successful, the information processing section 31 restores the
faces, whose parts have been switched in the process of step 91, to
the faces in the states of being originally captured (step 94).
[0532] As described above, based on the image processing program
according to the present embodiment, for example, the game is
started in the state where a face image of the user has been
acquired by the inner capturing section 24, and parts of the faces
have been switched between the acquired face image and another face
image. Then, when the user has succeeded in the game, for example,
when the user has won battles with the enemy objects EO, the face
images whose parts are switched are restored to the original face
images.
[0533] Therefore, for example, when the face image, a part of whose
face is switched, is a face image of the user themselves, or is a
face image of a person intimate with the user, the user is given a
high motivation to succeed in the game.
[0534] In addition, parts of the faces are switched at the start of
the game according to the present embodiment, in accordance with
the performance in another game, and therefore, it is possible to
give the user a handicap or an advantage based on the result of
said another game. Further, parts of the faces are switched in
accordance with the level of the game, and therefore, it is
possible to represent the difficulty level of the game by the
degree of the deformation of the faces.
[0535] In the fourth embodiment, for example, as in the processes
of step 30 of FIG. 42 and step 40 of FIG. 44, the cast
determination process in step 16 of FIG. 14, and the process of the
execution of the game in step 18, the description is given,
assuming the case where the game is executed by creating enemy
objects EO based on face images already stored in the saved data
storage area Do of the game. As the process of step 90 of FIG. 45,
however, for example, the processes of steps 100 through 105 of
FIG. 15 may be performed. That is, the game according to the
present embodiment may be executed in the state where a face image
has been acquired in the face image acquisition process, but has
yet to be stored in the saved data storage area Do of the game.
Then, as in steps 107 through 110 of FIG. 15, when the game has
been successful, the acquired face image may be stored in the saved
data storage area Do of the game. With such a configuration, as in
the first embodiment, the user tackles the game increasingly
enthusiastically in order to save the acquired face image.
Fifth Embodiment
[0536] With reference to FIGS. 47 through 51, a description is
given below of an image processing apparatus that executes an image
processing program according to a fifth embodiment of the present
invention. In the above embodiments, before the start of the game
for storing a face image in the saved data storage area Do (e.g.,
the first game), the face image, which is an acquisition target, is
acquired by the face image acquisition process that uses a camera
image captured by the inner capturing section 24 or the outer
capturing section 23. Then, after the face image has been acquired,
the game is executed, and when the user has succeeded in the game,
permission is given to store the face image in the saved data
storage area Do. In the fifth embodiment, during the execution of a
game where the user fights with enemy objects EO, a face image,
which is an acquisition target, is acquired from a camera image
captured by the inner capturing section 24 or the outer capturing
section 23. When conditions for succeeding in acquiring the face
image are satisfied, permission is given to store the face image
acquired during the game in the saved data storage area Do. That
is, in the fifth embodiment, a face image is acquired from a camera
image captured during the execution of a predetermined game, and
when conditions for succeeding in acquiring the face image have
been satisfied in the game, permission is given to store the face
image in the saved data storage area Do. Accordingly, this game
results in a game where permission is given to store the face image
in the saved data storage area Do. Similarly to the above
embodiments, the game according to the present embodiment is also
provided to the user by the information processing section 31 of
the game apparatus 10 executing the image processing program
expanded in the main memory 32. In the following descriptions, as
an example of the acquisition of a face image during the game, a
face image is acquired during the execution of the second game
described above (step 18 of FIG. 14).
[0537] FIG. 47 is an example of a screen displayed on the upper LCD
22 of the game apparatus 10 according to the present embodiment.
The procedure of the creation of this screen is similar to that of
the first embodiment. That is, the case is assumed where, for
example, the user holds the lower housing 11 with both hands as
shown in FIG. 4, such that the lower housing 11 and the upper
housing 21 of the game apparatus 10 are in the open state. At this
time, the user can view the display screen of the upper LCD 22.
Further, in this state, the outer capturing section 23 can, for
example, capture space ahead in the line of sight of the user.
During the execution of the game according to the present
embodiment, the game apparatus 10 displays in the background of the
screen a camera image CI captured by the outer capturing section
23. More specifically, the information processing section 31
texture-maps, on a frame-by-frame basis, an image captured by the
outer capturing section 23 onto the background portion of the
screen of the game. That is, on the upper LCD 22, a real-time real
world image (moving image) captured by the real camera built into
the game apparatus 10 is displayed in the background portion. When
the user has changed the orientation of the game apparatus 10 while
holding it in their hand, an image acquired through the outer
capturing section 23 in the direction of the line of sight of the
outer capturing section 23 after the change is displayed in the
background of the game. That is, in the background of the screen
shown in FIG. 47, an image acquired from the direction in which the
user has directed the outer capturing section 23 of the game
apparatus 10 is embedded. It should be noted that in the example of
the screen shown in FIG. 47, a person facing the outer capturing
section 23 in a full-face manner is included as a subject of the
camera image CI displayed as the background of the screen.
[0538] In addition, on the background of the screen, for example,
enemy objects EO and an aiming cursor AL are displayed, the enemy
objects EO and the aiming cursor AL created in accordance with the
procedure described in the above embodiments. Display is performed
such that face images selected in the cast determination process
and the like are texture-mapped on the facial surface portions of
the enemy objects EO. Then, when the user of the game apparatus 10
has pressed the operation button 14B (A button) corresponding to a
trigger button in the state where the aiming cursor AL is pointed
at the enemy objects EO, the user can fire a bullet at the enemy
objects EO.
[0539] Also during the game, the game apparatus 10 sequentially
performs a predetermined face recognition process on the camera
image CI captured by the real camera (e.g., the outer capturing
section 23), and determines the presence or absence of a person's
face in the camera image CI. Then, when the game apparatus 10 has
determined in the face recognition process that a person's face is
present in the camera image CI, and conditions for the appearance
of an acquisition target object AO have been satisfied, an
acquisition target object AO appears from the portion recognized as
a face in the camera image CI.
[0540] As shown in FIG. 48, the acquisition target object AO is
displayed by texture-mapping a face image extracted from the camera
image CI onto a predetermined portion of predetermined polygons
(e.g., the facial surface portion of a three-dimensional model
representing a human head shape. As an example, the acquisition
target object AO is displayed by attaching, as a texture, an image
of the portion recognized as a face in the camera image CI to the
surface of a three-dimensional model of a head shape formed by
combining a plurality of polygons. It should be noted that in the
game where the acquisition target object AO appears, the
acquisition target object AO is not limited to one obtained by
texture-mapping an image of a recognized face onto a
three-dimensional model. For example, the acquisition target object
AO may be displayed as a plate physical body, to the main surface
of which the image of the portion recognized as a face that has
been clipped from the camera image CI is attached, or may be
displayed as an image simply held in a two-dimensional pixel
array.
[0541] For example, similarly to the enemy objects EO, the
acquisition target object AO is placed in the virtual space
described above, and an image of the virtual space (virtual world
image), in which the acquisition target object AO and/or the enemy
objects EO are viewed from the virtual camera, is combined with a
real world image obtained from the camera image CI, whereby display
is performed on the upper LCD 22 as if the acquisition target
object AO and/or the enemy objects EO are placed in real space. In
accordance with an attack operation using the game apparatus 10
(e.g., pressing the button 14B (A button)), a bullet object BO is
fired in the direction of the aiming cursor AL, and the acquisition
target object AO also serves as a target of attack for the user.
Then, when the user has won a battle with the acquisition target
object AO, the user can store in the saved data storage area Do the
face image attached to the acquisition target object AO.
[0542] It should be noted that not only winning a battle with the
acquisition target object AO, but also completing the game where
the user attacks the enemy objects EO, that is, completing the game
that has already been executed when the face of the face image
attached to the acquisition target object AO has been recognized,
may be added to conditions for storing the face image of the
acquisition target object AO in the saved data storage area Do. For
example, possible conditions for completing the game where the user
attacks the enemy objects EO may be that a predetermined number or
more of enemy objects EO are defeated. In this case, the
specification of the game is that, during the execution of the game
where the user attacks the enemy objects EO, when the acquisition
target object AO has appeared in the middle of the game and has
been defeated, the face image of the acquisition target object AO
can be additionally acquired.
[0543] It should be noted that a face image used for the
acquisition target object AO may be a face image obtained from a
face recognized in the camera image CI (a still image), or may be a
face image obtained from a face recognized by repeatedly performing
face recognition on the repeatedly captured camera image CI (a
moving image). For example, in the second case, when the expression
and the like of the person's face repeatedly captured in the camera
image CI has changed, the changes are reflected on a texture of the
acquisition target object AO. That is, it is possible to reflect in
real time the expression of the person captured by the real camera
of the game apparatus 10, on the expression of the face image
attached to the acquisition target object AO.
[0544] In addition, the acquisition target object AO that appears
from the portion recognized as a face in the camera image CI may be
placed so as to always overlap the recognized portion when
displayed in combination with the camera image CI. For example,
changes in the direction and the position of the game apparatus 10
(i.e., the direction and the position of the outer capturing
section 23) in real space also change the imaging range captured by
the game apparatus 10, and therefore also change the camera image
CI displayed on the upper LCD 22. In this case, the game apparatus
10 changes the position and the direction of the virtual camera in
the virtual space in accordance with the motion of the game
apparatus 10 in real space. With this, the acquisition target
object AO displayed as if placed in real space is displayed as if
placed at the same position in real space even when the direction
and the position of the game apparatus 10 have changed in real
space. Further, on the upper LCD 22, a real-time real world image
captured by the real camera built into the game apparatus 10 is
displayed, and therefore, a subject may move in real space. In this
case, the game apparatus 10 sequentially performs a face
recognition process on the repeatedly captured camera image CI, and
thereby sequentially places the acquisition target object AO in the
virtual space such that the acquisition target object AO is
displayed so as to overlap the position of the recognized face when
combined with the camera image CI. Thus, even when changes in the
imaging direction and the imaging position of the game apparatus 10
or a change in the position of the captured person have changed in
the camera image CI the position and the size of the face image
having appeared as the acquisition target object AO, it is possible
to draw the acquisition target object AO so as to overlap the face
image by these processes.
[0545] It should be noted that the acquisition target object AO
displayed on the upper LCD 22 may be displayed by, for example,
enlarging, reducing, or deforming the face image actually captured
and displayed in the camera image CI, or may be displayed by
changing the display direction of the model to which the face image
is attached. Such image processing differentiates the actually
captured face image from the acquisition target object AO, and
therefore enables the user of the game apparatus 10 to easily
determine that the acquisition target object AO has appeared from
the camera image CI.
[0546] Next, with reference to FIGS. 49 through 51, a description
is given of specific processing operations performed by executing
the image processing program according to the fifth embodiment. It
should be noted that FIG. 49 is a subroutine flow chart showing an
example of a detailed operation of a during-game face image
acquisition process performed by executing the image processing
program. FIG. 50 is a subroutine flow chart showing an example of a
detailed operation of a yet-to-appear process performed in step 202
of FIG. 49. FIG. 51 is a subroutine flow chart showing an example
of a detailed operation of an already-appeared process performed in
step 208 of FIG. 49.
[0547] It should be noted that programs for performing these
processes are included in a memory built into the game apparatus 10
(e.g., the data storage internal memory 35), or included in the
external memory 45 or the data storage external memory 46, and the
programs are: loaded from the built-in memory, or loaded from the
external memory 45 through the external memory I/F 33 or from the
data storage external memory 46 through the data storage external
memory I/F 34, into the main memory 32 when the game apparatus 10
is turned on; and executed by the CPU 311
[0548] The processing operations performed by executing the image
processing program according to the fifth embodiment are performed
as follows. For the processing operations performed by executing
the image processing program according to the first embodiment, a
during-game face image acquisition process described later is
performed during the game processing described with reference to
FIG. 29, only in each cycle of the game processing (e.g., performed
once during steps 52 through 59). Thus, in the following
descriptions, only the processing operations added to the first
embodiment are described, and other processing operations are not
described in detail.
[0549] In addition, various data stored in the main memory 32 in
accordance with the execution of the image processing program
according to the fifth embodiment is similar to the various data
stored in accordance with the execution of the image processing
program according to the first embodiment, except that appearance
flag data, face recognition data, and acquisition target object
data are further stored. It should be noted that the appearance
flag data indicates an appearance flag indicating whether the
current state of the appearance of the acquisition target object AO
is "yet to appear", "during appearance", or "already appeared", and
the appearance flag is set to "yet to appear" in the initialization
in step 51 described above (FIG. 29). Further, the face recognition
data indicates the most recent face image obtained from faces
sequentially recognized in the repeatedly captured camera image CI,
and the position of the face image in the camera image CI.
Furthermore, the acquisition target object data includes: data of a
three-dimensional model corresponding to the acquisition target
object AO; texture data for performing mapping on the
three-dimensional model; data indicating the placement direction
and the placement position of the three-dimensional model; and the
like.
[0550] Referring to FIG. 49, the information processing section 31
determines whether or not the acquisition target object AO has yet
to appear (step 201). For example, with reference to the appearance
flag data, the information processing section 31 makes a
determination in step 201 described above, based on whether or not
the appearance flag is set to "yet to appear". When the acquisition
target object AO has yet to appear, the information processing
section 31 proceeds to the subsequent step 202. On the other hand,
when the acquisition target object AO is not in the state of having
yet to appear, the information processing section 31 proceeds to
the subsequent step 203.
[0551] In step 202, the information processing section 31 performs
a yet-to-appear process, and proceeds to the subsequent step 203.
With reference to FIG. 50, a description is given below of the
yet-to-appear process performed by the information processing
section 31 in step 203.
[0552] Referring to FIG. 50, the information processing section 31
performs a predetermined face recognition process on the camera
image indicated by the real camera image data Db, stores the face
recognition result in the main memory 32 (step 211), and proceeds
to the subsequent step. Here, the face recognition process may be
performed sequentially by the information processing section 31,
using the camera image, independently of the processing of the flow
chart shown in FIG. 50. In this case, when a person's face has been
recognized in the camera image, the information processing section
31 acquires the face recognition result in step 211 described
above, and stores the face recognition result in the main memory
32.
[0553] Next, the information processing section 31 determines
whether or not conditions for the appearance of the acquisition
target object AO in the virtual space have been satisfied (step
212). For example, the conditions for the appearance of the
acquisition target object AO, on an essential condition that a
person's face has been recognized in the camera image in step 221
described above, may be: that the acquisition target object AO
appears only once from the start to the end of the game; that the
acquisition target object AO appears at predetermined time
intervals; that in accordance with the disappearance of the
acquisition target object AO from the virtual world, a new
acquisition target object AO appears; or that the acquisition
target object AO appears at a random time. When the conditions for
the appearance of the acquisition target object AO have been
satisfied, the information processing section 31 proceeds to the
subsequent step 213. On the other hand, when the conditions for the
appearance of the acquisition target object AO have not been
satisfied, the information processing section 31 ends the process
of this subroutine.
[0554] In step 213, the information processing section 31 sets an
image of the face recognized in the face recognition process in
step 211 described above, as a texture of the acquisition target
object AO, and proceeds to the subsequent step. For example, in the
camera image indicated by the camera image data Db, the information
processing section 31 sets an image included in the region of the
face indicated by the face recognition result of the face
recognition process in step 211 described above, as a texture of
the acquisition target object AO, to thereby update the acquisition
target object data using the set texture.
[0555] Next, the information processing section 31 sets the
acquisition target object AO, using the face image obtained from
the face recognized in the face recognition process in step 211
(step 214), and proceeds to the subsequent step. As an example, in
accordance with the region of the image of the face recognized in
the face recognition process in step 211, the information
processing section 31 sets the size and the shape of a polygon
(e.g., a planar polygon) corresponding to the state of the start of
the appearance of the acquisition target object AO, and sets the
acquisition target object AO corresponding to the state of the
start of the appearance by attaching the texture of the face image
set in step 213 to the main surface of the polygon, to thereby
update the acquisition target object data.
[0556] Next, the information processing section 31 newly places the
acquisition target object AO in the virtual space (step 215), and
proceeds to the subsequent step. For example, when the camera image
is displayed on the upper LCD 22, the information processing
section 31 places the acquisition target object AO at the position
in the virtual space, at which a perspective projection is
performed such that the acquisition target object AO overlaps the
position of the face image obtained from the face recognized in
step 211, to thereby update the acquisition target object data.
[0557] In the present embodiment, an image is generated by
rendering with a perspective projection from the virtual camera the
virtual space where the acquisition target object AO is newly
placed in addition to the enemy objects EO, and a display image
including at least the generated image is displayed. Here, to make
representation such that the acquisition target object AO appears
from the face image in the camera image displayed on the upper LCD
22, the information processing section 31 places the acquisition
target object AO in the virtual space such that the acquisition
target object AO overlaps the region corresponding to the face
image in the boundary surface 3 on which the texture of the camera
image is mapped, and performs a perspective projection on the
placed acquisition target object AO from the virtual camera. It
should be noted that the method of placing the acquisition target
object AO in the virtual space is similar to the example of the
placement of the enemy object EO described with reference to FIGS.
33 through 39, and therefore is not described in detail.
[0558] Next, the information processing section 31 sets the
appearance flag to "during appearance" to thereby update the
appearance flag data (step 216), and ends the process of this
subroutine.
[0559] Referring back to FIG. 49, in step 203, the information
processing section 31 determines whether or not the acquisition
target object AO is appearing. For example, with reference to the
appearance flag data, the information processing section 31 makes a
determination in step 203 described above, based on whether or not
the appearance flag is set to "during appearance". When the
acquisition target object AO is appearing, the information
processing section 31 proceeds to the subsequent step 204. On the
other hand, when the acquisition target object AO is not appearing,
the information processing section 31 proceeds to the subsequent
step 207.
[0560] In step 204, the information processing section 31 performs
a during-appearance process, and proceeds to the subsequent step.
For example, in step 204, the information processing section 31
represents the state of the acquisition target object AO appearing,
by gradually changing the face image included in the camera image
to a three-dimensional object. Specifically, as in step 211
described above, the information processing section 31 sets the
face image as a texture of the acquisition target object AO, based
on the result of a face recognition performed on the camera image.
Then, as an example, the information processing section 31 sets the
acquisition target object AO by performing a morphing process for
changing a planar polygon to predetermined three-dimensional
polygons (e.g., a three-dimensional model formed by combining a
plurality of polygons so as to represent a human head shape). Then,
as in step 215, the information processing section 31 places the
acquisition target object AO subjected to the morphing process at
the position in the virtual space, at which a perspective
projection is performed such that the acquisition target object AO
overlaps the position of the face image obtained from the face
recognized in step 204, to thereby update the acquisition target
object data. When the acquisition target object AO appears from the
image of the face recognized in the real world image, the
acquisition target object AO is represented so as to gradually
change from planar to three-dimensional in the face image, by
performing such a morphing process.
[0561] It should be noted that the three-dimensional polygons, to
which the planar polygon is changed by the morphing process,
include polygons of various possible shapes. As a first example,
the acquisition target object AO is generated by performing the
morphing process to change the planar polygon to three-dimensional
polygons having the shape of the head of a predetermined character.
In this case, the image of the face recognized in the camera image
in the face recognition process is mapped as a texture onto the
facial surface of the head-shaped polygons. As a second example,
the acquisition target object AO is generated by performing the
morphing process to change the planar polygon to plate polygons
having a predetermined thickness. In this case, the image of the
face recognized in the camera image in the face recognition process
is mapped as a texture onto the main surface of plate polygons. As
a third example, the acquisition target object AO is generated by
performing the morphing process to change the planar polygon to
three-dimensional polygons having the shape of a predetermined
weapon (e.g., missile-shaped polygons). In this case, the image of
the face recognized in the camera image in the face recognition
process is mapped as a texture onto a part of the weapon-shaped
polygons (e.g., mapped onto the missile-shaped polygons at the head
of the missile).
[0562] Next, the information processing section 31 determines
whether or not the during-appearance process on the acquisition
target object AO has ended (step 205). For example, when the
morphing process on the acquisition target object AO has reached
its final stage, the information processing section 31 determines
that the during-appearance process has ended. Then, when the
during-appearance process on the acquisition target object AO has
ended, the information processing section 31 proceeds to the
subsequent step 206. On the other hand, when the during-appearance
process on the acquisition target object AO has not ended, the
information processing section 31 proceeds to the subsequent step
207. For example, when the polygon corresponding to the acquisition
target object AO has changed to a three-dimensional model by
repeating the morphing process in step 204, the three-dimensional
model formed by combining a plurality of polygons so as to
represent a human head shape, the information processing section 31
determines that the morphing process on the acquisition target
object AO is at the final stage.
[0563] In step 206, the information processing section 31 sets the
appearance flag to "already appeared" to thereby update the
appearance flag data, and proceeds to the subsequent step 207.
[0564] In step 207, the information processing section 31
determines whether or not the acquisition target object AO has
already appeared. For example, with reference to the appearance
flag data, the information processing section 31 makes a
determination in step 207 described above, based on whether or not
the appearance flag is set to "already appeared". When the
acquisition target object AO has already appeared, the information
processing section 31 proceeds to the subsequent step 208. On the
other hand, when the acquisition target object AO has not already
appeared, the information processing section 31 ends the process of
this subroutine.
[0565] In step 208, the information processing section 31 performs
an already-appeared process, and ends the process of the
subroutine. With reference to FIG. 51, a description is given below
of the already-appeared process performed by the information
processing section 31 in step 208 described above.
[0566] Referring to FIG. 51, the information processing section 31
performs a predetermined face recognition process on the camera
image indicated by the real camera image data Db, stores the face
recognition result as face recognition data in the main memory 32
(step 221), and proceeds to the subsequent step. Here, the face
recognition process may also be performed sequentially by the
information processing section 31, using the camera image,
independently of the processing of the flow chart shown in FIG. 51.
In this case, when a person's face has been recognized in the
camera image, the information processing section 31 acquires the
face recognition result in step 221 described above, and stores the
face recognition result as face recognition data in the main memory
32.
[0567] Next, the information processing section 31 sets an image of
the face recognized in the face recognition process in step 221
described above (an image included in the face area in the camera
image), as a texture of the acquisition target object AO (step
222), and proceeds to the subsequent step. For example, in the
camera image indicated by the real camera image data Db, the
information processing section 31 sets an image included in the
region of the face indicated by the face recognition result of the
face recognition process in step 101 described above, as a texture
of the acquisition target object AO, to thereby update the
acquisition target object data using the set texture.
[0568] Next, the information processing section 31 sets the
acquisition target object AO corresponding to the region of the
image of the face recognized in the face recognition process in
step 221 described above (step 223), and proceeds to the subsequent
step. For example, the information processing section 31 sets the
acquisition target object AO by attaching the texture of the face
image set in step 222 to the facial surface portion of a
three-dimensional model formed by combining a plurality of polygons
so as to represent a human head shape, to thereby update the
acquisition target object data. It should be noted that in step
223, the polygons to which the face image obtained from the face
recognized in the face recognition process in step 221 is attached
as a texture may be, for example, enlarged, reduced, or deformed,
or the texture of the face image may be deformed.
[0569] Next, the information processing section 31 places the
acquisition target object AO set in step 223 described above in the
virtual space (step 224), and proceeds to the subsequent step. For
example, as in step 215, when the camera image is displayed on the
upper LCD 22, the information processing section 31 places the
acquisition target object AO at the position in the virtual space,
at which a perspective projection is performed such that the
acquisition target object AO overlaps the position of the face
image obtained from the face recognized in step 221, to thereby
update the acquisition target object data. It should be noted that
in step 223, the acquisition target object AO may be placed such
that the facial surface portion to which the texture of the face
image is attached opposes the virtual camera, or the orientation of
the acquisition target object AO may be changed to a given
direction in accordance with the progression of the game.
[0570] Next, the information processing section 31 determines
whether or not the acquisition target object AO and the bullet
object BO have made contact with each other in the virtual space
(step 225). For example, using the position of the acquisition
target object AO indicated by the acquisition target object data
and the position of the bullet object BO indicated by the bullet
object data Dg, the information processing section 31 determines
whether or not the acquisition target object AO and the bullet
object BO have made contact with each other in the virtual space.
When the acquisition target object AO and the bullet object BO have
made contact with each other, the information processing section 31
proceeds to the subsequent step 226. On the other hand, when the
acquisition target object AO and the bullet object BO have not made
contact with each other, the information processing section 31
proceeds to the subsequent step 229.
[0571] In step 226, the information processing section 31 performs
a point addition process, and proceeds to the subsequent step. For
example, in the point addition process, the information processing
section 31 adds predetermined points to the score of the game
indicated by the score data Dh, to thereby update the score data Dh
using the score after the addition. Further, in the point addition
process, the information processing section 31 performs a process
of causing the bullet object BO having made contact based on the
determination in step 225 described above, to disappear from the
virtual space (e.g., initializing the bullet object data Dg
concerning the bullet object BO having made contact with the
acquisition target object AO, such that the bullet object BO is not
present in the virtual space).
[0572] Next, the information processing section 31 determines
whether or not the acquisition of the face image attached to the
acquisition target object AO having made contact with the bullet
object BO has been successful (step 227). As an example of means
for determining whether or not the acquisition of the face image
has been successful, the information processing section 31 performs
the process of step 227. Then, when the acquisition of the face
image has been successful, the information processing section 31
proceeds to the subsequent step 228. On the other hand, when the
acquisition of the face image has not been successful, the
information processing section 31 proceeds to the subsequent step
228.
[0573] Here, a success in the acquisition of the face image is, for
example, the case where the user has won a battle with the
acquisition target object AO. As an example, a predetermined life
value for existing in the virtual space is set for the acquisition
target object AO, and when the acquisition target object AO has
made contact with the bullet object BO, a predetermined number is
subtracted from the life value. Then, when the life value of the
acquisition target object AO has become 0 or below, the acquisition
target object AO is caused to disappear from the virtual space, and
it is determined that the acquisition of the face image attached to
the acquisition target object AO has been successful.
[0574] In step 228, when the acquisition of the face image has been
successful, the information processing section 31 saves the data
that indicates the face image obtained from the face recognized in
step 221 and is stored in the main memory 32, in addition to data
of the face image that has been saved in the saved data storage
area Do up to the current time, and proceeds to the subsequent step
229. As an example of the means for saving, the CPU 311 of the
information processing section 31 performs the process of step 228.
As described above, the saved data storage area Do is a storage
area in which the information processing section 31 can write and
read and which is constructed in, for example, the data storage
internal memory 35 or the data storage external memory 46. When
data of a new face image is stored in the saved data storage area
Do, the information processing section 31 can display the data of
the new face image on the screen of the upper LCD 22, for example,
in addition to the list of face images described with reference to
FIGS. 7 and 8.
[0575] At this time, to manage the face image newly saved in the
saved data storage area Do of the game, the information processing
section 31 generates and saves the face image management
information Dn1 described with reference to FIG. 12. That is, the
information processing section 31 newly generates face image
identification information, and sets the face image identification
information as a record of the face image management information
Dn1. Further, the information processing section 31 sets the
address and the like of the face image newly saved in the saved
data storage area Do, as the address of face image data.
Furthermore, the information processing section 31 sets the source
of acquiring the face image, the estimation of gender, the
estimation of age, pieces of related face image identification
information 1 through N, and the like.
[0576] In addition, the information processing section 31 may
estimate the attributes of the face image added to the saved data
storage area Do, to thereby update the aggregate result of the face
image attribute aggregate table Dn2 described with reference to
FIG. 13. That is, the information processing section 31 may newly
estimate the gender, the age, and the like of the face image added
to the saved data storage area Do, and may reflect the estimations
on the aggregate result of the face image attribute aggregate table
Dn2.
[0577] In addition, the information processing section 31 may
permit the user to, for example, copy or modify the data stored in
the saved data storage area Do, or transfer the data through the
wireless communication module 36. Then, the information processing
section 31 may, for example, save, copy, modify, or transfer the
face image stored in the saved data storage area Do in accordance
with an operation of the user through the GUI, or with an operation
of the user through the operation buttons 14.
[0578] In addition, in the process of step 228 described above, the
information processing section 31 may cause the acquisition target
object AO that is the target used to succeed in the acquisition of
the face image, to disappear from the virtual space. In this case,
the information processing section 31 initializes the acquisition
target object data concerning the acquisition target object AO that
is the target used to succeed in the acquisition of the face image,
such that the acquisition target object AO is not present in the
virtual space.
[0579] In step 229, the information processing section 31
determines whether or not the acquisition of the face image
attached to the acquisition target object AO present in the virtual
space has failed. Then, when the acquisition of the face image
attached to the acquisition target object AO has failed, the
information processing section 31 proceeds to the subsequent step
230. It should be noted that in the case where a plurality of
acquisition target objects AO are present, when any one of the face
images attached to the acquisition target objects AO has failed,
the information processing section 31 proceeds to the subsequent
step 230, On the other hand, when the acquisition of none of the
face images attached to the acquisition target objects AO has
failed, the information processing section 31 ends the process of
this subroutine.
[0580] Here, a failure in the acquisition of the face image is, for
example, the case where the user has lost a battle with the
acquisition target object AO. As an example, when the acquisition
target object AO has continued to be present in the virtual space
for a predetermined time or longer, it is determined that the
acquisition of the face image attached to the acquisition target
object AO has failed.
[0581] In step 230, when the acquisition of the face image has
failed, the information processing section 31 discards the data
that indicates the face image obtained from the face recognized in
step 221 described above and is stored main memory 32, and ends the
process of the subroutine. It should be noted that in the process
of step 230 described above, the information processing section 31
may cause the acquisition target object AO that is the target used
to fail in the acquisition of the face image, to disappear from the
virtual space. In this case, the information processing section 31
initializes the acquisition target object data concerning the
acquisition target object AO that is the target used to fail in the
acquisition of the face image, such that the acquisition target
object AO is not present in the virtual space.
[0582] As described above, based on the processes of FIGS. 49
through 51 according to the fifth embodiment described above, a
face image obtained from a face recognized in a camera image
captured during the game where the user attacks the enemy objects
BO, serves as a target to be newly saved in the saved data storage
area Do. Then, to save the face image acquired during the game in
the saved data storage area Do, the user requires a game result
sufficient to succeed in the acquisition of the face image in the
game already executed. In this game, display is performed such that
on a real world image obtained from a real camera, a virtual world
image showing an acquisition target object represented as if a face
image in the real world image slides out is superimposed. This
makes it possible to display a new image as if the acquisition
target object is present in real space. Further, in the game, for
example, in addition to character objects using face images that
have been saved in the saved data storage area Do up to the current
time, the acquisition target object AO to which the face image
acquired during the game is attached is caused to appear, whereby
the user who executes the game with the game apparatus 10 can
collect a new face image and add the new face image to the saved
data storage area Do, while reflecting the real world during the
execution of the game and human relationships in the real
world.
[0583] It should be noted that in the fifth embodiment described
above, as an example, when the user has attacked and defeated the
acquisition target object AO that appears during the game,
permission is given to store the face image attached to the
acquisition target object AO in the saved data storage area Do.
Alternatively, permission may be given to store the face image in
the saved data storage area Do, by executing another game where the
user fights with the acquisition target object AO. As an example,
in a game where the user competes with enemy objects in score, the
acquisition target object AO appears, to which a face image
included in a camera image captured during the game is attached.
Then, when the user has scored more points than the acquisition
target object AO that has appeared during the game, permission is
given to store the face image attached to the acquisition target
object AO in the saved data storage area Do. As another example, in
a game where the user overcomes obstacles set by enemy objects, the
acquisition target object AO appears, to which a face image
included in a camera image captured during the game is attached.
Then, when the user has overcome the obstacles set by the
acquisition target object AO that has appeared during the game, and
the user has reached a goal, permission is given to store the face
image attached to the acquisition target object AO in the saved
data storage area Do.
[0584] In addition, in the first through fifth embodiments
described above, as an example, a face image acquired in the image
processing based on the flow chart shown in FIG. 14 (a face image
acquired in the face image acquisition process before the execution
of the game for storing a face image in the saved data storage area
Do, in the first through fourth embodiments; and a face image
acquired during the execution of the game for storing a face image
in the saved data storage area Do, in the fifth embodiment) serves
as a target to be stored in the saved data storage area Do.
Alternatively, a face image already acquired in an application
different from the application of the image processing may serve as
a target to be stored in the saved data storage area Do. For
example, the game apparatus 10 includes a capturing section
(camera), and therefore, based on a camera capturing application
different from the application of the image processing based on the
flow chart shown in FIG. 14, can capture an image with the
capturing section, display the captured image on a screen, and save
data of the captured image in a storage medium, such as the data
storage internal memory 35 and the data storage external memory 46.
Further, the game apparatus 10 can receive data including a
captured image from another device, and can also save the received
data in a storage medium, such as the data storage internal memory
35 and the data storage external memory 46, by executing a
communication application. As described above, a face image
obtained from a face recognized in an image obtained in advance by
executing the camera capturing application or the communication
application may serve a target to be stored in the saved data
storage area Do.
[0585] When a face image already acquired by an application
different from the application of the image processing serves as a
target to be stored in the saved data storage area Do, at least one
face image is extracted by performing a face recognition process on
photographed images saved during the execution of the different
application, and the extracted face image serves as a target to be
stored. Specifically, to prompt selection of a character (face
image) to appear in the first game or the second game, when the
character is displayed on the upper LCD 22 and/or the lower LCD 12
(e.g., steps 30, 40, 90, 103, 126, 140, 160, and 162), at least one
character including a face image acquired in advance by executing
the different application is also displayed as a selection target.
In this case, before the character is displayed, at least one face
image is extracted by performing a face recognition process on the
photographed images saved in advance, and a given face image among
the extracted face images is displayed as a selection target in
addition to the character. Then, when the user has selected the
character including the face image obtained by the extraction to
appear in the game, and the user has won a battle with the
character, the face image is stored in the saved data storage area
Do.
[0586] As described above, a face image already acquired by an
application different from the application of the image processing
also serves as a target to be stored in the saved data storage area
Do. This increases the variations of face images that can be
acquired by the user, and therefore makes it easy to collect face
images. Additionally, a face image unexpected by the user is
suddenly added as a target to participate in the game, and
therefore, it is also possible to prevent weariness in collecting
face images.
[0587] In the above descriptions, as an example, the angular
velocities generated in the game apparatus 10 are detected, and the
motion of the game apparatus 10 in real space is calculated using
the angular velocities. Alternatively, the motion of the game
apparatus 10 may be calculated using another method. As a first
example, the motion of the game apparatus 10 may be calculated
using the accelerations detected by the acceleration sensor 39
built into the game apparatus 10. As an example, when the computer
performs processing on the assumption that the game apparatus 10
having the acceleration sensor 39 is in a static state (i.e.,
performs processing on the assumption that the acceleration
detected by the acceleration sensor 39 is the gravitational
acceleration only), if the game apparatus 10 is actually in a
static state, it is possible to determine, based on the detected
acceleration, whether or not the game apparatus 10 is tilted
relative to the direction of gravity, and also possible to
determine to what degree the game apparatus 10 is tilted. As
another example, when it is assumed that the game apparatus 10
having the acceleration sensor 39 is in a dynamic state, the
acceleration sensor 39 detects the acceleration corresponding to
the motion of the acceleration sensor 39 in addition to a component
of the gravitational acceleration. This makes it possible to
determine the motion direction and the like of the game apparatus
10 by removing the component of the gravitational acceleration by a
predetermined process. Specifically, when the game apparatus 10
having the acceleration sensor 39 is moved by being dynamically
accelerated with the user's hand, it is possible to calculate
various motions and/or positions of the game apparatus 10 by
processing the acceleration signals generated by the acceleration
sensor 39. It should be noted that even when it is assumed that the
acceleration sensor 39 is in a dynamic state, it is possible to
determine the tilt of the game apparatus 10 relative to the
direction of gravity by removing the acceleration corresponding to
the motion of the acceleration sensor 39 by a predetermined
process.
[0588] As a second example, the motion of the game apparatus 10 may
be calculated using the amount of movement of a camera image
captured in real time by the real camera built into the game
apparatus 10 (the outer capturing section 23 or the inner capturing
section 24). For example, when the motion of the game apparatus 10
has changed the imaging direction and the imaging position of the
real camera, the camera image captured by the real camera also
changes. Accordingly, it is possible to calculate the angle of
change in the imaging direction of the real camera, the amount of
movement of the imaging position, and the like, using changes in
the camera image captured by the real camera built into the game
apparatus 10. As an example, a predetermined physical body is
recognized in a camera image captured by the real camera built into
the game apparatus 10, and the imaging angles and the imaging
positions of the physical body are chronologically compared to one
another. This makes it possible to calculate the angle of change in
the imaging direction of the real camera, the amount of movement of
the imaging position, and the like, from the amounts of changes in
the imaging angle and the imaging position. As another example, the
entire camera images captured by the real camera built into the
game apparatus 10 are chronologically compared to one another. This
makes it possible to calculate the angle of change in the imaging
direction of the real camera, the amount of movement of the imaging
position, and the like, from the amounts of changes in the imaging
direction and the imaging range in the entire image.
[0589] As a third example, the motion of the game apparatus 10 may
be calculated by combining at least two of: the angular velocities
generated in the game apparatus 10; the accelerations generated in
the game apparatus 10; and a camera image captured by the game
apparatus 10. This makes it possible that in the state where it is
difficult to estimate the motion of the game apparatus 10 in order
to calculate the motion from one parameter, the motion of the game
apparatus 10 is calculated by combining this parameter with another
parameter, whereby the motion of the game apparatus 10 is
calculated so as to compensate for such a state. As an example, to
calculate the motion of the game apparatus 10 in the second example
described above, if the captured camera image has moved
chronologically in a horizontal direction, it may be difficult to
accurately determine whether the capturing angle of the game
apparatus 10 has rotated about the vertical axis, or the game
apparatus 10 has moved horizontally. In this case, it is possible
to easily determine, using the angular velocities generated in the
game apparatus 10, whether the game apparatus 10 has moved so as to
rotate or moved horizontally.
[0590] In addition, as a fourth example, the motion of the game
apparatus 10 may be calculated using so-called AR (augmented
reality) technology.
[0591] In addition, in the above descriptions, as an example,
mainly, a planar image (a planar view image, as opposed to the
stereoscopically visible image described above) of the real world
based on a camera image CI acquired from either one of the outer
capturing section 23 and the inner capturing section 24 is
displayed on the upper LCD 22. Alternatively, an image
stereoscopically visible with the naked eye (a stereoscopic image)
may be displayed on the upper LCD 22. For example, as described
above, the game apparatus 10 can display on the upper LCD 22 a
stereoscopically visible image (stereoscopic image) using camera
images acquired from the left outer capturing section 23a and the
right outer capturing section 23b. In this case, drawing is
performed such that the enemy objects EO are present in the
stereoscopic image displayed on the upper LCD 22, and the
acquisition target object AO appears from the stereoscopic
image.
[0592] For example, to draw the enemy objects EO and the
acquisition target object AO in the stereoscopic image, the image
processing described above is performed using a left-eye image
obtained from the left outer capturing section 23a and a right-eye
image obtained from the right outer capturing section 23b.
Specifically, in the image processing described above, either one
of the left-eye image and the right-eye image is used as the camera
image from which a face image is extracted by performing a face
recognition process, and the enemy objects EO or the acquisition
target object AO obtained by mapping a texture of the face image
obtained from the one of the images are set in the virtual space.
Further, a perspective transformation is performed from two virtual
cameras (a stereo camera), on the enemy objects EO, the acquisition
target object AO, and the bullet object BO, and the like that are
placed in the virtual space, whereby a left-eye virtual world image
and a right-eye virtual world image are obtained. Then, a left-eye
display image is generated by combining a left-eye real world image
with the left-eye virtual world image, and a right-eye display
image is generated by combining a right-eye real world image with
the right-eye virtual world image. Then, the left-eye display image
and the right-eye display image are output to the upper LCD 22.
[0593] In addition, in the above descriptions, a real-time moving
image captured by the real camera built into the game apparatus 10
is displayed on the upper LCD 22, and display is performed such
that the enemy objects EO and the acquisition target object AO
appear in the moving image (camera image) captured by the real
camera. In the present invention, however, the images to be
displayed on the upper LCD 22 have various possible variations. As
a first example, a moving image recorded in advance, or a moving
image or the like obtained from television broadcast or another
device, is displayed on the upper LCD 22. In this case, the moving
image is displayed on the upper LCD 22, and the enemy objects EO
and the acquisition target object AO appear in the moving image. As
a second example, a still image obtained from the real camera built
into the game apparatus 10 or another real camera is displayed on
the upper LCD 22. In this case, the still image obtained from the
real camera is displayed on the upper LCD 22, and the enemy objects
EO and the acquisition target object AO appear in the still image.
Here, the still image obtained from the real camera may be a still
image of the real world captured in real time by the real camera
built into the game apparatus 10, or may be a still image of the
real world photographed in advance by the real camera or another
real camera, or may be a still image obtained from television
broadcast or another device.
[0594] In addition, in the above embodiments, the upper LCD 22 is a
parallax barrier type liquid crystal display device, and therefore
is capable of switching between stereoscopic display and planar
display by controlling the on/off states of the parallax barrier.
In another embodiment, for example, the upper LCD 22 may be a
lenticular type liquid crystal display device, and therefore may be
capable of displaying a stereoscopic image and a planar image. Also
in the case of the lenticular type, an image is displayed
stereoscopically by dividing two images captured by the outer
capturing section 23, each into vertical strips, and alternately
arranging the divided vertical strips. Also in the case of the
lenticular type, an image can be displayed in a planar manner by
causing the user's right and left eyes to view one image captured
by the inner capturing section 24. That is, even the lenticular
type liquid crystal display device is capable of causing the user's
left and right eyes to view the same image by dividing one image
into vertical strips, and alternately arranging the divided
vertical strips. This makes it possible to display an image,
captured by the inner capturing section 24, as a planar image.
[0595] In addition, in the above embodiments, the descriptions are
given using the hand-held game apparatus 10. The present invention,
however, may be achieved by causing a stationary game apparatus or
an information processing apparatus, such as a general personal
computer, to execute the image processing program according to the
present invention. Alternatively, in another embodiment, not only a
game apparatus but also any hand-held electronic device may be
used, such as a personal digital assistant (PDA), a mobile phone, a
personal computer, or a camera. For example, a mobile phone may
include two display sections and a real camera on the main surface
of a housing.
[0596] In addition, in the above descriptions, the image processing
is performed by the game apparatus 10. Alternatively, at least some
of the process steps in the image processing may be performed by
another device. For example, when the game apparatus 10 is
configured to communicate with another device (e.g., a server or
another game apparatus), the process steps in the image processing
may be performed by the cooperation of the game apparatus 10 and
said another device. As an example, the game apparatus 10 performs
a face image acquisition process and game processing for permitting
face images to be saved in an accumulating manner, and the face
images that serve as targets to be permitted to be saved when the
game has been successful may be saved in another device. In this
case, a plurality of game apparatuses 10 save face images in
another device in an accumulating manner, and this further
encourages collection of face images. Additionally, this may also
possibly create a different enjoyment by browsing face images saved
by other game apparatuses 10. As another example, another device
may perform the processes of steps 52 through 57 of FIG. 29, and
the game apparatus 10 may perform the processes of steps 58 and 59
of FIG. 29, by the cooperation of the game apparatus 10 and said
another device. Thus, the image processing described above can be
performed by a processor or by the cooperation of a plurality of
processors, the processor and the plurality of processors included
in an information processing system that includes at least one
information processing apparatus. Further, in the above
embodiments, the processing of the flow chart described above is
performed in accordance with the execution of a predetermined
program by the information processing section 31 of the game
apparatus 10. Alternatively, some or all of the processing may be
performed by a dedicated circuit provided in the game apparatus
10.
[0597] In addition, the shape of the game apparatus 10, and the
shapes, the number, the placement, or the like of the various
buttons of the operation button 14, the analog stick 15, and the
touch panel 13 that are provided in the game apparatus 10 are
merely illustrative, and the present invention can be achieved with
other shapes, numbers, placements, and the like. Further, the
processing orders, the setting values, the criterion values, and
the like that are used in the image processing described above are
also merely illustrative, and it is needless to say that the
present invention can be achieved with other orders and values.
[0598] In addition, the image processing program (game program)
described above may be supplied to the game apparatus 10 not only
from an external storage medium, such as the external memory 45 or
the data storage external memory 46, but also via a wireless or
wired communication link. Further, the program may be stored in
advance in a non-volatile storage device of the game apparatus 10.
It should be noted that examples of the information storage medium
having stored thereon the program may include a CD-ROM, a DVD, and
any other optical disk storage medium similar to these, a flexible
disk, a hard disk, a magnetic optical disk, and a magnetic tape, as
well as a non-volatile memory. Furthermore, the information storage
medium for storing the program may be a volatile memory that
temporarily stores the program. Such storage media can be defined
as storage media that can be read by a computer or the like. For
example, a computer or the like is caused to read and execute the
program stored in each of these storage media, and thereby can
provide the various functions described above.
[0599] While the invention has been described in detail, the
foregoing description is in all aspects illustrative and not
restrictive. It is understood that numerous other modifications and
variations can be devised without departing from the scope of the
invention. It is understood that the scope of the invention should
be interpreted only by the appended claims. It is also understood
that one skilled in the art can implement the invention in the
equivalent range based on the description of the invention and
common technical knowledge, from the description of the specific
embodiments of the invention. Further, throughout the
specification, it should be understood that terms in singular form
include the concept of plurality unless otherwise specified. Thus,
it should be understood that articles or adjectives indicating the
singular form (e.g., "a", "an", "the", and the like in English)
include the concept of plurality unless otherwise specified.
Furthermore, it should be understood that terms used in the present
specification have meanings generally used in the art unless
otherwise specified. Therefore, unless otherwise defined, all the
jargons and technical terms have the same meanings as those
generally understood by one skilled in the art of the invention. In
the event of any contradiction, the present specification
(including meanings defined herein) has priority.
[0600] (Appended Notes)
[0601] The above embodiments can be exemplified by the following
forms (referred to as "appended notes"). The components included in
each appended note can be combined with the components included in
the other appended notes.
[0602] (Appended Note 1)
[0603] A computer-readable storage medium having stored thereon a
game program to be executed by a computer that displays an image on
a display device, the game program causing the computer to execute
as:
[0604] an image acquisition step of acquiring a face image;
[0605] a step of creating a first character object based on the
acquired face image; and
[0606] a game processing step of executing a game by displaying the
first character object together with a second character object, the
second character object being different from the first character
object,
[0607] the game processing step including: [0608] a step of
contributing to a success in the game by an attack on the first
character object, the attack made in accordance with an operation
of a player; and [0609] a step of invalidating an attack on the
second character object, the attack made in accordance with an
operation of the player.
[0610] (Appended Note 2)
[0611] A computer-readable storage medium having stored thereon a
game program to be executed by a computer that displays an image on
a display device, the game program causing the computer to execute
as:
[0612] an image acquisition step of acquiring at least one face
image;
[0613] a step of creating a first character object, the first
character object including one of the acquired face images; and
[0614] a game processing step of executing a game by displaying the
first character object together with a second character object and
a third character object, the second character object being smaller
in dimensions than the first character object and including the one
of the acquired face images, the third character object being
smaller in dimensions than the first character object and including
a face image other than the one of the acquired face images,
[0615] the game processing step including: [0616] a step of
advancing deformation of the face image included in the first
character object by an attack on the third character object, the
attack made in accordance with an operation of a player; and [0617]
a step of, by an attack on the second character object in
accordance with an operation of the player, reversing the
deformation such that the face image included in the first
character object approaches the acquired original face image.
[0618] (Appended Note 3)
[0619] A computer-readable storage medium having stored thereon a
game program to be executed by a computer that displays an image on
a display device, the game program causing the computer to execute
as:
[0620] an image acquisition step of acquiring a face image;
[0621] a step of creating a character object, the character object
including a face image obtained by deforming the acquired face
image;
[0622] a game processing step of receiving an operation of a
player, and advancing a game related to the face image;
[0623] a step of determining a success or a failure in the game,
the success or the failure made in accordance with an operation of
the player; and
[0624] a step of, when a result of the game has been successful,
restoring the deformed face image to the acquired original face
image.
[0625] (Appended Note 4)
[0626] An image processing apparatus connectable to a display
device, the image processing apparatus comprising:
[0627] image acquisition means for acquiring a face image;
[0628] means for creating a first character object based on the
acquired face image; and
[0629] game processing means for executing a game by displaying on
the display device the first character object together with a
second character object, the second character object being
different from the first character object,
[0630] the game processing means including: [0631] means for
contributing to a success in the game by an attack on the first
character object, the attack made in accordance with an operation
of a player; and [0632] means for invalidating an attack on the
second character object, the attack made in accordance with an
operation of the player.
[0633] (Appended Note 5)
[0634] An image processing apparatus connectable to a display
device, the image processing apparatus comprising:
[0635] image acquisition means for acquiring at least one face
image; means for creating a first character object, the first
character object including one of the acquired face images; and
[0636] game processing means for executing a game by displaying the
first character object together with a second character object and
a third character object, the second character object being smaller
in dimensions than the first character object and including the one
of the acquired face images, the third character object being
smaller in dimensions than the first character object and including
a face image other than the one of the acquired face images,
[0637] the game processing means including: [0638] means for
advancing deformation of the face image included in the first
character object by an attack on the third character object, the
attack made in accordance with an operation of a player; and [0639]
means for, by an attack on the second character object in
accordance with an operation of the player, reversing the
deformation such that the face image included in the first
character object approaches the acquired original face image.
[0640] (Appended Note 6)
[0641] An image processing apparatus connectable to a display
device, the image processing apparatus comprising:
[0642] image acquisition means for acquiring a face image;
[0643] means for creating a character object, the character object
including a face image obtained by deforming the acquired face
image;
[0644] game processing means for receiving an operation of a
player, and advancing a game related to the face image by
displaying the character object on the display device;
[0645] means for determining a success or a failure in the game,
the success or the failure made in accordance with an operation of
the player; and
[0646] means for, when a result of the game has been successful,
restoring the deformed face image to the acquired original face
image.
[0647] (Appended Note 7)
[0648] An image processing apparatus comprising:
[0649] a display device;
[0650] image acquisition means for acquiring a face image;
[0651] means for creating a first character object based on the
acquired face image; and
[0652] game processing means for executing a game by displaying on
the display device the first character object together with a
second character object, the second character object being
different from the first character object,
[0653] the game processing means including: [0654] means for
contributing to a success in the game by an attack on the first
character object, the attack made in accordance with an operation
of a player; and [0655] means for invalidating an attack on the
second character object, the attack made in accordance with an
operation of the player.
[0656] (Appended Note 8)
[0657] An image processing apparatus comprising:
[0658] a display device;
[0659] image acquisition means for acquiring at least one face
image;
[0660] means for creating a first character object, the first
character object including one of the acquired face images; and
[0661] game processing means for executing a game by displaying on
the display device the first character object together with a
second character object and a third character object, the second
character object being smaller in dimensions than the first
character object and including the one of the acquired face images,
the third character object being smaller in dimensions than the
first character object and including a face image other than the
one of the acquired face images,
[0662] the game processing means including: [0663] means for
advancing deformation of the face image included in the first
character object by an attack on the third character object, the
attack made in accordance with an operation of a player; and [0664]
means for, by an attack on the second character object in
accordance with an operation of the player, reversing the
deformation such that the face image included in the first
character object approaches the acquired original face image.
[0665] (Appended Note 9)
[0666] An image processing apparatus comprising:
[0667] a display device;
[0668] image acquisition means for acquiring a face image;
[0669] means for creating a character object, the character object
including a face image obtained by deforming the acquired face
image;
[0670] game processing means for receiving an operation of a
player, and advancing a game related to the face image by
displaying the character object on the display device;
[0671] means for determining a success or a failure in the game,
the success or the failure made in accordance with an operation of
the player; and
[0672] means for, when a result of the game has been successful,
restoring the deformed face image to the acquired original face
image.
[0673] (Appended Note 10)
[0674] An image processing system comprising:
[0675] a capturing device;
[0676] a display device that displays information including an
image acquired by the capturing device; and
[0677] an image processing apparatus that cooperates with the
capturing device and the display device,
[0678] the image processing apparatus including: [0679] image
acquisition means for acquiring a face image; [0680] means for
creating a first character object based on the acquired face image;
and [0681] game processing means for executing a game by displaying
on the display device the first character object together with a
second character object, the second character object being
different from the first character object,
[0682] the game processing means including: [0683] means for
contributing to a success in the game by an attack on the first
character object, the attack made in accordance with an operation
of a player; and [0684] means for invalidating an attack on the
second character object, the attack made in accordance with an
operation of the player.
[0685] (Appended Note 11)
[0686] An image processing system comprising:
[0687] a capturing device;
[0688] a display device that displays information including an
image acquired by the capturing device; and
[0689] an image processing apparatus that cooperates with the
capturing device and the display device,
[0690] the image processing apparatus including: [0691] image
acquisition means for acquiring at least one face image; [0692]
means for creating a first character object, the first character
object including one of the acquired face images; and [0693] game
processing means for executing a game by displaying on the display
device the first character object together with a second character
object and a third character object, the second character object
being smaller in dimensions than the first character object and
including the one of the acquired face images, the third character
object being smaller in dimensions than the first character object
and including a face image other than the one of the acquired face
images,
[0694] the game processing means including: [0695] means for
advancing deformation of the face image included in the first
character object by an attack on the third character object, the
attack made in accordance with an operation of a player; and [0696]
means for, by an attack on the second character object in
accordance with an operation of the player, reversing the
deformation such that the face image included in the first
character object approaches the acquired original face image.
[0697] (Appended Note 12)
[0698] An image processing system comprising:
[0699] a capturing device;
[0700] a display device that displays information including an
image acquired by the capturing device; and
[0701] an image processing apparatus that cooperates with the
capturing device and the display device,
[0702] the image processing apparatus including: [0703] image
acquisition means for acquiring a face image; [0704] means for
creating a character object, the character object including a face
image obtained by deforming the acquired face image; [0705] game
processing means for receiving an operation of a player, and
advancing a game related to the face image by displaying the
character object on the display device; [0706] means for
determining a success or a failure in the game, the success or the
failure made in accordance with an operation of the player; and
[0707] means for, when a result of the game has been successful,
restoring the deformed face image to the acquired original face
image.
[0708] (Appended Note 13)
[0709] An information processing method performed by a computer
that displays an image on a display device, the computer
executing:
[0710] an image acquisition step of acquiring a face image;
[0711] a step of creating a first character object based on the
acquired face image; and
[0712] a game processing step of executing a game by displaying the
first character object together with a second character object, the
second character object being different from the first character
object,
[0713] the game processing step including: [0714] a step of
contributing to a success in the game by an attack on the first
character object, the attack made in accordance with an operation
of a player; and [0715] a step of invalidating an attack on the
second character object, the attack made in accordance with an
operation of the player.
[0716] (Appended Note 14)
[0717] An information processing method performed by a computer
that displays an image on a display device, the computer
executing:
[0718] an image acquisition step of acquiring at least one face
image;
[0719] a step of creating a first character object, the first
character object including one of the acquired face images; and
[0720] a game processing step of executing a game by displaying the
first character object together with a second character object and
a third character object, the second character object being smaller
in dimensions than the first character object and including the one
of the acquired face images, the third character object being
smaller in dimensions than the first character object and including
a face image other than the one of the acquired face images,
[0721] the game processing step including: [0722] a step of
advancing deformation of the face image included in the first
character object by an attack on the third character object, the
attack made in accordance with an operation of a player; and [0723]
a step of, by an attack on the second character object in
accordance with an operation of the player, reversing the
deformation such that the face image included in the first
character object approaches the acquired original face image.
[0724] (Appended Note 15)
[0725] An information processing method performed by a computer
that displays an image on a display device, the computer
executing:
[0726] an image acquisition step of acquiring a face image;
[0727] a step of creating a character object, the character object
including a face image obtained by deforming the acquired face
image;
[0728] a game processing step of receiving an operation of a
player, and advancing a game related to the face image;
[0729] a step of determining a success or a failure in the game,
the success or the failure made in accordance with an operation of
the player; and
[0730] a step of, when a result of the game has been successful,
restoring the deformed face image to the acquired original face
image.
[0731] A storage medium having stored thereon a game program, an
image processing apparatus, an image processing system, and an
image processing method, according to the present invention can
generate a new image by combining a real world image with a virtual
world image, and therefore are suitable for use as a game program,
an image processing apparatus, an image processing system, an image
processing method, and the like that perform a process of
displaying various images on a display device.
* * * * *