U.S. patent application number 12/342716 was filed with the patent office on 2009-04-23 for information presentation system of visual field agreement type, and portable information terminal and server for use in the system.
This patent application is currently assigned to OLYMPUS CORPORATION. Invention is credited to Yuichiro Akatsuka, Takeo Asano, Takao Shibasaki, Mikihiko Terashima.
Application Number | 20090106126 12/342716 |
Document ID | / |
Family ID | 29405363 |
Filed Date | 2009-04-23 |
United States Patent
Application |
20090106126 |
Kind Code |
A1 |
Asano; Takeo ; et
al. |
April 23, 2009 |
INFORMATION PRESENTATION SYSTEM OF VISUAL FIELD AGREEMENT TYPE, AND
PORTABLE INFORMATION TERMINAL AND SERVER FOR USE IN THE SYSTEM
Abstract
A plurality of known markers is disposed in a specific district.
A marker detection section detects the known markers existing in
the image photographed by an image pickup device. A
position/orientation relation calculation section identifies the
detected known marker to calculate position and orientation
relations between the identified known marker and the image pickup
device. An associated information obtaining section obtains
predetermined information associated with the identified known
marker. An information processing section processes the obtained
predetermined information based on the calculated position and
orientation relations. A display device superimposes and displays
the processed information on the photographed image. At least the
image pickup device and display device are disposed in a portable
information terminal, and the other sections are disposed in a
server which can perform radio communication with the portable
information terminal.
Inventors: |
Asano; Takeo;
(Kunitachi-shi, JP) ; Shibasaki; Takao; (Tokyo,
JP) ; Akatsuka; Yuichiro; (Tama-shi, JP) ;
Terashima; Mikihiko; (Hachioji-shi, JP) |
Correspondence
Address: |
SCULLY SCOTT MURPHY & PRESSER, PC
400 GARDEN CITY PLAZA, SUITE 300
GARDEN CITY
NY
11530
US
|
Assignee: |
OLYMPUS CORPORATION
Tokyo
JP
|
Family ID: |
29405363 |
Appl. No.: |
12/342716 |
Filed: |
December 23, 2008 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10443932 |
May 22, 2003 |
|
|
|
12342716 |
|
|
|
|
Current U.S.
Class: |
705/26.1 ;
345/629; 705/40 |
Current CPC
Class: |
G06K 2009/3225 20130101;
G06K 9/3216 20130101; G09B 29/007 20130101; G06K 9/00664 20130101;
G06Q 20/102 20130101; G06T 7/73 20170101; G06Q 30/06 20130101; G06Q
30/0601 20130101 |
Class at
Publication: |
705/26 ; 345/629;
705/40 |
International
Class: |
G06Q 30/00 20060101
G06Q030/00; G09G 5/00 20060101 G09G005/00; G06Q 20/00 20060101
G06Q020/00 |
Foreign Application Data
Date |
Code |
Application Number |
May 24, 2002 |
JP |
2002-151161 |
Apr 4, 2003 |
JP |
2003-101644 |
Claims
1. A portable information terminal comprising: an image pickup
device configured to photograph an image; a display device
configured to display the image photographed by the image pickup
device; a marker detection section configured to detect a known
marker existing in the image photographed by the image pickup
device; a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section in a plurality of known markers disposed in a
specific district to calculate position and orientation relations
between the identified known marker and the image pickup device; an
associated information obtaining section configured to obtain
predetermined information associated with the known marker
identified by the position/orientation relation calculation
section; and an information processing section configured to
process the predetermined information obtained by the associated
information obtaining section based on the position and orientation
relations calculated by the position/orientation relation
calculation section, wherein the display device superimposes and
displays the information processed by the information processing
section on the image photographed by the image pickup device,
wherein the associated information obtaining section includes a
leading section configured to transmit information for arbitrarily
promoting leading to a specific facility or division in the
specific district to a user's portable information terminal
positioned in a division having high density or an arbitrary
portable information terminal, based on the confirmed density at a
current time or in response to a manager's request of the server,
and the portable information terminal displays the information in
the display device, when the information promoting the leading is
received.
2. The portable information terminal according to claim 1, further
comprising: an input device configured to perform various types of
instruction inputs by a user of the portable information terminal,
wherein a facility which is able to be reserved exists in the
specific district, and the associated information obtaining section
includes: a function of a communication section configured to
perform a radio or cable communication with the reservable
facility; and a function of a reservation section configured to
acquire information indicating a use situation of the facility by
the communication section in accordance with an operation of the
input device by a user of the portable information terminal, to
display the acquired information indicating the use situation in
the display device, and to transmit a request for making or
canceling the reservation of the facility to the facility by the
communication section in accordance with the user's operation of
the input device.
3. The portable information terminal according to claim 1, further
comprising: an input device configured to perform various types of
instruction inputs by a user of the portable information terminal,
wherein a sales facility which sells articles that are able to be
purchased by a user of the portable information terminal exists in
the specific district, and the associated information obtaining
section includes: a function of a communication section configured
to perform a radio or cable communication with the sales facility;
a function of an article information storage section configured to
store list information of the articles; and a function of a
purchase section configured to display the list information of the
articles stored in the article information storage section in the
display device in accordance with an operation of the input device
by the user of the portable information terminal and to transmit
one of a purchase reservation of the selected article to the sales
facility and a transfer request of the selected article to a
predetermined pick-up position in the specific district by the
communication section, in accordance with the article selection by
the user's operation of the input device.
4. The portable information terminal according to claim 3, wherein
the associated information obtaining section further functions as a
card number storage section configured to store a credit card
number of a user of the portable information terminal so that a
purchase cost of the article is collectively paid via a
corresponding credit company by the stored credit card number.
5. A portable information terminal comprising: an image pickup
device configured to photograph an image; a display device
configured to display the image photographed by the image pick-up
device; a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district existing in the image photographed by the image pickup
device; a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section to calculate position and orientation relations
between the identified known marker and the image pickup device;
and an associated information obtaining section configured to
obtain predetermined information associated with the known marker
identified by the position/orientation relation calculation
section, the associated information obtaining section
radio-transmitting the obtained predetermined information to a
server together with the position and orientation relations
calculated by the position/orientation relation calculation section
and allowing the server to process the transmitted predetermined
information based on the transmitted position and orientation
relations, wherein the display device radio-receives the processed
predetermined information from the server and superimposes and
displays the information on the image photographed by the image
pickup device, wherein the associated information obtaining section
includes a leading section configured to transmit information for
arbitrarily promoting leading to a specific facility or division in
the specific district to a user's portable information terminal
positioned in a division having high density or an arbitrary
portable information terminal, based on the confirmed density at a
current time or in response to a manager s request of the server,
and the portable information terminal displays the information in
the display device, when the information promoting the leading is
received.
6. A server which is able to be connected to a portable information
terminal including a display section by communication, said server
comprising at least one of: a marker extraction section configured
to extract a specific marker photographed in an observation image
acquired by an image input section attached to the portable
information terminal or disposed in the portable information
terminal; an orientation calculation section configured to use the
marker extracted by the marker extraction section to calculate
orientation information of the image input section; an information
presentation section configured to display predetermined
information based on the orientation information calculated by the
orientation calculation section in the display section of the
portable information terminal to present the predetermined
information; and a contents storage section configured to store
information to be presented as the predetermined information,
wherein the server transmits and receives necessary information
with the portable information terminal including the sections
excluding the sections disposed in the server or with the sections
of another server, wherein the contents storage section includes a
leading section configured to transmit information for arbitrarily
promoting leading to a specific facility or division in the
specific district to a user's portable information terminal
positioned in a division having high density or an arbitrary
portable information terminal, based on the confirmed density at a
current time or in response to a manager's request of the server.
Description
RELATED APPLICATION
[0001] This application is a continuation of U.S. Ser. No.
10/443,932, filed May 22, 2003, the entire contents of which are
incorporated herein by reference. This application is based upon
and claims the benefit of priority from the prior Japanese Patent
Applications No. 2002-151161, filed May 24, 2002; and No.
2003-101644, filed Apr. 4, 2003, the entire contents of both of
which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] The present invention relates to a visual field agreement
type information presentation system in which information related
to an image photographed by image pickup means is superimposed on
the photographed image and simultaneously displayed in a limited
specific district and in which the information can be presented at
a high density by an image observer, and a portable information
terminal and server for use in the system.
[0004] 2. Description of the Related Art
[0005] A calculation technique using a plurality of markers to
obtain position and orientation information of image pickup means
has heretofore been proposed.
[0006] For example, in Jpn. Pat. Appln. KOKAI Publication No.
2001-126051, a method is proposed comprising: reading information
included in a reference marker in order to obtain a schematic
position and orientation of an image acquisition section, so that
positions of the reference marker and a marker disposed around the
reference marker and/or a characteristic point can be confirmed.
According to the technique disclosed in the publication, the
positions of four or more markers and/or the characteristic point
are confirmed, and this information can be used to estimate the
position and orientation of the image pickup means.
[0007] Moreover, a method of obtaining the position/orientation of
the image pickup means from the reference marker position is
disclosed in "A High Accuracy Realtime 3D Measuring Method of
Marker for VR Interface by Monocular Vision" (3D Image Conference
'96 Proceeding pp. 67-172, Akira Takahashi, Ikuo Ishii, Hideo
Makino, Makoto Nakashizuka, 1996). That is, assuming in FIG. 1 that
O.sub.c is a visual point, p.sub.0 is a projected image diagonal
line intersection, and P.sub.0 is a rectangle diagonal image
intersection, x' is obtained so as to minimize:
F ( x ' ) = 4 - i = 1 4 ( n i n i ' ) 2 ##EQU00001##
so that the orientation of the image pickup means is obtained. It
is to be noted that here a variable vector x' is
x'=(.alpha..sub.1'.beta..sub.1',.alpha..sub.2'). For the position
of the image pickup means, minimum square estimation of d for
minimizing a distance between O.sub.cpi and vertex P.sub.i is
performed.
d = ( s 0 s i ) ( s i w i ) - ( s 0 w i ) 1 - ( s 0 s i ) 2 D
##EQU00002##
Next, the position of the image pickup means can be determined by
an average of depth values obtained from the respective
vertices.
[0008] Moreover, in "Mixed Reality" (ISBN 4-274-90283-8 Ohmsha,
Ltd.) Chapter 6: "Augmented Reality Tracking in Natural Feature
Environments" U. Nehmann, S. You, Y. Cho, J. Lee and J. Park 1999
p. 101-130, a known reference point called a "Fiducial Mark" is
used to obtain an initial position of the image pickup means. In
this case, a plurality of characteristic points is extracted to
obtain a projected position in an input image. The characteristic
point is tracked in response to movement of an input device, and an
optimum solution of the position of the image pickup means is
obtained.
[0009] On the other hand, many attractions or exhibits are usually
scattered in specific districts such as an amusement park, theme
park, and museum. Therefore, to directly visit the objective
attraction or exhibit, a general visitor employs a method of
consulting a booklet map distributed by a company managing the
specific district or a guide map in a main passageway to search a
destination.
[0010] Moreover, for shopping in a souvenir shop in the specific
district, it is usual to visit the shop and to buy articles over
the counter. In recent years, information indicating the shop and
the articles dealt in by the shop can be obtained beforehand by
guidebooks, but in actual the articles are usually bought only over
the counter.
BRIEF SUMMARY OF THE INVENTION
[0011] According to a fIrst aspect of the present invention, there
is provided an information presentation system, comprising:
[0012] a plurality of known markers distributed in a specific
district which is an object of the system;
[0013] an image pickup device configured to photograph an
image;
[0014] a marker detection section configured to detect the known
markers existing in the image photographed by the image pickup
device;
[0015] a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section to calculate position and orientation relations
between the identified known marker and the image pickup
device;
[0016] an associated information obtaining section configured to
obtain predetermined information associated with the known marker
identified by the position/orientation relation calculation
section;
[0017] an information processing section configured to process the
predetermined information obtained by the associated information
obtaining section based on the position and orientation relations
calculated by the position/orientation relation calculation
section; and
[0018] a display device configured to superimpose and display the
information processed by the information processing section on the
image photographed by the image pickup device,
[0019] wherein at least the image pickup device and display device
are disposed in a portable information terminal, and
[0020] the other sections are disposed in a server which performs
radio communication with the portable information terminal.
[0021] According to a second aspect of the present invention, there
is provided a portable information terminal comprising:
[0022] an image pickup device configured to photographs an
image;
[0023] a marker detection section configured to detect a known
marker existing in the image photographed by the image pickup
device;
[0024] a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section in a plurality of known markers disposed in a
specific district to calculate a position and orientation relations
between the identified known marker and the image pickup
device;
[0025] an associated information obtaining section configured to
obtain predetermined information associated with the known marker
identified by the position/orientation relation calculation
section;
[0026] an information processing section configured to process the
predetermined information obtained by the associated information
obtaining section based on the position and orientation relations
calculated by the position/orientation relation calculation
section; and
[0027] a display device configured to superimpose and display the
information processed by the information processing section on the
image photographed by the image pickup device.
[0028] According to a third aspect of the present invention, there
is provided a portable information terminal comprising:
[0029] an image pickup device configured to photograph an image,
the image pickup device radio-transmitting the photographed image
to a server and allowing the server to perform a function of
identifying a known marker existing in the transmitted image in a
plurality of known markers distributed in a specific district to
calculate position and orientation relations between the identified
known marker and the image pickup device and a function of
obtaining predetermined information associated with the identified
known marker to process the information based on the calculated
position and orientation relations; and
[0030] a display device configured to radio-receive the processed
predetermined information from the sewer and to superimpose and
display the information on the image photographed by the image
pickup device.
[0031] According to a fourth aspect of the present invention, there
is provided a portable information terminal comprising:
[0032] an image pickup device configured to photograph an
image;
[0033] a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district from the image photographed by the image pickup device,
the marker detection section radio-transmitting the image of a
portion corresponding to the known marker to a server and allowing
the server to perform a function of identifying the known marker in
the transmitted image to calculate position and orientation
relations between the identified known marker and the image pickup
device and a function of obtaining predetermined information
associated with the identified known marker to process the
information based on the calculated position and orientation
relations; and
[0034] a display device configured to radio-receive the processed
predetermined information from the server and to superimpose and
display the information on the image photographed by the image
pickup device.
[0035] According to a fifth aspect of the present invention, there
is provided a portable information terminal comprising:
[0036] an image pickup device configured to photograph an image,
the image pickup device radio-transmitting the photographed image
to a server and which allows the server to perform a function of
identifying a known marker existing in the transmitted image in a
plurality of known markers distributed in a specific district to
calculate position and orientation relations between the identified
known marker and the image pickup device and a function of
obtaining predetermined information associated with the identified
known marker;
[0037] an information processing section configured to
radio-receive the position and orientation relations and the
predetermined information from the server to process the
predetermined information based on the position and orientation
relations; and
[0038] a display device configured to superimpose and display the
information processed by the information processing section on the
image photographed by the image pickup device.
[0039] According to a sixth aspect of the present invention, there
is provided a portable information terminal comprising:
[0040] an image pickup device configured to photographing an
image;
[0041] a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district from the image photographed by the image pickup
device;
[0042] a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section to calculate position and orientation relations
between the identified known marker and the image pickup device,
the position/orientation relation calculation section
radio-transmitting the identified known marker and the calculated
position and orientation relations to a server and allowing the
server to perform a function of obtaining predetermined information
associated with the transmitted known marker to process the
information based on the transmitted position and orientation
relations; and
[0043] a display device configured to radio-receive the processed
predetermined information from the server and to superimpose and
display the information on the image photographed by the image
pickup device.
[0044] According to a seventh aspect of the present invention,
there is provided a portable information terminal comprising:
[0045] an image pickup device configured to photograph an
image;
[0046] a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district from the image photographed by the image pickup device,
the marker detection section radio-transmitting the image of a
portion corresponding to the known marker to a server and allowing
the server to perform a function of identifying the known marker in
the transmitted image to calculate position and orientation
relations between the identified known marker and the image pickup
device and a function of obtaining predetermined information
associated with the identified known marker;
[0047] an information processing section configured to
radio-receive the position and orientation relations and the
predetermined information from the server to process the
predetermined information based on the position and orientation
relations; and
[0048] a display device configured to superimpose and display the
information processed by the information processing section on the
image photographed by the image pickup device.
[0049] According to an eighth aspect of the present invention,
there is provided a portable information terminal comprising:
[0050] an image pickup device configured to photograph an
image;
[0051] a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district from the image photographed by the image pickup
device;
[0052] a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section to calculate position and orientation relations
between the identified known marker and the image pickup
device;
[0053] an associated information obtaining section configured to
obtain predetermined information associated with the known marker
identified by the position/orientation relation calculation
section, the associated information obtaining section
radio-transmitting the obtained predetermined information to a
server together with the position and orientation relations
calculated by the position/orientation relation calculation section
and allowing the server to process the transmitted predetermined
information based on the transmitted position and orientation
relations; and
[0054] a display device configured to radio-receive the processed
predetermined information from the server and to superimpose and
display the information on the image photographed by the image
pickup device.
[0055] According to a ninth aspect of the present invention, there
is provided a portable information terminal comprising:
[0056] an image pickup device configured to photograph an
image;
[0057] a marker detection section configured to detect presence of
any of a plurality of known markers distributed in a specific
district from the image photographed by the image pickup
device;
[0058] a position/orientation relation calculation section
configured to identify the known marker detected by the marker
detection section to calculate position and orientation relations
between the identified known marker and the image pickup device,
the position/orientation relation calculation section
radio-transmitting an identification result of the known marker to
a server and allowing the server to obtain predetermined
information associated with the transmitted identified known
marker;
[0059] an information processing section configured to
radio-receive the predetermined information from the server to
process the information based on the position and orientation
relations calculated by the position/orientation relation
calculation section; and
[0060] a display device configured to superimpose and display the
information processed by the information processing section on the
image photographed by the image pickup device.
[0061] According to a tenth aspect of the present invention, there
is provided an information presentation system comprising:
[0062] an image input section configured to acquire an observation
image;
[0063] a marker extraction section configured to extract a specific
marker photographed in the observation image acquired by the image
input section;
[0064] an orientation calculation section configured to use the
specific marker extracted by the marker extraction section to
calculate orientation information of the image input section;
[0065] an information presentation section configured to present
predetermined information based on the orientation information
calculated by the orientation calculation section; and
[0066] a contents storage section configured to store the
information to be presented as the predetermined information.
[0067] According to an eleventh aspect of the present invention,
there is provided an information presentation system
comprising:
[0068] an image input section configured to acquire an observation
image;
[0069] a marker design identification section configured to
identify design of a specific marker photographed in the
observation image acquired by the image input section;
[0070] a position calculation section configured to calculate
position information of the image input section from the design of
the specific marker identified by the marker design identification
section;
[0071] an information presentation section configured to present
predetermined information based on the position information
calculated by the position calculation section; and
[0072] a contents storage section configured to store the
information to be presented as the predetermined information.
[0073] According to a twelfth aspect of the present invention,
there is provided an information presentation system
comprising:
[0074] an image input section configured to acquire an observation
image;
[0075] a marker extraction section configured to extract a specific
marker photographed in the observation image acquired by the image
input section;
[0076] a marker design identification section configured to
identify design included in the specific marker extracted by the
marker extraction section;
[0077] a position/orientation calculation section configured to
calculates position and orientation information of the image input
section from the specific marker extracted by the marker extraction
section and the design of the specific marker identified by the
marker design identification section;
[0078] an information presentation section configured to present
predetermined information based on the position and orientation
information calculated by the position/orientation calculation
section; and
[0079] a contents storage section configured to store the
information to be presented as the predetermined information.
[0080] According to a thirteenth aspect of the present invention,
there is provided a portable information terminal comprising:
[0081] a display section configured to display information;
[0082] a marker extraction section configured to receive an
observation image acquired by an image input section attached to or
disposed in the portable information terminal to extract a specific
marker photographed in the observation image;
[0083] an orientation calculation section configured to use the
marker extracted by the marker extraction section to calculate
orientation information of the image input section;
[0084] an information presentation section configured to display
predetermined information based on orientation information
calculated by the orientation calculation section in the display
section to present the predetermined information; and
[0085] a contents storage section configured to store the
information to be presented as the predetermined information.
[0086] According to a fourteenth aspect of the present invention,
there is provided a portable information terminal comprising:
[0087] a display section configured to display information;
[0088] a marker design identification section configured to receive
an observation image acquired by an image input section attached to
the portable information terminal or disposed in the portable
information terminal to identify design of a specific marker
photographed in the observation image;
[0089] a position calculation section configured to calculate
position information of the image input section from the design of
the specific marker identified by the marker design identification
section;
[0090] an information presentation section configured to display
predetermined information based on the position information
calculated by the position calculation section in the display
section to present the predetermined information; and
[0091] a contents storage section configured to store the
information to be presented as the predetermined information.
[0092] According to a fifteenth aspect of the present invention,
there is provided a portable information terminal comprising:
[0093] a display section configured to display information;
[0094] a marker extraction section configured to receive an
observation image acquired by an image input section attached to
the portable information terminal or disposed in the portable
information terminal to extract a specific marker photographed in
the observation image;
[0095] a marker design identification section configured to
identify design included in the specific marker extracted by the
marker extraction section;
[0096] a position/orientation calculation section configured to
calculate position and orientation information of the image input
section from the specific marker extracted by the marker extraction
section and the design of the specific marker identified by the
marker design identification section;
[0097] an information presentation section configured to display
predetermined information based on the position and orientation
information calculated by the position/orientation calculation
section in the display section to present the predetermined
information; and
[0098] a contents storage section configured to store the
information to be presented as the predetermined information.
[0099] According to a sixteenth aspect of the present invention,
there is provided a server which is able to be connected to a
portable information terminal including a display section by
communication, comprising at least one of:
[0100] a marker extraction section configured to extract a specific
marker photographed in an observation image acquired by an image
input section attached to the portable information terminal or
disposed in the portable information terminal;
[0101] an orientation calculation section configured to use the
marker extracted by the marker extraction section to calculate
orientation information of the image input section;
[0102] an information presentation section configured to display
predetermined information based on the orientation information
calculated by the orientation calculation section in the display
section of the portable information terminal to present the
predetermined information; and
[0103] a contents storage section configured to store information
to be presented as the predetermined information,
[0104] wherein the server transmits and receives necessary
information with one of the portable information terminal including
the sections excluding the sections disposed in the server and the
sections of another server.
[0105] According to a seventeenth aspect of the present invention,
there is provided a server which is able to be connected to a
portable information terminal including a display section by
communication, comprising at least one of:
[0106] a marker design identification section configured to
identify design of a specific marker photographed in an observation
image acquired by an image input section attached to the portable
information terminal or disposed in the portable information
terminal;
[0107] a position calculation section configured to calculate
position information of the image input section from the design of
the specific marker identified by the marker design identification
section;
[0108] an information presentation section configured to display
predetermined information based on the position information
calculated by the position calculation section in the display
section of the portable information terminal to present the
predetermined information; and
[0109] a contents storage section configured to store information
to be presented as the predetermined information,
[0110] wherein the server transmits and receives necessary
information with one of the portable information terminal including
the sections excluding the sections disposed in the server and the
sections of another server.
[0111] According to an eighteenth aspect of the present invention,
there is provided a server which is able to be connected to a
portable information terminal including a display section by
communication, comprising at least one of:
[0112] a marker extraction section configured to extract a specific
marker photographed in an observation image acquired by an image
input section attached to the portable information terminal or
disposed in the portable information terminal;
[0113] a marker design identification section configured to
identify design included in the specific marker extracted by the
marker extraction section;
[0114] a position/orientation calculation section configured to
calculate position and orientation information of the image input
section from the specific marker extracted by the marker extraction
section and the design of the specific marker identified by the
marker design identification section;
[0115] an information presentation section configured to display
predetermined information based on the position and orientation
information calculated by the position/orientation calculation
section in the display section of the portable information terminal
to present the predetermined information; and
[0116] a contents storage section configured to store information
to be presented as the predetermined information,
[0117] wherein the server transmits and receives necessary
information with one of the portable information terminal including
the sections excluding the sections disposed in the server and the
sections of another server.
[0118] Advantages of the invention will be set forth in the
description which follows, and in part will be obvious from the
description, or may be learned by practice of the invention.
Advantages of the invention may be realized and obtained by means
of the instrumentalities and combinations particularly pointed out
hereinafter.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING
[0119] The accompanying drawings, which are incorporated in and
constitute a part of the specification, illustrate embodiments of
the invention, and together with the general description given
above and the detailed description of the embodiments given below,
serve to explain the principles of the invention.
[0120] FIG. 1 is an explanatory view of a prior-art method of
obtaining position and orientation of image pickup means from a
reference marker position;
[0121] FIG. 2 is a diagram showing a constitution of a visual field
agreement type information presentation system according to a first
embodiment of the present invention;
[0122] FIG. 3 is a function block diagram of the visual field
agreement type information presentation system according to the
first embodiment;
[0123] FIG. 4 is a flowchart showing a schedule preparation
operation;
[0124] FIG. 5 is a diagram showing an image of superimposed
information displayed in a display device;
[0125] FIG. 6 is a diagram showing a two-dimensional navigation
display example;
[0126] FIG. 7 is a flowchart showing an article purchase
operation;
[0127] FIG. 8A is a diagram showing an actual display state of an
ancient monument;
[0128] FIG. 8B is a diagram showing a superimposed display state of
a restored model of the ancient monument;
[0129] FIG. 9 is a diagram showing that detailed information of an
article is displayed;
[0130] FIG. 10 is a diagram showing the constitution of the visual
field agreement type information presentation system according to a
second embodiment of the present invention;
[0131] FIG. 11 is a diagram showing the constitution of the visual
field agreement type information presentation system according to a
third embodiment of the present invention;
[0132] FIG. 12 is a diagram showing the constitution of the visual
field agreement type information presentation system according to a
fourth embodiment of the present invention; and
[0133] FIG. 13 is a diagram showing the constitution of the visual
field agreement type information presentation system according to a
fifth embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0134] Embodiments of the present invention will be described
hereinafter with reference to the drawings.
First Embodiment
[0135] As shown in FIG. 2, a visual field agreement type
information presentation system according to a first embodiment of
the present invention includes: a large number of portable
information terminals 11 (only one is representatively shown in the
drawing) for use in a specific district; and a server 12 which can
communicate with these portable information terminals 11.
[0136] Here, the portable information terminal 11 is a small-sized
information terminal which can be carried by a user of a PDA with a
camera or a notebook-size personal computer. The portable
information terminal 11 includes at least: an image pickup device
11A which is image pickup means for photographing an image; a
display device 11B which is display means for displaying the image
photographed by the image pickup device 11A and various types of
information; and a button group 11C for inputting various types of
instructions or settings. Of course, communication means for
performing communication with the server 12 is also included.
Additionally, the "image" includes dynamic and static photograph
images, and the images are similarly handled in the following
description.
[0137] Moreover, the server 12 may be disposed inside or outside
the specific district. In the latter case, although not especially
shown, a relay device for relaying communication between the
portable information terminal 11 and server 12 is included in the
specific district.
[0138] Furthermore, the specific district is dotted with facilities
13 which are photographing objects each by the image pickup device
11A of the portable information terminal 11, and a marker 14 is
disposed in the vicinity of the facility. Here, in an information
display surface of the marker 14, a unique pattern or symbol is
shown so that the marker 14 can be distinguished from another
marker. Additionally, an installation position and direction of
each marker 14 are stored in a database (not shown) of the server
12. Moreover, information associated with each marker 14 is stored
in the database of the server 12.
[0139] It is to be noted that as the marker 14, information
indicating the position and direction disclosed in Jpn. Pat. Appln.
KOKAI Publication No. 2001-118187 is displayed as a pattern
indicating the information in the information display surface. In
this case, the installation position and direction do not have to
be stored in the database of the server 12. Moreover, the facility
13 is photographed beforehand from every direction and registered
in the database of the server 12, so that the direction and
facility photographed by the image pickup device 11A of the
portable information terminal 11 can be distinguished by pattern
matching. Then, the marker 14 can be omitted.
[0140] Moreover, although not shown, in each facility 13, a device
is disposed to perform radio or cable communication with the
portable information terminal 11 and/or the server 12. From the
portable information terminal 11 or the server 12, reservation and
reservation cancel of the facility, purchase of an article sold in
the facility, or a request for distribution to a predetermined
place in the specific district is possible.
[0141] As shown in a function block diagram of FIG. 3, the visual
field agreement type information presentation system according to
the first embodiment is constituted of: an image obtaining section
1 which is image pickup means; a marker detection (characteristic
point detection) section 2 which is marker detection means; a
marker recognition (characteristic point recognition) section 3
which performs a function of a position/orientation relation
calculation section as position/orientation relation calculation
means; an associated information obtaining section 4 which is
associated information obtaining means; a model data preparation
(rendering) section 5 which performs a function of an image
processing section as image processing means; and a display section
6 which is display means.
[0142] Here, the image obtaining section 1 corresponds to the image
pickup device 11A of the portable information terminal 11. The
marker detection section 2 detects the known marker 14 existing in
the image obtained by the image obtaining section 1 (i.e., detects
a characteristic point). The marker recognition section 3
recognizes the known marker 14 detected by the marker detection
section 2 and specifies the position and direction of the marker
14. Moreover, based on the specified information, a method
described above in the related art is used to calculate the
position and orientation of the image obtaining section 1 (image
pickup device 11A), that is, the position and direction of a user
of the portable information terminal 11. The associated information
obtaining section 4 obtains predetermined information associated
with the marker 14 identified by the marker recognition section 3.
This predetermined information is image data of the image such as
an arrow indicating a movement direction. The model data
preparation section 5 processes the predetermined information
obtained by the associated information obtaining section 4 based on
the position and orientation calculated by the marker recognition
section 3. For example, based on a route to a destination set
beforehand from a disposed position of the recognized marker 14,
and the position and orientation, that is, the direction of the
image pickup device 11A, the image prepared by the image data for
indicating the movement direction. Accordingly, a guide image
indicating the movement direction to the destination is prepared.
The display section 6 corresponds to the display device 11B of the
portable information terminal 11. The display section 6
superimposes and displays the information generated by the model
data preparation section 5 on the image obtained by the image
obtaining section 1.
[0143] Here, as described above, it is necessary to dispose the
image obtaining section 1 and display section 6 in the portable
information terminal 11. However, as in the following (1) to (8),
the marker detection section 2, marker recognition section 3,
associated information obtaining section 4, and model data
preparation section 5 may also be disposed in either the portable
information terminal 11 or the server 12.
[0144] (1) The image obtaining section 1, marker detection section
2, marker recognition section 3, model data preparation section 5,
and display section 6 are disposed in the portable information
terminal 11, and the associated information obtaining section 4 is
disposed in the server 12. This has no problem as long as a
processing capability of the portable information terminal 11 is
sufficient. In this case, only the information which identifies the
marker 14 may be transmitted from the portable information terminal
11 to the server 12, and an amount of transfer data is small. A
large storage capacity necessary for storing associated information
is not required in the portable information terminal 11.
Additionally, the model data preparation section 5 requires a large
processing capability for the processing, and therefore there is a
possibility that a processing time of another operation performed
in the portable information terminal 11 is influenced.
[0145] (2) The image obtaining section 1, marker detection section
2, marker recognition section 3, and display section 6 are disposed
in the portable information terminal 11, and the associated
information obtaining section 4 and model data preparation section
5 are disposed in the server 12. In this case, only the information
which identifies the marker 14 may be transmitted from the portable
information terminal 11 to the server 12, and the amount of
transfer data is small. The large storage capacity necessary for
storing the associated information is not required in the portable
information terminal 11. Furthermore, since the server 12 performs
the processing of the model data preparation section 5 requiring
the large processing capability, there is not the problem of the
above (1).
[0146] (3) The image obtaining section 1, marker detection section
2, model data preparation section 5, and display section 6 are
disposed in the portable information terminal 11, and the marker
recognition section 3 and associated information obtaining section
4 are disposed in the server 12. Accordingly, a detected result of
the marker is transferred from the portable information terminal 11
to the server 12, and the subsequent processing is performed by the
processing capability of the server 12. Also in this case, the
large storage capacity necessary for storing the associated
information is not required in the portable information terminal
11. Additionally, in the same manner as in (1), the model data
preparation section 5 requires the large processing capability for
the processing, and therefore there is a possibility that the
processing time of another operation performed in the portable
information terminal 11 is influenced.
[0147] (4) The image obtaining section 1, marker detection section
2, and display section 6 are disposed in the portable information
terminal 11, and the marker recognition section 3, associated
information obtaining section 4, and model data preparation section
5 are disposed in the server 12. In the same manner as in (2),
since the large storage capacity necessary for storing the
associated information is not required in the portable information
terminal 11, and the server 12 performs the processing of the model
data preparation section 5 requiring the large processing
capability, there is no problem of (1).
[0148] (5) The image obtaining section 1, model data preparation
section 5, and display section 6 are disposed in the portable
information terminal 11, and the marker detection section 2, marker
recognition section 3, and associated information obtaining section
4 are disposed in the server 12. The large storage capacity
necessary for storing the associated information is not required in
the portable information terminal 11. Additionally, since the
obtained image itself is transferred to the server 12, an increase
of traffic related to the transfer of the image itself becomes a
problem. In the same manner as in (1), since the processing of the
model data preparation section 5 requires the large processing
capability, there is a possibility that the processing time of
another operation performed in the portable information terminal 11
is influenced.
[0149] (6) The image obtaining section 1 and display section 6 are
disposed in the portable information terminal 11, and the marker
detection section 2, marker recognition section 3, associated
information obtaining section 4, and model data preparation section
5 are disposed in the server 12. In this case, in the same manner
as in (2), the large storage capacity necessary for storing the
associated information is not required in the portable information
terminal 11. Since the server 12 performs the processing of the
model data preparation section 5 requiring the large processing
capability, there is not the problem of the above (1). Therefore,
an inexpensive portable information terminal having a low
processing capability can be used. Additionally, in the same manner
as in (5), the increase of traffic related to the transfer of the
image itself becomes a problem.
[0150] (7) All the image obtaining section 1, marker detection
section 2, marker recognition section 3, associated information
obtaining section 4, model data preparation section 5, and display
section 6 are disposed in the portable information terminal 11. In
this case, the portable information terminal 11 is in a stand-alone
state. The associated information needs to be stored in the
portable information terminal 1, and it is supposed that the stored
data has a limitation.
[0151] (8) The image obtaining section 1, marker detection section
2, marker recognition section 3, associated information obtaining
section 4, and display section 6 are disposed in the portable
information terminal 11, and the model data preparation section 5
is disposed in the server 12. Since the server 12 performs the
processing of the model data preparation section 5 requiring the
large processing capability, there is not the problem of the above
(1).
[0152] It is to be noted that in the above (7) the server 12 is
unnecessary, but it is possible to provide various types of
information regardless of the marker recognized result, and it is
therefore preferable to leave the server 12.
[0153] An actual application example of the visual field agreement
type information presentation system according to the present
embodiment will be described hereinafter.
[0154] [Theme Park]
[0155] First, the specific district which is a theme park will be
described. It is to be noted that here the case (6) will be
described, but any of the other constitutions (1) to (5), (7), (8)
can of course be applied. In this example, as described later in
detail, the associated information obtaining section 4 disposed in
the server 12 also has a function of a management section which is
management means for storing/managing specific district information
including at least one of: installation place information of the
marker 14 in the specific district; use situation and reservation
situation obtained by the communication with the facility; the
situation of reservation for the facility by the user of the
portable information terminal; and action record of the user in the
specific district.
[0156] That is, when buying an admission ticket, the user of the
theme park applies to a ticket window for rental of the portable
information terminal 11 prepared by the theme park, and pays an
admission fee and rental fee. The payment can be settled by a
credit card. Subsequently, the user receives the bought admission
ticket, and enters the theme park which is the specific district
via an entrance gate by the admission ticket. It is to be noted
that a number unique to the ticket such as a serial number is
printed (preferably further magnetically recorded) in the admission
ticket. A person in charge of the ticket window operates a window
terminal to register the ticket number, the rental application
information of the portable information terminal, and the card
number of the credit card into the server 12 at an issuance time of
the admission ticket.
[0157] After entering the theme park, the user who has applied for
the rental of the portable information terminal 11 goes to a rental
window of the portable information terminal in the vicinity of the
entrance gate, and follows a rental procedure. That is, the user
transfers the admission ticket to a person in charge. The person in
charge who has received the admission ticket operates the rental
window terminal to transmit the printed (magnetically recorded)
ticket number to the server 12, and can confirm whether the user is
registered in a rental list of the portable information terminal.
After the confirmation is completed, ID exclusive for the portable
information terminal is issued, and this ID is added/registered
into the record of the server 12. Moreover, this ID is also printed
on the admission ticket. Subsequently, the printed admission ticket
is transferred to the user of the portable information terminal
together with the portable information terminal 11 for rent.
[0158] It is to be noted that a way to use the portable information
terminal 11 is preferably briefly lectured in transferring the
terminal to the user. Of course, when the way to use is unclear,
the user can call a HELP screen of the portable information
terminal 11 to instantly know the way to use, or can ask a staff
member in the theme park.
[0159] Moreover, the ID newly printed in the admission ticket
returned here does not have to be often referred to in the theme
park. However, when the user comes home, logs in a Web page
operated by the theme park, and refers to snapshots or action route
record, the ID is necessary. The ID has a time limitation, and
cannot be used with an elapse of a given period. When the ID is
continued to be used, a procedure for extension is performed on the
Web page. The same ID can continuously be used for the given period
or for an extended usable period. In again buying the admission
ticket, the user shows the admission ticket in which the
continuously usable ID is described, or applies for the continued
ID. In this case, the person in charge of the ticket window can
confirm user data of the credit card used in buying the admission
ticket on the day to judge whether the ID is really continuously
used or newly used. Additionally, when the credit card for use is
changed from the previously or continuously used card, the ID needs
to be declared. In this case, it is necessary to again register
association among the user and the ID of the portable terminal use
and the credit card number, but an admission ticket sell operation
is prevented from becoming largely intricate.
[0160] Then, the user carries the rented portable information
terminal 11 and moves forward in the theme park. Here, the user
calls a preparation function of schedule from menu of the portable
information terminal, and can prepare the tour schedule of events
and attractions on the day.
[0161] That is, as shown in FIG. 4, first a guidance is displayed
in the display device 11B to designate whether or not the user is a
repeater, and accordingly the user operates the button group 11C to
check the designated result (step S11). Here, when the user is not
a repeater, that is, when the user plays at the theme park for the
first time, the user is further allowed to input members of
visitors including the user in response to the guidance display
(step S12). Then, the portable information terminal 11 displays
recommended schedule prepared by the corporation of the theme park,
which is stored beforehand or obtained by the communication with
the server 12, in the display device 11B (step S13). For the
recommended schedule, the schedule for each of groups such as
family, lovers, and children, that is, a plan of participation in
recommended attractions or events for each user group is prepared,
and selected and provided in accordance with the inputted visitor
members. Subsequently, the schedule is amended so as to suit user's
taste by the user (step S14), and is completed (step S15). In this
manner, the display device 11B and button group 11C of the portable
information terminal 11 perform a function of a destination setting
section which is destination setting means or a schedule setting
section which is schedule setting means. It is to be noted that the
schedule completed in this manner is registered in the server 12 in
accordance with the ID. That is, the associated information
obtaining section 4 of the server 12 includes a function of a
schedule storage section which is schedule storage means. During
the amending of the schedule, a waiting time of the attraction or
event at the present time is displayed, and a change is possible
such that attractions or events having a long waiting time are
toured later and available attractions or events are first
participated in.
[0162] Moreover, when the user is the repeater (step S11), the
terminal further accepts the input of ID. When there is an ID input
having the time limitation (step S16), the previous schedule and
action record stored in the server 12 is obtained and displayed by
the ID (step S17). The user adds/prepares the schedule based on the
display (step S18), associates the completed schedule with the ID,
and registers the schedule in the server 12 (step S15). In this
case, it is possible to continuously support the action. For
example, the attractions or events missed at the previous visit of
the park are toured this time.
[0163] On the other hand, when the user is the repeater (step S11),
but when there is not any input of ID (step S16), the schedule is
arbitrarily prepared (step S19), associated with the ID of the
portable information terminal, and registered in the server 12. It
is to be noted that it is preferable for the repeater not only to
set the schedule on user's own way but also to present additional
information such as limited attractions or events which are held
only in this time.
[0164] Moreover, it is more preferable to make a restaurant
reservation at a schedule preparation time. That is, the schedule
can be prepared for the whole day including a time in a specific
restaurant between the attractions or events. Of course, a
reservation situation or congestion situation of the restaurant is
managed by the server 12 in a concentrated manner and can be
presented to the portable information terminal 11 as needed. The
reservation can be made while checking vacant seats or avoiding a
busy time zone. Of course, cancellation is possible at any time.
When the user participates in the attraction or the event and
cannot be in time for a scheduled time, the reservation may be
canceled on the portable information terminal. In this manner, a
function of a facility management section which is facility
management means can be imparted to the associated information
obtaining section 4 of the server 12. The image pickup device 11A
and button group 11C of the portable information terminal 11 can
perform a function of a reservation section which is reservation
means.
[0165] The user who has prepared a series of schedules in this
manner participates in the attraction or the event as scheduled.
According to the schedule, the next site is possibly sometimes far.
The repeater probably grasps all the sites where the respective
attractions or events are held, but a first visitor has to look at
the map distributed at the entrance and to confirm the positions
one by one before moving. However, in recent years, a sufficiently
broad space has been prepared for the theme park. The visitor needs
to walk a considerably long distance while moving end to end.
[0166] The markers 14 are installed in many places in the theme
park which is the specific district. As described above, when the
marker 14 is photographed by the image pickup device 11A of the
portable information terminal 11, the server 12 can grasp the
current position of the user of the portable information terminal.
Therefore, the server 12 judges an optimum route to the next
destination position from the current position based on the
prepared schedule or by the input of the next destination by the
user, and displays guidance in the display device 11B of the
portable information terminal 11. The associated information
obtaining section 4 of the server 12 has functions of a density
check section which is density check means for checking the
congestion situation of each attraction or event from the current
position of the portable information terminal 11, and a waiting
time storage section which is waiting time storage means for
storing the waiting time. Therefore, congestion information or
waiting time information is displayed in the portable information
terminal 11 in the place or in all the terminals, and thereby the
function of a leading section can be performed, which is leading
means for leading the user to another attraction or event. At this
time, the associated information obtaining section may also include
the function of a schedule preparation section which is schedule
preparation means for generating the schedule for efficiently
routing the respective attractions or events to display the
schedule in each portable information terminal 11. Furthermore, the
function of a schedule management section which is schedule
management means for obtaining a difference from the set schedule
from the current time and current position to display the
information based on the difference in the portable information
terminal 11 may be imparted to the associated information obtaining
section 4 of the server 12.
[0167] The guidance is three-dimensionally displayed on the
actually photographed image as described above. For example, as
shown in FIG. 5, an indication of an arrow 15 along a road is shown
in the screen of the display device 11B of the portable information
terminal 11. A usual map is two-dimensionally displayed. With a
bird's eye view of the whole theme park, the current position is
generally connected to the destination by a line and shown.
However, in the display of the present system, with the user's
movement, the direction of the indication by the arrow 15 also
changes. For example, when the user shakes head, the viewed road is
simultaneously moved in the visual field by the shaking. Similarly
in the display of the present system, the direction of the
indication is changed by the shaking, and a direction in which the
user has to move is necessarily pointed. The present system has a
so-called navigation function of detecting a visual line of the
portable information terminal 11, and accurately tracking the
change of the orientation of the portable information terminal 11
to display appropriate information. Of course, as shown in FIG. 6,
in the same manner as in the prior-art map, it is also possible to
two-dimensionally display the information. In FIG. 6, reference
numeral 16 denotes a walk track display, 17 denotes the
photographed marker, 18 denotes the current position, and 19
denotes the destination.
[0168] Moreover, similarly, it is also possible to photograph the
marker 14 and to obtain guide information in the facility. That is,
the marker 14 is photographed, and a guide menu of a building is
selected. Then, a three-dimensional superimposed model of the
building whose marker 14 has been photographed can be displayed
from the current position. For the model, a so-called plain display
is also possible. It is possible to graphically obtain the
information indicating the state inside the building such as the
position of a reception desk, and the position of a toilet. Of
course, the above-described navigation function is also effective
in the building. Even after the user enters the building, and when
the destination is a specific room, an accurate instruction
continues to be issued during the movement, and the user is led to
the room.
[0169] This marker recognition is left as the record of the user's
movement in background. That is, the data is associated with the ID
and stored as the record information of the day in the server 12
for a given period before the user goes out of the theme park. Even
after the user comes home, the user can browse the information as
the track of the user of the ID in the theme park on the day on the
above-described Web page. In this manner, the associated
information obtaining section 4 of the server 12 has the function
of a track route calculation section which is track route
calculation means.
[0170] Moreover, as described above, the portable information
terminal 11 includes the image pickup device 11A which is a camera,
and it is therefore possible to take snapshots. The position
information and photographing time of the recently photographed
marker 14 are attached, and the photographed snapshots are stored
in the server 12. Therefore, a photographer does not have to
memorize details of a photographed place. After coming home, the
photographer can recognize information such as snap data attached
to an image diagram of the whole facility, a shooting place, a
photographed object, and a shooting time at a glance in the Web
page. In this manner, the associated information obtaining section
4 of the server 12 includes the function of an image storage
section which is image storage means, and the function of a
download section which is downloading means.
[0171] Furthermore, a shop (stall) for selling high-value-added
articles which can be bought only in the theme park. For the
purchase of the article, the information presented in the portable
information terminal 11 can be used to buy the article in a remote
area without actually visiting the shop.
[0172] In this case, as shown in FIG. 7, on the portable
information terminal, it is possible to select a method of
searching the articles, a method of searching shops, and a method
of searching shops only of a sub-theme-land in the theme park. The
purchase methods are prepared in accordance with users' levels.
[0173] That is, when the article is searched (step S21), a list of
all the articles dealt in by the shops in the theme park (including
at least the image and price information) is transmitted to the
portable information terminal 11 from the server 12, and displayed
(step S22). Moreover, the user of the portable information terminal
11 selects a desired article from the list (step S23), and requests
the server 12 for a purchase procedure (step S24).
[0174] Furthermore, when the shop is searched (step S25), the list
of the shops in the theme park is transmitted and displayed into
the portable information terminal 11 from the server 12 (step S26).
Subsequently, the user of the portable information terminal 11
selects the desired shop from this list (step S27). When the
desired shop is selected in this manner, a list of articles dealt
in by the selected shop (including at least the image and price
information) is transmitted to the portable information terminal 11
from the server 12 and displayed (step S22). On receiving article
selection by the user (step S23), the portable information terminal
11 requests the server 12 for the purchase procedure of the article
(step S24).
[0175] Alternatively, when the shop in the sub-theme-land is
searched (step S25), a list of sub-theme-lands is transmitted to
the portable information terminal 11 from the server 12, and
displayed (step S28). Moreover, the user of the portable
information terminal 11 selects a desired sub-theme-land from the
list (step S29). In response to the selection of the
sub-theme-land, a list of shops in the sub-theme-land is displayed
(step S26). When the desired shop is selected (step S27), further a
list of articles dealt in by the selected shop (including at least
the image and price information) is displayed (step S22). On
receiving the article selection by the user (step S23), the
portable information terminal 11 requests the server 12 for the
purchase procedure of the article (step S24).
[0176] In this manner, the associated information obtaining section
4 of the server 12 includes the function of an article management
section which is article management means, and the image pickup
device 11A and button group 11C of the portable information
terminal 11 perform the function of a purchase section which is
purchase means.
[0177] It is to be noted that the purchase procedure may be
purchase reservation of the article, or a delivery request into a
predetermined place of the theme land such as a pick-up window near
an exit gate. The server 12 instructs a person in charge for
arrangement by the terminal in the shop which deals in the article
or in the pick-up window. In this case, for the payment for the
purchased article, the credit card is used as ID authentication at
a rental time of the portable information terminal 11, the card
number is stored in the associated information obtaining section 4
which performs the function of a card number storage section as
card number storage means, and the payment is more preferably
settled by the credit card. With this credit card settlement, it is
not necessary to line up at a cash register at an article pick-up
time. When the articles are collectively received at the pick-up
window near the exit gate on one's way home, a trouble of carrying
heavy baggage while moving in the theme park is eliminated.
[0178] [Museum]
[0179] Next, the specific district which is a museum will be
described. Additionally, in highly public facilities such as the
museum, business by the rental of the portable information terminal
11 is not assumed. However, when the rental is performed, user's
social standing needs to be guaranteed for the rental in some form.
Therefore, here, in the same manner as in the theme park, the
credit card is used to rent the portable information terminal. Of
course, a method of filling in a rental sheet, and presenting
identification to perform the rental operation is also considered.
Instead of freely carrying the portable information terminal, a
mode for using the terminal only in a certain room, or an
installation mode of the terminal is also considered. With the use
in the individual room, the terminal is used in the form of renting
the terminal in an entrance at an entrance time in an exhibition
room and returning the terminal at an exit time.
[0180] For example, the user rents the portable information
terminal 11 in the entrance of the exhibition room and proceeds in
the facility. The user proceeds to an exhibit in the museum, and
the marker 14 is attached beside an explanatory panel of the
exhibit, or beside the exhibit. When the marker 14 is photographed
by the image pickup device 11A of the portable information terminal
11 or a terminal installed in front of the exhibit, the information
associated with the marker 14 appears on the screen of the display
device 11B.
[0181] For example, for a skeleton specimen of a dinosaur, a
three-dimensional image of the dinosaur to which skin is attached
is displayed in the screen. For the display, in synchronization of
the visual line of the image pickup device 11A of the portable
information terminal 11, a drawn position is accurately changed
(processed). Therefore, even when the user observes the specimen
walking around the specimen, the image is drawn accurately in
synchronization with the users movement. That is, when the user
views the skeleton specimen moving toward a tail from a head side
of the specimen, the data of the image responds to the movement,
and the drawn data changes toward the tail from the head.
Accordingly, although the specimen only of the simple skeleton is
viewed, a sense can be obtained as if a stuffed specimen of the
dinosaur including the skeleton coated with the skin were exhibited
before the user in an actual scale.
[0182] Next, an example of an exhibit showing an ancient human life
will be described. For remains of habitation sites, land including
only holes of remains of columns of a house is dug, and the dug
remains are sometimes exhibited as house remains without being
changed (see FIG. 8A). Alternatively, restored houses are sometimes
built in the site. With the use of the present system, without
actually preparing restored objects, as shown in FIG. 8B, it is
possible to view the exhibit as if the house were constructed in
the position. That is, the system comprises: preparing a
three-dimensional model in which the remains of the holes form a
foundation of the columns as a restored image; and
drawing/displaying the image on the screen of the display device
11B of the portable information terminal 11. The marker 14 is
disposed in the house remains. When the marker 14 is observed, the
three-dimensional model is drawn. In the same manner as in the
example of the skeleton specimen, the image is synchronized with a
visual point. Therefore, the drawn image changes in accordance with
the direction in which the user views the image. The user feels as
if a real thing existed before the user. Here, when the user casts
eyes to actual scenery, real space spreads including only the
remains of the houses. A merit is also considered that it is
possible to accurately obtain the sense of scales of actual houses
or village by the contrast.
[0183] With the use of the present system, it is possible to
sophisticate usual explanations of the exhibit such as written
sentences and video explanation and to increase additional values.
For example, it is assumed that there is an explanation panel
indicating a habitat territory of a stuffed animal before the
stuffed animal. In general, there would be an explanation long
enough to be easily read together with a distribution map.
Additionally, the marker 14 is described in the explanation panel.
When the marker 14 is captured in the visual field by the image
pickup device 11A of the portable information terminal 11, more
detailed information is displayed in the screen of the display
device 11B of the portable information terminal 11. For example,
video may also be drawn/displayed in the portable information
terminal 11 indicating hierarchisation, living territory, staple
food, speed of growth, character, method of breeding, species to
which the animal belongs, evolution through which the animal has
passed, and actual living.
[0184] [Shopping Mall]
[0185] Next, the specific district which is a shopping mall will be
described. In this case, when renting the portable information
terminal 11 as in the theme park, a method of presenting the credit
card by the user for the rental is supposed to be most appropriate.
Additionally, since the purchase in the shopping mall is a main
purpose for the user, it is not efficient to incur expenses for
renting the terminal. It is important to bear the expenses on a
corporation side of the shopping mall and to eliminate financial
burden on a user side. When the user presents the credit card, the
reduction of the financial burden is not the main purpose, and the
card is used to mean the identification.
[0186] For the information to be presented to the user in the
portable information terminal 11, in the same manner as in the
theme park, examples of the information include: first the map
information of the shopping mall, that is, layout information of
the shops; and then navigation information to the objective shop.
Furthermore, individual information of each shop such as clearance
sale information is also included.
[0187] Concretely, the user stands in the entrance of the shopping
mall, photographs the marker 14 installed in the place with the
image pickup device 11A of the portable information terminal 11,
and selects, for example, the map information or the guide to the
objective shop from alternatives displayed in the screen of the
display device 11B, so that the selected content is displayed.
Following the navigation, the user moves in the mall and enters the
shop. There is the marker 14 in the entrance of the shop. When the
marker 14 is photographed by the image pickup device 11A, the
guidance unique to the shop is displayed. The guidance is a
guidance of the clearance sale information of the shop, or the
position of display of the dealt-in articles. The user can grasp
the display position of the objective article relying on the screen
display without asking a shop clerk any question, and can obtain
the clearance sale information without obtaining the information
beforehand by a folded leaf advertisement of a newspaper.
Furthermore, as shown in FIG. 9, the individual marker 14 is
disposed for an article on display 29, and the individual
information may also be registered as incidental information in the
marker 14. In this case, when the marker 14 is photographed by the
image pickup device 11A of the portable information terminal 1I to
obtain the detailed guidance, it is not necessary to separately
obtain any catalog, and detailed information 21 of the article can
be obtained/displayed. In this manner, convenience for the user is
much enhanced, because it is possible to obtain the desired article
or to know the content of the desired article without any advance
knowledge beforehand or without any explanation of a sales
assistant. A merit in management is large for the corporation
because it is unnecessary to employ the sales assistants more than
necessary.
Second Embodiment
[0188] As shown in FIG. 10, the visual field agreement type
information presentation system according to a second embodiment of
the present invention includes the specific marker 14, the portable
information terminal 11, and an image input section 1A which is
image input means. The portable information terminal 11 includes: a
marker extraction section 2A corresponding to marker extraction
means; an orientation calculation section 3A corresponding to
orientation calculation means; a contents storage section 4A
corresponding to contents storage means; an information
presentation section 5A corresponding to information presentation
means; and a display section 6A corresponding to display means.
[0189] Here, the image input section 1A is a camera
attachable/detachable with respect to the portable information
terminal 11. The section can be attached to the portable
information terminal 11 to transmit real-time image data to the
portable information terminal 11. Of course, as shown by a broken
line in FIG. 10, in the same manner as in the first embodiment, the
image input section 1A may integrally be disposed in the portable
information terminal 11, such as a PDA including the camera and a
cellular phone including the camera.
[0190] The user captures the specific marker 14 including a unique
pattern or symbol in an information display surface by the image
input section 1A so that the marker can be distinguished from
another marker. Accordingly, the image data is transferred to the
marker extraction section 2A disposed in the portable information
terminal 11, and a specific marker 14 portion in the image data is
extracted. A geometric attribute of the specific marker 14 is
registered beforehand in the orientation calculation section 3A.
The orientation calculation section 3A compares the appearance of
the marker 14 obtained as the image data with the geometric
attribute to calculate the spatial orientation of the image input
section 1A by which the image data has been obtained. Moreover, for
example, as a result, it is assumed that the image input section 1A
has rotated by (.theta..sub.x, .theta..sub.y, .theta..sub.z) with
respect to an axis of the marker 14. Here, it is assumed that
predetermined information stored beforehand in the contents storage
section 4A and associated with the marker 14 is the arrow. In this
case, the information presentation section 5A imparts an offset of
(.theta..sub.x, .theta..sub.y, .theta..sub.z) to the orientation at
a registration time of the arrow, and displays presented data in
the screen of the display section 6A.
[0191] An operation image of the present embodiment will be
described hereinafter. Signboards including the markers 14 are
installed in spots in a sightseeing area. The user photographs the
marker 14 with the image input section 1A attached to or built in
the portable information terminal 11 the user has. As a result, the
name and direction of the next recommended sightseeing spot, which
are the predetermined information associated with the marker 14,
are displayed in the screen of the display section 6A of the
portable information terminal 11, and the user continues the
sightseeing following the display. When the marker 14 is
photographed, the user does not have to necessarily capture the
marker 14 from the front surface. When the arrow is registered in
accordance with a spatial coordinate with the marker 14 arranged
therein, the marker is necessarily displayed in the portable
information terminal 11 without collapsing the spatial position
relation between the marker 14 and the arrow.
Third Embodiment
[0192] As shown in FIG. 11, in the visual field agreement type
information presentation system according to a third embodiment of
the present invention, the portable information terminal 11
includes a marker design distinction section 2B which is marker
design identification means, and a position calculation section 3B
which is position calculation means, instead of the marker
extraction section 2A and orientation calculation section 3A in the
second embodiment.
[0193] In the present embodiment, the image data is transferred to
the marker design distinction section 2B from the image input
section 1A, and the design of the specific marker 14 portion in the
image data is distinguished. The design attribute of the specific
marker 14 is registered beforehand in the marker design distinction
section 2B. The marker design distinction section 2B searches the
marker which matches with the presently obtained marker from the
appearance of the marker 14 obtained as the image data.
Subsequently, the position calculation section 3B knows the change
of the design of the obtained marker with respect to the registered
marker design to calculate the spatial position where the image
input section 1A has obtained the present data. As a result, for
example, it is assumed that translation of the image input section
1A by (.delta..sub.x, .delta..sub.y, .delta..sub.z) with respect to
the axis of the marker 14 has been calculated. It is further
assumed that the predetermined information registered beforehand in
the contents storage section 4A and associated with the marker 14
is the arrow. In this case, the information presentation section 5A
imparts the offset of (.delta..sub.x, .delta..sub.y, .delta..sub.z)
to the orientation of the arrow at the registration time, and
displays the presented data in the screen of the display section
6A.
[0194] The operation image of the third embodiment will be
described hereinafter. The signboard in which the marker 14 is
disposed is installed in an observatory of the sightseeing area.
The user photographs the marker 14 with the image input section 1A
attached to or built in the portable information terminal 11 the
user has. As a result, in the screen of the display section 6A of
the portable information terminal 11, a name of a mountain in a
faraway scenery and an arrow indicating the position of a peak of
the mountain, which are the predetermined information associated
with the marker 14, are displayed, and the user can enjoy the
scenery viewed from the observatory. When the marker 14 is
photographed, the user does not have to necessarily capture the
marker from the front surface. When the arrow is registered in
accordance with the spatial coordinate with the marker 14 arranged
therein, the marker is necessarily displayed in the portable
information terminal 1 without collapsing the spatial position
relation between the marker 14 and the arrow.
Fourth Embodiment
[0195] As shown in FIG. 12, in the visual field agreement type
information presentation system according to a fourth embodiment of
the present invention, the portable information terminal 11
includes: the marker extraction section 2A in the second
embodiment; the marker design distinction section 2B in the third
embodiment; and a position/orientation calculation section 3C which
is position/orientation calculation means including both the
functions of the orientation calculation section 3A in the second
embodiment and the position calculation section 3B in the third
embodiment.
[0196] That is, in the present embodiment, the image data from the
image input section 1A is transferred to the marker extraction
section 2A disposed in the portable information terminal 11, and
the specific marker portion in the image data is extracted.
Subsequently, the data of the extracted marker portion is
transferred to the marker design distinction section 2B, and the
design of the specific marker portion in the data of the marker
portion is distinguished. The geometric attribute and design
attribute of the specific marker 14 are registered beforehand in
the position/orientation calculation section 3C. The
position/orientation calculation section 3C compares the appearance
of the marker obtained as the image data with these attribute data
to calculate the spatial position/orientation by which the image
input section 1A has obtained the image data. Moreover, as a
result, for example, it is assumed that (.theta..sub.x,
.theta..sub.y, .theta..sub.z) rotation and (.delta..sub.x,
.delta..sub.y, .delta..sub.z) translation of the image input
section 1A with respect to the axis of the marker 14 have been
calculated. It is further assumed that the predetermined
information registered beforehand in the contents storage section
4A and associated with the marker 14 is the arrow. In this case,
the information presentation section 5A imparts the offset of the
(.theta..sub.x, .theta..sub.y, .theta..sub.z) rotation and the
(.delta..sub.x, .delta..sub.y, .delta..sub.z) translation to the
orientation at the registration time of the arrow, and displays the
presented data in the screen of the display section 6A.
[0197] The operation image of the present embodiment will be
described hereinafter. The signboard in which the marker 14 is
disposed is installed in the dug remains of Jomon remains (park).
The user photographs the marker 14 with the image input section 1A
attached to or built in the portable information terminal 11 the
user has. As a result, in the screen of the display section 6A of
the portable information terminal 11, CG image of a restored
building structure, which is the predetermined information
associated with the marker 14, is displayed on a base of a dug and
appearing ancient architecture. When the marker 14 is photographed,
the user does not have to necessarily capture the marker 14 from
the front surface. The restored building can be viewed from each
direction as if the user turned around the base, that is, the user
walked viewing around the building and viewed the marker 14 from
each direction. Even in this case, the marker is necessarily
displayed in the portable information terminal 11 without
collapsing the spatial position relation between the marker 14 and
the restored building.
Fifth Embodiment
[0198] As shown in FIG. 13, in the visual field agreement type
information presentation system according to a fifth embodiment of
the present invention, the marker extraction section 2A, marker
design distinction section 2B, information presentation section 5A,
and display section 6A in the fourth embodiment are disposed in the
portable information terminal 11. The position/orientation
calculation section 3C and contents storage section 4A in the
fourth embodiment are disposed in the server 12 which can
communicate with the portable information terminal 11.
[0199] That is, in the fourth embodiment, the respective function
sections including the (image input section 1A), marker extraction
section 2A, marker design distinction section 2B,
position/orientation calculation section 3C, information
presentation section 5A, and contents storage section 4A are
disposed in the portable information terminal 11, and the operation
of the single unit of the portable information terminal 11 is
performed. However, as described above in the first embodiment, the
group of the functions other than the image input section 1A
attached to or mounted on the portable information terminal 11 to
operate is not mounted in the portable information terminal 11, and
is mounted in the server 12 with which the portable information
terminal 11 can communicate. The processes of the respective
functions are performed in the server 12. Only the results are
transmitted to the portable information terminal 11. In this case,
the display section 6A can also present the information.
[0200] In this case, the respective function sections such as the
marker extraction section 2A, marker design distinction section 2B,
position/orientation calculation section 3C, and information
presentation section 5A operate independently of one another. A
final result is not influenced by whether any of the functions is
disposed in the portable information terminal 11 or in the external
server 12. Therefore, it is possible to determine the constitution
in consideration of the capability of the portable information
terminal 11 for use or the size or actual operation situation of
the distributed information. That is, the distribution shown in
FIG. 12 is only one example, and the present invention is not
limited to this. The present invention is not limited to one
server, and the means may also be distributed to a plurality of
servers.
[0201] For the image of the operation of the present embodiment,
the operation example of the fourth embodiment will be extended and
described hereinafter.
[0202] The signboard in which the marker 14 is disposed is
installed in the dug remains of Jomon remains (park). The user
photographs the marker 14 with the image input section 1A attached
to or built in the portable information terminal 11 the user has.
The marker extraction section 2A disposed in the portable
information terminal 11 extracts the marker portion from the image
data at this time. Next, the result is similarly transferred to the
marker design distinction section 2B disposed in the portable
information terminal 11. Subsequently, the detected data of the
marker obtained as the result of calculation is transferred to the
external server 12.
[0203] The external server 12 transfers the detected data of the
transferred marker to the position/orientation calculation section
3C, and this section calculates a coordinate value of the image
input section 1A in a spatial coordinate system in which the marker
14 is disposed. Additionally, the presented data which is the
predetermined information registered in the contents storage
section 4A by the marker design and associated with the marker 14
is searched, and the final presented information is prepared.
Subsequently, the presented information is distributed to the
information presentation section 5A of the portable information
terminal 11.
[0204] As a result, in the screen of the display section 6A of the
portable information terminal 11, the CG image of the restored
building, which is the predetermined information associated with
the marker 14, is displayed on the base of the dug and appearing
ancient architecture. When the marker 14 is photographed, the user
does not have to necessarily capture the marker 14 from the front
surface. The restored building can be viewed from each direction as
if the user turned around the base, that is, the user walked
viewing around the building and viewed the marker 14 from each
direction. Even in this case, the marker is displayed in the
portable information terminal 11 without collapsing the spatial
position relation between the marker 14 and the restored
building.
[0205] For an effect in the fifth embodiment, the portable
information terminal generally having a low processing capability
does not have to perform all the processes. The processes are
scattered, the system is efficiently operated, and real-time
properties of the processing and system operation can be
secured.
[0206] It is to be noted that the communication and distribution of
the data are not especially limited as long as the data can be
transferred in environments such as cable LAN, radio LAN, PHS
circuit, cellular phone circuit, infrared ray communication, and
Bluetooth.
[0207] The present invention has been described above based on the
embodiments, but the present invention is not limited to the
embodiments, and of course the present invention can variously be
modified or applied within the scope of the present invention.
[0208] For example, the present invention can also be applied to a
cellular phone and car navigation ubiquitous computing.
Furthermore, needless to say, the present invention is not limited
to a sightseeing purpose described in the embodiment, and can also
be applied to a construction/manufacturing industry.
[0209] Moreover, the specific district is not limited to the theme
park, museum, and shopping mall described in the embodiment, and
may include larger units such as a town, state, and country.
[0210] In recent years, a system in the cellular phone including
the camera has spread which comprises: recognizing the marker
(e.g., a specific signboard or a logo on the building) set in the
city from the camera; and superimposing/displaying the associated
information in the camera screen. In this system, as the applied
example of the present invention, for the recognition process and
position/orientation calculation process of the marker, it is
possible to set a process distribution (on a terminal side and on a
server side) different with each type of the cellular phone or to
appropriately distribute the respective function processes in
accordance with a carrier.
[0211] Additional advantages and modifications will readily occur
to those skilled in the art. Therefore, the invention in its
broader aspects is not limited to the specific details and
representative embodiments shown and described herein. Accordingly,
various modifications may be made without departing from the spirit
or scope of the general invention concept as defined by the
appended claims and their equivalents.
* * * * *