U.S. patent application number 11/817149 was filed with the patent office on 2009-11-12 for mobile holographic simulator of bowling pins and virtual objects.
Invention is credited to Silvia Zambelli.
Application Number | 20090280916 11/817149 |
Document ID | / |
Family ID | 35004335 |
Filed Date | 2009-11-12 |
United States Patent
Application |
20090280916 |
Kind Code |
A1 |
Zambelli; Silvia |
November 12, 2009 |
Mobile holographic simulator of bowling pins and virtual
objects
Abstract
A system with automatic positioning controls for the holographic
display of three-dimensional objects interacting in real time with
objects of the real world. The person observing the screen (18) has
the sensation that the objects are real, which is achieved thanks
to a real time perspective correction system (1). The system is
used in particular to simulate bowling pins (9). It is positioned
on the lane required by means of a dolly (6). The system determines
a relation between the information received from the sensors: video
cameras (1) (3), dimensions (26), weight (14) (27) and the
mathematical simulation models. When the ball is bowled, its
physical features are measured and when it hits the pin deck, some
holographic pins are displayed on the screen (19). The screen
holographically arranges the projected objects over what the bowler
(18) sees behind the screen (19). The simulated movements of the
pins and the physical interactions with the real objects are
determined by exploiting the theory of mechanics in
three-dimensional space.
Inventors: |
Zambelli; Silvia;
(Sant'Agata Bolognese, IT) |
Correspondence
Address: |
ZAMBELLI, Silvia
Via Benedetto XIV n.14
Sant'Agata Bolognese
40019
omitted
|
Family ID: |
35004335 |
Appl. No.: |
11/817149 |
Filed: |
March 2, 2005 |
PCT Filed: |
March 2, 2005 |
PCT NO: |
PCT/IT05/00116 |
371 Date: |
August 26, 2007 |
Current U.S.
Class: |
473/74 ; 463/49;
463/7 |
Current CPC
Class: |
A63D 1/00 20130101; A63D
3/00 20130101; A63D 5/04 20130101 |
Class at
Publication: |
473/74 ; 463/49;
463/7 |
International
Class: |
A63D 5/00 20060101
A63D005/00 |
Claims
1-43. (canceled)
44. System that offers the visual and sonorous perception of
virtual objects interacting dynamically and in real time with real
objects having known physical dimensions, characterised by a
physically and geometrically known surface on which the real
objects may move, featuring a transparent holographic or almost
transparent screen set between the real objects and the bowler with
the function of showing the bowler virtual perspective objects
interacting in real time with real objects and simultaneously
allowing the bowler to see the real objects situated on the part of
surface that he sees through the transparent holographic screen so
that he sees the virtual objects as real, characterised by a
projection system that projects a sequence of images showing
virtual objects on the holographic screen dynamically interacting
with real objects, characterised by a system that detects the
position of the real objects, characterised by a system that
re-creates sound effects that simulate the sounds produced when the
virtual objects collide with the real objects and when the virtual
objects collide with other virtual objects in real time,
characterised by a control unit that processes the signals of the
sensors working in real time and using a physical mathematical
model that reproduces the real situation with the aim to generate
and send the sequence of images from the projector to the screen
and to generate and send the signals to the system to create the
sound effects.
45. System that offers the visual and sonorous perception of
virtual objects interacting dynamically and in real time with real
objects having known physical dimensions, characterised by a
physically and geometrically known surface on which the real
objects may move, featuring a transparent holographic or almost
transparent screen set between the real objects and the bowler with
the function of showing the bowler virtual perspective objects
interacting in real time with real objects and simultaneously
allowing the bowler to see the real objects situated on the part of
surface that he sees through the transparent holographic screen so
that he sees the virtual objects as real, characterised by a
projection system that projects a sequence of images showing
virtual objects on the holographic screen dynamically interacting
with real objects, characterised by a system that detects the
position of the real objects, characterised by a system that
re-creates sound effects that simulate the sounds produced when the
virtual objects collide with the real objects and when the virtual
objects collide with other virtual objects in real time,
characterised by a control unit that processes the signals of the
sensors working in real time and using a physical mathematical
model that reproduces the real situation with the aim to generate
and send the sequence of images from the projector to the screen
and to generate and send the signals to the system to create the
sound effects, characterised by a system that detects the position
in the space of the bowler with the aim to use this point as the
perspective point in generating the images to be projected on the
holographic screen to provide a realistic perspective view on the
screen.
46. System that offers the visual and sonorous perception of
virtual objects interacting dynamically and in real time with real
objects having known physical dimensions, characterised by a
physically and geometrically known surface on which the real
objects may move, featuring a transparent holographic or almost
transparent screen set between the real objects and the bowler with
the function of showing the bowler virtual perspective objects
interacting in real time with real objects and simultaneously
allowing the bowler to see the real objects situated on the part of
surface that he sees through the transparent holographic screen so
that he sees the virtual objects as real, characterised by a
projection system that projects a sequence of images showing
virtual objects on the holographic screen dynamically interacting
with real objects, characterised by a system that detects the
position of the real objects, characterised by a system that
re-creates sound effects that simulate the sounds produced when the
virtual objects collide with the real objects and when the virtual
objects collide with other virtual objects in real time,
characterised by a control unit that processes the signals of the
sensors working in real time and using a physical mathematical
model that reproduces the real situation with the aim to generate
and send the sequence of images from the projector to the screen
and to generate and send the signals to the system to create the
sound effects, characterised by a system that determines the type
of object among all those of known physical dimensions,
characterised by a system that detects the angular speeds and that
transmits them to the control unit that uses them to increase the
realism of the dynamic simulation of the virtual objects.
47. System that offers the visual and sonorous perception of
virtual objects interacting dynamically and in real time with real
objects having known physical dimensions, characterised by a
physically and geometrically known surface on which the real
objects may move, featuring a transparent holographic or almost
transparent screen set between the real objects and the bowler with
the function of showing the bowler virtual perspective objects
interacting in real time with real objects and simultaneously
allowing the bowler to see the real objects situated on the part of
surface that he sees through the transparent holographic screen so
that he sees the virtual objects as real, characterised by a
projection system that projects a sequence of images showing
virtual objects on the holographic screen dynamically interacting
with real objects, characterised by a system that detects the
position of the real objects, characterised by a system that
re-creates sound effects that simulate the sounds produced when the
virtual objects collide with the real objects and when the virtual
objects collide with other virtual objects in real time,
characterised by a control unit that processes the signals of the
sensors working in real time and using a physical mathematical
model that reproduces the real situation with the aim to generate
and send the sequence of images from the projector to the screen
and to generate and send the signals to the system to create the
sound effects, characterised by a system that detects the position
in the space of the bowler with the aim to use this point as the
perspective point in generating the images to be projected on the
holographic screen to provide a realistic perspective view on the
screen, characterised by a system that determines the type of
object among all those of known physical dimensions, characterised
by a system that detects the angular speeds and that transmits them
to the control unit that uses them to increase the realism of the
dynamic simulation of the virtual objects.
48. Arrangement in accordance with patent claim 44, 45, 46, 47
characterized by a holographic screen that is positioned between
the bowler (18) and the position where the virtual objects should
be if they were real (9); the screen is able to refract the high
intensity rays emitted by the projection system to obtain visible
virtual objects and characterized by the fact that it lets through
any light of minor intensity without alteration or almost without
alteration, being the light typically reflected by the real
objects, showing a transparent view of the real objects just as
they are.
49. The system of claim 48 includes air screens, fog screens with
transparent liquids or gas, holographic crystals screens, made of
almost transparent material, screens made of almost transparent
fabric or of flexible transparent rubber.
50. In the system of claim 48 the screen is positioned in the point
with the smallest perspective error from the point of view of the
bowler and in that nearest the position in the space of the first
virtual object (9). The screen must not stop the movement of the
real objects on the surface where they move.
51. The system of claim 48 includes all types of solid and rigid
screens that cannot be crossed by real objects, so the screen is
raised off the known movement surface by at least the distance
necessary to let the real objects through.
52. The system of claim 48 includes all types of non-solid or
non-rigid screens that can be crossed by real objects, which are
positioned on the known movement surface in the real point
associated unmistakeably with the point of the mathematical model
in which the virtual object nearest the bowler is positioned.
53. Arrangement in accordance with patent claim 44, 45, 46, 47 with
the function of determining the position of the real objects; the
system consists of a camera or an optic system.
54. Arrangement in accordance with patent claim 53 made up of a
camera of which the images are processed to analyse the chromatic
variations to determine the position of the objects on the known
surface. This is determined using the mathematical model of the
known surface, the model of the real objects and the
physical/geometric/optical model of the camera that sets the images
in unmistakeable relation with the real system in order to deduce
the speed and the trajectory of the real objects in real time.
55. Arrangement in accordance with patent claim 53 made up of a
system of photocells that detect the trajectory and the speed of
the real object at a known time in a known point, made up of
photocells crossed with known position and direction with the aim
to detect the crossing times and to calculate the direction and the
linear speed of the objects that cross it based on the times.
56. Arrangement in accordance with patent claim 46,47 with the
function of determining the type of real object in order to extract
and use the physical data that are saved and that are unmistakeable
for each real object of the known set of objects. The system
includes a camera or a radio identification system.
57. Arrangement in accordance with patent claim 56 made up of a
camera of which the images are analysed in the chromatic variations
and based on the comparison of probabilistic methods and chromatic
filters and using the mathematical model of the system and of the
camera to define the shape, size and superficial chromatic
distribution of the object.
58. The information stated in claim 57 is used as an unmistakeable
access key to extract the physical dimensions of the object from a
limited archive.
59. Arrangement in patent claim 57 with the aim to determine the
angular speed of the real objects by analysing the variations in
the superficial chromatic distribution of the real object, since
their geometric shape and position in the space in relation to the
camera are known.
60. Arrangement in accordance with patent claim 56 with the
function of determining the type of real object with the aim to
extract and use the physical information, known unmistakeably, for
each real object and being part of a known set of objects. The
system consists of a receiver in radio frequency and of an rfid tag
with unmistakeable code applied to the real objects. Each rfid is
an unmistakeable access key to an archive of the physical/geometric
dimensions of the object to be used to generate the simulation.
61. Arrangement in accordance with patent claim 60 characterized in
that there is a coding system that enables each real object to send
a code in radio frequency that represents its information in terms
of mass, dimensions, type of material, balances, moment of inertia,
colour, friction and number of holes.
62. Arrangement in accordance with patent claim 56 made up of a
gyroscopic sensor situated in the real object to determine the
angular speed and to send this information in radio frequency to a
receiver that sends it to the control unit.
63. Arrangement in accordance with patent claim 45, 47 with the
function of detecting the position of the bowler in the space with
the aim to use this point as a prospective point in generating the
sequence of images projected on the screen and to thus obtain a
projection that the bowler perceives based on his position in the
space. A camera is used to obtain an unmistakeable relationship
between the chromatic variations of the camera and the position in
the space of the eyes of the bowler. This relationship is obtained
via probabilistic comparison methods of the chromatic variations
with the typical chromatic variations of human shapes, the
knowledge of the surface on which the bowler is standing and using
the mathematical model of the camera of which the position and
direction in the space are known.
64. Arrangement in accordance with patent claim 44, 45, 46, 47
including a projector used to project the images on the holographic
screen.
65. Arrangement in accordance with patent claim 44, 45, 46, 47
including two projectors used to project the images on the
holographic screens that can be crossed by the real objects and
with the aim to reduce the shadows generated by the real objects as
they move.
66. The projectors in patent claim 65 display the sequences of
images turned, reflected and distorted to be adapted to where the
projectors are positioned so that all the projectors display the
same image at the same time on the screen to obtain one single
projection on the screen; therefore where the light of one
projector is obscured by an object, the other projector completes
the image in the shadowed zone.
67. The projectors in patent claim 65 are positioned so that if an
object obscures the projector light and creates a shadow on the
screen, then the second projector reaches the zone of screen in the
shadow with its rays.
68. In accordance with patent claim 67 the projectors are
positioned behind the screen, one in the right side and one in the
left side and they project on the screen in the bowler's
direction.
69. In accordance with patent claim 67 the projectors are
positioned, one in front of the screen and one behind it, so that
one projects towards the bowler and one projects in the opposite
direction.
70. Arrangement in accordance with patent claim 44, 45, 46, 47
characterized by the fact that the control unit (8) receives the
data sent from the sensors and produces a physical simulation of
the virtual objects, as near as possible to reality, using a
mechanical three-dimensional model in real time.
71. Arrangement in accordance with patent claims 44, 45, 46, 47
characterized in that with this screen (19) we can change the
properties of the real objects, such as the colour, projecting a
virtual object over the real object in real time.
72. Arrangement in accordance with patent claim 44, 45, 46, 47
characterized in that the system acquires (8) and saves the
physical parameters and is used to collect useful statistics to
improve the bowler's ability; these parameters are transmitted to
the control unit.
73. Arrangement in accordance with patent claim 44, 45, 46, 47
characterized in that we can see a replay of the real and virtual
objects.
74. Arrangement in accordance with patent claim 44, 45, 46, 47
characterized in that we can modify some of the parameters of the
real objects to simplify the game at pleasure.
75. A method for creating an optical and sonorous illusion towards
a bowler standing in a limited space and far enough away from the
holographic screen to perceive the perspective simulation as
credible, to see the virtual objects interacting with the real
objects, tracing the movements of known real objects in real time
on a known surface, setting a transparent screen between the bowler
and the space where the objects move, on which the sequences of
images are projected that show virtual objects that simulate the
physical, dynamic, optical and spatial behaviour of real objects in
real time while the real objects are seen through the transparent
screen. The sequence of images generated are generated so that the
bowler perceives the holographic objects almost as if there were
actually real objects behind the screen interacting dynamically and
in real time with the real objects.
Description
BACKGROUND ART
Definitions and Basic Concepts
[0001] Tenpin bowling is played on a wooden lane or one of
synthetic material, according to the specifications of the American
Bowling Club ABC/WIBC. 10 pins are positioned at one end of the
lane while the player throws a ball from the other end in the
attempt to knockdown as many pins as possible. Please refer to the
ABC/WIBC specifications for the complete set of game rules, as this
is merely a generic description. There are many types of games,
which differ in the number of pins, shape, positions, score rules,
shape and dimensions of the lanes, balls and pins; here are the
most popular: 10 pin, 9 pin, candle pin, 5 pin, figure, duck pin,
red pin. The wooden lane is rectangular, 19 meters long and 3
meters wide and the pins are positioned on top of it. The game is
played on the lane, which consists in throwing bowling balls
towards the pins in the attempt to knockdown as many pins possible.
The pin deck is the minimum square area on which all the pins are
positioned (9). In the case of tenpin bowling, the pins are
positioned in the shape of a triangle. Pin 1 is the pin at the head
of the triangle. The player or bowler is the person who throws the
ball in the attempt to knock down the pins. The game of bowling is
played between one or more bowlers and has the purpose of scoring
the highest number of pins knocked down in a certain number of
throws. For example, in the case of tenpin bowling, each game is
split-up into 10 frames. One frame is a combination of two throws.
The bowler throws the ball from the bowling deck: from the opposite
end to where the pins are positioned. The bowler must not step over
the foul line, which is 18 meters away from the pins. A bowler is
said to have committed a foul if he steps over the foul line when
throwing the ball. Hdcp is a bonus starting score assigned to a
less expert player in order to make the game fair. The ball return
(17) (23) is a mechanical device that returns the ball back to the
same bowler so that he can make his next throw. The platform where
the ball falls (21) is the lower part of the lane; its purpose is
that of blocking and holding the ball thrown; it is set at a slant
so that the ball, through the mere force of gravity, rolls towards
the inlet of the ball return pit, through which the ball is
returned to the beginning of the lane. It is situated at the
opposite end of the lane compared to the deck where the ball is
thrown. The mechanical pinsetter is an automatic system that puts
the real pins knocked down back in place after each throw.
Mechanical pinsetters channel, move, turn and therefore position
the real pins in established positions exploiting mechanical
systems. The lane area in which the pins are positioned by the
mechanical pinsetters is marked specifically to point out the exact
position of the pins. An animation is a sequence of static pictures
that give the human eye the impression of moving objects. These
pictures are sequentially projected against a screen at a speed of
more than 25 pictures per second and differ from one another by
slight movements in the objects contained within the pictures. The
perspective view is the point in space from which we get the
impression of depth of objects contained within a picture, when
looking at a picture produced with the laws of perspective. The
physical simulator is a computer that exploits the axioms, theorems
and formulae of mechanics. It displays a perspective animation in
real time on a screen. This animation has the fundamental feature
of reproducing the behavior of objects in the real world according
to the laws of nature. This simulator receives information from the
real world through sensors that measure the speed, position and
mass of real objects. With this information it then simulates the
behavior of objects that in reality do not exist but that interact
with the real objects. The combination of the real objects and the
unreal objects is displayed on the screen with the dynamic
behaviour and sounds that they would have if they were all real.
The holographic pinsetter is a physical simulator that gives human
beings the illusion of the presence of pins. It simulates the
behaviour and the interaction of the virtual objects with the real
objects, such as the ball and the lane. This means that the
holographic pinsetter does not need all the mechanical parts, which
make mechanical pinsetters the systems most susceptible to wear,
slower, expensive, unstable, cumbersome, heavy and generally less
efficient. The holographic screen is made of transparent material,
which when hit by the light of a projector, diffuses the light
semi-spherically, thus covering the light that crosses it. The
effect perceived by the onlooker is that the pictures projected are
seen on top of the real pictures produced by the light reflected by
the bodies of the real objects. A reflecting screen reflects the
light sent from a luminous source, such as a projector, but does
not let the light through. The arrows (69) are drawn in the centre
of the lane at a few meters from the foul line in certain positions
and act as reference marks for the bowler. There are some black
spots (47) on the lane that point out the exact position in which
the pins are to be positioned. The visual angles of a video camera
are the angles where all the objects within the two half-lines of
the angle can be seen in the picture generated by the camera.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] FIG. 1: Side-on view of a lane with mobile holographic
pinsetter, where: 1. Video camera, 2. Speaker, 3. Video camera, 4.
RF reader, 5. Projector, 6. Dolly, 7. Track attachment brackets, 8.
Pinsetter control unit, 9. Pin deck, 10. Ball before the screen,
11. Ball after the screen, 12. Ball stopping cushion, 13. Ball
outlet hole, 14. Weight sensor, 15. Track control and data
acquisition unit, 16. Console, 17. ball return pit, 18. Bowler or
player, 19. Screen, 20. Foul line, 21. Slanted deck where the ball
falls, 22. False ceiling that conceals the holographic pinsetter,
23. Outlet of the ball return, 24. Connection wire between unit 8
and projector 5, 25. Support for screen, cameras, projector,
RF-reader, speaker and control unit, 26. Ball circumference
measuring unit, 27. Rotary movement measuring sensor.
[0003] FIG. 2: Overhead view of a pair of lanes with mobile
holographic pinsetter, where: 31. Photocell of the left-hand lane,
32. Photocell of the right-hand lane, 33. Track control and data
acquisition unit, 34. Dolly driving motor, 35. Cable, 37. Ball
stopping cushion, 38. Connection cable, 39. Weight sensors, 40.
Mobile dolly, 41. Reflex reflector for the right-hand lane
photocell, 42. Reflex reflector for the left-hand lane photocell,
43. Holographic pinsetter control unit, 44. Dolly transport tracks,
45. Cable return route, 46. 4 coloured spots, 47. Pin position
spots, 48. 4 coloured spots, 49. Edging between gutter and lane,
50. Gutters, 69. Arrows, 70. Foul line
[0004] FIG. 3: Cross section view of a ball and detail of the dolly
of the transport system, where: 51. Re-chargeable battery, 52.
Control unit, 53. Gyroscopic sensor, 54. Connection cables, 55.
Lights, 56. Lights, 57. Radio antenna, 58. Radio antenna, 59.
Battery charger attachment at the bottom of the finger holes. 60.
Photocell, 61. Reflex reflector, 62. Bracket used to secure the
tracks to the ceiling, 63. Transport tracks, 64. Motor driven
cable, 65. Rope attachment to dolly 16, 66. Dolly, 67. Reflex
reflector, 68. Support bracket for the holographic pinsetter
[0005] FIG. 4: Example animation: 71. The ball rolls under the
screen 77, 72. The ball strikes the virtual pins, 73. The ball
strikes the virtual pins, 74. The ball strikes the virtual pins,
75. The ball strikes the virtual pins, 76. The ball falls to the
slanted deck, 77. Screen
[0006] FIG. 5: Optical reaction of a holographic screen and a
screen of normal translucent material, where: 81. Light source, 82.
Holographic screen, 83. Projector, 84. Light beam, 85. Light beam
after it has passed through the transparent screen, 86. Light
emitted by the projector, 87. Light emitted by the projector after
it has passed through the screen, 88. Light emitted by the
projector, 89. External light, 90. Light emitted by the projector
after it has been refracted by a screen of normal translucent
material, 91. Light of the external source after it has been
refracted by a screen of normal translucent material.
[0007] FIG. 6: Screen with mobile bottom part: 101. Screen, 102.
Projector, 103. Ball, 104. Screen on the lane, 105. End of lane,
106. View point of bowler, 107. Outlet hole, 108. Screen, 111.
Flexible flap, 112. Rigid flap. 113. Lifting piston, 114. Fixed
screen, 115. Flaps in stand-by, 116. Ball, 117. Lane, 118. Screen,
119. Mobile part of the screen, 120. Lane, 121. Curved lane
DISCLOSURE OF INVENTION
[0008] When a mechanical pinsetter is faulty, the operator commands
the control unit (15) (33) using a keyboard and instructs it to
change the faulty lane. The control unit (15) (33) starts the motor
(34) that moves the holographic pinsetter, commencing by moving the
dolly (40) (6) (66) to which elements (1) (2) (3) (4) (5) (7) (8)
(19) (24) (25) of the holographic pinsetter are connected. The
control unit (15) (33) of the dolly starts to verify the position
of the dolly (40) (6) (66) by reading the information sent to it by
the position sensor (60), fitted on the tracks (63) (44) (77). The
control unit (15) (33) counts the inputs sent from the sensor (60)
and, considering that each lane has one, it is capable of deducing
on which lane it is situated. Once it detects that the dolly has
reached the position required, it stops sending the input that
operates the motor (34) of the dolly (66) (40) (6).
[0009] When reading the first and the last lane, the sensor
receives a double input, considering that the reflex reflector (61)
consists of two reflex reflectors separated by a small
non-reflecting gap. In this way, the control unit (15) (33)
realises whether it is at the end or the beginning of the tracks.
At this stage, the control unit (15)(33) sends an electrical signal
to the control unit of the holographic pinsetter (8)(43) to
instruct it to start the simulation. The above-mentioned transport
system may not be installed and, if this is the case, each lane has
a holographic pinsetter and the functions performed by the control
unit (15) (33) and described in this report, are carried out by
control units (8) and (43). This solution offers a few advantages:
it is quicker, since there is no need to transport it to the lane
required, in which case the costs are naturally higher.
[0010] The projector (5) starts to display the animated sequence of
the pins and the bowler (18) gets the impression that the pins are
actually on the pin deck (9).
[0011] The control unit (8) (43) continuously receives the pictures
sent from the camera (1) and exploits them to change the
perspective view of the animated sequence sent to the projector
(5). In this way the bowler (18) standing on the lane, always has a
perfect view of the pins, considering that it is that nearest to
reality: this is achieved thanks to the fact that the perspective
view of the projected picture is modified based on the position of
the bowler's eyes. The control unit (8) (43) has a
three-dimensional mathematical model of the position and direction
of the lane, of all the sensors, of the screen and of all the
objects involved in the simulation. It stores the dimensions and
positions relative to all the parts making up a lane and their
physical properties according to the ABC/WIBC specifications. The
known properties of the balls for example, are the weight, the
moment of inertia, the dimensions, the mass, the elasticity, the
friction, the three-dimensional model, the positions and the
directions in space. The pictures received from the video cameras
(1) (3) are related mathematically to projections on a flat square
surface, which corresponds to the three-dimensional model of the
video cameras; through the identification in the projected pictures
of at least 4 coplanar points known in the three-dimensional model,
we can determine an unmistakeable association between the pictures
and the three-dimensional model used in the simulation and
therefore we can deduce the position, the direction and the
inclination of the video cameras, of which we also know the optical
characteristics, such as their visual angles. We can therefore
determine an area of the picture, which corresponds to the deck
where the ball is bowled, of which the significant chromatic
variations are verified in order to identify the body of the bowler
and his position in space. This is done by associating the
chromatic variations with mathematical methods of probabilistic
identification of the silhouette of the bowler. Simplifying
principles are also assumed and, considering that we know the
direction of the camera, we can deduce that the chromatic
variations in the top part of the pictures relate to the upper body
of the bowler and vice versa for the bottom part of the chromatic
variations. The 4 points are determined by finding the foul line
(70), the arrows on the lane (69) and the edgings (49) of the lane
or by arranging coloured spots (46) (48) on the edges of the lane
at fixed and known distances/positions. The control unit (8)(43)
also controls the portion of picture corresponding to the adjacent
area around the foul line and can establish if the bowler steps
over the foul line (70) while throwing the ball: in other words, if
any significant chromatic variations are detected, the event is
saved and taken into consideration when calculating the score
displayed on the screen (19) and saved in the unit (15) and (8).
The chromatic variations must have a sufficiently long duration and
be measured experimentally to be associated with the bowler's foot,
otherwise they are associated with the ball passing the line as it
is thrown.
[0012] The bowler starts to throw the ball towards the pins, which
are projected on the holographic screen (19). Remember that the
bowler only has the impression of seeing real pins on the pin deck
(9).
[0013] The animated sequence produced takes into consideration some
objects, such as the ball (11), which are set over the pins; where
the animation takes place, space is left for the real objects, by
not projecting the virtual objects. When the ball passes under the
screen (19), the RF reader (4) receives information sent to it in
radio frequency by the ball (11) and this information is then sent
from the RF reader (4) to the control unit of the pinsetter
(8)(43).
[0014] From here on the ball enters the visual range of the camera
(3) and the control unit (8) exploits the pictures sent from the
camera (3) to determine when the ball has rolled passed,
calculating its speed, its trajectory, its dimensions and its
angular speed. These measurements can only be achieved on a picture
if there is a three-dimensional model and its relation with the
pictures of the camera.
[0015] To obtain this relation, at least 4 known and coplanar
points in the three-dimensional model must be identified on the
two-dimensional pictures. To do this, the control unit (8) analyses
the pictures and determines the gutters (50) (49) and the position
of the 10 spots (47) that point out the position of the pins on the
pin deck or 4 additional coloured spots (48) arranged on the edges
of the lane at fixed and known distances. If the control unit (8)
fails to identify the 4 optional coloured spots (48), it determines
the 4 coplanar points with the 10 spots (47) and the edges of the
lane (49) present on each lane. The angular speed for balls without
gyroscopic sensors is determined by analysing the superficial
movements of the ball, which can be enhanced for plain coloured
balls by using at least 6 spots of different colours arranged on
the ball surface and positioned so that at least one is always
visible. The Spots are stuck to the surface of the ball and are
crossed by 3 non-coinciding straight lines, arranged perpendicular
to each other. The 3 straight lines meet in the geometrical centre
of the ball. These movements are then reproduced in the
three-dimensional model as projections, on the optical surface of
the camera, of spots on the spherical surface of the ball
positioned on the lane deck. In this way we can determine the
angular speed of the ball with mathematical precision, using
formulae of geometry, perspective and of classical physics. The
control unit (8) receives all this information and processes it
with the mathematical models of mechanics. It starts sending the
signal that will show the simulation of the ball striking the pins
to the projector (5). As the ball reaches the pin deck (9), the
control unit (8) displays the pins colliding against the ball (72)
(73) (74) (75) and starts to move them as if they were moving in
reality.
[0016] The union in the animated sequence of the real ball and the
virtual pins, and all this in real time, produces a visual effect
that gives the bowler the impression of playing with physically
real pins, as seen in FIG. 4. The bowler will have the sensation of
playing with real pins, especially thanks to the use of holographic
screens.
[0017] In FIG. 4 we have illustrated a sequence of pictures that
represent an example of what the bowler sees after throwing a ball.
In FIG. 4, each pictures has a frame with black border (77) that
represents the transparent holographic screen of the holographic
pinsetter. Everything that you can see in each of the pictures of
FIG. 4 are objects that actually exist, with the exception of the
pins, which as you can see, are always within the black frame, in
other words within the transparent screen of the holographic
pinsetter. The pins are in actual fact projected by the projector
on the screen that sets them on top of the objects behind the
transparent screen. The real game is simulated by synchronising the
real objects, which move behind the screen (the ball), with the
objects projected on the screen by the projector (the pins). In the
sequence of pictures from (71) to (76) you can see that the real
ball strikes and moves the virtual pins just as if it would do if
the pins were actually on the lane.
[0018] You can also use non-transparent reflecting screens if you
want to hide everything situated on the other side; this means that
the animated sequence will also show objects covered by the screen,
such as the ball and lane. In this case, all the colours of the
ball and of the lane are captured by the pictures taken by the
video camera (3) and are reproduced in the virtual sequence on the
screen (19).
[0019] The real ball will then hit the end of the lane (12) (37)
and, thanks to the slanted platform at the end (21), it will fall
through the outlet hole (13). Its diameter will then be measured by
the dimensional measuring unit (26) (39) and finally it will be
weighed again by the weight sensor (14), of which there is one for
the pair of lanes. This sensor will send the information to the
control unit (15) (33) of the tracks (44) (63), which will then
send this information to the control unit (8) (43) of the
holographic pinsetter (FIG. 1) These sensors (14) (26) weigh and
measure the dimensions of bowling balls that do not have a radio
frequency transmission system of the weight/dimensions (52) (58)
(57), so that anybody having a standard ball can still play using
the holographic pinsetter (FIG. 1) without loosing any realistic
effects.
[0020] To be pointed out is the fact that the ball stopping cushion
(12) is equipped with a movement sensor (27) that measures the
quantity of motion absorbed each time it is hit by the ball. It
then sends this information to the control unit (8), which is aware
of the known, determined and constant physical parameters of the
cushion (12) and is capable of calculating the weight of the ball
that hit it. Considering however that the weight and dimensions of
the ball are measured after the ball reaches the end of the lane
(12) this information is registered by the control unit (8) and
utilised the next time the bowler throws his ball again. We
therefore assume that all bowlers tend to use balls of the same
weight and dimensions. This means that the system must have the
list of bowlers and keep track of their score to be able to foresee
whose turn it is to play and to be able to save the dynamic
information of the balls used by each bowler; this information is
entered either on the console (16) or the keyboard of the control
unit (15) at the beginning of each game. The dimensions of the ball
are also determined through a perspective calculation, using the
pictures of the camera (3) and the three-dimensional model
described previously.
[0021] Later, the ball will be returned to the bowler by the ball
return (17) and the control unit (8) will show the pins still
standing. The bowler can make his next throw, which will cause the
repetition of the events in the sequence just described.
[0022] Considering that the main feature of the holographic
pinsetter (FIG. 1) is that of displaying non-existent objects and
making them seem real and interacting with real objects, many
special effects can be created, such as that of showing a trail of
fire along the trajectory of the ball just thrown, objects that
score extra points if hit and so on.
[0023] The part of holographic pinsetter made up of elements (8)
(6) (7) (23) (1) (2) (3) (4) (5) is covered and concealed behind
the false ceiling (22), which has a triangular shape and hides the
equipment away from the bowler's sight (video cameras (1) (3),
control unit (8) and transport tracks (7) etc. Considering that the
screen (19) and the supports that connect it to the dolly (6) are
transparent, the bowler doesn't really see them.
[0024] The holographic screen (108) must remain a few meters away
from the end of the lane and must remain suspended from the ground
to leave enough room so that no interference is created with the
ball (103) as it rolls along the lane. This distance from the pin
set-up causes a perspective error that we can correct using the
camera (1) that observes the bowler. To avoid this correction we
can use a screen, as seen in (118) (119), where the picture is
projected on the screen that touches the lane (120). The bottom
part of this screen has some mobile elements (119) (115) so that
when the ball (116) touches them they raise (111) (112) to let the
ball through and return to their original position (115) as soon as
the ball has reached the other side. In this way we can project the
pins in their original position, thus minimising the perspective
error.
[0025] This mobile part consists of reflecting flaps (119) that
are; secured to the rigid screen (114). Their special feature is
that they are mobile (112) or flexible (115) and have the same
properties as a screen. To prevent the flaps from being hit by the
ball and becoming dirty, we can lift them before the ball actually
hits them (112) by installing a piston (113) that raises the flap.
The piston is controlled by the control unit (8), which determines
via camera (3), where the ball will hit the flaps (119); each flap
has a piston (113) that is independently controlled by the control
unit (8). Another variant is that of projecting also the picture on
the lane using an additional screen (104). In this way the
perspective error is reduced and the flaps eliminated (119). A
curved screen (121) is added at the end of the lane to improve the
perspective view. The transparent screen (101) is placed above the
pin deck instead of a few meters away from it, as in position
(108).
[0026] In this case there are two reflecting screens, one at the
end of the lane (104), on the pin deck, and one in position (101).
The picture projected by the projector (102) must be such to give
the observer (106) the impression of seeing real pins on the pin
deck. The two pictures projected on the screens (104) (101) are
composed by the bowler's eyes (f. 7.6) because he has the
impression that the pins are real.
Functional Description of the Project Elements
[0027] Screen (19): The function of the screen is that of
displaying the animated sequence received from the control unit and
of showing it to the bowler (18), giving the latter the impression
that the pins (9) are actually on the pin deck. This effect is
obtained by displaying a three-dimensional perspective animated
sequence that has the bowler's eye as the perspective point. The
screen is raised off the lane to let the ball past and is set at a
minimum distance of approximately 4 meters away from the first pin
so that the ball is hidden from the screen, thus enabling the
screen to set the light emitted by the projector over that
reflected from the ball and from the lane. This feature is
extremely important to give the simulated game sequence the best
possible realistic effect. FIG. 5 illustrates a holographic screen
on the left with the characteristics requested and a normal
transparent screen on the right. These screens are made of
polymers, the composition of which is known to producers such as
Hitachi or Holopro for example.
[0028] The light (84) emitted by an external source (81) reaches
the screen and passes through it without being reflected (85); in
this way the screen does not reflect the light of objects that are
not involved in the simulation. The light (86) sent from the
projector (83) is diffused towards the observer, as can be seen in
(87), so that the system can display objects that the observer
perceives to be behind the screen. This realistic sensation is
improved through a perspective correction of the objects projected,
which is done in real time, considering that we identify the
variations in position of the perspective point, which in our case
is the bowler's eye.
[0029] FIG. 5 shows that the light (88) (89) sent from the
projector and the external sources, is refracted in all directions
(90) (91) creating the undesired effects of normal transparent
material. To be pointed out is that holographic screens prove
useful if you wish to install the holographic pinsetter on a
standard bowling lane, leaving the end part of the lane visible,
while it proves useful to use opaque ones if you wish to hide
everything beyond the screen. [0030] Control unit (8)(43): This
unit receives the pictures from the video cameras (1)(3) and
determines the position in space of the video camera. It determines
when the ball rolls past, detecting its speed, trajectory,
dimensions and angular speed. It receives information from the
RF-Tag reader (4) to read the dimensions and weight of the ball. It
receives the picture from the camera 1(1) and determines the
position of the bowler's head and checks if the bowler steps over
the foul line (20). It saves the measurements sent to it by the
weight and dimension sensors (14). It processes the information and
produces a physical simulation of the game dynamics. It sends a
signal to the projector (5) that displays the simulation in real
time as the ball rolls past. It also sends some signals to the
speakers (2), which the latter convert into sounds feasibly
produced by the pins hitting each other or the ball or the lane. It
also controls the actions performed on the console (16) and
displays the requests on the screen (19), thus interacting with the
user. [0031] Projector (5): It receives the signal (24) from the
control unit and projects it on the screen (19) giving the visual
sensation of the pins and the ball. [0032] Camera (1): It aims at
the bowling deck and sends the pictures to the control unit, which
determines the three-dimensional position of the eyes of the
bowler. This enables an adjustment of the perspective view of the
animated sequence so that the bowler always has a real view of the
pin deck (9). This camera determines whether the player steps over
the foul line (20). It does so by controlling significant changes
in the chromatic range in the zone around the foul line. [0033]
Camera (3): It aims at the pin deck and the pictures sent are used
by the control unit (8) to measure when the ball rolls past, its
trajectory, dimensions, colour and speed of the ball (11) thrown by
the bowler. This information is transmitted to the control unit in
the form of pictures, which the latter then processes and converts
into a format that can be used within the processes of the
processing unit. The result obtained is a signal sent to the
projector and utilised by the projector to display the simulated
game sequence on the screen. To do this, the control unit knows the
position in the three dimensions of the camera (3). In the picture
sent to it by the camera (1), the control unit can determine the
height, position and angle of the video camera (3) in space. This
information is required to calculate the trajectory and speed of
the ball throughout the game. [0034] Ball stopping cushion (12):
This is required to stop the ball and let it drop into the slanted
surface (21), where it falls through the hole (13) and is
channelled into the tracks of the ball return pit (17) to return it
to the beginning of the lane (23). This cushion can also be
equipped with a movement sensor (27) (46) that measures the motion
absorbed each time it is hit by the ball; the information is sent
to the control unit (15)(33) via a cable. This information is
required to deduce the mass of the ball from its speed, from its
dimensions, from the mass of the cushion and its movement. The
dimensions of the ball and the speed are measured by the pictures
of the Camera (3) while the movement sensor (27) (46) is fitted on
one end of the cushion and connected to the control unit (33) (15).
This sensor is an alternative solution to the weight sensor (14)
(39) and is used to integrate or replace the RF-reader (4). Its
main purpose is that of measuring balls that are not equipped with
RF-transmitter and gyroscopic sensor. [0035] Weight sensor
(14)(39): It is positioned under the slanted surface (21) or near
the inlet hole (13) of the ball return; the sensor is used to weigh
the ball. This sensor represents an alternative to the RF sensor
(58), since it makes the same measurement. [0036] Ball dimension
measuring unit (26): The sensor is fitted at the beginning of the
ball return pit and is used to measure the diameter of the ball.
This sensor consists of an elastic cylinder, which when crossed by
the ball, stretches a position sensor fitted on the outer surface
of the elastic cylinder. This sensor sends an electrical input in
proportion with the amount it stretches and that will correspond to
the circumference of the ball. To be pointed out is that the
dimensions of the diameter of the ball are also measured by the
control unit of the holographic pinsetter via the picture of camera
1. [0037] RFreader (4): This device receives the information
transmitted by the ball (11) in radio frequency and sends it to the
control unit (8). This is repeated each time the ball passes under
the screen (19). [0038] Speakers (2): These convert the electric
signals created by the control unit into sounds that simulate the
sounds created by the pins when they hit each other and when they
collide with the ball and the lane. This is synchronised with the
physical simulation of the game and, in other words, with what is
seen on the screen. [0039] Console(16) : This is a keyboard that is
used to send the codes of the keys pressed by the user to the
control unit (8), which, based on the keys pressed, enables the
user to enter, modify and cancel information. Some examples: You
can enter names of the bowlers, you can modify the bowler's score
or you can modify the pins knocked down during the game, you can
choose the type of game and the type of pins, the level of
difficulty and so forth. [0040] Ball (f.3) (10) (11): The ball
contains a control unit (52), which transmits an unmistakable code
in radio frequency (57) (58) that depends on the weight and
dimensions of the ball. The control unit (52) sends a code to the
rf-reader (4) as the ball (11) passes under the screen (19), then
the control unit (8) converts the code into the real weight and
dimensions of the ball. In this way, each time the ball rolls past,
the control unit is capable of knowing with which weight and
diameter the physical simulation of the game is to be constructed.
The ball contains a gyroscopic sensor (53), which transmits the
angular speed of the ball to the control unit (52) each time the
ball passes under the screen; the control unit (52) then transmits
this speed to the control unit (8) via a radio frequency
transmitter (57) (58). The control unit also triggers a lamp that
is built-in the ball (55) (56) only when the ball is moving or
better still only when the gyroscopic sensor (53) transmits a
movement signal to the control unit (52). The lamp is battery
powered (51), which is again fitted inside the ball. This lamp is
positioned so that it illuminates the outer surface of the ball
from the inside, thus illuminating the actual ball as it rolls
along the lane. The battery is re-charged using two contacts (59)
arranged in the bottom of the finger holes, which all bowling balls
have. [0041] Movement tracks (63) (44): This structure is used to
position the holographic pinsetter on the lane required. A motor
(34), which is controlled by the control unit (33)(15), which in
turn is controlled by the user, transports the system onto the lane
chosen. The tracks are secured to the ceiling by rigid supports
(62), making the structure static. The screen (19), the control
unit (8), the camera (1), the camera (3), the RF reader (4) and the
speakers (2) are fitted to the structure (68) of a dolly (66)(40)
that runs along these tracks. The dolly (66)(40) is driven in both
directions by a motor (34) by means of the cable (64) arranged at
the end of the track that is operated by a control unit (33), which
30 performs the commands given by the user. A position sensor (60)
provides information on where the dolly is actually situated using
some reflex reflectors (61) (67) positioned on the support brackets
(62) of the tracks. This is needed to be able to stop the
servo-motor (34) of the dolly when it has reached the position
required. [0042] Foul Photocells: If the bowler steps over the foul
line (20), this is detected by the unit (8) that analyses the
picture sent by camera (1). Another method of detecting the foul is
carried out by more expensive photocells. The control unit (33)(15)
receives information from an infrared photocell (31) (32). This
photocell, by means of a reflex reflector (41) (42), sends
electrical inputs to the unit (33) (15), which sends the
information to the unit (8) (43). If the duration of this
electrical input is long enough, it means that the bowler has
stepped over the foul line, otherwise it means that it is just the
ball that has crossed the photocell. [0043] Track control unit (33)
(15): This unit receives commands from the user via the keyboard.
The user commands the control unit (33) of the tracks (44),
informing them on which lane the mobile holographic pinsetter is to
be positioned. This unit starts the motor (34) that moves the dolly
(40) of the mobile unit, in the direction required. This command
ends when the unit (33) receives information that the dolly (40)
has reached the position required. Information on the position of
the dolly is sent to the unit by the position sensor (60) installed
on the dolly (66) (40). This sensor (60) is an infrared photocell
which, as the reflex reflectors (61) (67), positioned on the
supporting brackets (62) of the tracks (63) are crossed, sends a
signal to the unit (33). An electrical input is thus sent from the
dolly to the unit, which the latter interprets as information on
the position. This unit (33) receives information from the weight
sensor (14), from the foul sensors (31)(32) and from the
dimensional sensors (26)(39) of all the lanes and saves all the
information and transmits it to the control unit of the holographic
pinsetter (43)(8). This unit (33) therefore has the task of
receiving the data from the fixed sensors of which each lane is
equipped with one for each type and that of sending the data to the
mobile unit (43)(8).To be pointed out is that if we have a solution
without tracks and therefore a permanent replacement of the
mechanical pinsetter, the functions performed by this unit (33)(15)
are carried out by unit (8)(43). The connections to the weight
sensors (14), to the foul sensors (31)(32) and to the dimensional
sensors (26) are accomplished through unit (8)(43). The track
system having the task of moving the holographic pinsetter onto the
lane required is no longer of any use, considering that each lane
has a dedicated and fixed holographic pinsetter.
BEST MODE FOR CARRYING OUT THE INVENTION
[0044] As described below.
* * * * *