U.S. patent application number 13/645692 was filed with the patent office on 2013-06-20 for system amd method for a song specific keyboard.
This patent application is currently assigned to REAL KEYS MUSIC INC. The applicant listed for this patent is BRUCE WILLIAM CICHOWLAS. Invention is credited to BRUCE WILLIAM CICHOWLAS.
Application Number | 20130157761 13/645692 |
Document ID | / |
Family ID | 48610659 |
Filed Date | 2013-06-20 |
United States Patent
Application |
20130157761 |
Kind Code |
A1 |
CICHOWLAS; BRUCE WILLIAM |
June 20, 2013 |
SYSTEM AMD METHOD FOR A SONG SPECIFIC KEYBOARD
Abstract
A melody based game system includes a controller, a computing
unit, and a display. The controller has a limited number of
programmable real keys configured to be manipulated by a player for
playing one or more song segments and thereby providing game input.
Each of the song segments includes one or more pitches and one or
more of the programmable real keys are programmed to generate the
one or more pitches comprised within each of the song segments,
respectively. The computing unit communicates with the controller
and includes a music application. The display is connected to the
computing unit and is configured to display prompts for
manipulating the one or more programmed real keys in coordination
with playing of the one or more pitches of each of the song
segments.
Inventors: |
CICHOWLAS; BRUCE WILLIAM;
(FRAMINGHAM, MA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
CICHOWLAS; BRUCE WILLIAM |
FRAMINGHAM |
MA |
US |
|
|
Assignee: |
REAL KEYS MUSIC INC
|
Family ID: |
48610659 |
Appl. No.: |
13/645692 |
Filed: |
October 5, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61543393 |
Oct 5, 2011 |
|
|
|
Current U.S.
Class: |
463/31 |
Current CPC
Class: |
A63F 13/42 20140902;
G09B 15/003 20130101; G10H 2240/325 20130101; G10H 2220/131
20130101; A63F 13/814 20140902; G10H 1/0016 20130101; G10H 1/368
20130101; G10H 2220/135 20130101; A63F 13/46 20140902; G10H
2220/241 20130101; G09B 15/023 20130101; A63F 13/2145 20140902;
G10H 2230/015 20130101; G10H 2210/091 20130101 |
Class at
Publication: |
463/31 |
International
Class: |
A63F 9/24 20060101
A63F009/24 |
Claims
1. A melody based game system configured to receive game input by
playing one or more song segments comprising: a controller
comprising a limited number of programmable real keys configured to
be manipulated by a player for playing one or more song segments
and thereby providing game input, and wherein each of said song
segments comprises one or more pitches and wherein one or more of
the programmable real keys are programmed to generate the one or
more pitches comprised within each of the song segments,
respectively; a computing unit communicating with said controller
and comprising a music application; a display connected to said
computing unit and configured to display prompts for manipulating
the one or more programmed real keys in coordination with playing
of the one or more pitches of each of the song segments; wherein
said music application simultaneously causes a song segment to be
played and provides prompts indicating a manipulation sequence of
the one or more programmed real keys corresponding to the playing
of the one or more pitches of the song segment; and wherein
manipulation of the one or more programmed real keys by the player
coinciding with the displayed prompts in pitch, intensity, and
timing during the playing of the one or more pitches of the song
segment causes a positive scoring event.
2. The melody based game system of claim 1, wherein said computing
unit comprises one of a computer, a tablet computer, an iPad.TM.,
an iPhone.TM., a Smartphone, a Playstation.TM., an Xbox.TM., a
Wii.TM., a PlayStation.TM., a Nintendo DS.TM., a game controlling
device or a handheld game controlling device.
3. The melody based game system of claim 1, further comprising a
database comprising musical recordings and data specific to each
musical recording that associate pitches of each musical recording
with a specific manipulation sequence of said real keys.
4. The melody based game system of claim 1, wherein said musical
recordings comprise one of song segments, song fragments, musical
arrangements, instrumental musical pieces, or vocal musical
pieces.
5. The melody based game system of claim 1, wherein no manipulation
of the one or more programmed real keys by the player coinciding
with the displayed prompts or manipulation of the programmed real
keys by the player not coinciding in pitch, intensity, and timing
with the simultaneously displayed prompts during the playing of the
one or more pitches of the song segment causes a negative scoring
event.
6. The melody based game system of claim 1, wherein the number of
the programmable real keys is in the range between 3 and 11.
7. The melody based game system of claim 1, wherein the number of
the programmable real keys is seven.
8. The melody based game system of claim 1, wherein the
programmable real keys comprise touch areas defined on a screen of
a touch enabled display.
9. The melody based game system of claim 1 wherein said computing
unit comprises a tablet computer having a touch enabled display
screen and wherein said controller and said programmable real keys
comprise touch areas defined in said touch enabled display
screen.
10. The melody base game system of claim 1, wherein said one or
more programmable real keys that are programmed to generate the one
or more pitches comprised within each of the song segments are
designated during the playing of each of the song segments with a
designation comprising one of light, colors, shapes, numbers,
letters, textures, font type, font size, or key relative
position.
11. The melody based game system of claim 1, wherein a melody of
any length comprising a plurality of song segments is played by
sequential reprogramming of the programmable real keys.
12. The melody based game system of claim 1, further comprising a
bidirectional communication mechanism between said computing unit
and said controller.
13. The melody based game system of claim 1, wherein said prompts
comprise lighted prompts that move along paths terminating onto the
programmed real keys.
14. A melody based game system configured to receive game input by
playing one or more song segments comprising: a tablet computer
comprising a touch enabled display screen and a music application;
a controller comprising a limited number of programmable real keys
configured to be manipulated by a player for playing one or more
song segments and thereby providing game input, and wherein each of
said song segments comprises one or more pitches and wherein one or
more of the programmable real keys are programmed to generate the
one or more pitches comprised within each of the song segments,
respectively; wherein said music application simultaneously causes
a song segment to be played and provides prompts indicating a
manipulation sequence of the one or more programmed real keys
corresponding to the playing of the one or more pitches of the song
segment; wherein said controller and said programmable real keys
comprise touch areas defined in said touch enabled display screen
and wherein said prompts are displayed on said touch enabled
display screen; and wherein manipulation of the one or more
programmed real keys by the player coinciding with the prompts in
pitch, intensity, and timing during the playing of the one or more
pitches of the song segment causes a positive scoring event.
15. A method for a melody based game system configured to receive
game input by playing one or more song segments comprising:
providing a controller comprising a limited number of programmable
real keys configured to be manipulated by a player for playing one
or more song segments and thereby providing game input, and wherein
each of said song segments comprises one or more pitches and
wherein one or more of the programmable real keys are programmed to
generate the one or more pitches comprised within each of the song
segments, respectively; providing a computing unit communicating
with said controller and comprising a music application; providing
a display connected to said computing unit and configured to
display prompts for manipulating the one or more programmed real
keys in coordination with playing of the one or more pitches of
each of the song segments; wherein said music application
simultaneously causes a song segment to be played and provides
prompts indicating a manipulation sequence of the one or more
programmed real keys corresponding to the playing of the one or
more pitches of the song segment; and wherein manipulation of the
one or more programmed real keys by the player coinciding with the
displayed prompts in pitch, intensity, and timing during the
playing of the one or more pitches of the song segment causes a
positive scoring event.
16. The method of claim 15, wherein said computing unit comprises
one of a computer, a tablet computer, an iPad.TM., an iPhone.TM., a
Smartphone a Playstation.TM., an Xbox.TM., a Wii.TM., a
PlayStation.TM., a Nintendo DS.TM., a game controlling device or a
handheld game controlling device.
17. The method of claim 15, further comprising providing a database
comprising musical recordings and data specific to each musical
recording that associate pitches of each musical recording with a
specific manipulation sequence of said real keys.
18. The method of claim 15, wherein said musical recordings
comprise one of song segments, song fragments, musical
arrangements, instrumental musical pieces, or vocal musical
pieces.
19. The method of claim 15, wherein no manipulation of the one or
more programmed real keys by the player coinciding with the
displayed prompts or manipulation of the programmed real keys by
the player not coinciding in pitch, intensity, and timing with the
simultaneously displayed prompts during the playing of the one or
more pitches of the song segment causes a negative scoring
event.
20. The method of claim 15, wherein the number of the programmable
real keys is in the range between 3 and 11.
21. The method of claim 15, wherein the number of the programmable
real keys is seven.
22. The method of claim 15, wherein said one or more programmable
real keys that are programmed to generate the one or more pitches
comprised within each of the song segments are designated during
the playing of each of the song segments with a designation
comprising one of light, colors, shapes, numbers, letters,
textures, font type, font size, or key relative position.
23. The method of claim 15, wherein a melody of any length
comprising a plurality of song segments is played by sequential
reprogramming of the programmable real keys.
24. The method of claim 15, wherein said computing unit comprises a
tablet computer having a touch enabled display screen and wherein
said controller and said programmable real keys comprise touch
areas defined in said touch enabled display screen.
Description
CROSS REFERENCE TO RELATED CO-PENDING APPLICATIONS
[0001] This application claims the benefit of U.S. provisional
application Ser. No. 61/543,393 filed on Oct. 5, 2011 and entitled
SYSTEM AMD METHOD FOR A SONG SPECIFIC KEYBOARD which is commonly
assigned and the contents of which are expressly incorporated
herein by reference.
FIELD OF THE INVENTION
[0002] The present invention relates to a system and a method for a
simplified "song specific" keyboard, and in particular to an
electronic game that utilizes an electronic board with a limited
number of buttons/keys for providing game input as prompted by a
game display and usually in coordination with a recorded recognized
song or song segment.
BACKGROUND OF THE INVENTION
[0003] The idea of teaching the playing of an actual keyboard
instrument through a computer has been discussed in prior art
documents, such as US2006/0252503. There are also several prior art
systems in the market that try to teach piano through a computer,
notably the Miracle Piano Systems, published by Software Toolworks,
Nintendo, Synthesia published byHDPiano.com, among others. However,
these devices are primarily intended for education, and in general
are not effective in having fun with music or learning to play
actual phrases and parts of music.
[0004] There are also several electronic games that receive game
input from musical instruments, such as Guitar Hero, published by
RedOctane, Rock Band and Dance Dance Revolution. However, these
systems are rhythm action games that utilize mock-type (i.e., not
"real") musical instruments and do not provide significant music
education value. A "real" musical instrument, as defined here, has
means for selecting the "pitches" of a song and includes both
actual instruments and musical instrument simulators that have
places on the instrument that can be associated with a "pitch".
"Pitch" represents the perceived fundamental frequency of a
sound.
[0005] Accordingly, there is a need for a musical instrument
teaching method and system that is fun to play as a game and more
effective than presently available music teaching systems and
methods.
SUMMARY OF THE INVENTION
[0006] The invention provides an electronic game that utilizes an
electronic board, or a controller, or a real musical instrument for
providing game input as prompted by a game display and usually in
coordination with a recorded recognized song or song segment.
[0007] In general, in one aspect, the invention features a melody
based game system configured to receive game input by playing one
or more song segments including a controller, a computing unit, and
a display. The controller has a limited number of programmable real
keys configured to be manipulated by a player for playing one or
more song segments and thereby providing game input. Each of the
song segments includes one or more pitches and one or more of the
programmable real keys are programmed to generate the one or more
pitches comprised within each of the song segments, respectively.
The computing unit communicates with the controller and includes a
music application. The display is connected to the computing unit
and is configured to display prompts for manipulating the one or
more programmed real keys in coordination with playing of the one
or more pitches of each of the song segments. The music application
simultaneously causes a song segment to be played and provides
prompts indicating a manipulation sequence of the one or more
programmed real keys corresponding to the playing of the one or
more pitches of the song segment. Manipulation of the one or more
programmed real keys by the player coinciding with the displayed
prompts in pitch, intensity, and timing during the playing of the
one or more pitches of the song segment causes a positive scoring
event.
[0008] Implementations of this aspect of the invention may include
one or more of the following features. The computing unit may be a
computer, a tablet computer, an iPad.TM., an iPhone.TM., a
Smartphone a Playstation.TM., an Xbox.TM., a Wii.TM., a
PlayStation.TM., a Nintendo DS.TM., a game controlling device or a
handheld game controlling device. The game system may further
includes a database comprising musical recordings and data specific
to each musical recording that associate pitches of each musical
recording with a specific manipulation sequence of the real keys.
The musical recordings may be song segments, song fragments,
musical arrangements, instrumental musical pieces, or vocal musical
pieces. No manipulation of the one or more programmed real keys by
the player coinciding with the displayed prompts or manipulation of
the programmed real keys by the player not coinciding in pitch,
intensity, and timing with the simultaneously displayed prompts
during the playing of the one or more pitches of the song segment
causes a negative scoring event. The number of the programmable
real keys is in the range between 3 and 11. The number of the
programmable real keys is seven. The programmable real keys are
touch areas defined on a screen of a touch enabled display. The
computing unit may be a tablet computer having a touch enabled
display screen and the controller and the programmable real keys
are touch areas defined in the touch enabled display screen. The
one or more programmable real keys that are programmed to generate
the one or more pitches comprised within each of the song segments
are designated during the playing of each of the song segments with
a designation comprising one of light, colors, shapes, numbers,
letters, textures, font type, font size, or key relative position.
A melody of any length including a plurality of song segments is
played by sequential reprogramming of the programmable real keys.
The game system further includes a bidirectional communication
mechanism between the computing unit and the controller.
[0009] In general, in another aspect, the invention features a
melody based game system configured to receive game input by
playing one or more song segments including a tablet computer and a
controller. The tablet computer has a touch enabled display screen
and a music application. The controller includes a limited number
of programmable real keys configured to be manipulated by a player
for playing one or more song segments and thereby providing game
input. Each of the song segments comprises one or more pitches and
the one or more of the programmable real keys are programmed to
generate the one or more pitches comprised within each of the song
segments, respectively. The music application simultaneously causes
a song segment to be played and provides prompts indicating a
manipulation sequence of the one or more programmed real keys
corresponding to the playing of the one or more pitches of the song
segment. The controller and the programmable real keys comprise
touch areas defined in the touch enabled display screen and the
prompts are displayed on the touch enabled display screen.
Manipulation of the one or more programmed real keys by the player
coinciding with the prompts in pitch, intensity, and timing during
the playing of the one or more pitches of the song segment causes a
positive scoring event.
[0010] In general, in another aspect, the invention features a
method for a melody based game system configured to receive game
input by playing one or more song segments including the following.
First, providing a controller including a limited number of
programmable real keys configured to be manipulated by a player for
playing one or more song segments and thereby providing game input.
Each of the song segments comprises one or more pitches and one or
more of the programmable real keys are programmed to generate the
one or more pitches comprised within each of the song segments,
respectively. Next, providing a computing unit communicating with
the controller and comprising a music application. Next, providing
a display connected to the computing unit and configured to display
prompts for manipulating the one or more programmed real keys in
coordination with playing of the one or more pitches of each of the
song segments. The music application simultaneously causes a song
segment to be played and provides prompts indicating a manipulation
sequence of the one or more programmed real keys corresponding to
the playing of the one or more pitches of the song segment.
Manipulation of the one or more programmed real keys by the player
coinciding with the displayed prompts in pitch, intensity, and
timing during the playing of the one or more pitches of the song
segment causes a positive scoring event.
[0011] Among the advantages of the invention may be one or more of
the following. The present invention combines melody and rhythm
action into a new generation of instrument games, expanding the
class of musical genre offered, and the demographics to be reached.
Real melody and its corresponding notes are associated with
colors/patterns.
[0012] The corresponding colors/patterns are then associated with
specific "keys"/positions on a musical instrument or a
"controller", so that when a "key"/position is pressed a
recognizable frequency (i.e., a note) is generated which is
associated with the desired note. When the right combinations of
these notes are pressed at the right time or rhythm, then the
melody is "played" correctly. The "played` melody may be audibly
heard by the player and/or scored for its correctness. In cases
where the "controller" is not an actual musical instrument, but is
able to select or generate different musical pitches (frequencies)
from its various "keys"/buttons that are programmed by the game
console, then in effect, the game has turned the controller into a
musical instrument. The colors/patterns may be displayed on a
screen/display and the screen/display may be integrated with the
"controller`/musical instrument or may be separate from the
"controller`/musical instrument.
[0013] Melody based action games are different from rhythm based
games because they have different pitches (frequencies) that are
actually being played, whereas rhythm only based games are
associated only with the timing of the actions. Melody based action
games have the advantage that they can be used for teaching music
while allowing the players to have fun with the game. This is
especially true, when the controller is a real musical instrument,
such as a piano, keyboard, guitar, or wind instrument, among
others. Melody based action games are also challenging and fun to
play because they allow the game players to hear the fruit of their
success in matching notes to colors/patterns to keys/positions and
thereby to produce real melodies. Appendix C lists some further
differences between this melody/rhythm game and existing games.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] FIG. 1A depicts a schematic diagram of the hardware
architecture for a simplified interactive musical instrument game
of this invention;
[0015] FIG. 1B is a schematic diagram of the hardware architecture
for another simplified interactive musical instrument game of this
invention;
[0016] FIG. 2A depicts a set of rectangular touch areas on an
electronic board of FIG. 1B;
[0017] FIG. 2B depicts a set of rectangular touch areas on an
iPad.TM.;
[0018] FIG. 3A depicts a song segment from "Tainted Love";
[0019] FIG. 3B depicts a set of rectangular touch areas programmed
for playing the song segment of FIG. 3A;
[0020] FIG. 4A depicts a song segment from "Mary had a little
lamb";
[0021] FIG. 4B depicts a set of rectangular touch areas programmed
for playing the song segment of FIG. 4A;
[0022] FIG. 5A depicts a song segment from "96 Tears";
[0023] FIG. 5B depicts a set of rectangular touch areas programmed
for playing the song segment of FIG. 5A;
[0024] FIG. 6A-FIG. 7D depict various touch areas arrangements and
designs;
[0025] FIG. 8A depicts a song (Doors "Light My Fire) with three
different song segments;
[0026] FIG. 8B depicts the lighting of the keys in a keyboard for
playing the song segments of FIG. 8A;
[0027] FIG. 8C depicts the lighting of the touch areas in a song
specific board for playing the song segments of FIG. 8A;
[0028] FIG. 8D depicts an implementation of the interactive musical
instrument game of this invention in an iPad.TM. tablet;
[0029] FIG. 8E-FIG. 8O depict various touch areas arrangements and
designs in the implementation of FIG. 8D;
[0030] FIG. 9A depicts the notes for Sweet Child O'Mine song;
[0031] FIG. 9B depicts the lighting of the touch areas in a song
specific board for playing the song segment of FIG. 9A;
[0032] FIG. 9C depicts the lighting of the keys in a keyboard for
playing the song segment of FIG. 9A;
[0033] FIG. 10A depicts the notes for Bohemian Rhapsody song;
[0034] FIG. 10B depicts the lighting of the touch areas in a song
specific board for playing the song segment of FIG. 10A;
[0035] FIG. 10C depicts the lighting of the keys in a keyboard for
playing the song segment of FIG. 10A;
[0036] FIG. 11 depicts a block diagram of the process of developing
the interactive musical instrument game of this invention;
[0037] FIG. 12 depicts a block diagram of the process of playing
the interactive musical instrument game of this invention;
[0038] FIG. 13 depicts a schematic diagram of the hardware
architecture for the interactive musical instrument game of this
invention;
[0039] FIG. 14 is an another embodiment of the hardware
architecture for the interactive musical instrument game of this
invention; and
[0040] FIG. 15A-15B depict typical examples of song segments;
[0041] FIG. 16A depicts the "Mary Had A Little Lamb" song
segment;
[0042] FIG. 16B-16C depict the corresponding multi-colour lighting
of the focused keyboard for the song segment of FIG. 16A played in
the C-key and Eb key, respectively;
[0043] FIG. 17-18 depict the lighting of focused keyboard for the
song segments of FIG. 8A;
[0044] FIG. 19A depicts a song segment of "Two Hearts That Beat As
One" by U2;
[0045] FIG. 19B depicts the lighting of the focused keyboard for
the song segment of FIG. 19A;
[0046] FIG. 20A depicts a song segment (Roy Orbison's Blue
Bayou);
[0047] FIG. 20B depicts the lighting of the focused keyboard for
the song segment of FIG. 20A;
[0048] FIG. 21A-21C depict a simplified right-hand accompaniment
pattern to John Lennon's "Imagine" and its representation on the
focused keyboard of this invention;
[0049] FIG. 21E depicts left (572) and right (574) groups of notes
accompaniment pattern to John Lennon's "Imagine" and their
corresponding representations on a non-focused keyboard;
[0050] FIG. 21F depicts a scale-oriented system for a song in the
key of C major and the corresponding non-focused keyboard with
multi-color key lighting;
[0051] FIG. 22 depicts the display of FIG. 1A and FIG. 13;
[0052] FIG. 23 depicts a game scoring indication in the display of
FIG. 22;
[0053] FIG. 24 depicts an alternative game scoring indication in
the display of FIG. 22;
[0054] FIG. 25A depicts an alternative keyboard for game 100 of
FIG. 13;
[0055] FIG. 25B depicts an arcade version of game 100 of FIG.
13.
[0056] FIG. 26 depicts a method of showing the players the audio
balance between what they are playing and what the recorded band is
playing;
[0057] FIG. 27A depicts a guitar with multi-color lighted fretboard
for the song segment of FIG. 16A played in the C-key;
[0058] FIG. 27B depicts a bass guitar with multi-color lighted
fretboard for the song segment of FIG. 16A played in the C-key;
[0059] FIG. 27C depicts a soprano sax with multi-color lighted keys
for the song segment of FIG. 16A played in the C-key;
[0060] FIG. 27D depicts an alto sax with multi-color lighted keys
for the song segment of FIG. 16A played in the C-key;
[0061] FIG. 27E depicts an Electronic Wind Instrument (EWI) with
multi-color lighted key for the song segment of FIG. 16A played in
the C-key;
DETAILED DESCRIPTION OF THE INVENTION
[0062] Referring to FIG. 1A, a simplified (or "song-specific")
interactive melody and rhythm action musical game 700 includes an
electronic board/controller 720 and a computing unit 702 connected
to the electronic board/controller 720 and to a display 706 and to
speakers 703a, 703b. The computing unit 702 may be a computer, an
Xbox.TM., a Wii.TM., a Playstation.TM., an iPad.TM., an iPhone.TM.,
a Smartphone, a tablet computer, or any other computing or game
controlling device. In one example, musical game 700 is implemented
in an iPad.TM. computer 702 that also includes the controller 720,
the display 706 and the speakers 703a, 703b. In this case, display
706 is a touch enabled display and controller 720 is implemented to
be an area of the touch enabled display. The computing unit 702
includes the Real Key Music (RKM) software 750 and a song library
760. Library 760 is a digital storage unit including recorded
songs, song fragments, musical arrangements, instrumental musical
pieces, and vocal musical pieces, among others. Examples of song
segments are shown in FIG. 4A. In other embodiments, library 760
may be a database stored in an external storage device or may be
downloaded from an online web-site via a network connection.
Speakers 703a, 703b may be separate speakers or may be integrated
within the computer 702, the electronic board 720, or the display
706. Speakers 703a, 703b broadcast sound generated from the playing
of the recorded musical pieces and or the playing of the electronic
board/controller 720. In some embodiments electronic
board/controller 720 includes a second set of speakers dedicated
for broadcasting music played on the electronic board/controller
itself. In FIG. 1A, the electronic board/controller 720 is shown to
be a rectangular board with four colored buttons 722a, 722b, 722c
and 722d. Buttons 722a, 722b, 722c, 722d are color coded with a
constant (non-changing) color. In the example of FIG. 1A button
722a is green, button 722b is red, button 722c is yellow and button
722d is blue. The buttons may be colored or they may have a colored
dot, sticker, or light. In other embodiments, electronic
board/controller 720 may have any shape including circular, oval,
triangular, hexagonal, polygonal, curved, spherical, cubical, among
others, or may have the shape of a keyboard, a Keytar (i.e., a
keyboard or synthesizer supported by a strap around the neck and
shoulders of the player), a typewriter keyboard, a computer
keyboard in which the keys are software mapped to correspond to
music notes, a saxophone or other wind-type instrument, a guitar,
or any other musical instrument. In other embodiments, the
electronic board/controller 720 is shaped as any game pad
controller or is integrated with a game controlling device or in
general with the computing unit 702.
[0063] Electronic board/controller 720 includes only as many
buttons 722a-722d as are needed to play any of the included songs
or a song segments. Some songs or song segments may not use all the
buttons. In some embodiments there is an indication that only some
of the buttons are needed for a particular song. In other
embodiments the buttons that are not used are implied by not using
buttons of certain color or shape. As was explained above, a song
segment is defined as a partial segment of a song in time, a
partial segment in terms of which instrumental or vocal parts were
included and/or a partial segment in that it is a simplification of
the actual musical notes played. In one example, an electronic
board/controller 720 configured to play "Mary Had A Little Lamb"
has only four buttons 722a-722d, as shown in FIG. 1A. In another
example, an electronic board/controller to play "Mary Had A Little
Lamb" has only three buttons, depending upon how the melody is
written. Each button corresponds to a "pitch" used in the song or
song segment being played. "Pitch" represents the perceived
fundamental frequency of a sound and in this case the song pitches
represent the set of perceived fundamental frequencies that
characterize the song or song segment. The buttons may still be
ordered by pitch, i.e., high to low or the reverse. As was
described above, almost any piece of music can be broken down into
song segments that involve a relatively small number of pitches and
therefore a small number of buttons can be used to play the son or
song segment.
[0064] One can play a melody of any length by using a sequence of
simplified "song-specific" electronic boards. A sequence of
"song-specific" electronic boards is actually a sequence of the
same electronic board 720 shown in FIG. 1A in which buttons
722a-722d are reprogrammed to generate different pitches
corresponding to pitches for a different song or different segment
of the song. In one embodiment, electronic board/controller 720
recognizes (i.e., "knows") it is time to reprogram itself by
observing what buttons the player has played and therefore knows
when the first segment of the song is over. Alternately, if the
player is playing along with a recorded version of the song or even
a prompting device as simple as a visual and/or audio metronome,
the electronic board "knows" when to switch to a different board
setup by knowing when the song started and the time elapsed. In yet
another embodiment, the player "tells" the electronic
board/controller 720 to switch to a different board setup by
pushing a button or a foot-switch, making a gesture or speaking a
certain word. The different "song-specific" electronic boards
within the same sequence may use a different number of buttons.
Therefore, sometimes there are some buttons that are not used. In
one example the electronic board/controller has thirteen buttons,
but one actually uses an average of seven buttons in a large number
of songs or song segments. Song segments that are suitable for
beginner players may only use four or five buttons and song
segments that are suitable for advanced players may use as many as
all thirteen buttons. The buttons do not need to be lighted and
usually only need an electrical contact closure. Electronic boards
with electrical contact buttons are usually manufactured
inexpensively. The unused buttons may be identified or not. The
unused buttons may be identified by their position, shape or color.
In one example, the unused buttons are selected to be at one or the
other "end" of the electronic board, or in a different row, or in a
specifically marked area of the electronic board. In another
example, it is indicated that the buttons/keys used for a specific
song are only the black buttons/keys or the first five
buttons/keys. In addition, the buttons/keys that are used may be
repositioned so that they are, for instance, all next to one
another or within easy reach. The group of buttons/keys 722a, 722b,
722d used in a song segment are shown in the display 706 as
buttons/keys 712a, 712b, 712d, as shown in FIG. 1A.
[0065] Since exactly the same set of physical buttons/keys is used
for the different sets of pitches of each song segment, it is
possible to always have these buttons/keys coded the same way for
identification purposes, regardless of the actual song or pitches,
provided that we always order the pitches used in the song in an
accommodating and corresponding way. In one example, within a song
segment, we always associate "blue" with the lowest pitch actually
used, "green" with the second lowest, "red" with the third lowest,
and light blue with the fourth lowest. New colors are added as
necessary. We then map those color/pitches to a linear set of
buttons in which the first is colored "blue", the second "green",
the third "red", and the fourth "light blue". In other embodiments,
the white keys of a music keyboard, or the adjacent strings or the
adjacent frets of an appropriately equipped stringed instrument or
the fingerings of a wind instrument, are tinted in a similar way as
"blue", "green", "red" and "light blue". In yet other embodiments,
footpads, button of a game controller, areas of a touch sensitive
surface, areas pointed at (with appropriate detection) or even
facial gestures detected are associated with the colors described
above. In some embodiments, a series of tinted or otherwise
differentiated light beams are used instead of buttons/keys and one
plays melodies by entering fingers, hands, legs, another object or
the shadow of any of the mentioned objects into the light beam.
These various implementations could lead to some novel and dramatic
ways to play melodies and other melodic parts on stage or
accommodate the limited physically expressive possibilities of a
handicapped person.
[0066] Library 160 includes, in addition to the recorded songs,
data specific to each song which associate the pitches of the song
with specific buttons in the electronic board, as will be described
below. The display 706 may be a computer display, a TV, a video
game console, an arcade machine display or any other display
device. Display 706 displays the image of a virtual electronic
board 708 corresponding to the actual electronic board 720, colored
lines 711a-711c, prompts 714a-714c and a target bar (or landing
area) 707. The virtual electronic board 708 is designed to have the
same number of buttons as the actual electronic board/controller
720. The displayed image is controlled by the RKM software 750. The
RKM software 750 causes specific buttons 712a-712c of the virtual
electronic board 708 to become highlighted with different colors
depending on the pitches of the song that is played. The
programming of the electronic board buttons 722a-722d is also
controlled by the RKM software 750. Lines 711a-711d diverge from a
common starting point and end at a point directly over the buttons
712a-712d, that are highlighted. They are spaced apart by non-equal
spaces and the common starting point may be off the screen. As
different buttons are highlighted during the playing of a song, the
end points of path lines 711a-711d move to point toward the buttons
that are highlighted while their starting point remains fixed. In
this way they act as visual pointers toward the buttons that are
hit on the virtual electronic board 708 and therefore need to be
hit on the actual electronic board 720. Prompts 714a-714c run along
path lines 711a-711c, respectively, and extinguish after they
arrive at the target bar/landing area 707. The time prompts
714a-714d hit the target bar 707 corresponds to the time the player
needs to hit the corresponding buttons on the actual electronic
board/controller 720 in order to have a successful score. Buttons
712a-712d, path lines 711a-711d and prompts 714a-714d have the same
color, respectively. In one example buttons 712a, path line 711a
and prompt 714a are blue, button 712b, path line 711b, and prompt
714b are green and button 712c, path line 711c, and prompt 714c are
red. Display 706 also includes a list of the available songs and
song segments 118, as shown in FIG. 22. In some embodiments, the
target bar/landing area 707 has the shape of the electronic
board/musical instrument and lights up if it is hit at the right
time to indicate the buttons/keys that are targeted. In yet other
embodiments, the target bar/landing area 707 is omitted or
coincides with the image of the electronic board 708. In these
cases the prompts 714a, 714b, 714d indicating the notes that are to
be played continue to move along the path lines 711a, 711b, 711d
onto the buttons/keys 712a, 712b, 712d, respectively, or change
direction after hitting the top of the board image 708 or target
bar 707. Path lines 711a-711d may continue and be visible under the
buttons/keys 712a-712d. In other embodiments, lines 712a-712d are
parallel to each other. In any of these embodiments, the score for
the note could simply appear on or near the related key rather than
following the path of the note.
[0067] Referring to FIG. 1B, in another embodiment, the simplified
(or "song-specific") interactive melody and rhythm action musical
game includes an electronic board/controller 720 and a computing
unit 702 connected to the electronic board/controller 720. No
display is included in this embodiment. The buttons/keys 722a-722d
that need to be played are indicated by lighted prompts 754a-754d
that move along paths 751a-751d, respectively, on the electronic
board 720. In one example, lighted prompts 754a-754b are mini light
emitting diodes (LEDs). In this embodiment, the electronic
board/controller 720, the indication prompts 754a-754d, the
computing unit 702 and the speakers 703a, 703b are integrated into
one unit 750.
[0068] In some embodiments, the system also includes a
bidirectional communication mechanism 704 between the computer 702
and the electronic board/controller 720, similar to the one shown
in FIG. 1A. Mechanism 704 carries signals that program buttons
722a-722d on the actual electronic board/controller 720 to
correspond to the specific pitches of a song or a song segment.
Mechanism 704 also transmits signals from the electronic board
buttons 722a-722d to the computer 702. In one example, mechanism
704 is implement via bidirectional musical instrument digital
interface (MIDI) communications. MIDI is an industry standard
protocol that enables electronic musical instruments and computers
to communicate, control, synchronize and exchange data with each
other. A MIDI instrument or controller transmits "event messages"
such as the pitch and intensity of the musical notes that are
played, control signals for parameters such a volume and vibrato
and clock signals to set the tempo. When a musical performance is
played on a MIDI instrument (or controller) it transmits MIDI
channel messages from its MIDI Out connector.
[0069] A reverse MIDI keyboard system may also be used to light up
buttons/keys 722a-722d in the electronic board/controller 720 from
a game controller. MIDI keyboards normally transmit pitch and
velocity (among other things) to a processor. The pitch is an
integer between 0 and 127 and corresponds to a chromatic pitch.
Similarly the velocity is also in that range with `0` being used to
indicate off. If we reverse this and consider the velocity number
to be a color instead (for instance a color selection from a list
of up to 127 colors with 0 representing off), then the processor
controlling the button lighting is analogous to a keyboard sending
data to the processor, except that it is in the reverse direction.
Both include pitch. Instead of 127 possible degrees of velocity we
send up to 127 variations of color, using the same bits in the
message format that would have been used to send velocity. It is
almost as if the computer is pressing a note in return and the
harder it presses the note, the more different colors it is able to
achieve.
[0070] In other embodiments, keys wired in a way similar to a
computer keyboard are used. In particular, most Windows PC and Mac
keyboards do an adequate job of relaying both key presses and
releases to the processor in a timely manner. In these cases, the
key-caps of keyboard are replaced with colored key caps. In one
example, a row of keys on a computer keyboard is programmed to
correspond to the pitches used in a song. In this example, the row
A-S-D-F-G-H-J-K-L-;- is used. The keys are marked with colored
stickers or are replaced with colored key-caps. One or more players
can use the programmed computer keyboard to play alone or in
cooperation with each other or against each other using the songs
stored in the library 760.
[0071] As was mentioned above, the computing unit 702 may be an
Xbox.TM., a Wii.TM., a Play station.TM., an iPad.TM., an
iPhone.TM., a Smartphone, a tablet computer, or any other computing
or game controlling device. These type of controllers have very
limited digital output bits, which makes it difficult to integrate
them with other gaming systems including the systems of this
invention. This integration problem is addressed by utilizing the
motor output signals of the controlling device. In particular, an
Xbox.TM. controller has two motor outputs. However, the output to
these motors is not a steady DC signal that could be considered to
be a zero or one. Instead, if a motor is on, the output to it is a
steady pulse wave with controllable duty cycle (pulse width). The
software specifies a value between 0.0 and 1.0 to the controller
and it delivers a wave to the motor with a duty cycle from about 5%
to 50%. A table shows what output values produce what duty cycle.
By measuring the width of the resulting pulses on the controller,
using digital or analog logic or a microprocessor, it is possible
to retrieve digital data. Approximately 40 pulses occur per second.
By timing the pulses, each pulse can be categorized into several
length categories. If there are, for example, eight distinguishable
length categories, that yields three bits of information, meaning
that we could use each of the two motor signals to transfer
reliably up to 120 bits/second, even if we distinguish only eight
different lengths of pulses. Combining the two motor signals gives
us 240 bits/second. For the lighted keyboard system 700, only one
bit of data need be transmitted for each key position since we need
only distinguish between on and off. If colors are used in a
predetermined successive order (as they can be by the game), then
we simply give each "on" key the next successive color. This means
that for a 25-key keyboard, we can relay the necessary information
for a change of lighting in about 1/10 second, which is timely
enough for these purposes.
[0072] Another scheme that allows us to control the keyboard
lighting with about 1/10 second response, even if the output
connection to the keyboard was limited to 10 bits/second includes
the following. Since the one or more successive lighting changes
for a particular song are known in advance of playing the song,
they could each be transmitted before the playing of the song
began, even if it took several seconds to do so. In this case, the
keyboard has a small amount of memory that remembers each of the
lighting configuration for each time a change in lighting occurs,
and then, by simply transmitting one bit to start the song and a
bit each time a lighting change is to occur the processor can
control the lighting of the keyboard even under extremely low data
communication conditions. These considerations are relevant for
most gaming platforms because they frequently only support a very
limited number of controllers. The easiest way to add a new piece
of peripheral hardware to such a system is to have it emulate a
controller that the gaming platform already supports.
[0073] The advantages of the simplified interactive melody and
rhythm action musical game includes the following. An electronic
board, or a controller or a musical instrument with a small number
of buttons/keys is less intimidating than one with many more
buttons/keys. An electronic board or a controller or a musical
instrument with a small number of buttons/keys is similar to a
simple video game control in complexity and therefore easy to use.
A player with relatively little experience can play a melody of any
length by using a succession of programmed simplified
"song-specific electronic keyboard pitch arrangement". In some
embodiments, the electronic board is designed to be easily used by
players with many types of physical disabilities. The electronic
board/controller/musical instrument is portable and is also
adaptable to other interesting interpretations (for instance,
waving arms or feet to get different pitches).
[0074] Referring to FIG. 2A and FIG. 2B, in this example a
touchscreen 120 (such as the Apple iPad.TM.) is used in the
prototype implementation of the electronic board. In touchscreen
120, touch areas 122a-122d are defined on the screen through a
software. These areas 122a-122d may be of any size, shape and
arrangement. In this example, areas 122a-122d are a series of
rectangular areas in a row. In this example, touch areas 122a-122d
are colored to make the illustrations cleaner and because it can be
an aid to someone playing a game or learning music. However, it is
not necessary to use a color for each button or even to use color
at all. And it is also possible to use other differentiators than
color, such as shape or size.
[0075] It is assumed that a "song segment" is a subset of the note
pitches of a piece of music, delimited optionally by time section
(such as "verse", "chorus", "intro" . . . ) within the performance,
instrument being used or some other criteria (such as "right hand
notes of the piano part"). A "song segment" from the well known
keyboard synthesizer "riff" of Tainted Love (made famous by Soft
Cell and others) is shown in FIG. 3A. This "song segment" spans a
range from G to Eb on the treble clef. This spans 8 chromatic
pitches (G-Ab-A-Bb-B-C-Db-D-Eb) and would normally be performed on
a keyboard of at least 25 keys (two octaves) since it spans two
C-to-C octaves. However, upon examination of the song segment, we
find that there are actually only 4 pitches played within this
particular song segment: [0076] G-Bb-C-Eb Thus, if the performer
has at least four different keys to press (or gestures to make or
pads to stomp or anything else discernibly different), they can
perform this song segment accurately provided that those four keys
or other mechanisms are tuned to play G, Bb, C and Eb. The other
keys are simply not required for this song segment and can
therefore be omitted. Using the example above, we could use the
setup in FIG. 3B to play the song of FIG. 3A.
[0077] In another example, if we were using the same physical or
touchscreen keyboard to play Mary Had A Little Lamb, shown in FIG.
4A, we could tune it as shown in FIG. 4B. If a song segment
requires a different number of pitches, we can programmatically add
additional buttons or use additional switches on a physical device.
For instance, this combo organ part from 96 Tears (made famous by
Question Mark and the Mysterians) uses 6 pitches from a range of 10
chromatic pitches and would typically be performed on a keyboard of
two octaves (25 keys) or more, as shown in FIG. 5A. However, here
we do it with just six buttons and still can accomplish the
original performance, as shown in FIG. 5B.
[0078] As was mentioned above, these "buttons" need not be actual
buttons or be colored or follow a regular pattern. For example,
they could be arranged in other ways in order to be more appealing
or more dramatic to a particular audience, to fit into a theme or
to coordinate with other activities or needs. Examples of the
various button shapes and arrangements are shown in FIG. 6A-FIG.
7C.
[0079] FIG. 8D shows an implementation on an iPad.TM., where we
take advantage of its self-contained touch screen, computer and
audio system. In FIG. 8D only the pitches used in the song have
colored keys 622a-622e. In FIG. 8E, this effect is enhanced by
fading the keys that are not used in the song somewhat, though
still maintaining a look similar to a standard keyboard. In FIG.
8F, the keys not used are almost shadow-like. In FIG. 8G, they are
not visible at all. Each step along the way makes it more obvious
to the player at the quickest glance which keys are going to be
used and which are not. Since this lighting is shown even before
the song begins, it also helps the player to mentally and
physically prepare for the song, all of which results in a better
performance and better learning retention by the player.
[0080] In FIG. 8H, we have completely removed the keys not used in
the song and adopted a simple setup and spacing for the keys
622a-622f used in the song. It is important to note that, unlike a
rhythm game, the player is still actually playing the melody when
they use this song specific keyboard because each key always plays
the same pitch for the duration of the song and all pitches used in
the melody are still present. Because this is a very distinct
visual simplification from the original piano-style keyboard,
novices play the songs much better. And since they are still
hearing themselves play and learning the timing and relative
pitches of the notes, they are able to transfer these skills
backwards with relative ease back to the conventional keyboard,
especially with the confidence they have gained in actually playing
the song.
[0081] In FIG. 8I, we see one further benefit of the song-specific
keyboard. Since only the pitches present in the song are used and
since the keys have a regular and simplified layout, we can
transfer this screen setup to a much smaller device such as an
iPhone.TM. and still leave the touch points far enough away from
one another that the player is able to select them easily and
accurately.
[0082] For some songs, the simplification is even more drastic,
eading to positive player experiences even more quickly. FIG.
81-FIG. 80 show the same succession of keyboard styles that would
be used for the melodic synthesizer riff of Tainted Love, which
only involved four distinct pitches and therefore only four keys
622a-622d are used.
[0083] FIG. 7A and FIG. 7B depicts an image of structures on a
stage set. The performer may stand in front of them and play on one
structure with each hand. Or, arranged differently, a performer
might touch one with a hand and the other with a foot. Or they
could be touched by two different performers. Even though the
colors might match from one to another, they could still be
different sets of pitches used in a particular song segment. And
since these are song-specific, they could be automatically
reprogrammed to different pitches depending on the song segment
being played at that particular time. For example, the performer
would be playing one melody at one part of the show and a different
melody at another part.
[0084] The coloring need not be different for each pitch and may
follow a color scheme based on some other performance or aesthetic
need. For instance, the implementation of FIG. 7C has only two
colors, but is decorative and could perform any song-segment with
six different pitches or less.
[0085] The song-specific concept can be used for song-segments with
any number of pitches. For example, FIG. 7D represents a decorative
touch pad designed to be able to play song segments with up to ten
pitches. The principle remains that the keyboard or other structure
need only be implemented and programmed to control as many pitches
as are used in a particular segment of a song rather than a larger
set of more general scalar or chromatic pitches. Then in another
segment, possibly following immediately after this one, the keys
are immediately reprogrammed to a different set of pitches
corresponding to the next song segment played.
[0086] People who have limited range of motions, such as having
only eye movement, moving only a few fingers or other limitations
often have limited chance for creative and/or playful activities.
The pitches of a song-specific keyboard are mapped by sensors or
other means to the few motions that a person can make, and this
allows the person with the limited range of motion to play musical
games or experiment with creating music using a set of chosen
pitches or using the same pitches as in a preferred song-segment.
The motion detection need not be through a physical device. A
mechanism such as Microsoft's Kinect.TM. may be used to detect the
motions and interpret them as different keys of a song-specific
keyboard. In one example, a patient's leg motions are converted to
the notes of a scale, similar to a slide trombone. However, adding
an implementation of song-specific technology, means that instead
of being limited to `n` notes of the range of a particular modal or
chromatic scale, the patient can now play any song segment with up
to `n` pitches, thereby greatly increasing the amount of music he
can perform.
[0087] A song-specific implementation can dramatically reduce the
number of keys needed for a particular song-passage, thereby
reducing the number of keys from what might be found on a
professional music keyboard to what might be inexpensively found on
a cereal box toy.
[0088] Referring to FIG. 8A, the well-known opening to "Light My
Fire" made popular by The Doors is divided into four "song
segments": 532a, 532b, 532c and 532d. FIG. 8B shows the keys needed
to play each part on a conventional 25-key keyboard. As can be
seen, this passage requires the larger part of a 25-key keyboard.
It uses chromatic pitches from C# (122a) to D an octave higher
(122c) for a range of 14 chromatic pitches
(C#-D-D#-E-F-F#-G-G#-A-A#-B-C-C#-D), 8 white keys and 6 black keys.
The corresponding song-specific keyboard is shown in FIG. 8C. The
pitches under the keys are for reference and are not displayed.
They show how the pitches change for each of the buttons during the
four sequential song segments (532a, 532b, 532c and 532d). The
performer need not be aware of the actual pitches or even that they
have changed. As long as they perform the correct buttons in time,
their performance will sound exactly as if they had performed it on
the 25-key keyboard shown. Far less awareness of key position and
agility is required on their part and a far less inexpensive
keyboard can be used. This could allow a performer who is not a
musician to more easily play a musical part as part of a
production. They could be actually playing the notes, adding their
own expressiveness in volume and note timing and duration as
opposed to a recording played from off-stage, for example.
[0089] As used in an iPad.TM. implementation, the song-specific
keyboard allows non-musician players to more easily play song
segments they might be required to play as part of a game. It also
makes it possible to do far more in the limited screen size of an
iPad.TM. as compared to the relative ease in positioning pitches on
a full-sized music keyboard (about 9'' wide for the iPad.TM.
compared to about 19'' wide for a conventional 25-key keyboard).
This is especially important because in a game the limited space of
the iPad.TM. screen must be shared with other elements of game
play.
[0090] The game display of Rock Band 3.TM., Guitar Hero.TM. or Tap
Tap.TM. may at first appear to be similar to the song-specific
keyboard. However, there is a fundamental and important difference.
The song-specific keyboard defined here has a one-to-one
correspondence between the buttons (or other mechanisms or
implementations) of the keyboard and the pitches being used in the
song segment being performed. That is not true of the three other
game displays mentioned above. This is very important because it
means that once someone has learned to play a song segment on the
song specific keyboard, that same song segment can be played on a
piano or similar instrument simply by putting one's fingers on top
of the keys for the pitches used in the song segment and repeating
the finger motions used on the song-specific keyboard. In other
words, once one's fingers are in place, that song-segment can be
played on a real piano keyboard in the exact same way it was
performed on the song-specific keyboard. That is not true of the
other products mentioned because one button on those games results
in the playing of different pitches at different times. If the same
motions from one of these games were attempted on a real piano
keyboard, the fingers would have to move from place to place within
the performance in order to get the multiple pitches corresponding
with the button. Not only is this more difficult, but the performer
would generally not have any idea of which piano key corresponded
to a particular button at any particular time.
[0091] To be specific, in the song game around "Sweet Child O'
Mine" in Expert mode Guitar Hero 2, within the first five seconds
of the performance sometimes the first button from the left results
in a musical `Db` pitch and sometimes it results in a musical `Gb
pitch. A video performance of this song in the game was available
at http://www.youtube.com/watch?v=VY_DOeCCbCg. The first two
seconds, approximately, of the performance described is shown in
FIG. 9. The apostrophe indicates pitches in the higher octave.
[0092] Similarly, a Tap Tap Revenge.TM. 3 performance of Fireflies
now exists at http://www.youtube.com/watch?v=1A9-sgX7qMg. In the
first part of the intro, note that the number of pitches used is
much more than the three buttons used by the player.
[0093] Similarly, in Rock Band 3.TM. a performance of Bohemian
Rhapsody, at about 2:50 a guitar lead line starts as shown below.
It has many more pitches than the five buttons available to the
player, but we will just consider the first five notes, shown in
FIG. 10A.
[0094] The numbers indicate the buttons numbered from the left, or
respectively green, red and yellow in color.
[0095] Note that button 2 plays an F at one point and then perhaps
a second later plays a Bb. Also note that the pitch of `G` is
played by button 3 at one point and by button 1 at another. Clearly
there is no one-to-one correspondence between buttons in the game
and pitches in the music.
[0096] In any of the last three existing products shown, it is not
possible to play the passage on a piano by merely placing your
fingers on the appropriate keys and making the same finger motions
in rhythm that you did in the game. Compare that to the two
examples of the same two passages illustrated above as they might
be performed on a song specific keyboard, each with an illustration
of a song-specific keyboard that could be used and the
corresponding finger positions to play the same passage on a piano,
as shown in FIGS. 9A, 9B and 9C and in FIGS. 10A, 10B and 10C.
[0097] Referring to FIG. 11, the process 200 of developing the game
includes the following steps. First a song or a song segment is
selected (210) and the main pitches of the song or song segment are
associated with specific keys in the electronic keyboard (202). The
data that associate the song "pitches" with keys are stored in a
data file (203) and a mechanism is provided for lighting specific
keys on the keyboard based on signals received from a
computer/controller (204). An electronic display is also provided
that receives signals from the computer and displays an image of a
virtual electronic keyboard and prompts directed to specific keys
of the displayed virtual keyboard (205).
[0098] Referring to FIG. 12, the process of playing of the game 210
involves selecting a song and recalling the song and its associated
pitches/keys data from the library (211). Next, the computer sends
signal to the electronic keyboard that activate lighting in
specific keys that are associated with the pitches of the selected
song (212). Each key is lighted with a different color. The image
of a virtual electronic keyboard is displayed on the display, as
well as prompts directed to the specific keys of the virtual
keyboard associated with the pitches fort he selected song (213).
Next, the playing of the song on the computer/controller is
initiated (214) and during the playing of the song the player
attempts to hit the specific lighted keys on the keyboard that
associated with the specific song pitches that are played (actual
hit) at the same time the electronic prompt hits the corresponding
specific lighted keys of the virtual keyboard (virtual hit) (215).
The player is rewarded based on the timing and duration of the
actual hit as compared with the virtual hit.
[0099] Referring to FIG. 13, in another embodiment, an interactive
melody and rhythm action musical game 100 includes an actual
musical instrument 120 and a computing unit 102 connected to the
musical instrument 120 and to a display 106 and to speakers 103a,
103b. The computing unit 102 may be a computer, an Xbox.TM., a
Wii.TM., a Playstation.TM., an iPad.TM., an iPhone.TM., an
Android.TM. phone or any other computing or game controlling
device. The computing unit 102 includes the Real Key Music (RKM)
software 150 and a library 160. Library 160 is a digital storage
unit including recorded songs, song fragments, musical
arrangements, instrumental musical pieces, and vocal musical
pieces, among others. Examples of song segments are shown in FIG.
4A. In other embodiments, library 160 may be a database stored in
an external storage device or may be downloaded from an online
web-site via a network connection. Speakers 103a, 103b may be
separate speakers or may be integrated within the computer 102, the
musical instrument 120, or the display 106. Speakers 103a, 103b
broadcast sound generated from the playing of the recorded musical
pieces and or the playing of the actual musical instrument 120. In
the embodiment of FIG. 14, musical instrument 120 includes a second
set of speakers 123a, 123b dedicated for broadcasting music played
on the musical instrument itself. In FIG. 13, the musical
instrument 120 is shown to be an actual electronic keyboard. In
this example, musical instrument 120 is a 25-key electronic
keyboard manufactured by M-Audio. It includes 15 white keys and 10
black keys. In other embodiments, musical instrument 120 may be any
type of electronic keyboard or synthesizer having different number
of keys and controls, a Keytar (i.e., a keyboard or synthesizer
supported by a strap around the neck and shoulders of the player) a
computer keyboard in which the keys are software mapped to
correspond to music notes, a saxophone, a guitar, or any other
musical instrument. Library 160 includes, in addition to the
recorded songs, data specific to each song which associate the
pitches of the song with specific keys in the keyboard, as will be
described below. "Pitch" represents the perceived fundamental
frequency of a sound and in this case the song pitches represent
the set of perceived fundamental frequencies that characterize the
song or song segment. The display 106 may be a computer display, a
TV, a video game console, an arcade machine display or any other
display device. Display 106 displays the image of a virtual
keyboard 108 corresponding to the actual keyboard 120, colored
lines 111a-111c, prompts 114a-114c and a target bar 107. The
virtual keyboard 108 is designed to have the same number of keys as
the actual keyboard 120. The displayed image is controlled by the
RKM software 150. The RKM software 150 causes specific keys
112a-112c of the virtual keyboard 108 to become highlighted with
different colors depending on the pitches of the song that is
played. Lines 111a-111c diverge from a common starting point and
end at a point directly over the keys that are highlighted. They
are spaced apart by non-equal spaces and the common starting point
may be off the screen. As different keys are highlighted during the
playing of a song, the end points of lines 111a-111c move to point
toward the keys that are highlighted while their starting point
remains fixed. In this way they act as visual pointers toward the
keys that are hit on the virtual keyboard 108 and therefore need to
be hit on the actual keyboard 120. Prompts 114a-114c run along
lines 111a-111c, respectively, and extinguish after they arrive at
the target bar 107. The time prompts 114a-114c hit the target 107
corresponds to the time the player needs to hit the corresponding
keys on the actual keyboard 120 in order to have a successful
score. Keys 112a-112c, lines 111a-111c and prompts 114a-114c have
the same color respectively. In one example key 112a, line 111a and
prompt 114a are red, key 112b, line 111b, and prompt 114b are green
and key 112c, line 111c, and prompt 114c are blue. Display 106 also
includes a list of the available songs and song segments 118, as
shown in FIG. 22.
[0100] The system also includes a bidirectional communication
mechanism 104 between the computer 102 and the instrument 120.
Mechanism 104 causes specific keys 122a-122c on the actual keyboard
120 to get highlighted based on signals controlled by the RKM
software 150 and received from the computer 102 during the playing
of a song. Mechanism 104 also transmits signals from the keyboard
keys to the computer 102. The highlighted specific keys 122a-122c
on the keyboard 120 correspond to the pitches of the song that is
played. In one example, mechanism 104 is implement via
bidirectional musical instrument digital interface (MIDI)
communications. MIDI is an industry standard protocol that enables
electronic musical instruments and computers to communicate,
control, synchronize and exchange data with each other. A MIDI
instrument or controller transmits "event messages" such as the
pitch and intensity of the musical notes that are played, control
signals for parameters such a volume and vibrato and clock signals
to set the tempo. When a musical performance is played on a MIDI
instrument (or controller) it transmits MIDI channel messages from
its MIDI Out connector.
[0101] A typical MIDI channel message sequence corresponding to a
key being struck and released on a keyboard includes the
following:
The user presses the middle C key with a specific velocity (which
is usually translated into the volume of the note) and the
instrument sends one Note-On message. The user changes the pressure
applied on the key while holding it down--a technique called
Aftertouch and the instrument sends one or more Aftertouch
messages. The user releases the middle C key, again with the
possibility of velocity of release controlling some parameters and
the instrument sends one Note-Off message. Note-On, Aftertouch, and
Note-Off are all channel messages. For the Note-On and Note-Off
messages, the MIDI specification defines a number (from 0-127) for
every possible note pitch (C, C #, D etc.), and this number is
included in the message.
[0102] In one example, a MIDI channel message includes the
following four parts:
(1) the "command" - - - in this case NoteOn (0x80) or NoteOff(0x90)
(2) the "channel" - - - any channel could be consistently used, for
example channel 1. (3) data1 - - - this is the MIDI pitch (4) data2
- - - for NoteOn, this is the "velocity", which in the present
invention is used to indicate the type of highlighting. NoteOff is
then used to turn off the highlighting for that note.
[0103] Many keyboards do not generate "Aftertouch" messages and
sometimes a NoteDown message with a velocity of zero actually
indicates a Note release, making it a substitute for the NoteUp
messages. In this implementation the velocity is used as the note
highlighting for messages that the processor sends to the keyboard.
If the velocity were non-zero, the velocity could be interpreted as
an indication of how a note should be highlighted, selecting one of
up to 127 possible colors, for example. Alternately, the seven bits
of the velocity could be interpreted as a color, using for instance
two bits for red intensity, three bits for green intensity and two
bits for blue intensity. This would result in three possible red
illumination levels (besides off), seven possible green intensity
levels besides off, and three possible blue levels besides off,
which can then be combined in a red-green-blue (RGB) fashion into
one of 127 possible colors. Since the colors need to be easily
distinguishable from one another in the described application, it
is hard to imagine a situation in which more than 127 color shades
might be needed. But if that should become necessary, additional
bits could be conveyed. For instance, if two notes on messages for
the key were sent in rapid succession, the velocity data of the
second message could be interpreted as additional bits, giving a
total of 14 bits instead of 7, and 17,367 shades besides completely
off.
[0104] In implementations where independent signals are used for
the three RGB components, one MIDI channel is used for red
intensity, one for green intensity and one for blue. Similarly, the
highlighting may utilize a subtractive color technology rather than
an additive one, and the intensities of concern will be cyan,
yellow and magenta instead of red, green and blue.
[0105] In alternate implementations, the "highlighting" method may
not use color at all and it may have very few or even just one
level (or color) other than off. For instance, keys may be
physically moved slightly to show that they were highlighted or
made to vibrate or physically "hum" or be of a different
temperature or conduct heat differently, which one could easily
discern through touch.
[0106] Another way to convey the control information through MIDI
is to convey it as "controller" information rather than as "note
channel" information. "SysX" technology may be used in this
implementation. The exact use of channels and message content is
defined almost arbitrarily and combined in various ways with other
messages, but the meaning attached to the messages includes the
highlighting message info above.
[0107] In any of these systems, there may be "panic" messages which
causes all highlighting to reset to off. In addition to or in place
of the messages above instructing single keys to highlight, "bulk"
messages may be used which would give the keyboard a new
highlighting scheme to use in place of the current one. These
messages may be implemented in MIDI, XML, JSON or some other
technology.
[0108] In one particular implementation, the music keys are
considered to be numbered from 0 successively left to right,
regardless of their MIDI pitch. In this implementation the f
instruction: "2,0;3,1;4,2;7,3;10,4;11,5" tells the keyboard to
discard any previous lighting and to
Highlight key number 2 in color 0 Highlight key number 3 in color 1
Highlight key number 4 in color 2 Highlight key number 7 in color 3
Highlight key number 10 in color 4 Highlight key number 11 in color
5
[0109] The remainder of the keys are not to be highlighted. In this
system, the actual colors are taken from a predefined, fixed list
of colors. For example, 0:blue, 1:green, 2:red, 3:yellow, 4:cyan,
5:orange. If the keyboard starts with a `C`, the corresponding
pitches will be D, D#, E, G, A# and B, respectively. However, it
would also be possible to have a system which downloaded the list
of colors at the beginning or periodically as necessary or
desirable.
[0110] In the embodiment of FIG. 13 mechanism 104 includes light
emitting diodes (LEDs) or any other lighting source used to light
keys 122a-122c in response to electronic signals received from the
computer 102. In the embodiment of FIG. 14, keys 122a-122c are
lighted by projecting light onto them through a projector 130.
Projector 130 receives the image of the lighted virtual keyboard
108 from the display 106 and projects it onto the actual keyboard
120. The image of the virtual keyboard is aligned with the actual
keyboard so that their corresponding keys coincide. The projector
beam may be directed directly onto the keys or may be reflected
onto the keys by a mirror. In other embodiments where multiple
keyboards are used, multiple projectors may be used or a single
projector with a multiple beam splitting. In yet other embodiments
the beam of a single projector is separated into multiple beams
with mirrors or prisms.
[0111] Referring to FIG. 15A and FIG. 15B, examples of song
segments used in playing the interactive game 100 include the
synthesizer part of the song "Whip it" by Devo 500 and the "bomp
bomp bomp" ending in the song "Sweet Caroline" by Neil Diamond 510.
The concept of a song segment, as used here, means a musical "part"
played by one or more instruments or voices for part or the
duration of a musical selection.
[0112] The division of a song and its vocal and instrumental parts
into song segments is arbitrary. A person or process breaking a
song into song segments can make choices to please the needs and
abilities of an intended audience, opting to try to keep the number
of pitches used near or below a particular number. In one example,
a simple automated process, could decide to use 16-bar segments of
tracks in a Musical Instrument Digital Interface (MIDI) file for
the song segments. That would work, but better results can be
achieved by considering the aspects of particular songs in a more
detailed manner, whether manually or in an automated process. In
one embodiment, represented by the examples above, the number of
keys used within a song segment is kept around seven. In other song
segments between three and eleven keys are used. This requires
skill and discretion in choosing the song segments. Once, the song
segments are defined, each can be seen to include certain pitches
(or "perceived fundamental frequencies" or keys) and not others and
that defines how it will appear and be used on the keyboard.
[0113] A "song segment" may contain a certain number of pitches,
some of which may be played more than once. In one example, the
melody to "Mary Had A Little Lamb", as commonly played, has four
different pitches, but all are used more than once, as shown in
FIG. 16A. This concept is similar to what would occur on one line
of a music score for a certain amount of musical time (possibly the
entire selection), as may be measured in musical beats or measures
or by Musical Instrument Digital Interface (MIDI) ticks or other
timekeeping system.
[0114] It is not necessary that this target song segment be
performed in the original work by the same type of instrument used
by the player. For instance, a keyboard could be used to play a
vocal melody line, a synthesizer used to play a brass line or a
guitar lead, a guitar used to play the notes of a string section
line, among others This sort of substitution often happens in real
musical ensembles, so it is not even unnatural to do it here.
[0115] The song segment may contain more than one pitch played at a
time. For instance, it may be a left or right hand part of a piano
selection. Alternatively, the pitches played by both hands maybe
considered together as the pitches of a song segment. Although, as
we shall see, there are advantages to defining the song segment in
such a way as to include a limited number of pitches, that
restriction is not included in how we are defining a song
segment.
[0116] It might even be played by more than one instrument or
player if, in the mind of the listener, it might meld together into
a musical concept. For example, a set of tympani is a set of
individual tuned drums that are physically independent from one
another, but the melodic line produced by using all of these drums
together might be considered to be a single song segment. In
another example, classical composers at times have written melodic
lines that run from the violins through the violas ending, for
instance, on the cello. The composer may conceive and the listener
may hear this as a single melodic line, even though it is played
serially by different players on different instruments. The part
played by the player may be transposed from the original. It might
even be simplified from the original.
[0117] As was mentioned above, the keyboard 120 has some means to
highlight a set of keys on the keyboard (or fret positions on the
fretboard). Though it is of some use to highlight all of them in
the same way, greater benefit can be obtained by varying each
highlighted key in a different way. For example, on a keyboard with
lighted keys, the non-highlighted pitches might be unlit and the
highlighted pitches lit. Though it is of some use for them to be
all one color, it would be a functional improvement if more than
one color were used and better still if each color used was unique
and easily distinguishable.
[0118] For instance, using the "Mary Had a Little Lamb" example in
the key of C, the pitches would be C, D, E and G from lowest pitch
to highest pitch. We might light them respectively blue, green, red
and yellow 122a 122b, 122c, 122d, respectively, as shown in FIG.
16B. Alternatively, if only two colors were available, we might
light them bright green, bright red, dim green and dim red.
Alternatively, even simply green, red, green, red if necessary,
referring to them when necessary as lowest green, lowest red,
highest green, highest red. If they were all one color, they could
be lowest selected pitch, 2nd lowest selected pitch, 2nd highest
selected pitch, and highest pitch. It is desirable to easily
distinguish each pitch uniquely, but some benefit can be derived
when designations on selected pitches need to be duplicated or even
in simply being able to very quickly distinguish the current
selected pitches from those not selected.
[0119] Using the four color representation, one could think of
"Mary Had a Little Lamb" as red-green-blue-green-red. However, on
keyboard 120, not every key is colored. In fact, all of the keys
which are not selected are not colored or colored the same or
identified by some unique changeable characteristic. The keys that
are used are colored, as was described above, and remain colored
during the entire duration of the song segment. Therefore at a
glance, the player can tell which keys do not have to be considered
for a song segment and which ones will be used. The keyboard 120
ideally illustrates this for all the keys even before the playing
of the song segment begins. This allows the player to quickly
consider how to place his hand or hands, or feet for a pedal
clavier, in such a way as to be able to easily and quickly access
those keys, ideally even before the playing of the song passage
begins. This described the concept of the "focused" keyboard.
[0120] This is very different from approaches that indicate one or
sometimes more notes on a keyboard or on a keyboard representation
just before they are used, such as Piano Wizard or Synthesia. In
these prior art cases, the keys have all the same or different
colors and there is not any form of constant highlighting of the
keys used in the song and, more importantly, constant
non-highlighting of all the keys not used in the song, even when
they occur physically between keys that are used, such as the F in
"Mary Has A Little Lamb".
[0121] In the focused keyboard 120 of this invention, it would be
considered functionally better if the same colors were used. The
present approach does not use one color for C's, another for D's,
and so on. Instead we use one color for the lowest note actually
used in a particular song segment, another for the second lowest
note actually used, and so on. Therefore, if we use a different
song segment, or a song segment played on a different key, we may
use the same colors, but they will generally be on different keys
since the keys used by the new song segment or the song segment
played on a different key would be different. In the example of
FIG. 16C the song segment of "Mary Has A Little Lamb" is shown
played in D-#(Eb) key and the pitches would be D#(Eb),F,G and
A#(Bb) from lowest pitch to highest pitch. We light them again as
blue, green, red and yellow 122a 122b, 122c, 122d, respectively,
but are on different keys than in FIG. 16B, where it was played in
the key of C major.
[0122] It is possible that a song segment might encompass an entire
song from start to finish, such as in "Mary Had A Little Lamb"
above. However, in many cases, more than one song segment might be
used in succession. For instance, there might be one song segment
for the intro, one for each verse, one for the first part of the
chorus, one for the remainder of the chorus and one for the ending.
Since it is desirable to have not more than a few pitches in each
song segment (for example, less than twelve and, when reasonable,
no more than six), in complicated musical passages or those
involving chords, it might be desirable to change to a new song
segment quite often, sometimes even within a measure.
[0123] Referring to FIG. 8A, in an example from the organ intro to
the Doors' "Light My Fire" there are four song segments 532a
(blue), 532b (green), 532c (pink), 532d (yellow). The colors used
are to distinguish one song segment from another and do not relate
to the keyboard color lighting. For each of the four song segments,
the keyboard lighting may respectively be 531a, 531b, 531c, 531d,
as shown in FIG. 8B. The coloring strategy here is to assign colors
consistently from left to right. As shown in FIG. 8B, the same key
does not always have the same color though out the different
segments.
[0124] Many alternative coloring strategies can be accommodated by
the focused keyboard 120. Referring to FIG. 17, here is the result
of a strategy that keeps a key the same color if it appears in two
successive song segments. However, even in this strategy, a color
is not constantly associated with a key or pitch name. For
instance, the `A` is green 122e at the start 531b and red 122c at
the bottom 531d. In other embodiments, the focused keyboard 120 may
employ a strategy in which each pitch remained the same color with
the colors perhaps chromatically assigned. That would result in
something similar to the set of pictures 531a-531d, shown in FIG.
18.
[0125] A focused keyboard's 120 color (or other designation)
selection could in some cases be set up specially for something
highly dependent on the particular song segment, for instance
making things easier to grasp by capitalizing on some symmetry in
the music. Referring to FIG. 19A, a passage played on bass
synthesizer in U2's pop song called "Two Hearts That Beat As One"
540 is represented on the focused keyboard 120, as shown in FIG.
19B. Here there are duplicated colors (which is usually a
disadvantage), but in this case they simplify the situation for the
player. If the player decides to play with two hands, the player
can see how to place the two hands and guess that they will be led
to play notes in similar ways. Even if the player uses only one
hand (as would be done in more professional situations), the layout
in this case still helps to explain how the notes are used in this
case. If this were played with the left hand, a professional would
generally use the thumb on the right three notes and other fingers
for the lower notes, possibly just the little finger, given the
nature of this example. In any event, the focused keyboard 120
helps the player to quickly see what is expected in
performance.
[0126] Referring to FIG. 20A, and FIG. 20B, in the example from the
marimba part of Roy Orbison's Blue Bayou 550, there's a similar
usage, but in this case identical or similar colors are used to cue
the user of keys that will be played together, even though they are
not the same pitch name.
[0127] There may be circumstances that make one scheme preferential
over another, such as cost considerations or a principle in a music
lesson being taught. For example, the scheme of FIG. 20A, FIG. 20B,
has the advantage that each key need only display one color. For
instance, twelve distinguishable shades are needed and similar
shades are on adjacent keys where, perhaps, contrasting colors
would be preferable for quick recognition, particular if
coordinating with rapidly moving colors on an associated computer
display. Also, there are only four notes highlights, but two of
them are F's and therefore have the same color. Also note that in
the first scheme shown in FIG. 19B, if one is playing the song
segments with right hand only, positioning one's thumb on the blue
note quickly starts the user towards what is often a favorable hand
position for that particular song segment. The user would get used
to associating `blue` with the lowest note. Yet, adjacently used
notes have contrasting colors whether they are very close in pitch
or not.
[0128] The controlling software or hardware 150 used with the
focused keyboard 120 may always consistently use the same approach
to key coloring. Alternately, it may use different strategies at
different times, either chosen by the user's preference or chosen
within the usage context, for instance, keeping pitch colors the
same when pitch-oriented lessons were given and keeping the color
for the thumb key the same when teaching lessons about
fingering.
[0129] Ideally the game or other mechanism allows the player to
first work with each song segment individually, possibly even
allowing it to be slowed down or advanced a note at a time. That
way, when the song segments are used together and the highlighting
changes (ideally as the last note of the previous segment is being
played), the user is ready and expecting the "focus" change since
they have already played the song segments individually. In another
possible embodiment, a focused keyboard 120 or fretboard could be
used without playing with recorded sounds or even trying to play in
rhythm. It would still help the player to only focus on the keys
used for the particular passage.
[0130] The present system doesn't lead the user through, note by
note. Instead, in system 100 the notes actually played during a
song segment are highlighted, whether or not they are a part of a
particular chord or scale. Rarely this might coincide to come up
with the same set of keys, but usually it would be quite different.
That's because many of the song segments are melody or counterpoint
lines and these usually do not simply follow the notes of a chord
or scale up and down. In one example, a song segment covers a
chordal accompaniment pattern that includes playing a C and F
chord, possibly in "broken fashion". The notes of the C-chord are
C-E-G, the notes of the F-chord are C-F-A and the notes for the
entire C-Major scale are C-D-E-F-G-A-B-C. The highlighting of the
notes on the keyboard is based on the notes that are actually
played in the song segment, i.e., C-E-F-G-A, as shown in FIG. 21B
and FIG. 21C. This is neither the notes of a chord nor the notes of
a scale.
[0131] Expanding further with a musical example, a simplified
right-hand accompaniment pattern to John Lennon's "Imagine" is
shown in FIG. 21A. Here we have included about twenty seconds of a
useful musical thing to play and will only need to highlight five
notes to cover that duration. Anybody will easily understand what
is being indicated and how to play it and most will succeed, in
this case even at full tempo, within a few tries of playing. Even
without any background recording, the playing of this song segment
at the tempo of the song with a piano sound is distinctive enough
that many listeners might quickly identify the song as John
Lennon's Imagine. That's unusual considering that the player may
not have any real musical experience, but it seems to be a common
occurrence with song segments learned from the game in this style.
On the focused keyboard instrument 120, the keys for these pitches
C-E-F-G-A are highlighted because these are exactly the ones
included in the above song segment, as shown in FIG. 21B. Ideally,
each of these keys 122a-122d are lit in a different color, i.e.
C-blue, E-green, F-red, G-yellow, A-cyan, respectively. These color
selections depend on the song segment, not the note name of the
key, i.e. C's would not always be blue every time they are used in
a song segment. A different song segment may likely result in a
different color for `middle C` (the lowest note in this example) if
it happened to use `middle C`. In the example of FIG. 21D each of
these keys 122a-122d are lit in the same color, i.e. C-red,
E-green, F-red, G-red, A-red, respectively. The keys that are not
used are not lit.
[0132] In contrast, a non-focused keyboard which highlights chords,
may display the left group of notes 572 for the odd-numbered
measures C-E-G and the right group of notes 574 for the even
measures C-F-A, shown in FIG. 21E. FIG. 21F depicts the notes in a
scale-oriented non-focused keyboard 578, even though `D` for
example, is not used in this song segment. In summary, the focused
keyboard instantly shows the player which keys will be played
during the song segment and which need not be considered at all,
thus showing the player where to focus their attention.
[0133] The scoring of the game is described with reference to FIG.
23 and FIG. 24. Each note can be worth up to 1000 points if it is
played accurately. Accuracy is judged both in pitch and in time. A
slightly wrong note played at the right time is worth approximately
as much as a right note played at a slightly wrong time. If a note
is neglected entirely, in the current implementation it turns into
an open circle `O` 117c when it becomes too late to play. If
"extra" notes are played that cannot be matched well enough with
any note in the song, an `X` 117e appears on or between the lines
showing this added note played. The software is programmable as to
how close a note must be in time and pitch to be a possible match.
At most, each song note can only be matched once. If it seems to
have been attempted more than once, the game attempts to match it
with the best match and the remaining notes are considered to
be"extra" notes and result in a penalty, similar to the penalty for
notes missed entirely.
[0134] If a note is matched pretty well, it results in an explosion
117a, as shown in FIG. 23. This note was close to ideal, but
perhaps off by a few milliseconds, so it scored 802 out of 1000 in
this implementation. If a note is matched, but not particularly
well, a tail 117b of various sizes is drawn on the note, as shown
in FIG. 23. In this case, the direction of the tail indicates that
the player played a note with a slightly lower pitch and played it
slightly late. This note was therefore not as good as the previous
one (117a), but still close enough to get 736 out of a possible
1000 points. In other embodiments scoring scales with a different
maximum point are used.
[0135] Referring to FIG. 24, in another embodiment the target bar
107 moves and the notes do not. It is also a horizontal,
non-perspective mode, though those attributes are not necessarily
associated with the target bar moving instead of the notes. In the
case shown here, the target bar 107 moves from left to right and a
short song segment has just completed. One advantage of this
particular mode is that it is easy for the user to look back after
the end of the song to see exactly what happened. In variations of
this mode, we alternate between two or more sets of lines, similar
to going from one line to the next in sheet music. That allows more
notes to be displayed at a time than would easily fit on one line
and also makes the rhythm clearer to the player. The "alpha" 117d
scoring notation is actually superposition of a missed note scoring
notation `O` 117c followed by an extra note scoring notation `X`
117e. The criteria for whether a note is a good enough match could
be altered by user preferences and also by such factors as the
speed and difficulty rating of the particular song.
[0136] Game 100 is able to accurately match and score notes in
real-time, as fast as they are played. Prior art games may appear
to do a similar thing in their scoring, but there is an important
difference. While prior art games may allow graduated scoring for
notes (or steps) that are slightly off in time, they do nothing
that compares to graduated scoring for notes that are different but
close in pitch. In the present game, any of the twenty-five keys on
the standard two-octave keyboard might be pressed at any time,
whether lit or not and partial score and matching will be
considered for even unlit keys that are played at nearly the right
time as the lighted note. This is doing matching in two dimensions
(time and pitch) rather than just in time, i.e., the problem space
is geometrically larger.
[0137] Since this is a music game, what the player hears when
playing is of paramount importance. There is actually some
flexibility in what sound might best be offered. Because the audio
sounds live and involving and closely matches what the player is
playing on the game keyboard, the player feels as if they are part
of making the sound.
[0138] This can be the case even when the sounds heard do not
reflect or include what the player is doing at all, particularly if
the player is doing well. A similar situation can happen with real
musicians in a group. If one is playing clarinet in a concert band
with a lot of clarinets and the nearby other clarinets are playing
the same part, it can be hard to discern one's own playing and even
if one stops playing, it may not sound that much different. All the
parts playing the same notes blend in together and it is difficult
to discern the instruments individually. Comparably, if the sound
played from the player's keyboard is very similar to the sound on
the recording and they are doing well at playing the same part with
the recording, they may feel as if they are hearing themselves
play, even when only the recording is being heard. This effect is
strengthened by the fact that the ear hears sound intensity in a
logarithmic way. When two instruments play instead of one, the
combined sound is three decibels louder, but that is not heard as a
doubling of the sound volume.
[0139] There are situations in which the player might really want
to blend in well and want the sound of their own "instrument" to
closely match the sound qualities of the associated instrument on
the recording and want to hear their part only quietly or not at
all. For instance, when showing off for friends, one might want to
sound just like the player in the recording, be heard distinctively
a little, but not be too obvious when wrong or mistimed notes were
played. On the other hand, when the player is alone and seriously
wanting to do better, they might want to be able to hear their part
clearly and distinctly by having it loud compared to the recording
and possibly of a somewhat different tone quality. For example, the
recording might be using an electric piano sound but the player
chooses to hear a different and distinguishable model of electric
piano (say Wurlitzer-like instead of Fender Rhodes-like) or even
chooses to hear their playing with an acoustic piano or perhaps a
marimba sound. This can be very helpful when learning, since the
player can relate to both the similarities between their playing
and the audio track as well as the differences. Another effect that
can be controlled by the computer and used separately or in
combination with this is to play the players part and the recorded
part in two different places spatially, for instance the audio
track in the left stereo channel and the player in the right stereo
channel.
[0140] In cases where the game is using an audio recording in which
it can separate the instrument that corresponds with the player
from the other instruments in the recording, there are even more
useful possibilities. For instance, the corresponding instrument in
the audio track may be dropped altogether, leaving only the player
playing that part in the result.
[0141] If the player can control the mix, they could also turn the
audio recording off altogether, only hearing their own playing. A
player might also find it fun, interesting and useful to remix the
levels of the instruments and voices in the recording, to the
extent the audio recording made it possible to do so. In the
current implementation, the player can change the mix between game
audio and their own playing at any time. On the screen, they can
see the current mix represented by the brightness of the keyboard
icon compared to the brightness of the band icon, as shown in FIG.
26.
[0142] In one embodiment of game 100, there is an audio track
(stereo pair of tracks, actually) that contains what the player is
to hear during the game. Most often, this is the original audio of
the song, as performed in a well known version by the creating
artists without modification. This is one reason why players and
bystanders seem to love the sound of the game. In some cases, this
audio is slowed down or even slightly speeded up for the benefit of
game ease or difficulty. In other cases, this audio may be
transposed (pitch changed) so that the player's part on the often
two-octave keyboard might correspond properly to the notes. The
only sound that is added in this case are a few "lead-in" beats
that alert the player that the audio is about to begin. The sound
of these metronome-like lead-in beats is generally percussive in
nature, though it could be a voice ("one, two, three, four") like a
band leader. Currently the game uses one hi-hat cymbal sound for
the downbeat and another for the other beats in the lead-in, though
other prototypes used bongo sounds or other instruments.
[0143] An alternative to supplying an audio track in the above
manner is to supply MIDI-tracks (played through some
MIDI-compatible synthesis method) or a combination of the two. One
advantage to the audio is that it sounds more natural to the player
and avoids the need to supply software and/or hardware to play the
described MIDI-tracks. In one embodiment, the above-mentioned audio
and the generated lead-in beats is all that the player hears during
the actual playing of the song during the game. Between songs, the
keyboard does other things, as described elsewhere. When the audio
needs to be slowed down excessively, it may sound unrealistic and
even grotesque. In these cases when a usable and pleasant slowed
down audio track is not available, it may be preferable to
substitute with a MIDI equivalent. That does require the presence
of MIDI software or hardware within the system or external to it
which might not be necessary otherwise, since the simple lead-in
beats can easily be done by other means. Another possibility to
support playing very slow (typically less than 50%) of the original
tempo is to use alternate audio tracks specifically designed to be
slower performances of the same songs. These might be recorded by
the original artists or by others.
[0144] There are other aural things that can be played along with
the playing of the song. The existence, strength and mix of these
might be determined as a player preference or fixed values might be
tested and built into the game. These possibilities include the
following:
[0145] a) Hearing what the player is actually playing on a similar
or dissimilar instrument. A similar instrument blends in better. A
dissimilar instrument makes it easier to hear the differences
between what the player is playing and the actual part in the song.
Either may be a good choice depending on circumstances. In some
embodiments, we have played the player's part in a different octave
to make it similar, but distinct that way.
[0146] b) Hearing the desired part played on a synthesized similar
instrument. In this case, audio may be used that deliberately does
not include the part of the desired instrument, which might require
audio specially produced for the game.
[0147] c) Hearing a deliberately somewhat dissonant version of the
desired track played if the player's performance has errors which
trigger this. This "error" track could be accomplished in either
audio or MIDI. After the player has met some criteria and/or after
some delay has passed and/or after some new part of the song is
reached, the playing of this "error" track would stop and the sound
would revert to what it had been.
[0148] d) Hearing some rhythm parts made louder or with more
emphasis in some or the entire song excerpt to make it easier to
follow the beat, though with current music this seldom seems
necessary.
[0149] e) Reinforcing or otherwise making the featured (often
keyboard) part of the song more emphatic and easily heard, possibly
by changing its strength or tone color or quieting or silencing
less relevant instruments or voices.
[0150] There are a lot of possible choices here. They could be
dictated by player preferences, considered preset choices,
circumstances within the game or some combination of these.
[0151] In the embodiment of FIG. 13, while the song is not actually
being played, the keyboard reverts to acting like a sampled
keyboard, using a "sample" that is similar to the instrument part
played in the previous or in the coming song. There are varying
degrees of sophistication possible in sampled keyboards and which
could be used to make this state of the game better. This
embodiment uses the same sound across the keyboard, only changing
it in pitch. Some sounds are looped, whereas others are not.
[0152] As was mentioned above, in the embodiment of FIG. 13,
keyboard 120 is a 25 key C to C keyboard. In other embodiments, one
may use a smaller or a larger keyboard. However, twenty-five keys
allow very many song parts to be easily played without much
modification. It is, of course, possible and somewhat desirable to
have even more keys, but that makes the keyboard more expensive,
especially if all the keys have all the lighting abilities and this
makes the instrument bigger and heavier. Keyboard 120 in FIG. 13
has full-size keys, though there is nothing that would prevent the
game from being played with keys of any size. FIG. 25A depicts two
examples of non-full-size keys 120a, 120b as are common on smaller
keyboards and even on some keyboard instruments, such as an
accordion. Game 100 is not dependent on the size of the keys and
any of these keyboard sized, along with other sizes, may be used.
However, playing the game with full-size keys provides more realism
and makes it easier to transfer any musical skills learned to other
instruments. Keyboard 120b may also be limiting because it has only
fifteen notes. Keyboards with limited number of notes, such as
120b, or keyboards using ordinary type buttons, are used in cases
where a low cost instrument option is desired. The game
automatically "transposes" the game music part to the octaves of
the attached keyboard. So a keyboard used for a bass part on one
song can play the high part on the next song without touching any
buttons on the keyboard.
[0153] Referring to FIG. 25B, in an arcade version of game 100 with
likely limited functionality, it might be more practical to light
up fewer keys. In an application like this where durability and
resilience to vandalism are important, the keys may even be large
lighted buttons on a panel that only depress slightly or even touch
sensitive lighted pads possibly flush with the panel surface.
Arcade machines generally use ruggedized panels as shown in FIG.
25B. Nonetheless, suitable buttons or touch surfaces are arranged
into a keyboard pattern and lit in multi-color style to implement
an arcade version of game 100.
[0154] Some keyboards have small speakers built-in, as shown n FIG.
25A, and others do not. Almost all have a headphone jack and most
have "line out" or "balanced line out" in mono, stereo or even quad
to external amplification and recording devices. Some keyboards
offer several power options. Any of these options are possible for
a keyboard 120 for game 100, though none are mandatory. It is a
matter of desirability vs. size and cost.
[0155] A minimum of four colors or other distinguishing mechanism
would be desired for a reasonable game. If only a small number of
colors were used, they could be repeated again in order on higher
notes without adding an intolerable amount of confusion. However,
the game works significantly better, when each key that is lit has
its own color. In some embodiments, between one and nine colors per
song excerpt have been used, but there is nothing magical about the
number "nine" and more colors could be available and used on some
songs.
[0156] From an electronic point of view, keyboard 120 (or any other
musical instrument) signals the pressing of notes electronically to
the game processor. It is highly desirable that it signals their
release as well, but one could still have a good game without that
feature. Keyboard 120 also accepts electronic signals telling it to
make a particular key a particular color.
[0157] Other desirable features of game 100 include the following
among others: [0158] Having a small display could be useful as a
means of communicating to the user, particularly if the keyboard
supports usage away from the game system. [0159] Having additional
controls, such as the pitch-bend and tonal effect controls
available on many keyboards could be used as an added feature in
some games and in general add to the enjoyment of playing the
instrument with or without the game attached. [0160] Touch
sensitivity and pressure sensitivity would be desirable and could
be used within the game, but these are not necessary. [0161] Having
an internal sound generation system with speakers and/or an
earphone jack would be useful particularly when the instrument is
used away from the game. [0162] If the sound generation system is
based on samples (such as PCM) or other means of capturing an
instrument sound, this is useful in making the keyboard sound like
it is associated with the game even when it is disconnected from
the game. [0163] Appropriate batteries and/or ac power adapter
would be necessary for use away from the game. The instrument could
be powered that way when used with the game, but it might more
likely be powered through the cable attaching it to the game
system. [0164] Wireless capability would add to the flexibility and
enjoyment of the game. [0165] The interface to the computer gaming
system might be through a standard compatible with many platforms,
such as USB, or some other standard could be used with the game
possibly connecting in a different way to the gaming system.
[0166] Keyboard 120 may have a small amount of memory that would
allow it to store the sounds of instruments recently used in the
game and/or instruments in the game specified by the player. This
will be useful in cases keyboard 120 is temporarily removed from
the game. It may also have other built-in sounds as well. With
enough memory and appropriate onboard intelligence, the keyboard
120 may even be able to download and store audio song samples from
the game so that the player could play along with them even without
the game. In some embodiments, the keyboard 120 incorporates the
computer 102 and/or display 106.
[0167] Other embodiments of the invention may include one or more
of the following. The keys in the focused keyboard 120 may be
designated with shapes, numbers, letters, textures or type font
size, instead of color. Another possibility is to reposition keys
to show their status in the song segment. For instance, the keys
not used could be slightly lowered. This may be done with or
without another designation system, such as colored keys. The
"prompting" display of the keys, as was described above, shows the
keys used by colored dots arranged in order of pitch in one
dimension and in order of times in the perpendicular or another
dimension. These dots move towards a target position 107, as shown
in FIG. 14 and FIG. 13. In other configurations the notes may be
steady and the target bar 107 may move across them instead. The
entire part being played could be represented by one continuous
line of these musical notes or several lines could be used in
sequence, proceeding to the next (or back to the first) when a line
has finished playing. This is particularly useful in cases where
the target moves instead of the notes because then it more
resembles the idiom of sheet music. The colored lines 111 may be
evenly spaced and instead of a model virtual keyboard 106 being
pictured at the bottom, two keyboards may be pictured at the left
as shown in FIG. 23. In this case the upper keyboard 108a shows
which colors are for which keys and the keys on this keyboard may
move up and down as the notes are played. The lower keyboard 108b
in this case echoes what notes are being played on the player's
actual keyboard 120 so that it is easily judged whether a correct
key is being used. Alternately, the upper keyboard 108a could just
show the colors and the lower keyboard 108b could show the timing
of the notes, i.e. whether they should be up or down at the moment.
A thin target bar 107 may be used on which the note should be
played when it is centered, as shown in FIG. 23. An arbitrary
scoring system may be used to score a game or even no scoring
system at all. Scoring systems could give bonuses, for instance,
for having played so many notes without error or for achieving,
say, exceptional timing accuracy. Mechanism 104 may be implement
via other means of transmitting music-related messages, such as
XML, JSON or a proprietary format. Game 100 may be played without
the actual instrument 120, but rather using the computer keyboard.
Game 100 may also be played with a keyboard that doesn't light up
by simply watching the lighting on the screen carefully and
matching notes to the keyboard by comparing it by shape with the
colored keys pictured on the screen. However, the ability to be
able to match the notes on the screen to those of the physical
color by color at any instant is something that gives particularly
starting players confidence and security and adds a lot to the
appeal and easy understanding of the game.
[0168] Furthermore, in other embodiments, two (or more) song
segments for the same song are shown and used simultaneously
together. For example, something with both a piano part and
synthesizer part, as is common in Elton John recordings.
Alternatively, it may be two or more synthesizers, or a piano and
organ, or even two or more string, horn or vocal parts. Both parts
are shown on the computer display or on two different displays or
even on two displays in two or more different locations connected
by wiring or some sort of network. Games/learning exercises that
can be played in this setup include the following among others:
[0169] One player plays two parts on a larger keyboard, one with
each hand. A third or even a fourth part could be played with
pedals in the style of a classical organist or some synthesizer
artists. One player plays two parts with a hand on each of two
keyboards. Again, adding pedal parts is possible. The players may
use the game to work up to this complexity one step at a time. Two
or more players may each play one part on a larger keyboard or
instruments. Two or more players may each play one or more parts on
separate keyboards or instruments.
[0170] Combinations of different instrument may be used, for
instance, keyboard and fretboard. The fact that each part is
presented in a focused manner to the player involved would help
keep the complexity manageable, and this would, of course, very
much be like real musicians playing together in ensemble.
[0171] In other variations, two or more players are involved with
the same song segment or segments. Some variations include the
following. The players alternately play the song segment in a
competitive manner. They might use the same keyboard (or
instrument) or they might each use their own, particularly if they
were in different locations. It is also possible to set up a
competitive situation if they are not playing the same song segment
scenario and/or level, but to be fair one would have to handicap if
the difficulties involved were different. Similarly, a handicap
system could give a weaker player a chance to compete meaningfully
against a stronger player. It would also be possible for a player
on a keyboard to compete with one on a fretboard given a handicap
system or other means to compensate, even though the ease of
playing the part might favor one type of instrument over
another.
[0172] In other embodiments, instruments other than keyboards are
used. They include guitar with lighted fretboard, bass guitar
lighted fretboard, soprano sax lighted keys, alto sax lighted keys
and EWI lighted keys, among others, as shown in FIG. 27A-FIG. 27E,
respectively.
[0173] The present invention also relates to a method and a system
that associates musical notes to colors and to "colored"
keys/buttons on an electronic board. The notes are associated with
the melody at specific frequencies (pitches) of a song or a song
segment. The notes are then associated with specific colors
displayed on a screen and the key/buttons are associated with the
specific colors. In one implementation, the system is used in a
Melody Rhythm Action Game application. The game is played by
hitting the keys/buttons corresponding to a specific color at the
same time the specific color is displayed on the screen, thereby
playing (matching) the notes that are associated with the melody of
the song or song segment. The key/buttons on the electronic board
are colored by light of a specific color (either reflected or
transmitted), or by paint.
[0174] The invention also relates to a method and a system that
converts any game controller/electronic board with a few buttons
into a musical instrument ("simplified interactive musical
instrument"). By definition a musical instrument is simply
something that generates notes/frequencies when struck or played at
different positions/keys. Thus when specific frequencies are
assigned to specific positions/keys/buttons, and sound comes out
when the keys/buttons are hit (i.e. played), then by definition a
musical instrument is created. In one example, a game controller
with 5 keys/buttons is used to play any pattern of melodies or
"music".
[0175] Several embodiments of the present invention have been
described. Nevertheless, it will be understood that various
modifications may be made without departing from the spirit and
scope of the invention. Accordingly, other embodiments are within
the scope of the following claims.
[0176] What is claimed is:
APPENDIX A
[0177] The notion of using an ordinary appearing piano (or other)
keyboard (or fretboard, etc.) as a song-specific keyboard (or
fretboard, etc.) might seem alien at first. However, considering it
from another point of view, it can actually be quite simple and be
viewed as a way of "tuning" a keyboard to a song, much like how
guitarists sometimes retune their instruments for different songs
or how a pedal steel guitar, orchestral harp or tympani are all
dynamically tuned by means of pedals.
[0178] I'm going to talk about one and two octave keyboards here
though the principles can be extended to keyboards larger and
smaller other variants.
[0179] Here's a conventioanl two-octave keyboard. On it, I have
written how the keys are ordinarily "tuned".
[0180] Note that as commonly used, the black keys can be called by
two different names, depending on whether they are referenced to
the white key to the left (in which case they are "sharps" or to
the white key to the right (in which case they are "flats".) The
notation G# is pronounced G-sharp. The notation Gb is pronounced
G-flat. Nothing new here so far. This is just how keys are usuallly
named on a piano keyboard. A keyboard with sharps and flats is
sometimes called "chromatic". Fretboard and wind instrument
fingering is similar in some ways and different in others but
accomplishes the same purpose.
[0181] Not all instruments can produce all of the chromatic scale
tones shown above. For instance, most harmonicas, bugles, and
tympani cannot without going to extraordinary measures within a
song. So it is possible for an instrument to be melodic without
being fully chromatic. In fact, to play a melody only really
requires being able to play the notes that are used within that
particular melody and that will be our point of departure from
typical keyboard usage, as we will see below.
[0182] On most musical instruments, the same key usually produces
the same pitch all the time, with the exceptions for some
instruments noted above. For some wind instruments, the wind
pressure and mouth position may also affect which pitch is
produced.
[0183] It is common for instruments to go from lower to higher
pitch from left to right and we will generally follow that
convention (though some artists such as Joe Zawinul arenoted for
setting up their keyboard from right to left, partly to encourage
experimentation).
The Song-Specific Keyboard
[0184] Let's consider the notes in what I have defined elsewhere as
a song segment. It could be a whole melody, a part of a melody or
some other significant and recognizable part using notes with
specific pitches.
[0185] We can consider all of the notes that are used anywhere
within a particular song segment. We can order them in pitch from
low to high and number the lowest one "1", the second lowest one
"2", etc. There could be dozens of pitches involved within a song
segment, but that would be unusual. If we limit how we define the
song segment in time, complexity or other means, we can scope it
small enough so that the number of pitches involved is thirteen or
less. In fact, very recognizable song segments often only have
about seven pitches used and some have as few as three or four.
(One well known samba has only two used in the entire chorus - - -
that's interesting because it means that we can produce recozniable
music sometimes with as few as two keys - - - so using up to
thirteen will actually cover a large amount of ground! Even seven
will allow us to do many interesting and recognizable things.)
[0186] Frequently we'd used colors instead of numbers, such as blue
for the lowest note, red for the second lowest, etc. following some
arbitrary scheme, but here I will continue using numbers because it
will make it clearer to illustrate the principles. But keep in mind
that if we used a coloring scheme such as the one just mentioned,
the coloring and key position for the lowest note, the second
lowest note, etc. would remain the same for each song, so keys
could simply be painted particular colors and then used for song
after song provided something like a computer "retuned" what
pitches the keys actually played for each song or song segment.
[0187] I'm going to use conventional piano-style keyboard to show
some of the possibilities for laying out the pitches in a logical
and playable manner but we could use the same principles on a
fretboard, wind instrument or anything at all that could be
logically divided into designated "pieces" that could be "played"
in some fashion.
[0188] Keyboards often run from "C to C", meaning that if pitches
were assigned in a conventional manner to the keyboard, it would
start and end with a C. For instance, here is a two octave C to C
keyboard, though I'm not going to label the keys because I want to
emphasize their pattern.
[0189] Note that in many of these examples, the final key on the
right is retangular, even though if the same pitch were in the
middle of the keyboard, it would have a chunk taken out of an upper
corner (or two) to accommodate the positioning of a black key.
[0190] There can also be a one octave C to C keyboard.
[0191] A one octave F to F keyboard is also a common possibility
and looks almost the same except for the positioning of the black
keys.
[0192] In fact, you can logically see a one octave F to F keyboard
as a subset of a two octave C to C keyboard.
[0193] To make things easier for the player/beginner musician, we
can logically pick out which keys to use. Usually we'll only use
about seven, so even on a one octave keyboard there will be unused
keys. In fact, even though I will usually show about a dozen keys
numbered, for a particular song we would only use as many as were
necessary. For instance, for "Mary Had A Little Lamb", a four-pitch
song, we would only use the keys numbered 1, 2, 3 and 4.
[0194] That said, here is one simple way to use a two-octave
keyboard. In fact, we will only use white keys:
[0195] I'll call this a two-octave linear mapping.
[0196] Since we generally only use seven or so notes, one might
comment that when we played a song-segment it would usually look as
if we were always playing down on the left side of the keyboard.
Since we have an excess or notes, let's do a mapping starting
nearer the center. If we run out of notes, we can always add a few
black keys or even go back and use some of the white keys that we
skipped:
[0197] If we only use about seven notes, it will seem as if we are
playing near the center of the keyboard most of the time. Ill call
this "Linear in G" since I started on a G with key number 1. I
would have started anywhere, but I liked how this was centered and
how I started with a group of two black keys between the first
thrree notes.
[0198] We could also have started one key to the left, for example.
When I do that, I call it Linear in F.
[0199] With two octave, we can usually get by just using the white
keys. If we were trying to save money on the design or construction
or coloring issues of such an instrument, the black keys would not
necessarily have to move or function at all! They might be merely
decorative or help the player find a particular key position by
pattern easier. (Of course, we don't neccessariy have to use the
two black key, three black key pattern of a piano. The keys could
be any fashionable pattern or might even be arranged like the
144-bass buttons of an accoridion or some entirely new
arrangement.
[0200] For size, economy or other reasons, it might be desirable to
scale down to a one octave or possibly even smaller keyboard. In
that case, merely using the white keys might still be enough since
there are eight of them, but sometimes we might need a few more
pitches or perhaps some players might enjoy using black keys as
well as white keys. We could use any arbitrary system to assign
pitches, but here are a few I have tried:
[0201] One octave C to C Chromatic order
[0202] This is what I call "Downstairs Upstairs in C" order, first
using the white keys, then moving up to use the black keys.
[0203] One octave F to F chromatic. (This can also be another way
to use the center section of a two-octave C to C keyboard.)
[0204] One Octave Downstairs Upstairs In F
[0205] I've also done it with typewriter keyboards and even with an
Xbox controller. If these are done with colored keycaps or buttons
instead of numbers they are much easier for a player to match, but
I am showing numbers here to be clear about order.
[0206] Note that there are so many keys on a Qwerty keyboard or on
a multi-octave music keyboard, that multiple players could play at
once on the same keyboard competitively or cooperatively, with the
same sound or different sounds. Or multiple players could each have
their own keyboard (music, qwerty or even Xbox).
[0207] Here's an Xbox controller mapping that I've used to play
melodies. (It felt pretty strange and reminded me of playing an
ocarina, but I think I could get used to it with a little
practive.)
[0208] Many other variations are possible. One might ask, why might
want to have a two-octave C to C keyboard when one is only using
the center F to F section. A number of reasons are possible. For
one, the player/musician, espcially "wannabe's", might like the
appearance of the larger keyboard. Another reason is that the game
or instruction might later advance to also using the keyboard in a
conventional way, in which case having the full two octaves or even
more might be a very handy thing. Or perhaps, at a later time,
possibly as an upgrade, the full two octave keyboard could be used
with added lighting in the way described in the Focused Keyboard,
teaching fingering skills for melodies that diectly transfer to
most keyboard instruments.
APPENDIX B
[0209] How this Melody/Rhythm Game Differs From Existing Music
Games
[0210] This is a summary of the major differences in our approach
to a music game compared to existing technology and games. Note
that in this discussion "key" refers to a key on a keyboard (or a
fret position on a fretboard or other physical position on other
instrument) where the key produces a musical "note" or a particular
"pitch" (roughly, frequency). Therefore, in this discussion, the
terms "key", "note" and "pitch" are closely associated and are
often interchangeable. (Note that "key" here does not refer to key
signature, i.e. C Major or F minor.)
Narrowing the Attention to the Keys Actually Used in a Song or Song
Segment
[0211] There have been a lot of keyboard games of various types in
which result in the player playing a melody or melodic part. Most
notably, there has been The Miracle Piano System (known by several
other names) and more recently Piano Wizard. Some of these games,
such as Piano Wizard, use a color coding that is related to the
pitch of the note (i.e. the keys with a C pitch are one color, the
keys with a C# pitch another, the keys with D still another, etc.).
Note that in a game like this the colors are not related to the
particular song played. Other methods, such as Piano Wizard and
Casio lighted keyboards, use a single color for all the keys or use
no color at all. Note that none of these systems distinguish
between the keys actually used for a particular song or song part
and those not used within that song or song part.
[0212] The games described here, on the other hand, do distinguish
between the keys actually used in a song or song segment and those
which are not. Accordingly, the designations on the keyboard and/or
designations referring to the keyboard change depending on the song
or song part played. In some cases, they may change within a song
from one part of the song to another. This differentiates this game
from others in this category.
[0213] Also note that in embodiments of these games that support it
(by the hardware used), the colors used are not related to the key
pitch. For instance, in one embodiment described, the lowest key is
always one particular color (say "Blue") regardless of whether the
lowest pitch is a C, a D, or an E-flat. This also differentiates
this game from others.
Melody/Rhythm Games compared to Rhythm Games
[0214] There are some games currently available, such as Guitar
Hero and Rock Band, in which the player taps buttons or performs
some similar action in time with and response to the rhythm of the
game in most cases also prompted with a visual display. While in
these game the player might tap out a melody of, say, a guitar solo
in rhythm with a recording of the guitar solo, there is not a
one-to-one correspondence between the pitches of the melody and the
buttons on the instrument, i.e. the same button might be used to
respond to a F pitch at one point and shortly thereafter respond to
a F-sharp, G, or some other pitch, generally without the player
being aware that anything had changed. In some cases, the same
pitch may be represented by a different button in another part of
the song or one button might correspond to more than one pitch
being played. In the games described here, the same key or button
always corresponds to the same pitch within the given song or song
segment. Within a song segment, one key means one particular pitch
and that particular pitch is only mapped to that one key.
Melody/Rhythm Games compared to Melody Games
[0215] There are some games, in particular on some Casio keyboards,
where a key is lit and when the player plays that key the next key
used in the song is lit up. Thus, the player is led note by note
through a song. (In some cases, more than one light might go on
indicating that two keys are to be played simultaneously). The only
lights that are on are, at most, the key or keys being played and
the key or keys to be played next. The game described here differs
from that in two significant ways: all the keys used in a song or
segment are designated at once for the length of the song or song
segment, not just immediately before they are needed - - - and - -
- in many of these other games, the player sets the rhythm by the
speed that they play speeding up or slowing down at their will
whereas in our game the rhythm of the song is steady and the player
must play at the same speed to get the best score--the speed of the
song does not vary with the speed of the player.
An Actual Musical Instrument May be Used in Some But Not All
Embodiments
[0216] The distinction between game controllers and actual musical
instruments can get quite fuzzy these days, particularly where
computers are involved enhancing and transforming either. Most
electronic games these days now involve a computer processor and
most electronic musical instruments also involve a computer
processor.
[0217] Therefore, what the player actually "plays" in the game
described can range from a game controller to a computer keyboard
to a toy musical keyboard to an actual musical keyboard to a
fretboard to the keys of a wind instrument or to the fingerboard of
a bowed instrument to moving ones hands in the air to other
embodiments. The game could be played using any distinguishable
movements that could be associated with pitches. The associated
instrument need not even be electronic, since its sound could be
detected, for instance, with a microphone and/or the player's
movements by a camera or other motion sensing mechanism.
[0218] Most of the descriptions here will concentrate on a game
being played using the keys of an electronic musical keyboard, but
in each case, it could be a fretboard, a toy instrument, a computer
keyboard, a game controller, a touchscreen or any of the
embodiments mentioned above. In the process of testing, this game
has already been played on a music keyboard, a computer keyboard, a
game controller and toy musical instruments models. This represents
only a few of the possible ways to implement this game.
[0219] It is also worth noting as a summary point that in some
implementations that keyboard (or fretboard, etc.) is used only as
a passive input device to the game, but in other embodiments, the
game controls features of the keyboard in a way to correspond with
game usage, most notably to illuminate or otherwise highlight (or
further highlight) the keys that are actually used within the
current song or song segment. Depending on the particular
embodiment, all the keys so used might highlight in the same way
or, alternatively, several means of highlight (such as several
colors) might be used to further distinguish one highlighted key
(or button, etc.) from another. Further, in embodiments where
highlighted keys are further distinguished from one another, the
highlight for each individual key might be unique (such as a unique
color) or it may be one of several types of highlighting (for
instance, when the hardware is incapable of supplying a unique
color for every highlighted key). In these latter cases, the
highlighting system may or may not correspond with the pitch
"letter" of the key, i.e. the same color of highlighting may or may
not be used for all pitch "C's" and if it is necessary to use the
same type of color or highlighting more than once, the keys
highlighted with one particular color may not necessarily be the
same pitch.
[0220] Appendix C illustrates several types of highlight schemes
possible for the song segment consisting of the first two lines of
the popular song "Take Me Out To The Ball Game".
APPENDIX C
Examples of Several Possible Highlighting Schemes for the Focused
Keyboard
[0221] The following examples use the melody of the first two lines
of the popular song Take Me Out To The Ball Game as a song segment.
Though the actual music notation and pitch letter names generally
do not appear directly in the game, here they are for
reference:
[0222] If the keyboard used for the game embodiment is one only
capable of highlight keys one way, the highlighting of the keyboard
for the playing of this song segment might look like this:
[0223] In another embodiment of the game, a keyboard might be used
that could light each key any one of three colors. For that
embodiment, this song segment might look like this:
[0224] Another embodiment of the game might have a rotating pattern
of, say, five colors repeated in a chromatic pattern, e.g.
C Blue
C# Purple
D Green
D# Orange
E Red
F Blue
F# Purple
G Green
G# Orange
A Red
A# Blue
B Purple
C Green
[0225] On such a keyboard, this song segment would look like:
[0226] On still another embodiment, there might be six colors
repeated in a chromatic pattern:
C Blue
C# Purple
D Green
D# Orange
E Red
F Yellow
F# Blue
G Purple
G# Green
A Orange
A# Red
B Yellow
C Blue
[0227] The song segment might then appear like this:
[0228] It might also appear that same way on an envisioned seven
color scheme that repeats on each octave yet gives each white key
pitch a distinct color:
C Blue
C# Purple
D Green
D# Orange
E Red
F Cyan
F# Blue
G Purple
G# Green
A Orange
A# Red
B Yellow
C Blue
[0229] The most important thing to note is that the keys actually
used in the song or song segment are highlighted in some way to
easily distinguish them from those that are not used. Also note
that the highlighting remains present throughout the entire song or
segment.
[0230] The most important effect of this method of highlighting is
to eliminate all of the keys that are not used in the song or song
segment from the player's consideration. Since the number of
pitches actually used in the song or song segment is generally much
smaller than the total number of pitches available, the player is
able to focus only on this smaller number of keys and therefore
success at playing the melody and rhythm involved is greatly
improved and the player experiences more satisfaction.
[0231] The keys or buttons need not be arranged in a way
corresponding to an actual musical instrument, but if they are, the
player will find it easy to transfer the patterns learned in
playing the game to performance on an actual conventional musical
instrument.
APPENDIX D
Using Keyboards or Buttons that are Permanently Colored and do not
in Themselves "Highlight"
[0232] For cost considerations or other reasons, it may be
desirable to use a keyboard (or buttons, fretboard, etc.) on which
keys are differentiated by color or some other means but on which
this highlighting is static and does not change.
[0233] For example, here is a one-octave "painted" keyboard on
which only the "white" keys are able to be used (we would not use
this as a "focused" keyboard for any songs or song segments that
required the use of any "black" keys if we were trying to use a
layout similar to an actual musical instrument).
[0234] We could still use this as a focused keyboard if we are able
to indicate which keys are actually used by some other distinctive
means. Following the "Take Me Out to the Ball Game" example, we
might for instance use this keyboard with a display that indicates
all the notes actually used in the song or song segment.
[0235] For instance, here is the above keyboard associated with a
screen or other display for the game:
[0236] Note that there are six keys used within this song segment
(the first two lines of Take Me Out To The Ball Game) and we can
easily see which ones they are by both the fact that the line
colors match those keys and by the fact that the lines are arranged
in the same order as the keys and physically approach them if the
keyboard is aligned below the display. Only six of the eight
possible "white" keys are used and none of the possible black keys
are used, thereby limited the player's consideration to only 6 of
13 possible pitches in the range of the keyboard. If the keyboard
were larger than an octave, this consideration would be even more
dramatic, being 6 of 25 for a two-octave keyboard or 6 of 88 for a
full sized piano. Even someone with no musical experience generally
has no trouble keeping track of only six possibilities, especially
when they are already familiar with the melody and rhythm of the
song. This level of concentration is similar to conventional video
games which might require the player to operate six different
buttons or parts of buttons within a short time period.
[0237] With some reduction in perceptual ease, this setup could be
reduced to even simpler hardware or logical embodiments. For
instance, we could eliminate the use of slanted lines, perhaps
making it possible to use a simpler display:
[0238] Note that the display (or panel) might be attached directly
to the keyboard. Also note that since the use of each color on this
display is restricted to a unique area, this display could, for
example, be achieved by a panel that selectively illuminate lines
and dots in white with a multi-color tinted filter placed over it.
(The display of the previous example could also be produced that
way, though the tinted areas would be trapezoids instead of
rectangles.)
[0239] In a more primitive form, the only important distinction
between notes might be whether they are included in the song or
not. This might lead to a very basic display like this:
[0240] Note that this form of display still makes it possible for
the player to easily tell which keys are included in the song or
song segment and which are not.
[0241] Also note that the examples used so far in this appendix
used white keys only and showed how a keyboard could be used
realistically even if the "black keys" did not operate. But the
examples could have used instead a painted keyboard with colored
black keys of unique colors or repeated colors, for instance this
one:
[0242] Please note that all of the examples in this appendix
operate the keyboard as a "focused keyboard", rather than a "song
specific" keyboard. Recall that on a "focused keyboard" the keys
(pitches) that are not used are not highlighted and the
configuration of pitches found on the keyboard resembles that of a
conventional musical keyboard. On the "song specific" keyboard,
pitches that are not used do not appear as keys at all on the
keyboard, the remaining keys are adjacent to one another and the
arrangement of pitches on the keys changes with each song or song
segment and does not resemble the arrangement of pitches on a
conventional music keyboard. The "song specific" keyboard is fully
described elsewhere in this document.
* * * * *
References