U.S. patent application number 14/499692 was filed with the patent office on 2016-03-31 for method for instant recognition of traffic lights countdown image.
The applicant listed for this patent is YUAN ZE UNIVERSITY. Invention is credited to DUAN-YU CHEN, YI-TUNG CHOU.
Application Number | 20160092742 14/499692 |
Document ID | / |
Family ID | 55584793 |
Filed Date | 2016-03-31 |
United States Patent
Application |
20160092742 |
Kind Code |
A1 |
CHEN; DUAN-YU ; et
al. |
March 31, 2016 |
METHOD FOR INSTANT RECOGNITION OF TRAFFIC LIGHTS COUNTDOWN
IMAGE
Abstract
A method for instant recognition of traffic lights countdown
image that can quickly scan and confirm the circular feature image
of a traffic light, and retrieve the countdown image thereof by
calculating the displacement ratio from the circular image, then
enhance, cut and converse the countdown image to display a feature
image thereof, and proceed similarity comparison with collected
data to calculate the percentage of similarity. The method
eventually brings out a result from the image comparisons, so as to
fulfill the effectiveness of searching and instantly recognizing
the countdown image of a traffic light.
Inventors: |
CHEN; DUAN-YU; (TAOYUAN
COUNTY, TW) ; CHOU; YI-TUNG; (TAOYUAN COUNTY,
TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
YUAN ZE UNIVERSITY |
TAOYUAN COUNTY |
|
TW |
|
|
Family ID: |
55584793 |
Appl. No.: |
14/499692 |
Filed: |
September 29, 2014 |
Current U.S.
Class: |
382/103 |
Current CPC
Class: |
G06K 9/00825 20130101;
G06T 7/90 20170101; G06T 2207/30252 20130101 |
International
Class: |
G06K 9/00 20060101
G06K009/00; G06K 9/62 20060101 G06K009/62; G06T 3/40 20060101
G06T003/40; G06K 9/46 20060101 G06K009/46 |
Claims
1. A method for instant recognition of traffic lights countdown
image, comprising: retrieving information of a real-time image that
is divided into a plurality of partitions, each having the four
corners as confirmed pixels which are processed within HSL (hue,
saturation, lightness) color space; scanning the color
features--hue, saturation, and lightness--of the confirmed pixels
and when the color features conform to the predetermined ones,
rendering the confirmed pixels as candidate pixels which turn the
neighboring four partitions into candidate regions; searching the
neighboring confirmed pixels of the candidate pixels that resemble
the features thereof and render them as candidate pixels as well
until every neighboring confirmed pixel is checked, and merging all
the candidate pixels and the candidate regions thereof together as
a group; conversing the merged candidate regions within HSL color
space by adjusting the threshold value of the lightness of said
merged candidate regions with adaptive threshold algorithm,
conversing into a binary image of lightness, and then conversing
the binary image of lightness into an edge image of lightness after
the edge detection processing, then intersecting the edge image of
lightness and a binary image of hue conversed from the candidate
regions in accordance with its range of color, and producing an
edge image which has a feature of circular image found after Hough
transform algorithm operation and to be compared with a
predetermined circular shape of a traffic light; confirming the
circular feature as the shape of a traffic light and then
retrieving a countdown image by calculating the displacement ratio
from the circular image; enhancing said countdown image by super
resolution algorithm and conversing into a greyscale image, then
adjusting the threshold value thereof by adaptive threshold
algorithm and conversing into an image of binary numbers; gathering
the horizontal and vertical projection information of the image of
binary numbers, finding the threshold value of the top and bottom
edge thereof and figuring out the width and the estimated cutting
curve thereof, and then cutting the image of binary numbers along
the cutting curve calculated by partial distribution statistics of
the vertical projection information and the estimated cutting
curve, then applying the block coding algorithm to display a
feature image by dividing the image of binary numbers into equal
rectangular blocks, calculating the ratio of black pixels and white
pixels of each rectangular block, and encoding the results; and
classifying and concluding all collected images of numbers by a
plurality of classifiers with machine learning algorithm to analyze
and compare with the feature image, and calculating the percentage
of similarity, then bringing out the image of the highest
percentage as the recognition result among the ones from the
classifiers.
2. The method as claimed in claim 1, wherein the range of the hue
is from 0.3 to 0.92 after normalization.
Description
BACKGROUND OF THE INVENTION
[0001] 1. Field of the Invention
[0002] The invention relates to a method for instant recognition of
traffic lights countdown image that can quickly scan and confirm
the circular feature of a traffic light, and retrieve the countdown
image thereof by calculating the displacement ratio from the
circular feature, then process the countdown image to display a
feature image thereof and bring out the recognition result.
[0003] 2. Description of the Related Art
[0004] Traffic lights have red, yellow, and green lights, among
which the red and green lights may have countdown function to
remind the drivers when the lights are about to change, so as to
eliminate safety concerns during the driving.
[0005] Nevertheless, the drivers may lose attention to the
countdown of traffic lights due to poor sight or distraction when
driving, and thus cause irreparable loss. Also, optical illusion
may happen owning to long time driving or exhaustion; but the
optical illusion can be remedied by machine vision to recognize the
status of the countdown.
[0006] However, there are still some defects for machine vision to
recognize the countdown image of traffic lights because of
unexpected situations caused by traffic and environment variation.
Firstly, the countdown has different image of numbers with the
passing of time; consequently, the machine vision has to process
different contours of the numbers individually. When the machine
vision cannot proceed with the image of numbers in time, it cannot
further proceed with the recognition, therefore decreasing its
reliability. Secondly, when the ambient light is weak, the
retrieved image may have diffraction areas and result in difficulty
in retrieving the exact image and conversing the image to binary
mode in accordance with its features. Such problem cannot be solved
even if the image is enhanced. Besides, the countdown image is
conversed from RGB image to greyscale image, but the LED light
source emitting the countdown numbers would cause detailed and
undetectable changes of the greyscale image. Hence, the countdown
image can be difficult to recognize due to the effects from the
ambient lights, and the contours of the figures are different,
making the recognizing process even more complicated and the entire
process is therefore slower.
SUMMARY OF THE INVENTION
[0007] It is a primary object of the present invention to provide a
method for instant recognition of traffic lights countdown image
that can quickly scan and confirm the circular feature of a traffic
light and retrieve the countdown image thereof by calculating the
displacement ratio from the circular feature, so as to provide a
solution to the problem of finding the countdown image of a traffic
light in the prior art.
[0008] Another object of the present invention is to provide a
method for instant recognition of traffic lights countdown image
that proceeds the countdown image with enhancing, cutting, and
recognizing operation, so as to provide a solution to the problem
of slow detection of the counting-down in the prior art.
[0009] In order to achieve the objects above, the present invention
comprises the following steps: retrieving information of a
real-time image that is divided into a plurality of partitions,
each having the four corners as confirmed pixels which are
processed within HSL (hue, saturation, lightness) color space;
scanning the color features--hue, saturation, and lightness--of the
confirmed pixels and when the color features conform to the
predetermined ones, rendering the confirmed pixels as candidate
pixels which turn the neighboring four partitions into candidate
regions; searching the neighboring confirmed pixels of the
candidate pixels that resemble the features thereof and render them
as candidate pixels as well until every neighboring confirmed pixel
is checked, and merging all the candidate pixels and the candidate
regions thereof together as a group; conversing the merged
candidate regions within HSL color space by adjusting the threshold
value of the lightness of said merged candidate regions with
adaptive threshold algorithm, conversing into a binary image of
lightness, and then conversing the binary image of lightness into
an edge image of lightness after the edge detection processing,
then intersecting the edge image of lightness and a binary image of
hue conversed from the candidate regions in accordance with its
range of color, and producing an edge image which has a feature of
circular image found after Hough transform algorithm operation and
to be compared with a predetermined circular shape of a traffic
light; confirming the circular feature as the shape of a traffic
light and then retrieving a countdown image by calculating the
displacement ratio from the circular image; enhancing said
countdown image by super resolution algorithm and conversing into a
greyscale image, then adjusting the threshold value thereof by
adaptive threshold algorithm and conversing into an image of binary
numbers; gathering the horizontal and vertical projection
information of the image of binary numbers, finding the threshold
value of the top and bottom edge thereof and figuring out the width
and the estimated cutting curve thereof, and then cutting the image
of binary numbers along the cutting curve calculated by partial
distribution statistics of the vertical projection information near
the estimated cutting curve, then applying the block coding
algorithm to display a feature image by dividing the image of
binary numbers into equal rectangular blocks, calculating the ratio
of black pixels and white pixels of each rectangular block, and
encoding the results; and classifying and concluding all collected
images of numbers by a plurality of classifiers with machine
learning algorithm to analyze and compare with the feature image,
and calculating the percentage of similarity, then bringing out the
image of the highest percentage as the recognition result among the
ones from the classifiers.
[0010] In the process mentioned above, the range of the hue is from
0.3 to 0.92 after normalization.
[0011] As stated above, the present invention can quickly scan and
confirm the circular feature of a traffic light, and retrieve the
countdown image thereof by calculating the displacement ratio from
the circular feature, then enhance, cut and converse the countdown
image to display a feature image thereof so as to fulfill the
effectiveness of searching and instantly recognizing the countdown
image of a traffic light.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] FIG. 1 is a flow diagram of the present invention;
[0013] FIG. 2A is a schematic diagram illustrating the settling of
confirmed pixels of the present invention;
[0014] FIG. 2B is a schematic diagram illustrating a confirmed
pixel becoming a candidate pixel and rendering its neighboring area
as candidate regions;
[0015] FIG. 2C is a schematic diagram illustrating a resembling
confirmed pixel becoming another candidate pixel and rendering its
neighboring area as candidate regions;
[0016] FIG. 3A is a schematic diagram of a binary image of
lightness conversed from the hue of the candidate regions;
[0017] FIG. 3B is a schematic diagram of an edge image of lightness
according to the present invention;
[0018] FIG. 3C is a schematic diagram illustrating the hue of the
candidate regions conversing into a binary image;
[0019] FIG. 3D is a schematic diagram of an edge image produced
after intersecting process according to the present invention;
[0020] FIG. 3E is a schematic diagram illustrating the circular
feature found by the method of the present invention;
[0021] FIG. 4 is a schematic diagram of a countdown image retrieved
by the method of the present invention;
[0022] FIG. 5A is a schematic diagram of a greyscale image of the
image retrieved by the method of the present invention;
[0023] FIG. 5B is a schematic diagram of a binary image of the
image retrieved by the method of the present invention;
[0024] FIG. 6A is a horizontal projection histogram of the binary
image according to the present invention;
[0025] FIG. 6B is a vertical projection histogram of the binary
image according to the present invention;
[0026] FIG. 6C is a schematic diagram of a numeral image according
to the present invention with the surrounding blank area
excised;
[0027] FIG. 6D is a schematic diagram of cut numeral image
according to the present invention; and
[0028] FIG. 7 is a schematic diagram of a feature image according
to the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0029] Referring to FIGS. 1 to 7, a preferred embodiment of the
present invention includes steps as following. Step 1 S1:
initiating the process. Step 2 S2: retrieving information of a
real-time image that is divided into a plurality of partitions,
each having the four corners as the confirmed pixels which are
processed within HSL (hue, saturation, lightness) color space. FIG.
2A is a partial image of the real-time image that is divided into
12 partitions C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, in
which the fifth partition C5 has its four corners as the first,
second, third, and fourth confirmed pixels P1, P2, P3, and P4, and
the sixth partition has its four corners as the second, fourth,
fifth, and sixth confirmed pixels P2, P4, P5, and P6.
[0030] Step 3 S3: scanning the color features--hue, saturation, and
lightness--of the confirmed pixels and when the features conform to
the predetermined ones, rendering the confirmed pixels as candidate
pixels which turn the neighboring four partitions into candidate
regions. In this embodiment, the range of the hue is from 0.3 to
0.92 after normalization, but it is not limited to such
application. In FIG. 2B, the first confirmed pixel P1 conforms to
the predetermined color features and becomes the first candidate
pixel T1, rendering the neighboring first, second, fourth, and
fifth partitions C1, C2, C4, and C5 as the first, second, fourth,
and fifth candidate regions R1, R2, R4, and R5.
[0031] Step 4 S4: searching the neighboring confirmed pixels of the
candidate pixels that resemble the features thereof and render them
as candidate pixels as well until every neighboring confirmed pixel
is checked, and merging all the candidate pixels and the candidate
regions thereof together as a group. As shown in FIG. 2C, in this
embodiment, the color features of the second confirmed pixel P2
resemble the ones of the first candidate pixel T1, rendering the
second confirmed pixel P2 as the second candidate pixel T2. When
there is no other candidate pixel, the first and second candidate
pixels T1, T2 merge into a group together with the first, second,
third, fourth, fifth, and sixth candidate regions R1, R2, R3, R4,
R5, and R6.
[0032] Step 5 S5: conversing the merged candidate regions within
HSL color space by adjusting the threshold value of the lightness
of said merged candidate regions with adaptive threshold algorithm,
conversing into a binary image of lightness W1 as shown in FIG. 3A,
and then conversing the binary image of lightness W1 into an edge
image of lightness E1 after the edge detection processing as
illustrated in FIG. 3B; then intersecting the edge image of
lightness E1 and a binary image of hue presented in FIG. 3C,
conversed from the candidate regions in accordance with its range
of color; the result of intersecting is shown in FIG. 3D. With
reference to FIG. 3E, an edge image E2 is produced which has a
feature of circular image F1 found after Hough transform algorithm
operation and to be compared with a predetermined circular shape of
a traffic light.
[0033] Step 6 S6: referring to FIG. 4, confirming the feature of
circular image F1 as the shape of a traffic light and then
retrieving the center and the diameter D1 thereof to calculate the
displacement ratio D2 from the circular image F1 and then retrieve
the countdown image F2; if the confirmation failed, go to step 7
S7: reexamining the candidate regions for another confirmation
process as in step 6 S6. If there is another area within the
regions to be confirmed, go to step 4 S4 and run the process again
therefrom; if every area within the candidate regions is examined
and the confirmation still failed, go to step 11 S11: terminating
the process.
[0034] Step 8 S8: as shown in FIG. 5A, enhancing said countdown
image F2 by super resolution algorithm and conversing into a
greyscale image V, then adjusting the threshold value thereof by
adaptive threshold algorithm and conversing into an image of binary
numbers A as presented in FIG. 5B.
[0035] Step 9 S9: with reference to FIGS. 6A, 6B, 6C and 6D,
gathering the horizontal and vertical projection information of the
image of binary numbers A, finding the threshold value of the top
and bottom margin thereof, excising the surrounding blank area and
figuring out the width and the estimated cutting curve G thereof,
and then cutting the image of binary numbers along the cutting
curve calculated by partial distribution statistics of the vertical
projection information near the estimated cutting curve.
[0036] In this embodiment, the image of binary numbers A includes a
digit of the units A1 and a digit of the tens A2. To find out the
estimated cutting curve, the position of the digit of the units A1
has to be located first, according to which the one of the digit of
the tens A2 is located as well; then find out the right-hand margin
of the digit of the units A1, according to which the left-hand
margin thereof as well. The estimated cutting curve G is therefore
confirmed as the left-hand margin of the digit of the units A1.
Then perform an operation on the vertical projection of the area
within 10 pixels extending from the estimated position to both
sides thereof, so as to find the exact cutting curve.
[0037] Referring to FIG. 7, dividing the image of binary numbers A
equally into twenty-one rectangular blocks and assign as
B1.about.B12; calculating the ratio of black pixels X and white
pixels Y of each rectangular block and encoding the results to
converse into a feature image Z. The amount of the white pixels Y
within each block is the descriptor. Besides, in order to have an
identical standard for processing, each side of the blocks is
defined as 20 pixels, thus defining each block having 400 pixels
within.
[0038] Step 10 S10: classifying and concluding all collected images
of numbers by a plurality of classifiers with machine learning
algorithm to analyze and compare with the feature image Z, and
calculating the percentage of similarity, then bringing out the
image of the highest percentage as the recognition result among the
ones from the classifiers. Step 11 S11: terminating the
process.
[0039] The method for instant recognition of traffic lights
countdown image can be written as a program and further applied
when being installed on devices with shooting function. For
example, it can be applied to navigation systems for best route
analysis, instant traffic monitor, or traffic warning system for
drivers. Also, it can be applied to cloud systems for keeping track
of the countdown of other traffic lights nearby. With the
applications, the present invention has achieved effectiveness of
instant recognition and expanded tracking and monitoring in the
practical fields.
* * * * *