U.S. patent application number 13/954373 was filed with the patent office on 2014-01-23 for location based services quality assessment.
This patent application is currently assigned to AT&T Mobility II LLC. The applicant listed for this patent is AT&T Mobility II LLC. Invention is credited to Jeffrey A. Krinsky, Leslie Paul Leonard, Linh Nguyen, Falguni Sarkar.
Application Number | 20140024363 13/954373 |
Document ID | / |
Family ID | 40583491 |
Filed Date | 2014-01-23 |
United States Patent
Application |
20140024363 |
Kind Code |
A1 |
Krinsky; Jeffrey A. ; et
al. |
January 23, 2014 |
LOCATION BASED SERVICES QUALITY ASSESSMENT
Abstract
A technique to gather and assess the quality of a location based
services provided to a mobile wireless device via a wireless
network includes providing location-based service information by
referring to a target location of a requested item of interest
within a proximity of the present location of the mobile wireless
device. Subjective quality or accuracy information can be submitted
to the wireless network from the from the wireless device. The
subjective quality or accuracy information is correlated with event
and condition information affecting the wireless network. Reports
can be generated which indicate the quality and accuracy
information of the target location information. The reports are
useful for troubleshooting location-based service inaccuracy.
Inventors: |
Krinsky; Jeffrey A.;
(Redmond, WA) ; Leonard; Leslie Paul; (Kingston,
WA) ; Nguyen; Linh; (Bellevue, WA) ; Sarkar;
Falguni; (Redmond, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
AT&T Mobility II LLC |
Atlanta |
GA |
US |
|
|
Assignee: |
AT&T Mobility II LLC
Atlanta
GA
|
Family ID: |
40583491 |
Appl. No.: |
13/954373 |
Filed: |
July 30, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12142629 |
Jun 19, 2008 |
8509761 |
|
|
13954373 |
|
|
|
|
11227972 |
Sep 15, 2005 |
8396468 |
|
|
12142629 |
|
|
|
|
Current U.S.
Class: |
455/422.1 |
Current CPC
Class: |
H04W 4/02 20130101; H04W
4/029 20180201; H04W 24/08 20130101 |
Class at
Publication: |
455/422.1 |
International
Class: |
H04W 24/08 20060101
H04W024/08 |
Claims
1. A method comprising: receiving subjective feedback corresponding
to an accuracy of a target location with respect to a location of a
device; generating correlated information by correlating the
subjective feedback with event and condition information affecting
the device; and providing an indication of a quality of service for
the device, the quality of service being based on the correlated
information.
2. The method of claim 1, further comprising: requesting
information pertaining to the target location.
3. The method of claim 1, further comprising: receiving a request
for the target location.
4. The method of claim 1, further comprising: receiving a
subjective quality rating.
5. The method of claim 1, further comprising: receiving a
time-stamped subjective quality rating, wherein the subjection
quality rating is time-stamped by the device.
6. The method of claim 1, further comprising: receiving multiple
instances of subjective quality ratings.
7. The method of claim 1, wherein: the correlated information
comprises the subjective feedback correlated with at least one of a
user call data record, a road traffic record, a weather record,
location data, network performance data, or video server
performance data.
8. The method of claim 1, wherein: the location of the device is
determined via at least one of global positioning system data, time
differential of arrival data, angle of arrival data, or user input
data.
9. The method of claim 1, further comprising: providing the quality
of service in association with data specifically related to network
performance data during a time of the received subjective user
feedback.
10. An apparatus comprising; a processor; and memory coupled to the
processor, the memory comprising executable instructions that when
executed by the processor cause the processor to effectuate
operations comprising: receiving subjective feedback corresponding
to an accuracy of a target location with respect to a location of a
device; generating correlated information by correlating the
subjective feedback with event and condition information affecting
the device; and providing an indication of a quality of service for
the device, the quality of service being based on the correlated
information.
11. The apparatus of claim 10, the operations further comprising:
requesting information pertaining to the target location.
12. The apparatus of claim 10, the operations further comprising:
receiving a request for the target location.
13. The apparatus of claim 10, the operations further comprising:
receiving a subjective quality rating.
14. The apparatus of claim 10, the operations further comprising:
receiving a time-stamped subjective quality rating, wherein the
subjection quality rating is time-stamped by the device.
15. The apparatus of claim 10, the operations further comprising:
receiving multiple instances of subjective quality ratings.
16. The apparatus of claim 10, wherein: the correlated information
comprises the subjective feedback correlated with at least one of a
user call data record, a road traffic record, a weather record,
location data, network performance data, or video server
performance data.
17. The apparatus of claim 10, wherein: the location of the device
is determined via at least one of global positioning system data,
time differential of arrival data, angle of arrival data, or user
input data.
18. The apparatus of claim 10, the operations further comprising:
providing the quality of service in association with data
specifically related to network performance data during a time of
the received subjective user feedback.
19. An apparatus comprising; a processor; and memory coupled to the
processor, the memory comprising executable instructions that when
executed by the processor cause the processor to effectuate
operations comprising: providing subjective feedback corresponding
to an accuracy of a target location with respect to a location of
the apparatus; generating correlated information by correlating the
subjective feedback with event and condition information affecting
the device; and receiving an indication of a quality of service for
the apparatus, the quality of service being based on the correlated
information that was correlated by correlating the subjective
feedback with event and condition information affecting the
apparatus.
20. The apparatus of claim 19, wherein: the correlated information
comprises the subjective feedback correlated with at least one of a
user call data record, a road traffic record, a weather record,
location data, network performance data, or video server
performance data.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] The instant application is a continuation application of,
and claims priority to U.S. patent application Ser. No. 12/142,629,
filed Jun. 19, 2008. U.S. patent application Ser. No. 12/142,629 is
a continuation-in-part application of U.S. patent application Ser.
No. 11/227,972, filed Sep. 15, 2005. U.S. patent application Ser.
No. 11/227,972 issued on Mar. 12, 2013 with U.S. Pat. No.
8,396,468. U.S. patent application Ser. No. 12/142,629 is
incorporated by reference herein in its entirety. U.S. patent
application Ser. No. 11/227,972 is incorporated by reference herein
in its entirety. U.S. Pat. No. 8,396,468 is incorporated by
reference herein in its entirety.
TECHNICAL FIELD
[0002] The technical field is generally related to communication
systems and more specifically related to assessing the quality of a
service provided via a communications system.
BACKGROUND
[0003] Maintaining quality of service in the eyes of customers is
an important factor in services industries. This is the case in the
wireless/mobile communications industry, as well as the traditional
wireline telephone industry where customers often base their
decisions to join, or stay with a particular service provider based
on the quality of the services provided. For example, with respect
to wireless telephones, events such as recurring dropped calls,
poor sound quality during calls and unexpected unavailability of
service may drive customers to seek a new service provider,
especially given rising standards for call quality.
[0004] Currently, various techniques exist for monitoring quality
of service in the wireless communications industry. Some of these
techniques, such as PESQ (Perceptual Evaluation of Speech Quality)
obtain objective quality of service information. For example, PESQ
measures voice quality by comparing an input test signal with the
signal output across a connection. Another objective technique,
ITU-T E-model (e.g., ITU-T G. 107) predicts conversational MOS
(mean opinion score) from IP networks and/or terminal parameters.
Subjective techniques also exist where test mobile devices are
established to monitor sound quality. Service providers often
arrange for such testing immediately following deployment of new
networks, new network elements (such as base stations) and network
changes or upgrades, etc.
[0005] The techniques described above are often difficult to
implement, may be limited in their capability to monitor an entire
network, and may be expensive, especially in the case of call
quality monitoring techniques that utilize specialized
infrastructure in the wireless environment. Many of these
techniques may also cause unwanted load on the network. In
addition, it is often not practical to implement such techniques on
a regular basis. In addition, current quality monitoring techniques
are difficult to implement throughout the entire network,
especially when networks may span large and diverse geographical
area and the test devices have to be frequently mobile to simulate
as much as the real user experience. For example, current quality
monitoring techniques may be implemented so that a single sector is
used to report on the quality of an entire region. Accordingly, it
may be difficult to accurately monitor all areas of the network
using such techniques.
SUMMARY
[0006] Described herein is a means for user feedback concerning
observed accuracy and quality of location-based services in a
wireless networked system. In one embodiment, a wireless network
provides a capability to accept user-provided quality feedback
information concerning a target location data provided by the
network in response to a query for location information from the
user. Feedback data provided by the user can be correlated to other
aspects of system operation such as the preset location of the
wireless receiving device, the call traffic of the system, and
other relevant information. Reports may be generated to assist a
system engineer or system administrator in troubleshooting and
adjusting the wireless network to lessen or eliminate the target
location data provided by the wireless system.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1 is a block diagram showing an example of an
environment in which location based services quality assessment can
be implemented.
[0008] FIG. 2 is a block diagram showing the mobile communication
device of FIG. 1 in one embodiment.
[0009] FIG. 3 is block diagram showing components of the mobile
service provider administration system of FIG. 1 in one
embodiment.
[0010] FIGS. 4A and 4B are flow diagrams showing examples of high
level call quality assessment routines in some embodiments.
[0011] FIG. 5 is a flow diagram showing an example of an automatic
callback routine for reporting on call quality in one
embodiment.
[0012] FIG. 6 is flow diagram showing an example of an optional
callback routine for reporting on call quality in one
embodiment.
[0013] FIG. 7 is flow diagram showing an example of routine at the
mobile device of FIGS. 1 and 2 for reporting on call quality using
a questionnaire in one embodiment.
[0014] FIG. 8 is flow diagram showing an example of routine at the
mobile service provider system of FIGS. 1 and 3 for reporting on
call quality using a questionnaire in one embodiment.
[0015] FIG. 9 is a display diagram showing an example of a user
interface for acquiring call quality information from a user of a
mobile device in one embodiment.
[0016] FIG. 10 is a display diagram showing a second example of a
user interface for acquiring call quality information from a user
of a mobile device in one embodiment.
[0017] FIG. 11 is a block diagram of an example location quality
assessment system.
[0018] In the drawings, the same reference numbers identify
identical or substantially similar elements or acts. To facilitate
the discussion of any particular element or act, the most
significant digit or digits in a reference number refer to the
figure number in which that element is first introduced (e-g.,
element 204 is first introduced and discussed with respect to FIG.
2).
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
[0019] Location based services quality assessment will now be
described with respect to various embodiments. The following
description provides specific details for a thorough understanding
of, and enabling description for, these embodiments. However, one
skilled in the art will understand that location based services
quality assessment may be practiced without these details. In other
instances, well-known structures and functions have not been shown
or described in detail to avoid unnecessarily obscuring the
description of the embodiments.
[0020] It is intended that the terminology used in the description
presented be interpreted in its broadest reasonable manner, even
though it is being used in conjunction with a detailed description
of certain specific embodiments. Certain terms may even be
emphasized below; however, any terminology intended to be
interpreted in any restricted manner will be overtly and
specifically defined as such in this Detailed Description
section.
I. REPRESENTATIVE SYSTEM
[0021] FIGS. 1-3 and the following discussion provide a brief,
general description of a suitable computing/network environment in
which the call quality monitoring can be implemented. Although not
required, aspects of the call quality monitoring are described in
the general context of computer-executable instructions, such as
routines executed by a general-purpose computer, e.g., a server
computer, wireless device, or personal computer. Those skilled in
the relevant art will appreciate that location based services
quality assessment can be practiced with other communications, data
processing or computer system configurations, including Internet
appliances, hand-held devices (including PDAs), wearable computers,
all manner of cellular or mobile phones, multi-processor systems,
microprocessor-based, ASIC-based, or PGA-based systems,
programmable consumer electronics, set-top boxes, network PCs,
minicomputers, mainframe computers, and the like. Indeed, the terms
"computer," "device," and "component" are generally used broadly
and interchangeably, and refer to any of the above devices and
systems, as well as any data processor.
[0022] Aspects of the call quality monitoring can be embodied in a
special purpose computer or data processor that is specifically
programmed, configured, or constructed to perform one or more of
the computer-executable instructions explained in detail herein.
Aspects of the call quality monitoring can also be practiced in
distributed computing environments where tasks or modules are
performed by remote processing devices, which are linked through a
communications network, such as a LAN, WAN, or the Internet. In a
distributed computing environment, program modules may be located
in both local and remote memory storage devices (e.g., including
memory associated with field programs, gate arrays, EPROM memory,
etc.).
[0023] Aspects of the call quality monitoring may be stored or
distributed on computer-readable media, including magnetically or
optically readable computer discs, hard-wired or preprogrammed in
chips (e.g., EEPROM semiconductor chips), nanotechnology memory,
photonic memory, biological-based memory, or other data storage
media, including but not limited to any one or more of the
following: magnetic, optical, elecro optical, semiconductor, and
supper conducting. Indeed, computer implemented instructions, data
structures, screen displays, and other data under aspects of
location based services quality assessment may be distributed over
the Internet or over other networks (including wireless networks),
on a propagated signal on a propagation medium (e.g., an
electromagnetic wave(s), a sound wave, etc.) over a period of time,
or may be provided on any analog or digital network (packet
switched, circuit switched, or other scheme). Those skilled in the
relevant art will recognize that portions of location based
services quality assessment can reside on a server computer, while
corresponding portions reside on a client computer such as a mobile
or portable device, and thus, while certain hardware platforms are
described herein, aspects of location based services quality
assessment are equally applicable to nodes on a network.
[0024] Referring to FIG. 1, a system 100 on which the quality of
service assessment system may be implemented is shown. The system
of FIG. 1 is an example of a GPRS (general packet radio service)
system based on GSM (global system for mobile communication).
However, location based services quality assessment may be
implemented on other types of systems and technologies including a
variety of cellular systems, UMTS, HPDSA, . . . , as well as other
broadcast, multicast and point to point communication systems that
are any combination of simplex, half duplex, full duplex including
terrestrial broadcast, cable, satellite broadcast, satellite cell,
and any one or any combination of PAN, LAN, WAN, MAN, or/and other
network(s) including Bluetooth, and 802.11 family. The system 100
includes a mobile device 102 (e.g., mobile phone, PDA, wireless
laptop, etc.) in communication with a base station 104. A base
station controller 106 in communication with a serving GPRS support
node (SGSN) 108 and a gateway GPRS support node (GGSN) 110 together
support packet switched transactions, which are handled separately
from circuit switched traffic that is supported by a mobile
switching center (MSC) 111. The MSC 111 also serves as an access
point for the Public Switched Telephone Network.
[0025] The SGSN 108, GGSN 110, and MSG 111 interact with a home
location register 112 (HLR). In some embodiments, the HLR 112 is
the primary database of permanent subscriber customer information
for the service provider's mobile network. In the context of
activated devices, the HLR 112 may contain pertinent user
information, including address information, account status, and
preferences. In some embodiments, a visiting location register
(VLR) 114 manages requests from out-of-area subscribers who are out
of the area covered by their home system.
[0026] In the illustrated embodiment, the system 100 includes
components associated with quality of service assessment including
a mobile service provider administration system 120. The mobile
service provider administration system 120 may include a quality
monitoring system 122, a call quality database 124, a customer care
system 126, and a billing system 128, described in more detail with
respect to FIG. 3. Customers may interact with the mobile service
provider administration system 120 and its various components via
the mobile device 102 and a wireless services network 116, as well
as through other means, such as the internet 118.
[0027] FIG. 2 shows a block diagram of a typical mobile
communication device 200, such as a mobile handset. While a mobile
phone is shown as the mobile communication device in FIG. 1, those
skilled in the relevant art will appreciate that location based
services quality assessment can be practiced with other devices and
configurations, including Internet appliances, hand-held devices,
wearable computers, multi-processor systems, microprocessor-based
or programmable consumer electronics, set-top boxes, PDAs, portable
laptop computers, and the like. The term "mobile device" is
intended to include all such devices.
[0028] The mobile device 200 has one or more internal or external
antennas 202 for receiving and transmitting electromagnetic signals
such as radio frequency signals. A transceiver 204 is connected to
the antenna(s) 202 and typically provides modulation and
demodulation of the transmitted and received signals, respectively.
A processor unit 206 connected to the transceiver 204 may comprise
a signal processor, microprocessor, ASIC, or other control and
processing logic circuitry. The processor unit 206 may perform
signal coding, data processing, input output processing, power
control, and other functions necessary for implementing a mobile
communication device. A customer may provide input to the processor
unit 206 via a keypad 208, microphone 210, or display touchpad 212.
In turn, the processor unit 206 may provide information to the
customer via the display touchpad 212 or a speaker 214.
[0029] The processor unit 206 may access information from, and
store information in, a nonremovable memory 216 or a removable
memory 218. The non-removable memory 216 may consist of RAM, ROM,
EPROM, a hard disk, or other memory storage technologies. The
removable memory 218 may consist of Subscriber Identity Module
(SIM) cards, which are well known in GSM communication systems, or
other well-known memory storage technologies, such as "smart
cards." Various applications, including text message applications
220 and quality reporting applications 220, could be implemented in
either the removable memory 218 or the non-removable memory 216.
For example, the applications may include a user interface
application (e.g., a Java applet) that allows a user to rate a call
or other transaction (e.g., by pushing a particular button). The
applications may allow rating in real time (e.g., while a call is
in progress) or, alternatively, after call has been completed. In
some embodiments, an application used to rate a call or other
communication may be located on a device that is separate from the
device used to place the call itself (e.g., a smart device with GPS
compatibility so that the precise location of the user can be
identified if the phone itself does not have such
capabilities).
[0030] In some embodiments, a device location component 224 allows
the location of the device to be known to the wireless service
provider, so that the wireless service provider can use this
information (or pass it along) for the purpose of assessing call
quality factors.
[0031] Referring to FIG. 3, a more detailed view of the mobile
service provider administration system 120 of FIG. 1 is shown. The
mobile service provider administration system 120 may include a
call quality monitoring system 122 (also shown in FIG. 1). Various
components of the quality monitoring system 122 may include a
quality reporting customer interface 302 and a quality reporting
processing and analysis component 304. Both of these components may
communicate with a call quality database 124 (also shown in FIG.
1). For example, in some embodiments the quality reporting customer
interface 302 may provide a way for customers to interact with the
quality monitoring system 122 and provide information that can then
be processed by the quality reporting processing and analysis
component 304.
[0032] Both the data collected from the customer using the quality
reporting customer interface 302 and the data resulting from
processing by the quality reporting processing and analysis
component 304 may be stored in the call quality database 124. In
some embodiments the quality reporting customer interface 302 may
be limited to a physical device interface (as opposed to possessing
application-based user interface characteristics). For example, if
the user's mobile device includes a comprehensive interface for
providing quality reporting information, the quality reporting
customer interface 302 may simply be an interface to retrieve
information from the device. In contrast, if the user's mobile
device does not include a comprehensive reporting interface, the
quality reporting customer interface 302 may include various
features that allow users to answer questions and/or provide
statements about the quality of their calls. For example, in some
embodiments, quality reporting customer interface 302 may include
automated telephone questioning systems, a web server and related
applications for providing online questionnaires, etc.
[0033] In some embodiments, the quality monitoring system 122 may
also include other quality control components 306. For example
these other quality control components may use objective standards
for providing quality control information. Examples include PESQ
systems, systems employing test mobile devices, etc. In some
embodiments, the information collected by the other quality control
components 306 may be compared with information obtained and
retrieved by the quality reporting customer interface 302 and
processed by the quality reporting processing and analysis
component 304.
[0034] As described with respect to FIG. 1, the mobile service
provider administration system 120 may also include a customer care
system 126 and a billing system 128, which may optionally interact
with the quality monitoring system 122 to address customer concerns
such as dropped calls, credits to service plans, etc.
[0035] Unless described otherwise below, aspects of location based
services quality assessment may be practiced with conventional
systems. Thus, the construction and operation of the various blocks
shown in FIGS. 1-3 may be of conventional design, and need not be
described in further detail herein to make and use location based
services quality assessment, because such blocks will be understood
by those skilled in the relevant art. One skilled in the relevant
art can readily make any modifications necessary to the blocks in
FIGS. 1-3 (or other embodiments or Figures) based on the detailed
description provided herein.
II. SYSTEM FLOWS
[0036] Referring to FIG. 4A, a high level call quality assessment
routine 400 may provide a means for customers to report on call
quality in accordance with some embodiments. At block 401 a call is
initiated, which may include initiation at the network level and/or
at the device level. At block 402 the call is in progress. At block
403 the call is terminated. For example, the user may have hung up
the device to end the call, or the call may have been dropped due
to inadequate service.
[0037] At block 404 a quality reporting option is initialized upon
termination of the call. For example, the user may receive a
message requesting input on the quality of the call or may be given
the option to make a call to an automated quality monitoring
reporting system. Many options for reporting are available without
departing from the scope of location based services quality
assessment. At block 405 the quality monitoring system collects
call information. At block 406 the quality monitoring system
performs exploratory and/or automated data analysis. As a result of
this data analysis, the quality monitoring system may identify
issues not detected by objective assessment of individual network
modes (block 406a); quantify actual customer experience across the
entire communication network (block 406b); assess the impact of
changes made to the communication network (block 406c); identify,
localize, and resolve call quality degradation issues (block 406d),
etc.
[0038] FIG. 4B shows a high level call quality assessment routine
420, similar to the routine 400 of FIG. 4A, but where the call
quality information is collected from the user while the call is
still in progress. For example at block 421 a call is initiated,
which may include initiation at the network level and/or at the
device level. At block 422 the call is in progress. At block 423,
the quality monitoring system collects call information during the
call. For example, the user may be able to push a button on the
phone each time he or she feels that call quality during the call
is below an acceptable level. At block 424 the call is terminated.
For example, the user may have hung up the device to end the call,
or the call may have been dropped due to inadequate service. At
block 425 the quality monitoring system performs exploratory and/or
automated data analysis based on the information collected at block
423. Some of this analysis may also occur while the call is in
progress. As a result of this data analysis, the quality monitoring
system may identify issues not detected by objective assessment of
individual network modes, quantify actual customer experience
across the entire communication network, assess the impact of
changes made to the communication network, identify, localize, and
resolve call quality degradation issues, etc.
[0039] FIG. 5 is a flow diagram showing an example of a routine 500
for collecting quality information is collected by an automatic
callback feature. At block 501 the call is terminated. At block 502
the quality monitoring system sends a message to the device to
perform auto callback upon completion of a call. Alternatively, the
device may be programmed to automatically place such a call,
depending on the conditions present at the device (e-g.,
termination of a call by the user, termination of a call due to
loss of service or other network problem, etc.
[0040] At block 503 the quality monitoring system receives a
callback from the mobile device. When this occurs, mobile device
may ring as the call is being placed automatically by the mobile
device. In some cases the user may have the option of hanging up or
discontinuing the feedback call if the user does not want to
provide input at this time. At block 504 the quality monitoring
system receives user input via the automatic call. The routine 500
ends after block 504.
[0041] FIG. 6 is a flow diagram showing an example of a callback
option routine 600 for providing an opportunity for the user to
optionally provide feedback on a call that was recently made. In
this routine 600, the user has the option of whether or not to
continue with a reporting call before the call is made. At block
601 a call is terminated. At block 602 the quality monitoring
system sends a message to the device to prompt the user for a
callback. Alternatively, the device may be programmed to display
such a message without a prompt from the quality monitoring system.
For example the user may receive a message displayed on his or her
device screen requesting whether he or she would like to call back
and report on call quality at the current time.
[0042] At decision block 603, if the user selects to go forward
with the callback option, the routine 600 continues at block 604.
Otherwise the routine ends without reporting. At block 604 the
routine receives an auto callback call from the device in response
to the user's selection to provide feedback. At block 605 the
quality monitoring system receives user input. After block 605 the
routine ends.
[0043] FIG. 7 is a flow diagram showing an example of a routine 700
for a post-call questionnaire, where the questionnaire is executed
by a program on the mobile device itself. At block 701 the call is
terminated. At block 702 the routine 700 begins executing at the
mobile device. At block 703 the routine 700 poses a next question
to the user. At decision block 704 if the routine 700 has posed a
last question to the user, the routine 700 continues at block 705.
Otherwise, the routine 700 loops back to block 703 to pose the next
question to the user. At block 705, assuming all questions have
been posed to the user, the routine 700 sends a response to the
call quality monitoring system. For example, the routine 700 may
provide data comprising a completed series of questions or
statements by the user on call quality.
[0044] FIG. 8 is a flow diagram showing an example of a routine 800
for presenting a post call questionnaire to the user of a mobile
device, where the routine 800 is executed from the quality
monitoring system. At block 801 the call is terminated. At block
802 the routine 800 contacts the device to initiate connection with
the questionnaire application. At block 803 the routine 800
provides a next question to the device. At block 804 the routine
800 receives a response from the device (e.g., provided by the user
as a response to a question in the questionnaire). At decision
block 805, if the last question has been posed, the routine 800
ends. Otherwise, the routine 800 moves back to block 803 to provide
the next question to the device.
III. USER INTERFACE FOR COLLECTIONS CALL QUALITY INFORMATION
[0045] FIG. 9 is a display diagram showing various examples of
providing a user of a mobile device with an option to answer
questions relating to call quality. With respect to display 902, a
call has been dropped due to a service problem, and the user is
asked:
[0046] "Would you like to report on call quality now?" In this
example, the user may select either "yes" or "no" using appropriate
device keys.
[0047] In another example shown in display 904, a message provides:
"Rate the quality of your last call now and receive free airtime
minutes." The user may then respond "yes" or "no" as to whether he
or she wishes to rate the call at the present time.
[0048] In a third example shown in display 906, a message provides
three rating options: "Select 1 to report on call quality now";
"Select 2 to report on call quality later"; and "Select 3 to opt
out of reporting call quality in relation to the current call." If
the user selects the first option, the device may present a text
questionnaire to the user or make a call to an automated quality
reporting system, as described in more detail in the preceding flow
diagrams and with respect to FIG. 10. Alternatively if the user
selects to report on call quality later, the user may be given the
option to provide feedback at a later time via any one of a number
of means (e.g., Internet questionnaire, automated, call, text
message on device screen, etc.).
[0049] FIG. 10 is a display diagram showing an example of a user
interface for reporting call quality. Display 1002 shows a text
mode where the user is asked to report on call quality after a
dropped call using text messaging or similar means. For example the
user may be requested to answer the question: "Did the signal
strength vary during your call?" To this the user may respond
either "yes" or "no" using appropriate buttons on his or her
device.
[0050] Display 1004 illustrates reporting on call quality using an
audio mode (e-g., automated telephone system). For example, the
user may be asked to "describe the quality of your last call." The
user's response may then be recorded and or automatically
processed. In another example, the user may be asked: "What was the
signal strength just prior to the time that the call was dropped?
For poor select or say 1, or moderate . . . " Another question that
the user may be asked is "Have you had problems with dropped calls
in this area before?"
[0051] While specific examples are given here for the purpose of
illustration, regardless of the mode used (e.g., text mode 1002, in
an audio mode 1004, or in another mode not illustrated here), any
number of questions may be asked to the user for quality reporting.
While by no means all inclusive, the table below shows examples of
other types of questions that may be posed to users in determining
call quality:
TABLE-US-00001 Sound Quality On a scale of 1-5, with 5 being the
best quality sound, how would you rate the quality of the sound of
your call? Were you able to hear the person on the other end of the
line clearly during the call? Never? Rarely? Most of the time?
Always? Was the person on the other end of the line able to hear
you during the call? Never? Rarely? Most of the time? Always? Do
you have repeated problems with sound quality when placing calls in
this area? Noise On a scale of 0-5 with 0 being no detectable
background noise, how would your rate the amount of background
noise of your call? How often were you able to hear unwanted
background noise during your call? Never? Rarely? Most of the time?
Always? What was the severity of background noise during your call?
Greatly affected ability to hear content of call. Somewhat affected
ability to hear content of call. Did not affect the ability to hear
content of call but was annoying. Was audible but not distracting.
No detectable background noise present Do you have repeated
problems with sound quality when placing calls in this area?
Dropped Call Did the signal strength vary during your call? What
was the signal strength at the time the call was placed? What was
the signal strength just prior to the time the call was dropped?
Have you had problems with dropped calls in this area before? How
many calls do you make in the area that the call was dropped? How
many times did you attempt to reconnect the same call? Subjective
Reponses Describe the quality of your last call. Describe your
overall satisfaction with the quality of your last call. Describe
how the quality of your last call could have been improved.
Describe your overall experience with your wireless communication
service provider. Describe your overall satisfaction with your
wireless communication service provider. Other Did you have dial
tone problems when making this call? Press 1 for yes. Press 2 for
no. Rate the call set up time for this call. Please indicate if
there were issues disconnecting from this call. Please indicate if
there were problems handing off between cells during this call.
[0052] In addition to the above, other factors and techniques may
be used in rating calls. For example, various different types of
rating scales may be implemented. In another example, users may be
provided with descriptive icons or graphics to select from to make
call rating fast and easy.
IV. LOCATION ACCURACY ASSESSMENT
[0053] In another aspect of location based services quality
assessment, location service quality may be assessed from a user
perspective. In this aspect, the user can provide feedback on the
perceived quality of location services provided by a wireless
telephone network that are received on a user wireless device. It
is possible that the present location of a user, as provided by her
mobile device, is incorrect. And thus, location services based on
the present location will be in error. Errors may occur in a
location based service in multiple ways. For example, if the
location based service provides to a user a target location of a
nearby restaurant within easy commuting distance of the user's
mobile device but the information is in error, there can be
multiple causes for the error. A mobile device can include a
traditional mobile and cell phones and other devices including
smart phones, multifunction devices, dedicated devices (e.g., music
and video players) wireless PDAs and computers (including handheld,
palm held, laptop, desktop, wearable, etc.) and other electronic
content storage and delivery devices and media.
[0054] One such cause for error in a location service that provides
a target location with respect to a present location of the user
wireless device is that the data representing the present location
of the target, in the above case, a restaurant, can be in error.
That is, the database containing the location of the target
restaurant could be in error in that the restaurant is correlated
with incorrect location data as read by the database having the
restaurant location data stored thereon. A second source of error
could be that the cellular-based location determination equipment
could have inaccurate location data resulting in an incorrect
present location of the user equipment. An example of this type of
error could be that the cell-tower location information, stored in
a database, is in error. If so, present location determination of a
cell phone or other mobile device would, become inaccurate if the
location determination equipment used the erred cell tower location
as a basis for triangulation location of the cellular phone or
other mobile device. A third source of error for cellular-based
location services is in the actual determination of the resent
location using the information provided. For example, if the
location based-services use time differential of arrival (TDOA),
angle of arrival (AOA), or triangulation principles to determine a
user mobile unit location, then the calculation itself may be in
error, which provides false present location determination
information. Another example of error is in the use of assisted
global positioning system (AGPS) information. If AGPS is used, the
GPS spacecraft ephemeris is provided to a GPS receiver/processor in
the mobile device which receives the spacecraft identifiers. The
AGPS cell system provides ephemeris and other reference information
about the GPS satellites and ground position directly to the mobile
device via the cellular network so that the GPS receiver processor
can more quickly calculate position information at the mobile
device instead of waiting many minutes for ephemeris information to
be down-linked from the GPS spacecraft. Thus, if the ephemeris,
reference altitude, or reference position information supplied by
the cellular system is in error, so will be the present location
information derived from that information.
[0055] The user of the mobile device that receives location-based
services, such as the location of nearby target points of interest
(restaurants, hotels, gas stations, etc.) generally knows if the
target location given to the mobile device is in error. This
becomes evident if the user takes the mobile device to the
suggested target location, and the target item of interest is not
at the expected target location. Thus, the user of the mobile
device is in a unique position to first determine if the
cellular-system-provided target location information is incorrect
and, if so, provide feedback on the quality of the received target
location information.
[0056] In one aspect of location based services quality assessment,
the mobile device user can provide user-feedback to validate the
accuracy of target location-based services in a cellular network.
After utilizing a location based service, such as a search for a
nearby target point of interest using a mobile device, the user is
provided user-equipment based means to comment upon the accuracy of
the location-based service. Various implementations are envisioned.
In one embodiment, the user can provide feedback via the mobile
device onto which the location-based service was provided. In
another embodiment, the user can use separate means, such as the
internet, to provide user feedback, or a combination of both means.
Other embodiments include the user answering questions via a
network interface, sending and/or reply to an SMS message
indicating the quality of the location-based service, the user can
make a packet switched and/or circuit switched call to an automated
attendant to provide information pertaining to the quality of the
location based service target location data or a combination
thereof. The user's response can be stored in a database or the
like, for further analysis and comparison with other users'
feedback.
[0057] In one aspect of location based services quality assessment,
the location service quality assessment does not have to compare an
absolute location accuracy with the location information provided
by the mobile device. In general, correlating the mobile device
user's observations of acceptable target location accuracy is
sufficient to determine if the location-based services offered by
the telephone network provider has become de-valued because of
target location accuracy problems. For example, a measured
inaccuracy of 30 feet may be of little consequence to a user and a
user may not wish to report such an error. Indeed, a user may not
even consider this level of target location accuracy to be a
problem. But a target location accuracy error equivalent to a city
block length or greater is most likely of great significance to a
user. The provision of present location (spatial) and time
(temporal) information further increases the value of the collected
target location accuracy information. For example, in the case of
wireless services, knowing where and when poor or good location
service was received can help correlate the user reported data with
other infrastructure data such as outage or geographic coverage
issues and reports from other users. Thus, the use of accurate time
(day/hour/minute) and a location service quality assessment can
enhance the subjective location accuracy assessment value.
[0058] In application, a subjective location service quality
assessment system may be applied to a variety of system
applications. Example systems include conventional wired systems
having a LAN or WAN connection, such as a voice over IP system, a
terrestrial telephone system, a point to point system, and a
theater system. Wireless applications for the location service
quality assessment discussed herein may include terrestrial
cellular, general RF communications systems, and satellite
systems.
[0059] One such embodiment is in a wireless telephone network
having wireless user equipment which can be a mobile telephone, a
PDA, a laptop computer, or other wireless end-user equipment having
location-based services capability. FIG. 11 depicts an example
system 1100 useful to subjectively assess location accuracy. In one
typical scenario, a user operating the user equipment 1106,
requests a location-based service via the network 1104. The
location-based service is typically a service offered via the
network to subscribers. Location-based services include queries for
the location of items of interest nearby the present location of
the user equipment. For example, such requests can include queries
for the locations of banks, schools, churches, shops, businesses,
gas stations, town centers, bazaars, flea markets, recreational
areas, scenic areas, wireless centers (hot spots), and the like. A
typical wireless location-based service may accept a query from a
wireless device user for a target location. The request is received
by the wireless network and routed to a location-based services
server(s) 1102. The query is answered by determining the location
of the wireless device producing the request. This is performed
using location determination equipment as part of the location
based services server(s) 1102 or may be determined using present
location determination equipment in the user's wireless device.
Whatever means is used, the present location information is
provided to the location-based service and a location of the target
item of the request is sent to the user wireless equipment 1106 for
display 1106A.
[0060] In other embodiments, the network 1104 may be either a
wireless network, such as a cellular telephone network or other
type of digital services network supporting such user equipment
such as an e-mail devices, PDAs, and laptop computers. The network
1104 may also be a wired network such as a point to point system, a
LAN or a WAN. At the user equipment 1106, the location based
service information is received and displayed 1106A to the user. An
example service would be to locate nearby gas stations for a
traveler requesting such information. In this instance, the
estimated present location of the mobile device is determined by
the network and its present location estimation equipment 1102.
Based on the estimate of the user equipment by the location based
service equipment, the locations of nearby gas stations can be
provided to the user equipment and displayed 1106A.
[0061] In this scenario, if the target location of the nearby gas
stations is correctly provided to the user equipment, then the user
would simply go to the nearest fueling station, and re-fuel the
vehicle to continue her travels. However, if the location
information of the nearby gas stations is incorrect or inaccurate,
the user has an option to report the inaccurate location
information provided to her. As indicated above, correlation of a
location inaccuracy with other information is useful to aid systems
engineers and administrators in locating the cause for the errant
target location information.
[0062] The system 1100 permits a user to provide subjective
feedback concerning the location information displayed 1106A on the
user equipment 1106. The subjective user input 1106B, also called
subjective user feedback, may be provided in a number of different
ways. In one embodiment, a manual evaluation mode of subjective
feedback may be used. In this mode, a user would construct a
message that is sent to the user data feedback data store 1116. In
the fully manual mode, a user may input the time, and conditions of
the observed location inaccuracy as text entered through the keypad
of the user equipment 1106. Another embodiment in manual mode is to
record a voice message having the same information as the text
message. The voice message would be sent to the user feedback store
1116 for evaluation.
[0063] In another embodiment, the user equipment 1106 may solicit
user subjective information using an automated mode via the
evaluation input function 1106B. In the automatic mode, the user
may select a menu option while evaluating the location information.
In this mode the local time may be automatically entered using
automated user equipment 1106 resources such as an internal clock
for time-tagging the feedback information.
[0064] In one embodiment, where the user equipment 1106 is a
wireless mobile device 200 as shown in FIG. 2, the quality
reporting application 222 provides the interface for the manual and
automatic modes of the subjective user input to the user equipment
1106. In general, the process of collecting and analyzing
subjective user feedback comports with the method of FIG. 4A. In
the specific example of the acquisition of user feedback during the
time that location information is being assessed by the user, the
flow of FIG. 4B may be employed. In the specific instance when an
auto callback is used to collect the user subjective information,
the flow of FIG. 5 or 6 may be used. Post call questions may also
be employed to collect user information and the basic flow for that
user interface may be conducted as per the flows of FIGS. 7 and
8.
[0065] Returning to FIG. 11, in one embodiment, time information
may be available as well as some local or present location
information. Present location information may take the form of a
GPS receiver processor or an AGPS location mechanism. In accordance
with the types of errors listed above, even precise location
information, when combined with faulty database information, can
result in a poor quality location based service such as the
location of a desired store, shop, park, and the like. Thus, as an
aspect of location based services quality assessment, the user
equipment 1106 may also contain a time and/or location data source
1106C as a resource for accurate time and location information. If
implemented within the user equipment, the time function 1106C can
provide a time tag for subjective information input by the user.
For example, in one scenario, where a user is evaluating search
results for a restaurant, the user may, in real time, press menu
keys on the user equipment to rate the quality of the location
accuracy. The user may input a 5 for excellent quality and a 1 for
very poor quality. As the user actuates a soft key or other input
device, a time function 1106C preferably located within the user
equipment 1106 time tags the information so that the subjective
rating information may be correlated temporally. As part of the
quality evaluation process, the user may make multiple requests for
a location based service. After or during each request, a quality
assessment of the location service accuracy may be made. In this
manner, the wireless network support single or multiple queries for
a location-based service and the same wireless network supports the
provision of feedback on the service from a user wireless
device.
[0066] Considering the location function 1106C, it may reside
within the user equipment or outside of the user equipment.
Location determination may be performed in a variety of ways. In
one embodiment, if the location function resides within the user
device, a global positioning system (GPS) receiver/processor may be
built into the user device such that location information is
provided and tagged to one or more user subjective rating inputs.
Differential GPS (DGPS) and Assisted GPS (AGPS) schemes may be
implemented. Other satellite-based systems may also provide
location information as well. In another embodiment within the user
equipment, the location function may be such that the user is
prompted to enter present location information manually onto the
user equipment. The user-entered present location information, such
as a street name, for example, may then be sent out along with the
subjective user rating to assist in the location of the user
device.
[0067] The location function 1106C may also be located outside of
the user device and be part of the wireless network. An example of
a location function attached to the user network is a time
difference of arrival (TDOA) calculation scheme. Another
implementation is an angle of arrival (AOA) scheme. Those of skill
in the art will recognize that terrestrial-based locations
mechanisms such as TDOA and AOA location schemes can provide the
location of the user equipment using measurements from multiple
base stations. This location information can be used to augment the
subjective rating input information from the user concerning the
quality of location based services received on the user
equipment.
[0068] Subjective user feedback may be sent back through the
network 1104 to be received and stored on a database 1116 on the
network 1104 which accumulates the user feedback data for storage
or buffering. The storage 1116 may be local or remote storage
component as part of a data acquisition server, or may be
implemented as an integral part of a base station. The user
feedback information is accessed from storage 1116 by a data
correlator 1114. The data correlator can associate the user
feedback information with other network performance information to
provide useful event and condition information of the network 1104.
In one embodiment, the data correlator is part of the quality
reporting, processing and analysis block 304 of the quality
monitoring system 122 shown in FIG. 3.
[0069] In on aspect of location based services quality assessment,
network performance data is acquired by observing network events
and conditions. In one example, a network monitor 1108 collects
network performance in a wireless network 1104. Network events such
as call setups, teardowns, dropped packets and calls, switching
failures, and outages are collected using the network performance
monitor 1108. Also collected are performance parameters and
conditions such as bandwidth utilization, packet loss, call volume
and call traffic handling, data link routing and switching element
performance and switching configurations for base station and cell
tower configurations as well as capacity information. Such network
events and conditions represent the performance, capacity, and
configuration of the network 1104. The network performance and
capacity data is recorded using a storage device 1110.
[0070] The data correlator can input the user feedback information
1116 with the network performance data from database 1110. By
performing a correlation between the subjective user feedback and
the performance of the network, an association can be made whereby
a cause and effect may be identified. For example, if a network
switch lost power or otherwise failed during transmission of
location information to a user, then the user may experience a loss
of signal or corruption of data at that time. By correlating the
failure event to the subjective experience data provided by the
user, a cause (the switch power failure) and the effect (flawed
data) can be identified. In another instance, a change in the
network performance, such as when a marked increase in call volume
occurred for a base station at the same time as a data request, and
a mis-read of location based data. Such a correlation can provide
insight to a system administrator that a network traffic routing
adjustment may be needed for a group of base stations at a specific
time of day.
[0071] Correlated information can be useful to determine the cause
and effect of problems that cover a wide geographic area during by
identifying a common element such as latency on a server or
switching or routing element, problems that correlate to a specific
area (e.g., a single cell site sector) indicating a coverage,
capacity or perhaps interference issue, problems that correlate to
specific types of video content, or problems that correlate to
specific types of handsets or combinations of handsets. Thus, the
value of correlating network performance data with subjective video
quality data becomes apparent.
[0072] In addition to the network performance data 1108 and the
user feedback 1116, the data correlator 1114 can access other
information that is useful for an assessment of location
information displayed at the user device. In one embodiment,
location based service information 1102 server performance and
capacity data is recorded on a storage device 1112 which stores
performance information concerning the location based services
server. In one scenario, if the server has a fault, such as an
inability to retain proper response time due to network faults or
high traffic demands, then the performance of the location based
services 1102 server could be adversely affected which would result
in poor or disrupted service at the user device 1106. Under such
conditions, knowledge of the location based services 1102 server
conditions would be useful. The data correlator 1114 can also
access storage device 1112 to obtain the location based services
server performance and capacity information for correlation with
the subjective information available in storage device 1116.
[0073] Another source of information that is useful to augment the
user feedback data and the network performance data is the road
traffic and weather data available via database 1118. Here, as can
be appreciated by one of skill in the art, weather aberrations can
cause perturbations in performance of user equipment 1106. Thus,
data correlator 1114 can access the weather and road traffic data
and associate the local weather conditions between a relevant base
station and the user equipment to determine if the atmospheric
conditions are affecting reception at the user equipment. Road
traffic information may like wise be useful because road congestion
may be an external source of wireless system usage. For example,
given a congested section of roadway, cell phone usage may peak and
aggregate to one or two cell towers or base stations. To
accommodate the above-normal call volume, bandwidth available for
each call being routed through those base stations, and thus a
location-based service performance may be compromised. Knowledge of
such road traffic conditions can thus support an analysis of
degraded location service.
[0074] Another source of information useful to correlate with user
feedback data is call data record information available at database
1120. Call data record information contains information as to what
communications links were established, broken down, or dropped by
the user equipment 1106. When accessed, call data record
information 1120 can be associated by the data correlator 1114 in
order to provide an event reference for the location service
quality rating information with respect to the specific user
equipment.
[0075] The data correlator 1114 can access information from the
user feedback database 1116 and correlate the subjective
information with information from a variety of databases; the
network performance database 1110, the server performance database
1112, the weather and road traffic database 1118, and the call data
record database 1120 along with time and location information. All
or any of this databases may be accessed by the data correlator
1114. The data correlator can then associate the subjective
location service quality rating from the user with time and
location information, network and location server performance
information, call records, and environmental condition data and
identify relevant perturbations which can have a cause and effect
upon the received location service quality presented by the user
equipment to the user. These results can then be reported to a
system administrator via report or in graphical display form 1122.
The system administrator can then take the results and determine if
the location service quality data can be improved by system
hardware or software adjustments, or take some other action.
Reports concerning the location-based service accuracy and quality
assessment can be generated at 1102 to assist the administrator or
system engineer in both troubleshooting and fixing unacceptable
location-based service errors reported by wireless device
users.
IV. CONCLUSION
[0076] Unless the context clearly requires otherwise, throughout
the description and the claims, the words "comprise," "comprising,"
and the like are to be construed in an inclusive sense as opposed
to an exclusive or exhaustive sense; that is to say, in the sense
of "including, but not limited to." Additionally, the words
"herein," "above," "below" and words of similar import, when used
in this application, shall refer to this application as a whole and
not to any particular portions of this application. When the claims
use the word "or" in reference to a list of two or more items, that
word covers all of the following interpretations of the word: any
of the items in the list, all of the items in the list, and any
combination of the items in the list.
[0077] The above detailed description of embodiments of location
based services quality assessment is not intended to be exhaustive
or to limit location based services quality assessment to the
precise form disclosed above. While specific embodiments of, and
examples for, location based services quality assessment are
described above for illustrative purposes, various equivalent
modifications are possible within the scope of location based
services quality assessment, as those skilled in the relevant art
will recognize. For example, while processes or blocks are
presented in a given order, alternative embodiments may perform
routines having steps, or employ systems having blocks, in a
different order, and some processes or blocks may be deleted,
moved, added, subdivided, combined, and/or modified. Each of these
processes or blocks may be implemented in a variety of different
ways. Also, while processes or blocks are at times shown as being
performed in series, these processes or blocks may instead be
performed in parallel, or may be performed at different times.
Where the context permits, words in the above Detailed Description
using the singular or plural number may also include the plural or
singular number, respectively.
[0078] The teachings of location based services quality assessment
provided herein can be applied to other systems, not necessarily
the system described herein. The elements and acts of the various
embodiments described above can be combined to provide further
embodiments. Aspects of location based services quality assessment
can be modified, if necessary, to employ the systems, functions,
and concepts of the various related technologies to provide yet
further embodiments of location based services quality
assessment.
[0079] These and other changes can be made to location based
services quality assessment in light of the above Detailed
Description. While the above description details certain
embodiments of location based services quality assessment and
describes the best mode contemplated, no matter how detailed the
above appears in text, location based services quality assessment
can be practiced in many ways. As noted above, particular
terminology used when describing certain features or aspects of
location based services quality assessment should not be taken to
imply that the terminology is being redefined herein to be
restricted to any specific characteristics, features, or aspects of
location based services quality assessment with which that
terminology is associated. In general, the terms used in the
following claims should not be construed to limit location based
services quality assessment to the specific embodiments disclosed
in the specification, unless the above Detailed Description section
explicitly defines such terms. Accordingly, the actual scope of
location based services quality assessment encompasses not only the
disclosed embodiments, but also all equivalent ways of practicing
or implementing location based services quality assessment under
the claims.
[0080] While certain aspects of location based services quality
assessment are presented below in certain claim forms, the
inventors contemplate the various aspects of location based
services quality assessment in any number of claim forms. For
example, while only one aspect of location based services quality
assessment is recited as embodied in a computer-readable medium,
other aspects may likewise be embodied in a computer-readable
medium. Accordingly, the inventors reserve the right to add
additional claims after filing the application to pursue such
additional claim forms for other aspects of location based services
quality assessment.
* * * * *