U.S. patent application number 11/351085 was filed with the patent office on 2007-01-04 for queueing events in an interactive media environment.
This patent application is currently assigned to Microsoft Corporation. Invention is credited to Olivier Colle, James C. Finger, Arthur William James Freeman, Khurshed Mazhar, John Andre Yovin.
Application Number | 20070006233 11/351085 |
Document ID | / |
Family ID | 37591410 |
Filed Date | 2007-01-04 |
United States Patent
Application |
20070006233 |
Kind Code |
A1 |
Finger; James C. ; et
al. |
January 4, 2007 |
Queueing events in an interactive media environment
Abstract
An arrangement is provided where all applications in an
interactive media environment run on a single application thread in
a media player. Event queues are utilized to schedule the
application thread's processing of workitems corresponding to
events that occur in the environment. Workitems include methods to
be invoked when the workitem is processed and arguments for the
method. Workitems further include a begin time and an end time and
are ordered in the event queue first by begin time followed by the
order in which they were inserted into the queue. The application
thread marks workitems whose begin times corresponds to the current
or previous time and then processes marked workitems from the queue
in order. Such processing is committed so that once the application
thread begins processing of a workitem it does not stop. Workitems
are dropped from the event queue if their end times have been
passed.
Inventors: |
Finger; James C.; (Kirkland,
WA) ; Yovin; John Andre; (Woodinville, WA) ;
Mazhar; Khurshed; (Kirkland, WA) ; Colle;
Olivier; (Redmond, WA) ; Freeman; Arthur William
James; (Kirkland, WA) |
Correspondence
Address: |
MICROSOFT CORPORATION;ATTN: PATENT GROUP DOCKETING DEPARTMENT
ONE MICROSOFT WAY
REDMOND
WA
98052-6399
US
|
Assignee: |
Microsoft Corporation
Redmond
WA
|
Family ID: |
37591410 |
Appl. No.: |
11/351085 |
Filed: |
February 9, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60695944 |
Jul 1, 2005 |
|
|
|
Current U.S.
Class: |
718/100 ;
715/210 |
Current CPC
Class: |
G06F 9/542 20130101;
G06F 9/4843 20130101 |
Class at
Publication: |
718/100 ;
715/500.1 |
International
Class: |
G06F 9/46 20060101
G06F009/46; G06F 17/00 20060101 G06F017/00 |
Claims
1. A method to manage processing of workitems corresponding to
events that occur in an interactive media environment where time is
counted using a sequence of ticks, the method comprising the steps
of: marking workitems disposed in an event queue whereby workitems
having a begin time that is at a current tick or before the current
tick are marked; processing marked workitems that have an end time
that is after the current tick; and dropping from the event queue
marked workitems having an end time that is before the current
tick.
2. The method of claim 1 in which workitems represent one or more
events fired by an interactive media application.
3. The method of claim 2 in which the one or more events are fired
by a markup in the interactive media application
4. The method of claim 2 in which the one or more events are fired
by a script in the interactive media application.
5. The method of claim 1 in which workitems represent one or more
events generated by an interactive media player.
6. The method of claim 1 in which workitems represent one or more
events fired by user interaction in the interactive media
environment.
7. An application scheduler arranged to manage processing of
workitems corresponding to events occurring in an interactive media
environment where time is counted using a sequence of ticks, each
of the workitems having a timestamp that includes a begin time and
an end time, the application scheduler comprising: a event queue
for queuing the workitems, the event queue having a head end and a
tail end and arranged so that workitems are processed from the head
end of the event queue; and an application thread arranged for
marking workitems whose begin time corresponds to a current tick or
a previous tick, processing marked workitems, and inserting
workitems associated with new events that occur during the
processing at the tail end of the event queue so that they follow
after marked workitems.
8. The application scheduler of claim 7 wherein the inserting is
performed irregardless of a timestamp for the new events.
9. The application scheduler of claim 7 where each workitem
includes a field for indicating a method to be invoked and
arguments for the method.
10. The application scheduler of claim 7 where each workitem
includes a clock selector field that indicates whether the
timestamp is measured in title time, page time or application
time.
11. The application scheduler of claim 7 where the application
thread drops all workitems from the event queue when an application
running in the interactive media environment reloads its page.
12. The application scheduler of claim 7 where all applications in
the interactive media environment run on the same thread of a
playback engine.
13. A computer-readable medium, which when executed by one or more
processors disposed in an electronic device, performs a method to
manage processing of work items associated with repetitive events
and non-repetitive events that occur in an interactive media
environment, each workitem having a timestamp that includes at
least a begin time, the method comprising the steps of: ordering
workitems in an event queue by begin time followed by entry time
into the event queue; processing workitems from the event queue in
order and on a committed basis so that an event associated with a
processed workitem is handled before processing is terminated; and
dropping only workitems from the event queue that are associated
with repetitive events.
14. The computer-readable medium of claim 13 where the repetitive
event is a periodic event.
15. The computer-readable medium of claim 13 where the periodic
event is represented by a workitem having an endtime equal to a
next scheduled begin time.
16. The computer-readable medium of claim 13 where the ordering is
performed during insertion of workitems into the event queue.
17. The computer-readable medium of claim 13 where the ordering is
performed during extraction of workitems from the queue.
18. The computer-readable medium of claim 13 including the further
step of inserting a workitem into the event queue for calling a
markup engine in an interactive media application to evaluate
timing changes.
19. The computer-readable medium of claim 13 including the further
step of inserting a workitem into the event queue for calling a
markup engine in an interactive media application to reflow a
markup in an interactive media application and render the markup on
a display device.
20. The computer-readable medium of claim 19 in which time is
counted in the interactive media environment using a sequence of
ticks and the inserted workitem is processed last in each tick.
Description
STATEMENT OF RELATED APPLICATION
[0001] This application claims the benefit of provisional
application number 60/695,944, filed Jul. 1, 2005, which is
incorporated by reference herein.
TECHNICAL FIELD
[0002] The described arrangements, systems and methods relate
generally to interactive media and more particularly to queuing
events in an interactive media environment.
BACKGROUND
[0003] Interactive media environments are typically resource
constrained in terms of available processing power, memory and
other resources that are available to applications running in the
environment. One common example of interactive media is video
encoded on DVD (digital versatile disc) where users can interact
with graphical menus or other controls to navigate to specific
video content or invoke special features that are authored into the
DVD.
[0004] In more complex interactive media environments, despite the
limited resources, applications need to respond to users in
real-time manner that is frame-accurate with the video. However,
the use of real-time multithreaded programming to accomplish such a
goal would place a large burden on interactive media authors in
dealing with issues such as thread management and
synchronization.
SUMMARY
[0005] An arrangement is provided where all applications in an
interactive media environment run on a single application thread in
a media player. Event queues are utilized to schedule the
application thread's processing of workitems corresponding to
events that occur in the environment. Workitems include methods to
be invoked when the workitem is processed and arguments for the
method. Thus, the scheduling and processing of workitems from the
event queues determines what work get done and when in the
environment. Typical events include user events that are fired from
user interaction with the media player, system events fired by the
media player, and events that are fired by the applications.
[0006] In various illustrative examples, workitems further include
a begin time and an end time and are ordered in the event queue
first by begin time followed by the time in which they were
inserted into the queue. In one illustrative example, the
application thread marks workitems whose begin time corresponds to
the current or previous time and then processes marked workitems
from the queue in order. All processing is performed on a committed
basis so that once the application thread begins processing a
workitem it does not stop. Workitems are dropped from the event
queue if their end times have been passed. Workitems for new events
that occur during processing are inserted at the end of the queue
to be handled after the committed workitems. In another
illustrative example, workitems for new events are inserted into
the event queue based on their relative begin times to be handled
after the committed workitems. Repetitive events, like timer events
and application drawing events, may have individual occurrences
dropped from the event queue. However, one-shot (i.e., single
occurrence, non-repetitive) events are never dropped.
[0007] Advantageously, the event queuing model with single
application thread provide a stable and predictable methodology for
interactive media authors to manage events in a real-time
frame-accurate manner where hardware resources, including processor
cycles and memory, are limited.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] FIG. 1 is an illustrative block diagram showing the elements
making up an application used in an interactive media
environment;
[0009] FIG. 2 is an illustrative diagram which shows the
relationship among multiple markup documents and script;
[0010] FIG. 3 is a block diagram of an illustrative interactive
media player including an interactive content processor, a video
content processor and a mixer;
[0011] FIG. 4 is a block diagram of a second illustrative
interactive media player;
[0012] FIG. 5 is a block diagram of an illustrative arrangement
having a plurality of event queues and a single application
thread;
[0013] FIG. 6 is a block diagram of an illustrative event queue
showing the ordering of workitems first by BeginTime, and then by
the time in which workitems were inserted into the event queue;
[0014] FIG. 7 is a block diagram of an illustrative arrangement
where the application thread automatically inserts two workitems at
the end of the event queue; and
[0015] FIG. 8 is a flow chart of an illustrative method for queuing
workitems associated with events that occur in an interactive media
environment.
DETAILED DESCRIPTION
[0016] Referring to FIG. 1, an illustrative block diagram of the
elements making up an application 110 used in an interactive media
environment is shown. Applications are typically used in the
interactive media environment to enable interaction between a user
and an interactive media player rendering graphics and video on a
coupled display device (such as a television or monitor) through a
user interface such as a remote control. More specifically,
applications control presentation behavior of various content
objects, including video playback, in the environment. Presentation
of graphic objects such as menus and interactive buttons over the
video is also realized using applications. Applications further
manage and control audio playback and sounds in the environment. It
is contemplated that multiple applications will generally be
running simultaneously in most interactive media settings. However,
there is no requirement the multiple applications run
simultaneously and the decision to divide or aggregate applications
in a particular setting is a design choice of the interactive media
author. Applications may also be logically subdivided into
application pages depending on the requirements of a specific
setting.
[0017] The application 110 comprises a script host 115 containing
zero or more script files 117 and 119 and zero or more markup
documents 120 that is used to generate a document object model
(DOM). The markup documents 120 include information relating, for
example, to content, style, timing and layout of graphic objects.
Thus, the markup context is used generally to provide graphics on a
graphics plane in the interactive media environment.
[0018] In this illustrative example, the markup documents are XML
document files in accordance with W3C standards. As indicated in
FIG. 1, multiple physical XML files may be accessed using the
<include>element in the <head>section of the markup. In
some settings it may be preferable for an application to not have
more than one active markup at a time. However, an application may
switch its markup 120 by using a <link>element in the markup.
Alternatively, an application may switch its markup 120 by
utilizing an application programming interface (API) that enables
applications to gain access to functional objects within a current
application. Using a loadMarkup ( ) call through the API, an
application may switch markup files 120 by passing the Uniform
Resource Identifier (URI) of the new markup through an API.
[0019] In cases where an application accesses a new markup, the API
call takes effect only after a current event handler in the
application finishes executing its current task. Any current
markup-related event handlers that are pending are also cancelled
as the new markup, once loaded, will invalidate those event
handlers.
[0020] In this illustrative example, script host 115 contains
script files 117 and 119 which are used along with the markup 120
to implement interactive media experiences. Script files 117 and
119 may be implemented, for example, using ECMAScript as defined by
Ecma International in the ECMA-262 specification. Common scripting
programming languages falling under ECMA-262 include JavaScript and
JScript. In some settings, it may be desirable to implement scripts
117 and 119 using a subset of ECMAScript 262, in particular
ECMA-327, along with a host environment and a set of common APIs.
Script context in most settings is utilized to deal with
interactive control issues from user along with system events,
graphics control, video playback, resource management (e.g. use of
caching or persistent store resources) and other issues that are
not readily or efficiently implemented using solely markup 120.
[0021] The availability of APIs and resources to application 110 is
indicated by reference numeral 125 in FIG. 1. Resources include,
for example, audio and video files, fonts, pictures and images
(e.g., in common file formats including PNG, JPEG, GIF, BMP, TIFF
etc.) and other resources as may be required by an application
according to the circumstances of a specific setting.
[0022] Each application 110 maintains its own script host 115 that
maintains the context for the script's variables, functions and
other states. In most settings, variables and functions in one
application are not visible to another application unless the
applications are specifically set up to enable such
cross-application visibility, for example, by using an object that
is shared across all applications. For example, in this
illustrative example, the interactive media player object has a
single instance that is shared across all applications. Optionally,
therefore, special objects may be placed inside script host
115--for example, using a C++ object--to implement singletons
(i.e., a objects having limited instantiation) where the special
objects all reference the same internal function, for example, of
the player. This optional aspect enables interactive media script
authors to logically treat common objects as singletons while still
allowing the script host 115 to implement the functionality
necessary to expose an object to the single script host.
[0023] Referring now to FIG. 2, an illustrative diagram showing the
relationship among multiple markup documents and script is
provided. An application manifest 230 interacts with applications
which, as noted above, are defined generally by resources 125,
script 205, and markup documents 251, 260 and 275 as shown. Each
application typically uses a single application manifest file in
most settings, but the application manifest is not part of the
runtime state of the application. In this illustrative example, the
application manifest 230 is encoded as an XML document file.
[0024] The application manifest 230 describes the initial markup
file 251 to be used by the application 110 (FIG. 1) as well as the
script files--collectively indicated by the rectangle with
reference numeral 205 in FIG. 2--contained in script host 115 (FIG.
1). If the application manifest 230 lists more than one script, as
in this illustrative example, then all the scripts are loaded into
a script handling engine in the interactive media player. Thus, the
multiple script files are treated and behave as if the script
author had concatenated all of the script files into a single large
file in the order listed in the application manifest 230.
[0025] As shown in FIG. 2, the application manifest 230 refers to
resources 125. The resources available to an application in an
interactive media environment form a directed graph, rooted by the
resources 125 referenced in the application manifest 230. The
allowed extent of the graph for each application is proscribed by
the application manifest 230.
[0026] FIG. 2 shows three applications running in the interactive
media environment. As noted above, each application may only have
one active markup at a time and application content is kept
separate by the applications. As indicated by the arrows between
the markup pages 251, 260 and 275, via script 205, the application
is able to advance from markup page 251 to 260, and then later from
page 260 to 275.
[0027] The progression of context execution by applications in the
interactive media environment is guided by a playlist 290 which
describes, among other things, the relationship among objects in
the environment including presentation objects that are rendered by
the player onto the display device. These presentation objects
typically include video (which may include multiple streams as
described in more detail below) and graphics produced by the
applications.
[0028] Playlist 290 further manages resources across the
interactive media environment as a single management entity in
order to efficiently allocate and control the consumption of
resources by applications. As with the application manifest 230 the
playlist 290 may be advantageously embodied as an XML document file
in most settings.
[0029] The markup pages in FIG. 2 may be used in some settings to
fire events into an execution context (created by the script files
117 and 119 in FIG. 1). The execution context then manipulates the
DOM created by the current application markup. As the markup is
used in the interactive media environment to specify style,
content, timing and layout of graphical objects in the environment
(as represented by elements 253 262 and 277 in FIG. 2), the
combination of script and markup enables the creation of a
comprehensive set of capabilities.
[0030] FIG. 3 is a block diagram of a first illustrative
interactive media player 305 including an interactive content
processor (ICP) 335, video content processor (VCP) 310, and mixer
339. It is noted that the arrangement presented in FIG. 3 provides
a logical model which describe features and functions of the
illustrative interactive media player 305 that are pertinent to
application state management. Thus, an actual implementation of an
interactive media player may utilize various structural forms while
still operating as described herein to achieve the benefits of
application state management. The interactive media player 305 is
typically realized in dedicated hardware such as standalone
consumer electronic device, or alternatively using a software
implementation employing computer readable media with a general
purpose processor such as that found in a personal computer.
[0031] VCP 310 manages one or more media streams that may be
received from multiple sources including a local optical drives
such as a DVD drive or a high-definition DVD (HD-DVD) drive, a
local memory or a remote broadband source over a network. VCP 310,
in this illustrative example, includes one or more media processors
1, 2 . . . N as indicated by elements 304 and 306 in FIG. 3. Media
processors 304 and 306 process the received media streams, which
typically include audio and video, to decode and render the
corresponding images and sound which are output as an audio/video
stream on line 325. Audio/video stream 325 may represent a
plurality of video elements, for example to render multiple
separate video windows using a "picture in picture" type
configuration.
[0032] Media processors 304 and 306 each comprise a media source
interface, demultiplexer and decoder. Media processors 304 and 306
may optionally include decryption capabilities as well. A display
device 355 is coupled to receive and display the audio/video
stream.
[0033] A media clock 312 is utilized so that each received media
has an associated "Media Time." When a video stream is paused on
the interactive media player 305 then the media clock 312 is paused
as well. When the video stream is set by a user to go faster or
slower than real time (for example, when the video is put into fast
forward, rewind or slow-motion modes--using any of these modes is
referred to as "trick play"), then the media clock 312 speeds up or
slows down accordingly. The Media Time is thus derived from the
media clock and the operation of the media processors 304 and 306.
The Media Time is passed to the playlist manager 337 in ICP 335
over line 315. Time in the interactive media environment, including
Media Time, is typically counted in units of "ticks."
[0034] ICP 335 performs all application-related processing and may
be arranged from several components that may be realized in
hardware, software, firmware or a combination thereof. The
components of ICP 335 include, for example, a markup engine, script
language interpreter, and an XML parsing component (not shown). ICP
335 outputs a graphics stream on line 321 which is synchronous with
the audio/video stream 325. Mixer 339 takes the graphics stream on
line 321 and the audio/video stream on line 325 so that the
graphics are rendered in a graphics layer over the video stream to
implement an interactive media session for a user.
[0035] In most settings, ICP 335 outputs graphics that are
synchronized on a frame-by-frame basis with the video stream.
However, such synchronization may be performed using other bases,
including, for example, time (including Title Time and Media time
as defined below), content in the video, or other metadata embedded
in the video that is used to indicate or mark a particular point in
the stream.
[0036] ICP 335 includes a playlist manager 337 and a task manager
330. The playlist manager 337 is responsible for controlling
presentation objects in the environment. These objects include
video playback on the player 305 along with applications that are
running to generate interactive graphics. Playlist manager 337
manages the playlist 290 which is described above in the text
accompanying FIG. 2.
[0037] The playlist manager 337 also computes the "Title Time"
associated with each portion of content in a media stream. A title
is a unique sequence of video and audio content with a start and
end time that is typically defined by the DVD author. However, what
such author defines as a title can be arbitrary. Thus, particular
content which is perceived in a video may be part of one title, a
complete title, or run across multiple titles.
[0038] One example of a title is the copyright warning that
precedes all pre-recorded video in both analog and digital format
in the United States. The featured attraction (e.g., the main
movie) on a DVD is another example and is often the longest title.
In some settings, individual chapters in a movie might be
designated as separates titles by the DVD author. For all such
titles, Title Time is defined as the time elapsed since a given
title started playing as shown on the media clock 312.
[0039] A presentation clock 360 is coupled to the playlist manager
on line 362. The presentation clock 360 is a clock whose time
changes at the same pace as a real-world clock (i.e., it takes one
second of real time for the presentation clock 360 to advance by
one second). In contrast to the media clock 312, the presentation
clock 360 never stops and cannot be sped up or slowed down. The
Presentation Time from the presentation clock 360 is passed to the
task manager 330 which uses it to calculate "Application Time" and
application "Page Time."
[0040] Application Time is the time elapsed since an application
started (or enters an "Active" state as described in more detail
below). When multiple applications are in runtime, each application
has a notion of its own Application Time. For each application,
Application Time always starts at zero when an application is
started in the environment.
[0041] For example, if an application App1 starts at Presentation
Time of 20 arbitrary time units (which is 0 time units for App1)
and application App2 starts at Presentation Time of 25 time units
(which is 0 time units for App2), then at Presentation Time of 35
time units, App1's Application Time is 15 time units and App2's
Application Time is 10 time units. For applications that are
logically subdivided into pages, the Page Time is the time elapsed
since a page of an application has been loaded.
[0042] FIG. 4 is a block diagram of a second illustrative media
player 405 including an ICP 435, VCP 410, and mixer 439.
Interactive media player 405 is similar in form and function to the
interactive media player 305 shown in FIG. 3. Notably, however, VCP
435 includes media processors, 1, 2 . . . N (as indicated by
elements 404 and 406 in FIG. 4) that are arranged to provide
separate feeds 425 and 427 to mixer 439. Such arrangement may be
desirable in some settings where manipulation of the individual
media streams is performed prior to mixing. For example, image
processing/selection techniques such panning and zooming of video
in a media stream may be independently implemented on one or more
of the N separate feeds represented by reference numerals 425 and
427 in FIG. 4.
[0043] The audio/video feeds 425 and 427, along with the
synchronous graphics stream from ICP 435 are mixed in mixer 439 and
output on line 441 to a display device 455. The other elements in
FIG. 4 including ICP 435 (comprising playlist manager 437 and task
manager 430), media clock 412 in VCP 410 and presentation clock 460
are configured and function in a similar manner as their
counterparts shown in FIG. 3 and described in the accompanying
text.
[0044] FIG. 5 is a block diagram of an illustrative arrangement 500
having with a plurality of event queues 1, 2 . . . N as designated
by reference numerals 510, 515, and 518, respectively, and a single
application thread 523. In this illustrative arrangement, all
applications running on an ICP (such as ICP 435 in FIG. 4) are
single threaded, and application thread 523 is dedicated for such
purpose. However, ICP 435 does not necessarily need to be single
threaded itself. In alternative implementations, ICP 435 may
utilize other threads, for example for pre-fetching resources into
a cache.
[0045] Each of the event queues 510, 515, and 518 are arranged to
feed into application thread 523 from their head ends (located at
the right side of FIG. 5). A plurality of applications App1, App2 .
. . AppN as designated by reference numerals 527, 530 and 532,
respectively, are arranged to post workitems, representatively
designated by reference numeral 535, into the queues 510, 515 and
518 from their tail ends (on the left side of FIG. 5).
[0046] Application events are events which are fired by an
application. These may include events fired by either script (e.g,
script host 115 in FIG. 1) or by markup (e.g., markup 120 in FIG.
1). Application events, in most scenarios, are handled only by
script. However, applications 527, 530 and 532 do not invoke script
or markup functionality directly. Instead, all such functionality
is posted to the applications' respective event queues in the form
of workitems and are invoked when the application thread 523
processes the workitem.
[0047] In alternative arrangements, events from sources other than
applications are also scheduled using event queues. For example,
user events are fired by user interaction with a remote control.
System events are events fired by the interactive media player such
as player 405 shown in FIG. 4 and described in the accompanying
text.
[0048] Each workitem in events queues 510, 515 and 518 contains
fields as shown in FIG. 5. These fields include an application
association field 540, a method field 545, a BeginTime field 552,
an EndTime field 555, and an optional ClockSelector field 558.
[0049] The application association field 540 indicates the
particular application to which a workitem applies. The method
field 545 contains a method that is invoked when the workitem is
processed by the application thread 523. Method field 545 also
includes arguments for the method.
[0050] The BeginTime field 552 and EndTime field 555 are used,
respectively, to indicate when the workitem's method begins and
ends. In this illustrative example, time is expressed using
Application Time. However, in alternative examples, the BeginTime
field 552 and EndTime field 555 contain values which may be
alternatively expressed in Title Time, Application Time or Page
Time depending on the requirements of a particular setting. In such
cases, the particular timeframe used by a workitem is indicated in
the ClockSelector field 558. Regardless of the timeframe utilized,
a BeginTime for a workitem must always be less than the
EndTime.
[0051] FIG. 6 is a block diagram of event queue 515 showing the
ordering of workitems contained therein. The other event queues 510
and 518 (FIG. 5) are not shown in FIG. 6 for ease of clarity in
illustration. However, the ordering methodology described below is
equally applicable to such other event queues.
[0052] Event queue 515 includes workitems 1, 2, 3 . . . N as
indicated by reference numerals 605, 610, 615 and 620,
respectively. Each workitem includes the fields shown in FIG. 5 and
described in the accompanying text.
[0053] Workitem 605 includes a BeginTime.sub.1, and an associated
time of insertion t.sub.1, into the event queue 515, as indicated
in block 630 in FIG. 6. Similarly, workitem 610 includes a
BeginTime.sub.2 and an associated time of insertion t.sub.2 into
the event queue 515 as indicated in block 635. Workitem 615
includes a BeginTime.sub.3 and an associated time of insertion
t.sub.3 into the event queue 515 as indicated in block 640. And,
workitem 620 includes a BeginTime.sub.N and an associated time of
insertion t.sub.N into the event queue 515, as indicated in block
645.
[0054] In this illustrative example, workitems are ordered in the
event queue 515 first by BeginTime and then by the time in which
workitems were inserted into the event queue. Such ordering results
in the application thread 523 processing workitems in order of
BeginTime, or when two workitems have the same begin time, then in
FIFO (first in, first out) order.
[0055] Thus, as workitem 620 is at the head of event queue 515 the
implication is that BeginTime.sub.N<BeginTime.sub.3; or if
BeginTime.sub.N=BeginTime.sub.3, then t.sub.N<t.sub.3 (i.e.,
workitem 620 was inserted into event queue 515 before workitem 615.
Following this same reasoning for workitems 605, 610 and 615, then:
TABLE-US-00001 BeginTime.sub.3 < BeginTime.sub.2 ; or if
BeginTime.sub.3 = BeginTime.sub.2, then t.sub.3 < t.sub.2; and
BeginTime.sub.2 < BeginTime.sub.1 ; or if BeginTime.sub.2 =
BeginTime.sub.1, then t.sub.2 < t.sub.1
[0056] The ordering of workitems in an event queue is performed
using two alternative methods: workitems may be ordered when
inserted into an event queue or when workitems are extracted from
the event queue prior to processing. Either arrangement is equally
usable so long as the processing of workitems from the event queue
is performed by BeginTime followed by queue insertion time.
[0057] FIG. 7 is a block diagram of an illustrative arrangement
where the application thread 523 automatically inserts two
workitems 705 and 715 at the tail end of the event queue 515. The
other event queues 510 and 518 (FIG. 5) are not shown in FIG. 7 for
ease of clarity in illustration. However, the automatic insertion
of workitems by the application thread as described below is
equally applicable to such other event queues. As shown, the
automatically inserted workitems 705 and 715 follow after workitems
605 and 620 in the event queue 515. In an illustrative example, the
automatic insertion of the two workitems is performed when an
application starts and the workitems are rescheduled after each
tick.
[0058] Workitem 705 includes calls into the markup engine (e,g, a
markup engine disposed in ICP 435 in FIG. 4) to process timing for
a page in application App2 530 as indicated in block 730. In block
735, workitem 715 includes calls into the markup to reflow
application App2's markup to reflect processed events and then
render the markup on the display device (e.g., display 455 in FIG.
4). Workitems 705 and 715 are always the last workitems processed
in an application's tick by application thread 523.
[0059] FIG. 8 is a flow chart of an illustrative method for queuing
workitems associated with events that occur in an interactive media
environment. In an illustrative example of event queuing with a
single application thread, the method is performed by the
arrangements shown in FIGS. 4-7 and described in the accompanying
text. The method shown is typically performed iteratively for each
tick.
[0060] The process starts at block 805. At block 810, when the
application thread 523 (FIGS. 5-7) is free to process workitems, it
first marks each workitem in the event queue 515 whose BeginTime
corresponds to the current or previous ticks. Application thread
523 will only process marked workitems. Thus a workitem in event
queue 515 will never be processed before its BeginTime.
[0061] At decision block 816, if a marked workitem's EndTime has
already been passed then it is dropped from event queue 515 as
indicated in block 819. No processing on that workitem will be
performed in such a case. Should application App2 530 reloads its
page, the application's page clock is reset to zero and all
outstanding (i.e., queued) workitems based on the application's
page clock are dropped from event queue just as if they had reached
their EndTime.
[0062] If at decision block 816 a marked workitem's EndTime has not
been passed, then control is passed to block 822 and the
application thread 523 processes the workitem. As noted above in
the description accompanying FIG. 6, each workitem is processed in
order from the event queue 515: first by BeginTime, followed by the
time each workitem was inserted into the event queue 515.
[0063] Both repetitive events and one-shot (i.e., single
occurrence, non-repetitive) events are managed using the method
shown in FIG. 8. A repetitive event may include a periodic event
where the associated workitem has an EndTime that is equal to the
next scheduled BeginTime. That is, each periodic event has a
duration equal to the event's period.
[0064] Periodic events typically include events like timer events
and application drawing events. For example, if an application's
script (e.g., in script host 115 in FIG. 1) creates a timer that
will call back once every 10 seconds, it will add a timer workitem
to the event queue 515 with a BeginTime equal to the current time
plus 10 seconds. The EndTime will be set to the BeginTime plus 10
seconds. Once the timer workitem is executed out of the event queue
515, the BeginTimes and EndTimes will be adjusted by adding another
10 seconds and the workitem will be reinserted into the event queue
515 at the proper location based on the new BeginTime.
[0065] Periodic events are invoked whenever possible. But if they
cannot be processed by the application thread 523 before the
EndTime in their associated workitems expires, then that particular
invocation is dropped and the next invocation is scheduled with a
new workitem.
[0066] Advantageously, the event queuing method enables a parameter
may be passed to timer events to indicate the time that the event
is to be invoked. This parameter must be the same as the BeginTime
in the associated workitem. Script associated with a periodic timer
event might not be run exactly at the invoked time, as noted above.
However, as each workitem includes a method field 545 (FIG. 5) that
specifies arguments to the method, the argument's value will
reflect an intended time of invocation and not the actual time.
Accordingly, the handler for a timer event will know what time
(i.e., tick) it is handling.
[0067] A one-shot event has a corresponding workitem with an
EndTime of INFINITE. Therefore, a one-shot event will never be
dropped from the event queue 515. For example, if a one-shot event
is an input event, then that event's handler is scheduled as a
workitem in the event queue 515 with an EndTime of INFINITE.
[0068] As indicated in block 822, the processing is performed on a
committed basis. That is, once the application thread 523 begins
processing a workitem from the event queue 515, it does not stop
processing. For example, script which may be long running is not
aborted nor are exceptions injected into the script in order to
throw it out. While such a scheme can tie up the application thread
while it processes script, as noted above the ICP (e.g., ICP 435 in
FIG. 4) may be arranged to include other threads which continue to
run during the committed processing of workitems. In alternative
arrangements, it may be desirable to handle workitems in a manner
such that a new event is inserted into the event queue 515 based on
its relative BeginTime to be handled after the commited
workitems.
[0069] At block 830, any new workitems that are created during the
processing of marked workitems are inserted into the event queue
515 after the marked workitems, regardless of their BeginTime. The
process of marking workitems, committing to them and inserting new
workitems after the committed workitems in an event queue (as shown
in blocks 810, 822 and 830) ensures that the applications are
always afforded some visible progress.
[0070] As indicated at block 835 and 828 in FIG. 8, the application
thread automatically inserts two workitems into each application's
event queue for each tick, as shown in FIG. 7 and described in the
accompanying text. These workitems call into the markup engine for
each application to evaluate application timing and then reflow and
render the markup on a display device. As noted above, the
workitems are inserted upon application start and a rescheduled
after each tick. In addition, these two workitems are always the
last two to be processed for an application's tick and are treated
as periodic events that may be dropped from the event queue
515.
[0071] It is noted that for the sake of clarity and ease of
illustration in the description above that data, programs, and
other executable program components such as operating systems are
shown is discrete blocks, boxes or other elements although it is
recognized and emphasized that such programs and components may
reside at various times in different storage, memory or processing
components of any hardware host used and are executed by one or
more processors in such host hardware.
[0072] Although various illustrative arrangements and methods for
managing application states in an interactive media environment
have been shown and described, it should be understood that the
scope of the claims appended hereto shall not necessarily be
limited to the specific features, arrangements or methods
described. Instead, the specific features, arrangements or methods
are disclosed as illustrative forms of implementing managed
applications states in an interactive media environment as more
particularly claimed below.
* * * * *