U.S. patent application number 14/727226 was filed with the patent office on 2016-05-12 for programmatic user interface generation based on display size.
This patent application is currently assigned to Microsoft Technology Licensing, LLC. The applicant listed for this patent is Microsoft Technology Licensing, LLC. Invention is credited to Andrew Fomichev, Alexandre Grigorovitch, Ben Howell, Luan Nguyen, Vlad Riscutia, Maya Rodrig, Julie Seto.
Application Number | 20160132301 14/727226 |
Document ID | / |
Family ID | 55912229 |
Filed Date | 2016-05-12 |
United States Patent
Application |
20160132301 |
Kind Code |
A1 |
Riscutia; Vlad ; et
al. |
May 12, 2016 |
PROGRAMMATIC USER INTERFACE GENERATION BASED ON DISPLAY SIZE
Abstract
Non-limiting examples of the present disclosure describe
programmatic generation of a user interface for display on a
processing device. A display class is determined from a plurality
of display classes based on a detected display size of a processing
device on which the user interface is to display. Prior to
instantiating a user interface window, a stored user interface
definition is identified and interpreted. The stored user interface
definition comprises at least one programmed command object. A
displayed user interface is instantiated on the processing device,
where the displayed user interface comprises at least one user
interface element. The user interface element is programmatically
generated by translating the programmed command object of the user
interface definition into the user interface element based on
operations set in accordance with the determined display class.
Other examples are also described.
Inventors: |
Riscutia; Vlad; (Redmond,
WA) ; Seto; Julie; (Duvall, WA) ; Nguyen;
Luan; (Seattle, WA) ; Grigorovitch; Alexandre;
(Woodinville, WA) ; Fomichev; Andrew; (Bellevue,
WA) ; Howell; Ben; (Seattle, WA) ; Rodrig;
Maya; (Seattle, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Microsoft Technology Licensing, LLC |
Redmond |
WA |
US |
|
|
Assignee: |
Microsoft Technology Licensing,
LLC
Redmond
WA
|
Family ID: |
55912229 |
Appl. No.: |
14/727226 |
Filed: |
June 1, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62076368 |
Nov 6, 2014 |
|
|
|
Current U.S.
Class: |
715/763 |
Current CPC
Class: |
G06F 3/048 20130101;
G06T 2200/16 20130101; G06F 3/0483 20130101; G06F 3/04886 20130101;
G06F 2203/04803 20130101; G06F 9/451 20180201; G06T 3/4092
20130101; G06F 3/017 20130101; G06F 3/0484 20130101; G06F 8/38
20130101; G06T 2200/24 20130101 |
International
Class: |
G06F 9/44 20060101
G06F009/44; G06F 3/0482 20060101 G06F003/0482; G06F 3/0484 20060101
G06F003/0484 |
Claims
1. A computer-implemented method comprising: determining a display
class from a plurality of display classes based on a detected
display size of a processing device on which a user interface is to
display; prior to instantiating a user interface window,
identifying and interpreting a stored user interface definition
that comprises at least one programmed command object; and
instantiating, on the processing device, a displayed user interface
that comprises at least one user interface element, wherein the
user interface element is programmatically generated by translating
the programmed command object into the user interface element based
on operations set in accordance with the determined display
class.
2. The computer-implemented method according to claim 1, wherein
the display size is a value determined from at least one of a
screen diagonal of the display of the processing device and a
resolution of the display of the processing device.
3. The computer-implemented method according to claim 1, wherein
the identifying and interpreting further comprises occurs at
runtime of an application associated with the user interface.
4. The computer-implemented method according to claim 1, wherein
the stored user interface definition comprises definitions for the
programmed command object and grouping data for presenting the
programmed command object as the user interface element.
5. The computer-implemented method according to claim 1, wherein
the translating of the programmed command object into the user
interface element further comprises setting a size of the user
interface element based on the operations corresponding with the
determine display class.
6. The computer-implemented method according to claim 1, wherein
the translating of the programmed command object into the user
interface element further comprises setting a style of the user
interface element based on the operations corresponding with the
determine display class.
7. The computer-implemented method according to claim 1, wherein
the translating of the programmed command object into the user
interface element further comprises setting a layout of the user
interface element based on the grouping data of the user interface
definition and the operations corresponding with the determine
display class.
8. The computer-implemented method according to claim 1, further
comprising creating a user interface definition file comprising one
or more programmed command objects, and storing the user interface
definition file in a memory of the processing device.
9. The computer-implemented method according to claim 1, further
comprising receiving update to the stored user interface
definition, and instantiating the displayed user interface based on
the updated stored user interface definition.
10. A system comprising: a memory; and at least one processor
operatively connected with the memory, executing operations
comprising: determining a display class from a plurality of display
classes based on a detected display size of a processing device on
which a user interface is to display, prior to instantiating a user
interface window, identifying and interpreting a stored user
interface definition that comprises at least one programmed command
object, and instantiating, on the processing device, a displayed
user interface that comprises at least one user interface element,
wherein the user interface element is programmatically generated by
translating the programmed command object into the user interface
element based on operations set in accordance with the determined
display class.
11. The system according to claim 10, wherein the display size is a
value determined from at least one of a screen diagonal of the
display of the processing device and a resolution of the display of
the processing device.
12. The system according to claim 10, wherein processor executes
the identifying and interpreting of the stored user interface
definition at runtime of an application associated with the user
interface.
13. The system according to claim 10, wherein the stored user
interface definition is stored in the memory and comprises
definitions for the programmed command object and grouping data for
presenting the programmed command object as the user interface
element.
14. The system according to claim 10, wherein the translating of
the programmed command object into the user interface element
further comprises setting a size of the user interface element
based on the operations corresponding with the determine display
class.
15. The system according to claim 10, wherein the translating of
the programmed command object into the user interface element
further comprises setting a style of the user interface element
based on the operations corresponding with the determine display
class.
16. The system according to claim 10, wherein the translating of
the programmed command object into the user interface element
further comprises setting a layout of the user interface element
based on the grouping data of the user interface definition and the
operations corresponding with the determine display class.
17. The system according to claim 10, wherein the executed
operations further comprising creating a user interface definition
file comprising one or more programmed command objects, and storing
the user interface definition file in the memory of the system.
18. The system according to claim 10, wherein the executed
operations further comprising receiving update to the stored user
interface definition, and instantiating the displayed user
interface based on the updated stored user interface
definition.
19. A computer-readable storage device including executable
instructions, that when executed on at least one processor, causing
the processor to perform a process comprising: determining a
display class from a plurality of display classes based on a
detected display size of a processing device on which a user
interface is to display; prior to instantiating a user interface
window, identifying and interpreting a stored user interface
definition that comprises a plurality of groupings of programmed
command objects; and instantiating, on the processing device, a
displayed user interface that comprises at least one user interface
control palette, wherein the user interface control palette is
programmatically generated by translating the plurality of
groupings of programmed command objects into a plurality of
groupings of user interface elements based on operations set in
accordance with the determined display class.
20. The computer-readable storage device according to claim 19,
wherein the process comprising receiving update to the stored user
interface definition, and instantiating the displayed user
interface based on an updated stored user interface definition.
Description
PRIORITY
[0001] This application claims the benefit of U.S. Provisional
Application No. 62/076,368, filed on Nov. 6, 2014, which is hereby
incorporated by reference in its entirety.
BACKGROUND
[0002] Devices such as personal computers (PCs), laptops, slates,
and phones offer a wide range of screen sizes. However, there is no
established method for scaling a user interface (UI) across a large
range of screen sizes, from very large displays down to smaller
displays. It is with respect to this general technical area that
the present application is directed.
SUMMARY
[0003] Non-limiting examples of the present disclosure describe
programmatic generation of a user interface for display on a
processing device. A display class is determined from a plurality
of display classes based on a detected display size of a processing
device on which the user interface is to display. Prior to
instantiating a user interface window, a stored user interface
definition is identified and interpreted. The stored user interface
definition comprises at least one programmed command object. A
displayed user interface is instantiated on the processing device,
where the displayed user interface comprises at least one user
interface element. The user interface element is programmatically
generated by translating the programmed command object of the user
interface definition into the user interface element based on
operations set in accordance with the determined display class.
[0004] In other non-limiting examples, a user interface is
generated that comprises a user interface control palette usable to
control an application. A display class is determined from a
plurality of display classes based on a detected display size of a
processing device on which the user interface is to display. Prior
to instantiating a user interface window, a stored user interface
definition is identified and interpreted. The stored user interface
definition comprises a plurality of groupings of programmed command
objects. A displayed user interface is instantiated on the
processing device, where the displayed user interface comprises at
least one user interface control palette. The user interface
control palette is programmatically generated by translating the
plurality of groupings of programmed command objects of the user
interface definition into a plurality of groupings of user
interface elements based on operations set in accordance with the
determined display class.
[0005] This Summary is provided to introduce a selection of
concepts in a simplified form that are further described below in
the Detailed Description. This Summary is not intended to identify
key features or essential features of the claimed subject matter,
nor is it intended to be used to limit the scope of the claimed
subject matter. Additional aspects, features, and/or advantages of
examples will be set forth in part in the description which follows
and, in part, will be apparent from the description, or may be
learned by practice of the disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Non-limiting and non-exhaustive examples are described with
reference to the following figures.
[0007] FIG. 1 is a block diagram illustrating an example of a
computing device with which aspects of the present disclosure may
be practiced.
[0008] FIGS. 2A and 2B are simplified block diagrams of a mobile
computing device with which aspects of the present disclosure may
be practiced.
[0009] FIG. 3 is a simplified block diagram of a distributed
computing system in which aspects of the present disclosure may be
practiced.
[0010] FIG. 4 is an exemplary method for managing user interface
definitions with which aspects of the present disclosure may be
practiced.
[0011] FIG. 5 is an exemplary method for presenting a user
interface with which aspects of the present disclosure may be
practiced.
[0012] FIG. 6 is a diagram illustrating an exemplary method for
setting a layout of a user generated interface palette with which
aspects of the present disclosure may be practiced.
[0013] FIG. 7 is a diagram illustrating display for exemplary
processing devices of different sizes with which aspects of the
present disclosure may be practiced.
[0014] FIG. 8 is a diagram illustrating user interface examples
with which aspects of the present disclosure may be practiced.
[0015] FIG. 9 is a diagram illustrating user interface examples
with which aspects of the present disclosure may be practiced.
DETAILED DESCRIPTION
[0016] Users of processing devices desire applications to be
optimized in a form-factor manner. However, there is no established
method of scaling a user interface (UI) across a large range of
screen sizes with such an approach. Simply attempting to merge a
large screen version of an application with a small screen version
of an application creates complications. As an example, when large
screen versions of applications are executed on devices having
smaller display sizes, the UI gets too crowded and touch targets
become too small. Additionally, another complication is that UIs
are not traditionally scalable across devices having different
display sizes. For instance, a user of a processing device may be
viewing an application on a device having a smaller display size
(e.g., mobile phone) and proceed to connect the device having the
small screen display to a device having a larger display size
(e.g., PC). Attempted resizing of an application across differing
display sizes may drastically affect the display and operation of
the UI for an application and/or UI control. In other cases where
different versions of an application are developed (e.g., mobile
version and desktop version), systems are typically unable to
recognize that a UI is to be scaled to a different programmed
version to account for display size changes. Other instances of
building UI packages may incorporate a scaling model for large and
small screen devices but are only able to show a single type of UI
(e.g., phone version or slate version) once an application is
installed. This may limit a user's ability to connect to large
display screens and enjoy UI that takes advantage of available
display space.
[0017] In non-limiting examples, systems and methods describe
programmatic generation of user interfaces in a form factor manner
that accounts for display size of the processing device upon which
an application/UI is displayed. Programming operations applied
generate multiple sets of UI controls and layouts to use and based
on the size of a display, determine which of the UI controls and
layout algorithms to apply when instantiating a user interface.
Examples of the present disclosure comprise evaluation of display
class information associated with an application UI at runtime of
the application to identify a class of display (e.g., large
screen/tablet/slate/phablet/phone, etc.). Display class information
may be used to determine whether to display a UI optimized for
larger screen devices, smaller screen devices or something
in-between. One or more UI layouts may be generated for each of a
plurality of display classes. When a display class is identified
based on the detection of a display size of a processing device
upon which an application/UI is running, a user interface may be
displayed based on a UI layout associated with determined display
class. Examples described enable a UI to be tailored at run-time in
a form factor manner without requiring any changes to code or a
user interface definition. Exemplary UI examples generated provide
visually different experiences tailored to the processing device
than an application/UI is executing upon. At the same time,
examples preserve semantic meaning of commands/user interactions
and advantages of a definition-based UI.
[0018] A number of technical advantages are achieved based on the
present disclosure including but not limited to: improved
scalability of UI for applications, consistent functionality for a
UI that accounts for varying display sizes, visually appealing
presentation of UI controls, enhanced processing capability across
devices of varying display sizes including improved efficiency and
usability for UI controls, improved efficiency in navigation and
access to control content for applications/UIs, improved efficiency
in programming application command objects, and improved user
interaction with applications/UI controls, among other
examples.
[0019] FIGS. 1-3 and the associated descriptions provide a
discussion of a variety of operating environments in which examples
of the invention may be practiced. However, the devices and systems
illustrated and discussed with respect to FIGS. 1-3 are for
purposes of example and illustration and are not limiting of a vast
number of computing device configurations that may be utilized for
practicing examples of the invention, described herein.
[0020] FIG. 1 is a block diagram illustrating physical components
of a computing device 102, for example a mobile processing device,
with which examples of the present disclosure may be practiced. In
a basic configuration, the computing device 102 may include at
least one processing unit 104 and a system memory 106. Depending on
the configuration and type of computing device, the system memory
106 may comprise, but is not limited to, volatile storage (e.g.,
random access memory), non-volatile storage (e.g., read-only
memory), flash memory, or any combination of such memories. The
system memory 106 may include an operating system 107 and one or
more program modules 108 suitable for running software
programs/modules 120 such as JO manager 124, other utility 126 and
application 128. As examples, system memory 106 may store
instructions for execution. Other examples of system memory 106 may
store data associated with applications. The operating system 107,
for example, may be suitable for controlling the operation of the
computing device 102. Furthermore, examples of the invention may be
practiced in conjunction with a graphics library, other operating
systems, or any other application program and is not limited to any
particular application or system. This basic configuration is
illustrated in FIG. 1 by those components within a dashed line 122.
The computing device 102 may have additional features or
functionality. For example, the computing device 102 may also
include additional data storage devices (removable and/or
non-removable) such as, for example, magnetic disks, optical disks,
or tape. Such additional storage is illustrated in FIG. 1 by a
removable storage device 109 and a non-removable storage device
110.
[0021] As stated above, a number of program modules and data files
may be stored in the system memory 106. While executing on the
processing unit 104, program modules 108 (e.g., Input/Output (I/O)
manager 124, other utility 126 and application 128) may perform
processes including, but not limited to, one or more of the stages
of the operations described throughout this disclosure. Other
program modules that may be used in accordance with examples of the
present invention may include electronic mail and contacts
applications, word processing applications, spreadsheet
applications, database applications, slide presentation
applications, drawing or computer-aided application programs, photo
editing applications, authoring applications, etc.
[0022] Furthermore, examples of the invention may be practiced in
an electrical circuit comprising discrete electronic elements,
packaged or integrated electronic chips containing logic gates, a
circuit utilizing a microprocessor, or on a single chip containing
electronic elements or microprocessors. For example, examples of
the invention may be practiced via a system-on-a-chip (SOC) where
each or many of the components illustrated in FIG. 1 may be
integrated onto a single integrated circuit. Such an SOC device may
include one or more processing units, graphics units,
communications units, system virtualization units and various
application functionality all of which are integrated (or "burned")
onto the chip substrate as a single integrated circuit. When
operating via an SOC, the functionality described herein may be
operated via application-specific logic integrated with other
components of the computing device 502 on the single integrated
circuit (chip). Examples of the present disclosure may also be
practiced using other technologies capable of performing logical
operations such as, for example, AND, OR, and NOT, including but
not limited to mechanical, optical, fluidic, and quantum
technologies. In addition, examples of the invention may be
practiced within a general purpose computer or in any other
circuits or systems.
[0023] The computing device 102 may also have one or more input
device(s) 112 such as a keyboard, a mouse, a pen, a sound input
device, a device for voice input/recognition, a touch input device,
etc. The output device(s) 114 such as a display, speakers, a
printer, etc. may also be included. The aforementioned devices are
examples and others may be used. The computing device 104 may
include one or more communication connections 116 allowing
communications with other computing devices 118. Examples of
suitable communication connections 116 include, but are not limited
to, RF transmitter, receiver, and/or transceiver circuitry;
universal serial bus (USB), parallel, and/or serial ports.
[0024] The term computer readable media as used herein may include
computer storage media. Computer storage media may include volatile
and nonvolatile, removable and non-removable media implemented in
any method or technology for storage of information, such as
computer readable instructions, data structures, or program
modules. The system memory 106, the removable storage device 109,
and the non-removable storage device 110 are all computer storage
media examples (i.e., memory storage.) Computer storage media may
include RAM, ROM, electrically erasable read-only memory (EEPROM),
flash memory or other memory technology, CD-ROM, digital versatile
disks (DVD) or other optical storage, magnetic cassettes, magnetic
tape, magnetic disk storage or other magnetic storage devices, or
any other article of manufacture which can be used to store
information and which can be accessed by the computing device 102.
Any such computer storage media may be part of the computing device
102. Computer storage media does not include a carrier wave or
other propagated or modulated data signal.
[0025] Communication media may be embodied by computer readable
instructions, data structures, program modules, or other data in a
modulated data signal, such as a carrier wave or other transport
mechanism, and includes any information delivery media. The term
"modulated data signal" may describe a signal that has one or more
characteristics set or changed in such a manner as to encode
information in the signal. By way of example, and not limitation,
communication media may include wired media such as a wired network
or direct-wired connection, and wireless media such as acoustic,
radio frequency (RF), infrared, and other wireless media.
[0026] FIGS. 2A and 2B illustrate a mobile computing device 200,
for example, a mobile telephone, a smart phone, a personal data
assistant, a tablet personal computer, a phablet, a slate, a laptop
computer, and the like, with which examples of the invention may be
practiced. For example, mobile computing device 200 may be
implemented to execute applications and/or application command
control. Application command control relates to presentation and
control of commands for use with an application through a user
interface (UI) or graphical user interface (GUI). In one example,
application command controls may be programmed specifically to work
with a single application. In other examples, application command
controls may be programmed to work across more than one
application. With reference to FIG. 2A, one example of a mobile
computing device 200 for implementing the examples is illustrated.
In a basic configuration, the mobile computing device 200 is a
handheld computer having both input elements and output elements.
The mobile computing device 200 typically includes a display 205
and one or more input buttons 210 that allow the user to enter
information into the mobile computing device 200. The display 205
of the mobile computing device 200 may also function as an input
device (e.g., a touch screen display). If included, an optional
side input element 215 allows further user input. The side input
element 215 may be a rotary switch, a button, or any other type of
manual input element. In alternative examples, mobile computing
device 200 may incorporate more or less input elements. For
example, the display 205 may not be a touch screen in some
examples. In yet another alternative example, the mobile computing
device 200 is a portable phone system, such as a cellular phone.
The mobile computing device 200 may also include an optional keypad
235. Optional keypad 235 may be a physical keypad or a "soft"
keypad generated on the touch screen display or any other soft
input panel (SIP). In various examples, the output elements include
the display 205 for showing a GUI, a visual indicator 220 (e.g., a
light emitting diode), and/or an audio transducer 225 (e.g., a
speaker). In some examples, the mobile computing device 200
incorporates a vibration transducer for providing the user with
tactile feedback. In yet another example, the mobile computing
device 200 incorporates input and/or output ports, such as an audio
input (e.g., a microphone jack), an audio output (e.g., a headphone
jack), and a video output (e.g., a HDMI port) for sending signals
to or receiving signals from an external device.
[0027] FIG. 2B is a block diagram illustrating the architecture of
one example of a mobile computing device. That is, the mobile
computing device 200 can incorporate a system (i.e., an
architecture) 202 to implement some examples. In one examples, the
system 202 is implemented as a "smart phone" capable of running one
or more applications (e.g., browser, e-mail, calendaring, contact
managers, messaging clients, games, and media clients/players). In
some examples, the system 202 is integrated as a computing device,
such as an integrated personal digital assistant (PDA), tablet and
wireless phone.
[0028] One or more application programs 266 may be loaded into the
memory 262 and run on or in association with the operating system
264. Examples of the application programs include phone dialer
programs, e-mail programs, personal information management (PIM)
programs, word processing programs, spreadsheet programs, Internet
browser programs, messaging programs, and so forth. The system 202
also includes a non-volatile storage area 268 within the memory
262. The non-volatile storage area 268 may be used to store
persistent information that should not be lost if the system 202 is
powered down. The application programs 266 may use and store
information in the non-volatile storage area 268, such as e-mail or
other messages used by an e-mail application, and the like. A
synchronization application (not shown) also resides on the system
202 and is programmed to interact with a corresponding
synchronization application resident on a host computer to keep the
information stored in the non-volatile storage area 268
synchronized with corresponding information stored at the host
computer. As should be appreciated, other applications may be
loaded into the memory 262 and run on the mobile computing device
200 described herein.
[0029] The system 202 has a power supply 270, which may be
implemented as one or more batteries. The power supply 270 might
further include an external power source, such as an AC adapter or
a powered docking cradle that supplements or recharges the
batteries.
[0030] The system 202 may include peripheral device port 230 that
performs the function of facilitating connectivity between system
202 and one or more peripheral devices. Transmissions to and from
the peripheral device port 230 are conducted under control of the
operating system (OS) 264. In other words, communications received
by the peripheral device port 230 may be disseminated to the
application programs 266 via the operating system 264, and vice
versa.
[0031] The system 202 may also include a radio interface layer 272
that performs the function of transmitting and receiving radio
frequency communications. The radio interface layer 272 facilitates
wireless connectivity between the system 202 and the "outside
world," via a communications carrier or service provider.
Transmissions to and from the radio interface layer 272 are
conducted under control of the operating system 264. In other
words, communications received by the radio interface layer 272 may
be disseminated to the application programs 266 via the operating
system 264, and vice versa.
[0032] The visual indicator 220 may be used to provide visual
notifications, and/or an audio interface 274 may be used for
producing audible notifications via the audio transducer 225. In
the illustrated example, the visual indicator 220 is a light
emitting diode (LED) and the audio transducer 225 is a speaker.
These devices may be directly coupled to the power supply 270 so
that when activated, they remain on for a duration dictated by the
notification mechanism even though the processor 260 and other
components might shut down for conserving battery power. The LED
may be programmed to remain on indefinitely until the user takes
action to indicate the powered-on status of the device. The audio
interface 274 is used to provide audible signals to and receive
audible signals from the user. For example, in addition to being
coupled to the audio transducer 225, the audio interface 274 may
also be coupled to a microphone to receive audible input, such as
to facilitate a telephone conversation. In accordance with examples
of the present invention, the microphone may also serve as an audio
sensor to facilitate control of notifications, as will be described
below. The system 202 may further include a video interface 276
that enables an operation of an on-board camera 230 to record still
images, video stream, and the like.
[0033] A mobile computing device 200 implementing the system 202
may have additional features or functionality. For example, the
mobile computing device 200 may also include additional data
storage devices (removable and/or non-removable) such as, magnetic
disks, optical disks, or tape. Such additional storage is
illustrated in FIG. 2B by the non-volatile storage area 268.
[0034] Data/information generated or captured by the mobile
computing device 200 and stored via the system 202 may be stored
locally on the mobile computing device 200, as described above, or
the data may be stored on any number of storage media that may be
accessed by the device via the radio 272 or via a wired connection
between the mobile computing device 200 and a separate computing
device associated with the mobile computing device 200, for
example, a server computer in a distributed computing network, such
as the Internet. As should be appreciated such data/information may
be accessed via the mobile computing device 200 via the radio 272
or via a distributed computing network. Similarly, such
data/information may be readily transferred between computing
devices for storage and use according to well-known
data/information transfer and storage means, including electronic
mail and collaborative data/information sharing systems.
[0035] FIG. 3 illustrates one example of the architecture of a
system for providing an application that reliably accesses target
data on a storage system and handles communication failures to one
or more client devices, as described above. Target data accessed,
interacted with, or edited in association with programming modules
108, applications 120, and storage/memory may be stored in
different communication channels or other storage types. For
example, various documents may be stored using a directory service
322, a web portal 324, a mailbox service 326, an instant messaging
store 328, or a social networking site 330, application 128, IO
manager 124, other utility 126, and storage systems may use any of
these types of systems or the like for enabling data utilization,
as described herein. A server 320 may provide storage system for
use by a client operating on general computing device 102 and
mobile device(s) 200 through network 315. By way of example,
network 315 may comprise the Internet or any other type of local or
wide area network, and client nodes may be implemented as a
computing device 102 embodied in a personal computer, a tablet
computing device, and/or by a mobile computing device 200 (e.g.,
mobile processing device). Any of these examples of the client
computing device 102 or 200 may obtain content from the store
316.
[0036] FIG. 4 is an exemplary method 400 for managing user
interface definitions with which aspects of the present disclosure
may be practiced. As an example, method 400 may be executed by an
exemplary system such as shown in FIGS. 1-3. In examples, method
400 may be executed on a device comprising at least one processor
configured to store and execute operations, programs or
instructions. However, method 400 is not limited to such examples.
In at least one example, method 400 may be executed (e.g.,
computer-implemented operations) by one or more components of a
processing device or components of a distributed network, for
instance, web service/distributed network service (e.g. cloud
service). System components may be utilized to perform the
operations described herein with respect to method 400.
[0037] In examples, method 400 may be performed in associated with
an application. An application is a software component that
executes on the processing device, interfacing with hardware and
software components of the device. An application comprises one or
more programs designed to carry out operations, and an application
is associated with a UI. A user interface (Up provides interactive
or non-interactive dialog windows, task panes, etc., for managing
user interaction with an application or service. Examples of tasks
performed by a UI comprise presenting information and receiving
input from a user. An example UI may contain one or more
elements/controls, or groups of controls, such as, for example,
push buttons, radio buttons, check boxes, edit boxes, text labels,
list boxes, etc. An example UI includes any number of controls or
other elements, such as, for example, text, vector graphics,
images, video, animations, and audio. In one example, a UI exists
separately from an application with which an exemplary UI is to be
used. Alternately may be either integrated with the source code, or
included with standard resources used by the application, such as
dynamic linked library (DLL) files, among other examples.
[0038] In examples, a UI may provide application command control
though UI controls. User interface control is a graphical control
element that interfaces with an application that executes on the
processing device (e.g., memory, processor and functions of mobile
device) and software components such as an operating system (OS),
applications executing on a mobile device, programming modules,
input methods (e.g., soft input panel (SIP)) and command container
such as a pane or contextual menu, among other examples. As an
example, an UI control is used to control execution of
actions/commands for the application. An SIP is an on-screen input
method for devices (e.g., text input or voice input), and a pane is
a software component that assists function of other software
running on the device such as the OS and other software
applications, among other examples. In some examples, a UI control
may be integrated within an application. For instance, a UI control
may be able to be launched, closed, expanded or minimized when an
application is launched, closed, expanded or minimized. In other
examples, UI control is executable as its own application that
interfaces with another application. For instance, UI control may
be able to be launched, closed or minimized separately from the
launching of an application that is controlled by the UI
control.
[0039] Method 400 begins at operation 402 where a programmed
command object is received or updated. A programmed command object
is code that represents a command and/or command group that is to
be rendered as a UI element. A programmed command object may be
included in a UI definition that is used to translate the
programmed command objects into displayable and usable portions of
a UI. UI elements are the displayable and usable portions of the UI
that act as controls to trigger actions/commands associated with a
programmed command object. In operation 402, developers may create
new programmed command objects or update existing command objects
for inclusion in a user interface definition that can be used to
define how programmed command objects/groupings of programmed
command objects are presented for display. In examples, developers
that create programmed command objects may focus on
commanding/function of a programmed command object do not
necessarily need to be specify how a command object may display as
it is scaled across processing devices having varying display
sizes. In that example, programmed command objects may be included
in a UI definition data and programmatically generated as UI
elements in a form factor manner using other programming operations
(e.g., application programming interface (API)) that evaluate a UI
definition data.
[0040] Once programming command objects are created or updated,
flow may proceed to operation 404 where a programming command
object is stored. In operation 404, data associated with
programming command objects may be stored in one or more memories,
storages, libraries, files, or databases, etc., for use in creation
of a UI definition. In one example, stored program command objects
may be stored on a processing device upon which an application/UI
is executing. In another example, stored program command objects
may be stored on one or more separate processing devices that may
be used to manage applications/UIs and UI definition data, for
example. In any example, storages for programmed command objects
(e.g., database) are editable in order to add or delete controls or
other elements, or to change the appearance or behavior of one or
more controls or other elements.
[0041] Flow may proceed to operation 406 where a UI definition is
created or updated. A UI definition is a collection of programmed
command objects for commanding an application. The UI definition is
data used to translate the programmed command objects into
displayable and usable portions of an application user interface.
User interface definitions may be data stored in any form, for
example any type of file format including linked file libraries. In
examples, a UI definition includes properties for each programmed
command object, including, for example, position, dimension,
visibility, text, colors, opacity, borders, accessibility
information, enabled state, and other states commonly associated
with conventional UI controls. In one example, programmed command
objects specified by the UI definition files are stored (operation
408) in a library or database comprised of predefined controls and
other elements, i.e., vector graphics, images, video, etc. as
identified in operation 404. Alternately, a storage, memory,
database etc. may contain pointers to some or all of the controls
or other programmed command objects. Such an example is useful
where command objects having a relatively large file size, such as
a video clip, are to be included in a rendered UI window.
[0042] In addition, the UI definition may also reference one or
more event handlers or "listeners" that are to be associated with
particular controls, groups of controls, other elements, or entire
UI windows so that the controls, elements, or UI windows are
capable of interacting with an associated application. In alternate
examples, these listeners are either stored along with the UI
definitions, or in a separate listener memory, storage, file,
database, etc. After reading the user interface descriptions,
either at or during application run time, the controls or other
elements described by the UI definition files are read from a
library or database file of controls and elements, associated with
the specified listeners, and then used to automatically instantiate
extensible user interfaces.
[0043] Flow may proceed to operation 408 where user definitions are
stored for use in generation of user interfaces for display. In one
example, user interface definitions or descriptions may be stored
(operation 408) separate from the application code with which the
user interfaces are intended to be used. User interface definitions
or definition files can be changed at any time prior to running an
underlying application, or alternately, at any time prior to
displaying the user interface to provide uniquely configured user
interface windows which are then automatically instantiated when
rendering the user interface windows. This concept offers several
major advantages. For instance, changes to any of the user
interface windows do not require editing and recompiling of the
associated application source code. Consequently, the number of
potential errors that may be introduced into an application are
dramatically reduced because the application itself is not edited
to modify the UI windows associated with that application. Further
the use of separate UI definition files serves to allow for a
common baseline application source code, regardless of the user
interfaces that are associated with that application. Alternately,
as noted above, the UI descriptions are stored/included (operation
408) in either the application code itself, or in one or more
linked files, such as a DLL file, or other files which are included
in the application as the application is compiled, rather than
including the descriptions in separate UI definition files.
Regardless of where the UI descriptions are located, UI
descriptions are interpreted and treated in the same manner prior
to automatic instantiation of the UI windows.
[0044] FIG. 5 is an exemplary method 500 for presenting a user
interface with which aspects of the present disclosure may be
practiced. As an example, method 500 may be executed by an
exemplary system such as shown in FIGS. 1-3. In examples, method
500 may be executed on a device comprising at least one processor
configured to store and execute operations, programs or
instructions. However, method 500 is not limited to such examples.
In at least one example, method 500 may be executed (e.g.,
computer-implemented operations) by one or more components of a
processing device or components of a distributed network, for
instance, web service/distributed network service (e.g. cloud
service). System components may be utilized to perform the
operations described herein with respect to method 500.
[0045] Method 500 begins at operation 502 where a display class is
determined based on a detected display size of a processing device
upon which an application/UI is to be displayed. Display class
determination provides an abstraction for determining the size of a
display. Operations (e.g., code) can query display class
information to determine a UI instance to instantiate depending on
the display size of the processing device that an application
window is to execute upon. A display class can be defined for
processing devices having display sizes that fall within the range
associated with the display class. That is, display classes act as
transition points for UI experiences. Display class is a value that
is determined based a maximum display size. The value for display
class may be in any form including numeric values and elements of
speech, as examples. For instance, display classes may be set to
correspond with different types of processing devices (e.g.,
laptops, PCs, tablets, phones, etc.) where an exemplary display
class may be "<=Phone" or "<=Tablet". In another example,
display classes may be set based on numeric values. For example, a
display class may be identified using numeric values (e.g., 0 to 3
inches). In any examples, display classes are used to classify
processing devices in accordance with display size. For example, a
display class may be set for processing devices having a display
size falling in a range from 0 to 3 inches where another display
class may be set for processing devices having a display size in a
range from 3.1 to 5 inches, and so on. A range for values of
display classes may fall between 0 and infinity. In one example,
operations for display class determination are written in style of
successive less than or equal to (<=) checks, with an else for
everything greater than a defined display class. In this example,
additional display class designations may be easily added without
having to change operational code behavior. However, one skilled in
the art will recognize that display class designations including
minimum and/or maximum values for ranges of display classes can be
defined in any possible way that can be useful in defining user
interface interaction. In examples, a minimum value of a display
class may be a value that is equal to or greater than a maximum
value of a display class which is directly smaller than the display
class being defined. For instance, as in an example above, a first
display class may correspond to a range for devices having displays
between 0 and 3 inches and a minimum value of a second display
class may take into account a maximum value of the first display
class (e.g., 3 inches) and set the minimum value of the second
display class at 3.1 inches, for instance. Display classes may be
changes over time based on programmer prerogative,
analysis/testing/use cases, etc.
[0046] A plurality of display classes may be set and operation 502
may determine a display class from the plurality of available
display classes. In one example, operation 502 occurs upon or
before launching of a new application or UI window or when a
display size change is detected such as when a display is moved to
a display of a different size. However, one skilled in the art will
recognize that detection of a display size and determination of a
display class may occur at any time after powering on of a
processing device. For instance, operation 502 may execute in a
background of an operating system (OS) while other applications are
operating in the foreground of the OS. This may minimize the delay
in launching of a UI instance and improve an overall user
experience.
[0047] In one example, operation 502 utilizes API accessing a
shared library of data (e.g., dynamic link library (DLL) to
determine a display class. As one example, exemplary operational
code associated with a display class determination (e.g., display
class event) is not limited to but may be similar to:
TABLE-US-00001 /** Interface to register against for display class
change events */ struct IDisplayClassInformation : public
Mso::IRefCounted { public: /** Returns the event store for display
class change events. This event store will be invoked- Whenever the
running application changes to a different display with a new
display-Whenever the active display changes its DPI */ virtual
DisplayClassChangedEvent& DisplayClassChanged( ) = 0; virtual
DisplayClass GetCurrentDisplayClass( ) const = 0; }; /** Get a
DisplayClasslnformation reference on the active UI thread */
MSOCPPAPI_(Mso::TCntPtr<Mso::DisplayClassInformation::IDisplayClassInfo-
rmation>) MakeDisplayClassInformation( );.
[0048] A processing device may be any device comprising a display
screen, at least one memory that is configured to store operations,
programs, instructions, and at least one processor that is
configured to execute the operations, programs or instructions such
as an application/application command control. Display size is a
measurement of viewable area for display on a processing device. As
an example, display size is a measurement associated with active
viewable image size of a processing device. In other examples,
display size may be associated with a nominal size value. In one
example, detecting of the display size comprises detecting a
measurement value for screen diagonal of a display of a processing
device. However, display size determination is not limited to such
an example. Factors that may be evaluated to determine a display
size include but are not limited to: screen diagonal of a display,
dot density (e.g., dots per inch (DPI), pixel density (e.g., pixels
per inch (PPI), physical size of a screen/display, use case
distance of a display from a user, display length, and display
width, among other examples. As an example, display size may be a
measurement value associated with effective resolution of a display
for a processing device. Measurement of effective resolution
enables is an example of a value used to evaluate display form
factors with a common metric, and enables UI scaling to be
classified into different display classes. However, one skilled in
the art will recognize that any common metric relative to display
size can be applied in exemplary method 500. In alternative
examples, other factors other than display size may impact UI
adaptation. Examples include but are not limited to: processing
device orientation, processing device operational mode (e.g.,
keyboard mode, touch mode, ink mode, etc.), application window
size, screen aspect ratio, and screen effective resolution, among
other examples.
[0049] In one example, operation 502 may comprise a program
instruction or module that can identify and evaluate system
specifications for a processing device such as a mobile device or
personal computer. For instance, a programming instruction
implemented in operation 502 identifies a type or version of the
processing device and executes a fetch of data to identify system
information of the processing device. In another example, a
programming instruction or module may reference manufacturer
specification information to determine a value associated with
display size of a processing device.
[0050] Flow may proceed to operation 504 where a stored UI
definition is identified and interpreted. Creating and storing of
UI definitions are described in the description of method 400 of
FIG. 4. A UI definition may define commanding objects and groupings
of commanding objects to be laid out as UI elements when a UI is
generated. As described above, a UI definition is data used to
translate the programmed command objects into displayable and
usable portions of an application user interface. In examples,
operation 504 occurs at runtime of an application/UI prior to
instantiating a user interface window for display.
[0051] In examples, operation 504 may comprise parsing the stored
UI definition and generating an object model for UI commanding upon
which an actual user interface can be laid out for display to a
user. An object model is a collection of programmed command objects
or groupings/classes through which programming operations can be
used to parse, and manipulate the programmed command objects. The
object model represents commanding objects and groupings of
commanding objects that are to be rendered as UI. In examples
describe herein, UI listeners may work with a generated object
model and be notified of user interactions with programmed command
objects.
[0052] Flow may proceed to operation 506 where a user interface in
instantiated and displayed on a processing device in a form factor
manner. Operation 506 may comprise generating a user interface that
comprises one or more user interface elements. As identified above,
UI elements are the displayable and usable portions of the UI that
act as controls to trigger actions/commands associated with a
programmed command object. In generating a displayable UI,
operation 506 programmatically generates the UI element(s) by
translating the programmed command object into the user interface
element based on operations set in accordance with the determined
display class. In examples, operation 506 may employ a component
configured to creation a UI element that triggers an
action/operation of the programmed command element, the component
execution operations to create UI elements through
programs/code/applications/APIs/modules, etc. One skilled in the
art will recognize that UI elements may have varying levels of
complexity depending on a type of UI element that is being created
and the properties of that UI including but not limited to:
actions, data and associations, button controls, combo boxes,
galleries, color, effect, caching, triggers, text/font,
height/width, style, alignment, focus, language, and margins, among
other examples. In one example, UI elements may be controls
generated in extensible application markup language (XAML).
However, one skilled in the art will recognize that UI elements may
be generated in any form including any programming language
including markup languages, among other examples.
[0053] In examples, operation 506 generates a UI and presents on a
display of a processing device based on display class rules (e.g.,
display class rule set) for interpreting the UI definition. A
display class rule set is a set of pre-defined dynamic layout rules
for display classes that account for a detected display size of a
processing device running an application/UI. A display class rule
set may be defined for generating a UI from a stored UI definition
taking into account a display class designation as determined in
operation 502. In examples, display class rule sets may be
programmed to be associated with one or more display classes. In
one example, a display class rule set may be programmed for each of
a plurality of display classes. In other examples, a display class
rule may be programmed to cover more than one display class. For
instance, a display class rule set may be utilized to set a UI
layout for small screen processing devices or large screen
processing devices, being classifications that cover multiple
display classes. Operation 506 generates and displays a UI
comprising one or more UI elements by evaluating a UI definition in
accordance with display class rules. In one example, a generated UI
is an application command control that comprises a plurality of
palettes (UI palettes) programmed for application control. A
palette is a hierarchical organization of UI elements including a
plurality of groupings of UI elements usable for UI command
control. Further description regarding palettes is provided in the
description of FIGS. 6-9.
[0054] Dynamic layout of the automatically instantiated UI windows
is accomplished by applying the display class rule set to a UI
definition. In one example, display class layout rules are provided
in a UI layout rule field or database which may be read out at the
time that a UI is being instantiated. In examples, layout rules for
display classes may be separately maintained from stored UI
definitions and application code. In that instance, programmers who
create and update commanding objects do not need to provide
consideration for scaling of programmed commanding objects across
processing devices of differing display sizes. Dynamic layout rules
for display classes may be stored in a storage, memory, database,
etc, of a processing device upon which a UI is to be instantiated
or one or more other processing devices usable to manage user
interface programming. The display class layout rules define how a
programmed command object may be translated to be generated as a UI
element. That is, presentation of a programmed command object may
vary depending on the determined display class associated with the
processing device upon which the UI is to be displayed. In
operation 506, translating of the programmed command object into
the user interface element may comprise setting a size of the user
interface element based on the operations corresponding with the
determine display class. For example, a programmed command object
may appear in a first size (e.g., number of pixels, length by
width, etc.) when displayed upon a processing device having a large
display (identified by display class determination) and the same
programmed command object may appear in a different size when
displayed upon a processing device having a small display
(identified by display class determination). In operation 506,
translating of the programmed command object into the user
interface element may comprise setting a style of the user
interface element. For example, a programmed command object may be
presented as a drop-down menu in one instance (e.g., where display
space may be limited) and the same programmed command object may be
presented more robustly as a plurality of visible icons in another
instance, depending on the layout rules defined for the display
class. In operation 506, translating of the programmed command
object into the user interface element may comprise setting a
layout of the user interface element based on the operations
corresponding with the determine display class. For example, a
grouping of programmed command objects may display only in a first
arrangement in one instance and the arrangement may be different in
another instance. That is, layout for each displayable UI window is
interpreted in light of such display class rules.
[0055] Consequently, in examples, the layout rules for any user
interface can be changed, edited, or otherwise updated any time
prior to rendering the UI window to provide uniquely configured
user interface windows. This may be accomplished without the need
to modify the underlying application code in any way. Display class
layout rules allow for dynamic resizing and repositioning of UI
windows, with a corresponding real-time automatic dynamic
adjustment of UI items within already rendered UI windows.
[0056] Flow may proceed to decision operation 508 where it is
determined whether a UI definition is updated. In one example,
display class layout rules provide for the automatic addition or
removal of one or more UI elements from a displayed. UI window
based either on user interaction with the UI window, or automatic
programmatic interaction with the IA window by one or more active
applications. That is, in examples, user of a UI may make changes
or update a UI definition and a UI may be dynamically adjusted to
account for such changes/updates. Alternatively,
developers/programmers of a UI may make changes or update a UI
definition and a UI may be dynamically adjusted to account for such
changes updates. If operation 508 determines that a displayed
definition has not been updated, flow branches NO and processing of
method 500 ends. In an example where operation 508 determines that
a UI definition has changed, flow may return to operation 504 where
the UI definition is identified and interpreted. Flow may proceed
to operation 506 where a displayed UI is instantiated based on the
updated UI definition and the display class rules.
[0057] FIG. 6 is a diagram illustrating an exemplary method 600 for
setting a layout of a generated user interface palette with which
aspects of the present disclosure may be practiced. As an example,
method 600 may be executed by an exemplary system such as shown in
FIGS. 1-3. In examples, method 600 may be executed on a device
comprising at least one processor configured to store and execute
operations, programs or instructions. However, method 600 is not
limited to such examples. In at least one example, method 600 may
be executed (e.g., computer-implemented operations) by one or more
components of a processing device or components of a distributed
network, for instance, web service/distributed network service
(e.g. cloud service). System components may be utilized to perform
the operations described herein with respect to method 600.
[0058] In examples associated with the present disclosure, a
displayed UI (e.g., application command control) hosts a majority
of an application's command set, organized in a hierarchical
structure of individual palettes. An exemplary UI may comprises a
plurality of palettes (UI palettes) programmed for application
control. A palette is a hierarchical organization of UI elements
including a plurality of groupings of UI elements usable for UI
command control. Exemplary palettes for application command control
are illustrated in FIGS. 8-9 and further described in the
description of FIGS. 7-9. Further, UI palettes may be programmed to
dynamically interact with an application and display simultaneously
with applications and/or user interface components such as an SIP
or on screen keyboard.
[0059] Flow of method 600 begins at operation 602 where a layout of
a UI palette is set or changed based on the display class rules as
applied to a UI definition. In examples, operation 602 may occur
when a UI is being generated for display on a processing device.
Dynamic layout of the automatically instantiated UI windows is
accomplished by applying a display class rule set to a UI
definition as described in the description of FIG. 5. Setting
(operation 602) a layout for a UI palette may further comprise
operations such as operations shown in operations 604-610.
[0060] In operation 604, a maximum number of rows of UI elements
for a palette are determined. Setting (operation 602) the layout of
UI palette may comprise determining (operation 604) a maximum
number of rows of UI elements for a UI palette displayed by the UI.
Operation 604 enables an UI developer to best layout palettes and
UI elements for users given the limited space that may available on
processing devices. In operation 604, scaling for display of UI
elements in UI palettes may be a programming operation associated
with a display class rule set. Operation 604 determines a maximum
number of rows of commands for a palette in association with the
display class rule set which is based on the display size of the
processing device. Operation 602 may utilize such data in
determining an optimal layout for a UI palette.
[0061] In operation 606, a maximum number of UI elements per row
may be determined for a UI palette. Setting (operation 602) the
layout of UI palette may comprise determining (operation 606) a
maximum number of UI elements per row for a UI palette displayed by
the UI. Operation 606 enables an UI developer to best layout
palettes and UI elements for users given the limited space that may
available on processing devices. In operation 606, scaling for
display of UI elements in UI palettes may be a programming
operation associated with a display class rule set.
[0062] As an example, display class rule sets may act as scaling
plateaus. For instance, evaluation of a display size of a
processing device may determine that a diagonal of a display for a
processing device is four inches wide, and the display of the
processing device has a resolution that is 360 pixels in width by
640 pixels in length. Thus, the display class rule set may
determine that for a display having an effective pixel width of 360
pixels, a maximum of 8 rows should be displayed each having a
maximum of 4 commands per row. In another example, evaluation of
the processing device may determine that a diagonal of a display
for a processing device is 5.5 inches wide, and the display of the
processing device has a resolution that is 512 pixels in width by
853 pixels in length. Thus, the application command control 402,
based on its scaling plateaus, may determine to display up to 12
rows and up to 6 commands per row of commands in palettes that are
displayed.
[0063] In operation 608, a height and/or width is determined for a
UI palette that is being generated. Setting (operation 602) the
layout of UI palette comprises determining a height/width for a UI
palette displayed by the UI. In operation 608, scaling for display
of UI elements in UI palettes may be a programming operation
associated with a display class rule set. The determined height is
an amount of space the UI palette takes up length-wise on a display
of a processing device when a UI window is in an open state
displaying the UI palette. The determined width is an amount of
space the UI palette takes up from side to side on a display of a
processing device when a UI window is in an open state displaying
the UI palette. In one example, height/width values of a palette
are set by a program developer of the UI and may be specified in
the display class rule set for generating a form factor UI to
display on a processing device. In one example, users of the UI may
be prevented from resizing a UI palette. In another example,
height/width of a palette may be adjustable, for example during
operation by a user. In any example, height/width of a UI palette
may be set for cases where the UI is displayed in different states
(e.g., open state, minimized state, invisible state, etc.). In
different states, a height/width of one palette or all palettes
displayed may be set to a predetermined value. For instance, a
palette height may be set to a percentage (e.g., 55%) of a height
of a display for the processing device when the UI is operating in
an open state. In another example, a height of a palette may be set
to different percentage (e.g., smaller percentage) of a display
when the UI is operating in a minimized state. In examples, a
height of the palette may be set as a percentage or as a fixed
number of pixels. Height or width values of a palette may also be
conditional based on launching of other software components by the
OS such an SIP, and panes. For example, conditions may be set such
as when the palette sits above an SIP or below a pane, its
height/width is equal to the remaining space for display of a UI
palette on the processing device. When a software component such as
an SIP or pane is closed or minimized, this may adjust the height
and/or width of a UI palette.
[0064] In operation 610, a size, style and layout of UI elements
may be determined for a UI palette. Setting (operation 602) the
layout of UI palette may comprise setting (operation 610) of a
size, style and layout of UI elements in a palette displayed by the
UI. Description relates to setting of a size, style and layout of a
UI element is described in operation 506 of FIG. 5. Operation 610
encompasses operations described in operation 506 as applied to
generation of a UI palette. Operation 610 enables an UI developer
to best layout palettes and UI elements for users given the limited
space that may available on processing devices. In operation 610,
scaling for display of UI elements in UI palettes may be a
programming operation associated with a display class rule set.
Setting (operation 610) of UI elements comprises setting a layout
for at least one UI palettes such as a top-level palette and/or
drill-in palette. Each of the top-level palettes and the drill-in
palettes is a collection or grouping comprising one or more
selectable UI elements. An exemplary top-level palette is shown in
FIG. 8 and described in the description of FIG. 8. An exemplary
drill-in palette is shown in FIG. 9 and described in the
description of FIG. 9.
[0065] Operation 610 may utilize a UI definition in determining a
layout of a UI palette. A UI definition may specify how programmed
command objects can be grouped or nested and actual layout of the
command objects as UI elements may be set by applying a display
class rule set associated with a determined display class. In
examples, a UI definition may comprise command grouping data being
information relating to the grouping of command objects including
associations between command objects. For example, text editing
features such as bolding, underlining, italicization, superscript
and subscript may be associated and commonly used. Ideally, a UI
window would like to include all of these commonly used functions
on the same UI palette. However, due to limitations on the screen
size, certain commands may need to be separated. Command grouping
data is information that identifies associations and what commands
should or should not be separated from each other. For example, it
may be determined that the maximum number of rows and commands
allows displaying of text formatting commands including a
superscript editing command in a top-level palette but would not
also allow displaying of a subscript command. Using the command
grouping data, it may be identified that from a functionality
and/or usability standpoint, that it is best not to separate the
superscript and subscript editing commands. For instance, a user
who makes a subscript text edit may later look to make a
superscript edit or visa-versa. Thus, in setting the layout of
commands for palettes, operation 610 may take into account command
grouping data of a UI definition to determine an optimal UI display
window that accounts for the display size of a device upon which
the UI is displaying. In one example, operation 610 may display a
higher-level command for text editing in a top-level palette and
the superscript and subscript editing commands may be included in a
drill-in palette (child palette) of that top-level palette (parent
palette) so they such commands remain associated.
[0066] Once a UI palette is set and instantiated, flow may proceed
to decision operation 612 where it is determined whether a display
size change has occurred. If a display size change is not detected
flow branches NO and method 600 ends. Method 600 may be
implemented/re-implemented any time a UI window is to be
instantiated or a display size change is detected. If a display
size change is detected (e.g., resolution change or connection of
another device is detected) flow branches YES and returns back to
operation 602 where a layout of palettes of a UI may be set or
changed.
[0067] FIG. 7 is a diagram illustrating display for exemplary
processing devices of different sizes with which aspects of the
present disclosure may be practiced. Examples shown in FIG. 7
comprise processing devices having varying sizes and/or varying
screen/display sizes, for example processing device 702, processing
device 704, processing device 706 and processing device 708.
[0068] As shown in FIG. 7, a UI control (e.g., application command
control) and an application/canvas are displayed in exemplary
processing devices 702-708. An application command control and an
application/canvas are examples of components of a UI with which
the present disclosure may apply. In examples, the UI is programmed
to efficiently scale itself to utilize display space of processing
devices of different sizes and/or operating size of display
windows. For example, presentation of a UI control and/or
application/canvas may vary across the different processing devices
702-708. UI control and/or an application/canvas may be scaled
according to a determined display class associated with a
processing device.
[0069] An application/canvas is a portion of a display of a
processing device that is designated for display of an application
executing on the device. The application/canvas region is the
application UI that shows effects implemented by actions executed
via a UI control. That is, the application/canvas is the content
consisting of but not limited to the pages in workspace or editable
portions of an application.
[0070] A UI control (application command control) hosts a majority
of an application's command set, organized in a hierarchical
structure of individual palettes, chunks, and commands. Further,
application command control may be programmed to dynamically
interact with an application and display simultaneously with
applications and/or user interface components such as a soft input
panel (SIP) or on screen keyboard. In one example, a UI control may
intelligently adapt based on content of an application (e.g.,
displayed or selected on an application canvas). A UI control
comprises a plurality of palettes (command palettes) programmed for
application control. In one example, palettes of an application
command control comprise top-level palettes and drill-in palettes.
Each of the top-level palettes and the drill-in palettes is a
collection or grouping of rows comprising one or more selectable UI
elements. As an example, a top-level palette may comprise a highest
level grouping of UI elements including commands that are more
frequently used/more likely to be used by users. A top-level
palette may display command listings that can be drilled into and
displayed in drill-in palettes. FIG. 8 illustrates an exemplary
top-level palette of an application command control. A drill-in
palette is a collection or grouping of commands that may be used
less frequently/or likely to be used less frequently compared to
the commands displayed on a top-level palette. As an example,
drill-in palettes host over-flow commands that, due to constraints
resulting from a limited amount of display space for an application
command control, are not included in a top-level palette. FIG. 9
illustrates an exemplary drill-in palette of an application command
control. Using a word processing application as an exemplary
application, a top-level palette may comprise high-level UI
elements for text editing, font editing, paragraph formatting, word
finder, spell-check etc. that may be frequently used by users of an
application. As an example, a drill-in palette for a word
processing application may comprise sub-elements of such high-level
commands of the top-level palette, for example, subscript or
superscript commands for a font command/function. In examples,
organization of palettes and UI elements may be editable, for
example, where an element or a chunk of a palette can be pulled
from one palette and added/displayed in another. For instance, an
overflow command of a drill-in palette can be added to a top-level
palette.
[0071] Examples of common components that make up a top-level
palette or a drill-in palette include but are not limited to: a
palette bar and palette title, palette switching feature (including
one touch target that launches palette switcher from title of
palette bar), UI element to dismiss palette (e.g., visual
representation of ellipses), quick command elements (e.g., undo or
redo), palette canvas comprising a plurality of elements, chunk UI
elements (e.g., groupings of commands) and chunk dividers (e.g.,
dividing different groupings of commands), drill-in features to
access drill-in palettes (when applicable).
[0072] In one example, palettes of a UI are presented in a vertical
layout. For example, a top-level palette and a drill-in palette are
vertically scrollable and comprise a collection of rows comprising
one or more selectable UI elements. However, in other examples,
setting of the layout of a palette may also comprise presenting UI
elements in a horizontal layout where palettes are horizontally
scrollable. In some examples, no limit is set on the scrollable
height of a palette. Scrolling position may be kept on top-level
palettes when switching between top-level palettes however
scrolling position may or may not be kept for drill-in palettes. UI
elements set and displayed may include labels identifying an
element and may be configured to take up an entire row of a
palette. In other examples, multiple elements may be displayed in
one row of a palette. Scaling is applied to setting and displaying
commands in palette rows. In some other examples, UI elements may
not have labels, for example, commands that are well known or have
images displayed that are well known to users. Separators or
spacers (either horizontal or vertical depending on layout of
palette) may be displayed to break up different elements or chunks
of elements.
[0073] In FIG. 8, UI control is illustrated an exemplary top-level
palette 802. In FIG. 9, UI control is illustrated an exemplary
drill-in palette 902. For example, UI control shown in FIG. 9 is a
drill-in palette 902 of the UI top-level palette 802 shown in FIG.
8. That is, UI top-level palette 802 is a parent palette of the UI
drill-in palette 902 (e.g., child palette of the top-level
palette). As shown in top-level palette 802, a row showing a "font
formatting" command includes a caret indicative of a drill-in
feature. When the drill-in feature is selected, a drill-in palette
of drill-in palette 902 is displayed on a display of a processing
device. For instance, in FIG. 9, font formatting sub-elements
"superscript" and "subscript" are displayed. In this way,
application command control and/or an application/canvas may be
scaled in accordance with a determined display class associated
with a processing device.
[0074] Reference has been made throughout this specification to
"one example" or "an example," meaning that a particular described
feature, structure, or characteristic is included in at least one
example. Thus, usage of such phrases may refer to more than just
one example. Furthermore, the described features, structures, or
characteristics may be combined in any suitable manner in one or
more examples.
[0075] One skilled in the relevant art may recognize, however, that
the examples may be practiced without one or more of the specific
details, or with other methods, resources, materials, etc. In other
instances, well known structures, resources, or operations have not
been shown or described in detail merely to observe obscuring
aspects of the examples.
[0076] While sample examples and applications have been illustrated
and described, it is to be understood that the examples are not
limited to the precise configuration and resources described above.
Various modifications, changes, and variations apparent to those
skilled in the art may be made in the arrangement, operation, and
details of the methods and systems disclosed herein without
departing from the scope of the claimed examples.
* * * * *