U.S. patent application number 13/893031 was filed with the patent office on 2014-03-13 for automated graphical user-interface layout.
This patent application is currently assigned to APPLE INC.. The applicant listed for this patent is Apple Inc.. Invention is credited to Brian R. Frick, Joe R. Howard, Timothy B. Martin, Christopher John Sanders.
Application Number | 20140075324 13/893031 |
Document ID | / |
Family ID | 50233242 |
Filed Date | 2014-03-13 |
United States Patent
Application |
20140075324 |
Kind Code |
A1 |
Howard; Joe R. ; et
al. |
March 13, 2014 |
Automated Graphical User-Interface Layout
Abstract
Embodiments of the present disclosure relate to automatic
generation of dynamically changing layouts for a graphical
user-interface. Specifically, embodiments of the present disclosure
employ analysis of an image associated with the view (e.g., either
the current view or a future view) of the graphical user-interface
to determine colors that are complementary to the image. The colors
are applied to the view, such that the color scheme of the view
matches the image.
Inventors: |
Howard; Joe R.; (San Jose,
CA) ; Frick; Brian R.; (Gaithersburg, MD) ;
Martin; Timothy B.; (Morgan Hill, CA) ; Sanders;
Christopher John; (San Jose, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Apple Inc. |
Cupertino |
CA |
US |
|
|
Assignee: |
APPLE INC.
Cupertino
CA
|
Family ID: |
50233242 |
Appl. No.: |
13/893031 |
Filed: |
May 13, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61699758 |
Sep 11, 2012 |
|
|
|
Current U.S.
Class: |
715/738 ;
715/765 |
Current CPC
Class: |
H04L 67/10 20130101;
H04L 67/42 20130101; G06F 9/451 20180201; G06F 3/04817 20130101;
G06T 2200/24 20130101; G09G 2340/12 20130101; G06T 11/60 20130101;
G06F 3/0484 20130101; G09G 5/026 20130101; G09G 5/377 20130101;
H04L 12/18 20130101 |
Class at
Publication: |
715/738 ;
715/765 |
International
Class: |
G06F 3/0484 20060101
G06F003/0484 |
Claims
1. A non-transitory computer-readable medium, comprising
computer-readable instructions to: provide a graphical
user-interface to display content; analyze an image associated with
a current view of the content to determine a set of colors found in
the image; and update a layout of the current view of the graphical
user-interface based at least in part upon the set of colors.
2. The non-transitory computer-readable medium of claim 1,
comprising computer readable instructions to: determine a primary
text color from the set of colors; determine a secondary text color
from the set of colors; apply the primary text color to a first
level of text in the graphical user-interface; and apply the
secondary text color to a second level of text in the graphical
user-interface.
3. The non-transitory computer-readable medium of claim 2,
comprising computer readable instructions to: generate a tertiary
color by modifying a characteristic of the primary or secondary
text color; and apply the tertiary color to a third level of text
in the graphical user-interface.
4. The non-transitory computer-readable medium of claim 3
comprising modifying a transparency characteristic, a brightness
characteristic, or both of the primary or secondary text color to
generate the tertiary color.
5. The non-transitory computer-readable medium of claim 2,
comprising computer readable instructions to: update the layout
using a default set of colors when analysis of the image does not
result in a suitable set of colors for the layout.
6. The non-transitory computer-readable medium of claim 1,
comprising computer readable instructions to: determine a
background color from the set of colors, determine a background
image based upon the analysis of the image, or both.
7. The non-transitory computer-readable medium of claim 1,
comprising computer readable instructions to: determine and apply a
background image for the current view based upon the image
associated with the current view; wherein the background image is a
faded version of the image associated with the current view that
becomes increasingly faded towards a body of the current view.
8. The non-transitory computer-readable medium of claim 1,
comprising computer readable instructions to: detect whether the
image associated with the current view contains a border; crop the
border if one is detected; and determine a background color from
the image after the boarder has been cropped.
9. The non-transitory computer-readable medium of claim 1; wherein
the graphical user-interface is configured to display media content
comprising audio content, video content, information relating to
the audio content, information relating to the video content, or a
combination thereof;
10. A system comprising: a server comprising a processor configured
to provide content comprising an image associated with a user
selection in a content player hosted by a client; and a layout
generator configured to generate machine-readable layout
instructions configured to define changes to at least a portion of
a layout of a graphical user interface of the content player based
upon the image.
11. The system of claim 10, comprising the client, wherein the
client comprises the layout generator.
12. The system of claim 10, wherein the server comprises the layout
generator.
13. The system of claim 10, comprising a non-transitory cache
storage configured to store the layout instructions for subsequent
retrieval and use by the content player.
14. The system of claim 13, wherein the cache storage comprises
non-transitory cloud-based storage.
15. The system of claim 13, wherein the server, the client, or both
comprise the cache storage.
16. A method, comprising: obtaining, via a processor, an input
image; analyzing, via the processor, the input image for potential
background colors for a graphical user interface; analyzing, via
the processor, the input image to determine dominant colors of the
input image; determining, via the processor, a set of graphical
user interface element colors based upon the dominate colors, by
determining which of the dominant colors or variations of the
dominant colors compliment the potential background colors;
applying, via the processor, one of the potential background colors
to the graphical user interface; and applying, via the processor,
the set of graphical user interface element colors to one or more
elements of the graphical user interface.
17. The method of claim 16, wherein analyzing the input image for
potential background colors comprises: resizing the input image;
generating hue and grayscale histograms from edges of the input
images, wherein some edges of the input images may be weighted with
higher importance; condensing the hue and grayscale histograms;
choosing dominant hue and dominate grayscale values from the hue
and grayscale histograms; selecting the higher of the dominant hue
or the dominate grayscale values; determining a dominate color
based upon the dominate hue when the dominate hue is selected;
validating the dominant color when the dominant color is determined
or validating the dominant grayscale when the dominate grayscale is
selected; and setting the dominant color or the dominate grayscale
as a background color of the graphical user interface when the
validation is successful.
18. The method of claim 17, wherein validating the dominant color
or dominant grayscale comprises: selecting multiple locations along
each edge of the input image; repeating for each edge of the input
image: moving from the edge of the locations inward to compare
pixel colors with the dominant color or dominant grayscale; finding
a length of a border for an edge location when the pixel colors
contrast with the dominant color or dominant grayscale; sorting the
locations based on the length of the border; comparing the longest
and shortest border lengths for each edge; and determining that the
edge is a border when the length difference is within a certain
threshold or the longest length is zero or otherwise determining
that the edge is not a border; determining that the input image is
bordered when 3 or 4 edges are a border and otherwise determining
that the input image is not bordered; determining that the dominant
color or dominant grayscale is not valid, cropping the image by the
longest border length, and repeating analysis of the input image
for potential background colors when the image is bordered; and
determining that the dominant color or dominant grayscale is valid
when the input image is not bordered.
19. The method of claim 17, wherein edges are weighted based upon
where the input image will be displayed on the graphical user
interface.
20. The method of claim 16, wherein analyzing the input image to
determine the dominant colors, comprises: generating hue and
grayscale histograms; compressing the hue and grayscale histograms;
filtering the hue and grayscale histograms; sorting the hue and
grayscale histograms; determining dominate colors for each hue in
the hue historgram; creating grayscale color based upon the
grayscale historgram; and merging the dominate colors for each hue
and the grayscale color based on a dominate of the dominate colors
in the input image.
21. The method of claim 16, wherein the one or more elements
comprise primary text, secondary text, or both and the element
colors comprise a primary text color, a secondary text color, or
both.
22. The method of claim 21, comprising determining the primary text
color by: iterating through the dominant colors in the input image
to determine whether any of the dominant colors contrasts with the
background color; choosing the primary text color from one of the
dominate colors that contrasts with the background color or
choosing a fallback color for the primary text color when none of
the dominate colors contrasts with the background color.
23. The method of claim 22, wherein determining the primary text
color, comprises artistically modifying the primary text color.
24. The method of claim 21, comprising determining the secondary
text color by: iterating through the dominant colors in the input
image to determine whether any of the dominant colors is close in
color to the primary text color and contrasts with the background
color; selecting the secondary text color from the dominant colors
that are not close to the primary text color and contrast with the
background color; when all of the dominant colors are close to the
primary text color and do not contrast with the background color,
iterating through the dominant colors in the input image to
determine whether any of the dominant colors contrasts with the
background color; selecting the secondary text color from one of
the dominant colors that contrast with the background color; when
none of the dominant colors contrast with the background color,
selecting the secondary text color from a fallback color.
25. The method of claim 24, comprising artistically modifying the
secondary text color.
Description
BACKGROUND
[0001] This application is a Non-Provisional of U.S. Provisional
Patent Application No. 61/699,758, entitled "Automated Graphical
User-Interface Layout", filed Sep. 11, 2012, which is herein
incorporated by reference.
[0002] The present disclosure relates generally to a graphical
user-interface, and, more particularly, to automatically generating
a layout of the graphical user-interface.
[0003] This section is intended to introduce the reader to various
aspects of art that may be related to various aspects of the
present disclosure, which are described and/or claimed below. This
discussion is believed to be helpful in providing the reader with
background information to facilitate a better understanding of the
various aspects of the present disclosure. Accordingly, it should
be understood that these statements are to be read in this light,
and not as admissions of prior art.
[0004] As the prevalence of electronic devices has increased,
content, such as movies, music, images, etc., has largely become
available in a digital format, displayable by these electronic
devices. Accordingly, digital content players for electronic
devices have become very prevalent. Because of the popularity of
these players, manufacturers of these content players oftentimes
strive to enhance the user experience. Unfortunately, despite this
focus, content players generally have a static layout (e.g., static
color-scheme, background images, etc.) that does not change based
upon content that the content player is displaying.
SUMMARY
[0005] A summary of certain embodiments disclosed herein is set
forth below. It should be understood that these aspects are
presented merely to provide the reader with a brief summary of
these certain embodiments and that these aspects are not intended
to limit the scope of this disclosure. Indeed, this disclosure may
encompass a variety of aspects that may not be set forth below.
[0006] Embodiments of the present disclosure relate to devices and
methods for dynamically altering a graphical user-interface layout
based upon an icon representing a currently displayed portion of
the graphical user-interface. In some embodiments, a digital
content player may define layout parameters based upon cover art or
another image associated with the currently displayed content.
Accordingly, the layout of content player may be dynamically
enhanced to compliment the currently displayed digital content.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Various aspects of this disclosure may be better understood
upon reading the following detailed description and upon reference
to the drawings in which:
[0008] FIG. 1 is a block diagram of components of an electronic
device, in accordance with an embodiment;
[0009] FIG. 2 is a schematic diagram of an example of an electronic
device, in accordance with an embodiment;
[0010] FIG. 3 is a flowchart describing a process for applying a
dynamic layout, in accordance with an embodiment;
[0011] FIG. 4 illustrates a system where a server, external to the
content player, generates a layout to be displayed by the content
player, in accordance with an embodiment;
[0012] FIG. 5 illustrates a system where a client, hosting the
content player, generates the layout, in accordance with an
embodiment;
[0013] FIG. 6 illustrates a flowchart illustrating a process for
determining background and dominant colors from an image associated
with the digital content, in accordance with an embodiment;
[0014] FIG. 7 illustrates a process for selecting a primary text
color from the dominant colors of the process of the process of
FIG. 6, in accordance with an embodiment;
[0015] FIG. 8 illustrates a process for selecting a secondary color
from the dominant colors of the process of FIG. 6, in accordance
with an embodiment;
[0016] FIG. 9 illustrates an example of a content player graphical
user-interface, using the colors determined in the processes of
FIGS. 7 and 8, in accordance with an embodiment; and
[0017] FIG. 10 is a flowchart illustrating a process for edge
analysis of the input image, such that borders may be cropped, in
accordance with an embodiment.
DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS
[0018] One or more specific embodiments will be described below. In
an effort to provide a concise description of these embodiments,
not all features of an actual implementation are described in the
specification. It should be appreciated that in the development of
any such actual implementation, as in any engineering or design
project, numerous implementation-specific decisions must be made to
achieve the developers' specific goals, such as compliance with
system-related and business-related constraints, which may vary
from one implementation to another. Moreover, it should be
appreciated that such a development effort might be complex and
time consuming, but would nevertheless be a routine undertaking of
design, fabrication, and manufacture for those of ordinary skill
having the benefit of this disclosure.
[0019] The present technique allows for the dynamic creation of a
layout for a graphical user-interface based upon an icon associated
with the current view of the graphical user-interface. For example,
a content player, such as the iTunes.RTM. application, available
from Apple Inc. of Cupertino, Calif., may be enabled to provide a
dynamic layout based upon an icon, such as a user-supplied graphic
or content cover art. The dynamic layout may include changes to the
background image and/or color as well as text color based upon the
icon.
[0020] With these foregoing features in mind, a general description
of suitable electronic devices for implementing aspects of the
present techniques are described below. In FIG. 1, a block diagram
depicting various components that may be present in electronic
devices suitable for use with the present techniques is provided.
In FIG. 2, one example of a suitable electronic device, here
provided as a desktop computer, is depicted. This type of
electronic device, and other electronic devices providing
comparable storage and/or processing capabilities, may be used in
conjunction with the present techniques. For example, these and
similar types of electronic devices may be used to display a
dynamic layout graphical user-interface, in accordance with the
teachings of the present disclosure.
[0021] An example of a suitable electronic device may include
various internal and/or external components which contribute to the
function of the device. FIG. 1 is a block diagram illustrating the
components that may be present in such an electronic device 10 and
which may allow the device 10 to function in accordance with the
techniques discussed herein. The various functional blocks shown in
FIG. 1 may comprise hardware elements (including circuitry),
software elements (including computer code stored on a
machine-readable medium) or a combination of both hardware and
software elements. It should further be noted that FIG. 1 is merely
one example of a particular implementation and is merely intended
to illustrate the types of components that may be present in a
device 10. For example, in the presently illustrated embodiment,
these components may include a display 12, I/O ports 16, input
structures 18, one or more processors 20, a memory device 22, a
non-volatile storage 24, a networking device 26, and a power source
28.
[0022] With regard to each of these components, the display 12 may
be used to display various images generated by the device 10. The
display 12 may be any type of display such as a cathode ray tube
(CRT), a liquid crystal display (LCD), a light emitting diode (LED)
display, an organic light emitting diode (OLED) display, or other
suitable display. Additionally, in certain embodiments of the
electronic device 10, the display 12 may include a touch-sensitive
element, such as a touch screen.
[0023] The I/O ports 16 may include ports configured to connect to
a variety of external devices, such as a power source or other
electronic devices (such as handheld devices and/or computers,
printers, projectors, external displays, modems, docking stations,
and so forth). The I/O ports 16 may support any interface type,
such as a universal serial bus (USB) port, a video port, a serial
connection port, a IEEE-1394 port, an Ethernet or modem port,
and/or an AC/DC power connection port.
[0024] The input structures 18 may include the various devices,
circuitry, and pathways by which user input or feedback is provided
to the processor 20. Such input structures 18 may be configured to
control a function of the device 10 when actuated. For example, the
input structures 18 may include buttons, sliders, switches, control
pads, keys, knobs, scroll wheels, keyboards, mice, touchpads, and
so forth.
[0025] In certain embodiments, an input structure 18 and display 12
may be provided together, such as in the case of a touchscreen
where a touch sensitive mechanism is provided in conjunction with
the display 12. In such embodiments, the user may select or
interact with displayed interface elements via the touch sensitive
mechanism. In this way, the displayed user interface may provide
interactive functionality, allowing a user to select, by touch
screen or other input structure, from among options displayed on
the display 12.
[0026] User interaction with the input structures 18, such as to
interact with a user or application interface displayed on the
display 12, may generate electrical signals indicative of the user
input. These input signals may be routed via suitable pathways,
such as an input hub or bus, to the processor(s) 20 for further
processing.
[0027] The processor(s) 20 may provide the processing capability to
execute the operating system, programs, user and application
interfaces, and any other functions of the electronic device 10.
The processor(s) 20 may include one or more microprocessors, such
as one or more "general-purpose" microprocessors, one or more
special-purpose microprocessors and/or ASICS, or some combination
of such processing components. For example, the processor 20 may
include one or more reduced instruction set (RISC) processors, as
well as graphics processors, video processors, audio processors
and/or related chip sets.
[0028] The instructions or data to be processed by the processor(s)
20 may be stored in a memory 22. The memory 22 may be provided as a
volatile memory, such as random access memory (RAM), and/or as a
non-volatile memory, such as read-only memory (ROM). The memory 22
may store a variety of information and may be used for various
purposes. For example, the memory 22 may store firmware for the
electronic device 10 (such as a basic input/output instruction or
operating system instructions), other programs that enable various
functions of the electronic device 10, user interface functions,
processor functions, and may be used for buffering or caching
during operation of the electronic device 10.
[0029] The components may further include a non-volatile storage 24
for persistent storage of data and/or instructions. The
non-volatile storage 24 may include flash memory, a hard drive, or
any other optical, magnetic, and/or solid-state storage media. The
non-volatile storage 24 may be used to store data files such as
personal information, software (e.g., an application used to play
digital content on the electronic device 10 or on another
electronic device), wireless connection information (e.g.,
information that may enable the electronic device 10 to establish a
wireless connection, such as a telephone or wireless network
connection), and any other suitable data.
[0030] The components depicted in FIG. 1 also include a network
device 26, such as a network controller or a network interface card
(NIC). In one embodiment, the network device 26 may be a wireless
NIC providing wireless connectivity over any 802.11 standard or any
other suitable wireless networking standard. The network device 26
may allow the electronic device 10 to communicate over a network,
such as a Local Area Network (LAN), Wide Area Network (WAN), or the
Internet. Further, the electronic device 10 may connect to and send
or receive data with any device on the network, such as portable
electronic devices, personal computers, printers, and so forth.
Alternatively, in some embodiments, the electronic device 10 may
not include a network device 26. In such an embodiment, a NIC may
be added as an expansion card to provide similar networking
capability as described above.
[0031] Further, the components may also include a power source 28.
In one embodiment, the power source 28 may be one or more
batteries, such as a lithium-ion polymer battery. The battery may
be user-removable or may be secured within the housing of the
electronic device 10, and may be rechargeable. Additionally, the
power source 28 may include AC power, such as provided by an
electrical outlet, and the electronic device 10 may be connected to
the power source 28 via a power adapter. This power adapter may
also be used to recharge one or more batteries if present.
[0032] FIG. 2 illustrates an electronic device 10 in the form of a
desktop computer 50, such as an iMac.RTM. by Apple Inc., that may
be used to generate and/or present a dynamic layout graphical
user-interface. It should be noted that while the techniques will
be described below in reference to illustrated electronic device 50
(which may be a desktop computer), the techniques described herein
are usable with any electronic device employing a display. For
example, other electronic devices may include a laptop computer, a
tablet computer, a viewable media player, a mobile phone, a
personal data organizer, a workstation, a standalone display, or
the like. In certain embodiments, the electronic device may include
a model of an iMac.RTM., Mac.RTM. mini, Mac Pro.RTM., MacBook.RTM.,
a MacBook.RTM. Pro, MacBook Air.RTM., Apple Cinema Display.RTM.,
Apple Thunderbolt Display.RTM., iPad.RTM., iPod.RTM. or iPhone.RTM.
available from available from Apple Inc. of Cupertino, Calif. In
other embodiments, the electronic device may include other models
and/or types of electronic devices, available from any
manufacturer.
[0033] As illustrated in FIG. 2, the desktop computer 50 includes a
housing 52 that supports and protects interior components, such as
processors, circuitry, and controllers, among others, that may be
used to generate images to display on display 12. Desktop computer
50 also includes user input structures 18, shown here as a keyboard
and a mouse, that may be manipulated by a user to interact with
desktop computer 50. For example, user input structures 18 may be
employed to operate a graphical user interface (GUI) 54 of an
operating system running on the desktop computer 50 as well as a
GUI 56 of applications running on desktop computer 50. Input
structures 18 may be connected to the electronic device 10 through
a wired or wireless configuration. Further, in certain embodiments,
electronic device 10 may include other types of user input
structures, such as a touchscreen or trackpad, among others.
[0034] The GUI 54 may include various layers, windows, screens,
templates, or other graphical elements that may be displayed in
all, or a portion, of the display 12. Generally, the GUI 54 may
include graphical elements 58 that represent applications and
functions of the electronic device. The graphical elements 58 may
include icons and other images representing buttons, sliders, menu
bars, and the like. The icons may correspond to various
applications of the electronic device that may open upon selection
of a respective icon. Furthermore, selection of an icon may lead to
a hierarchical navigation process, such that selection of an icon
leads to a screen that includes one or more additional icons or
other GUI elements 58. The icons may be selected via a touch screen
included in the display 12, or may be selected by a user input
structure 18, such as a wheel or button.
[0035] When an icon is selected, the desktop computer 50 may be
configured to open an application associated with that icon and
display a corresponding screen. For example, when the "iTunes" icon
60 is selected, the desktop pc 50 may be configured to open an
application GUI 56 for the iTunes.RTM. application.
[0036] As discussed above the GUI 54 and/or the GUI 56 may include
dynamic layout functionality that enables the GUI 54 and/or 56 to
dynamically change the GUI layout based upon an icon that
represents the current view of the GUI 54 and/or 56. For example,
the processor 20 of the desktop computer 50 may be enabled to
analyze album artwork or other images displayed in the GUI 56 to
determine a complementary layout (e.g., a layout with a color
scheme that at least partially matches the displayed image) and
apply the complementary layout to the GUI 56. Similarly, the layout
of the operating system GUI 54 may be dynamically altered based
upon an analysis of the background image 62 displayed in the GUI
54.
[0037] FIG. 3 illustrates a process 90 for applying a dynamic
layout to a GUI based upon a graphical image displayed in the GUI.
The process 90 may be executed by a computer, such as the computer
50 of FIG. 2. In some embodiments, a setting in the GUI may
initiate the process 90. For example, the GUI may include an option
to activate and/or deactivate dynamic layout adjustments. When
activated, the process 90 may be implemented by the computer 50.
When deactivated, the computer 50 may, at least partially, halt the
process 90. For example, the computer 50 may halt the entire
process 90 or may continue to determine complementary layouts for
the different GUI views, but not apply the complementary views
until the dynamic layout adjustments are re-activated.
[0038] Process 90 begins by obtaining an input image (block 92).
The input image may be an image related to a user selection and/or
the current view within the GUI. For example, in an iTunes.RTM.
library, a user may select a particular piece of digital content,
such as a movie, song, album, etc. In other cases, the user may
select a genre, artist, or other page related to digital content.
The iTunes.RTM. GUI may provide a view pertaining to the user's
particular selection. In such cases, an image associated with the
user's selection may be prominently displayed on the view. Because
the image is prominently displayed, it may be desirable to
coordinate the layout of the current view with the image.
Accordingly, the input image may be prominently displayed image of
the current view.
[0039] Once the input image has been obtained, an analysis of the
image is performed to determine potential background colors as well
as colors for other elements of the GUI (block 94). For example, as
will be described in more detail below with regards to FIG. 6, a
computer may determine a background color as well as other dominant
colors of the input image. Further these potential color choices
may be further extended by developing additional colors through one
or more modifications to characteristics of the potential color
choices. For example, the translucence and/or brightness of a
potential color choice may be modified to create an additional
color that may be used in the GUI.
[0040] Once the background color is determined, the computer may
determine a combination of the potential color choices that would
be within acceptable parameters for a color scheme of the layout
(block 96). For example, the computer may include instructions that
describe optimal color combinations and/or color combinations to
avoid. For example, the combination of certain red and blues may
create a "push-in" or "pop-out" effect that is undesirable. The
instruction may specifically avoid these effects by limiting the
blue and red combinations that are acceptable color combination
selections.
[0041] Further, the instructions may include characteristics useful
for a desirable color scheme and/or characteristics that do not
result in a desirable color scheme. For example, the instruction
may provide that high-contrast colors create a desirable color
scheme and/or that low-contrast colors result in an undesirable
color scheme. Many different attributes may be included in
instructions used for color selection. In some embodiments, the
instructions may factor for multiple color attributes working in
combination with one another. For example, the instructions may
dictate that for high-contrast color selections, an increased
amount of transparency may be allowed, while less transparency may
be allowed when low-contrast colors are selected.
[0042] Once the color selections are made, the layout (e.g., the
background color, the color selections, and/or a version of the
input image) are applied to the GUI (block 98). For example, in
some embodiments a single background color and two additional color
selections are determined (e.g., a primary color selection and a
secondary color selection). Based upon the computer's selections,
the GUI layout may be altered. For example, higher prominence text
may have the primary color selection applied, while less prominent
text may have the secondary color applied. Additionally, in some
embodiments, a tertiary color may also be generated based upon
changing a color attribute of one of the color selections (e.g.,
modifying brightness, transparency, or other attribute of the color
selection). This tertiary color may be used to provide an
additional color choice for lower prominence text.
[0043] FIGS. 4 and 5 illustrate potential locations where the
dynamic layout may be determined In FIG. 4, the layout is generated
external to a client computer hosting the content player and in
FIG. 5 the layout is generated at the same client that hosts the
content player. Starting first with FIG. 4, a digital content
distribution system 120 is illustrated. The system 120 includes a
client computer 122 that hosts a content player 124, such as
iTunes.RTM., that is enabled to apply a dynamic layout. In the
illustrated embodiment, the client 122 is communicatively coupled
via a communications network 126 (e.g., the Internet), to a content
distribution server 128, such as the iTunes.RTM. store hosted by
Apple Inc. The server 128 may include one or more services 130 to
supply digital content and/or content information to the client
122. Further, the server 128 may include a service 132 that
generates layouts for particular pieces of content and/or content
information views, based upon image information obtained by the
server 128. For example, for each piece of content and/or content
information (e.g., album, song, artist, genre, etc.) that has an
icon/image, the server 128 may attempt to generate a layout 134,
according to the process 90 described in FIG. 3. The generated
layout 134 may be associated with the digital content/content
information and may be served directly to the client 122 (as
illustrated by the arrow 136 or may be stored in cloud storage 138
(as illustrated by arrow 140) for retrieval by the client 122,
either as a batch process (e.g., download multiple layouts at once)
or as an instantaneous process (e.g., download the currently needed
layout for the current view displayed in the content player
124).
[0044] Turning now to FIG. 5, a content distribution system 160 is
provided. In the current embodiment, the client 162 hosts the
content player 124 and also hosts a layout generation service 164
that is enabled to generate layouts based upon the current view of
the content player 124 and/or the content stored in the content
player 124.
[0045] The client 162 may be optionally connected, via a
communications network 166 (e.g., the Internet), to a server 168
that hosts a service 170 for distributing digital content and/or
digital content information (e.g., the iTunes.RTM. store). When
connected to the server 168, the client may provide digital content
and/or content information selections 172 to the server 168. In
return, the server 168 may provide a content graphic 174, such as
related album artwork, an artist's picture, a genre icon, etc. back
to the client 162. Upon receiving the content graphic 174, the
client may generate a layout using the content graphic 174.
[0046] In some embodiments, the server 168 may provide the content
graphic 174 without receiving a particular content selection 172
from the user. For example, a list content and/or content
information selections may be provided from the client 162 based
upon content stored in the media player 124. Accordingly, the
client may receive content graphics 174 for these pieces of digital
content and/or content information and generate layouts in the
background, such that they can be applied the next time the user
selects a piece of content in the content player 124.
[0047] When offline (e.g., the client 162 is not connected to the
server 168) and/or when an image is already associated with digital
content and/or content information at the client 162,
communications with server 168 may be unnecessary. Indeed, the
pre-existing image may be used as the input image, enabling a
dynamic layout to be generated without any interaction with the
server 168.
[0048] The layouts derived by the layout generator may be cached,
thus reducing a need to generate layouts every time a user
selection changes. For example, results from the layout generator
may be cached in the cloud storage and/or at the client 162 or
server 168.
[0049] Having generally discussed the process of applying a dynamic
layout and where the input image may be provided from, the
discussion now turns to a more detailed look at color selection
based upon the input image. FIG. 6 is a flowchart 190 depicting
processes 192 and 194 for determining a background color and
dominant colors, respectively. To initiate the processes 192 and
194, the input image may be resized to a size that will be
displayed in the GUI and/or a uniform size expected by the
computer-implemented algorithm executing processes 192 and 194
(block 196).
[0050] After the input image is resized, the background color may
be selected using process 192. First, hue and grayscale histograms
from the input image edges are generating, taking into account edge
weighting (blocks 198 and 200, respectively). The hue may describe
a particular color's association with a defined color, while the
grayscale may define a particular color's intensity. Depending on
the position of the image within the GUI view, different portions
(e.g., the edges and/or the center of the image) may be weighted
differently, in terms of importance in determining colors. For
example, edge weighting may be defined when the input image will be
displayed on the upper-left corner of the GUI. For example, it may
be more important to focus on the lower and right edges of the
input image, as these edges may be closer to text or other layout
elements that may be affected by dynamic layout changes based upon
the input image. Further, as will be discussed in more detail
below, the input image may be used as a background image that fades
into the background color. Accordingly, the edges that are more
centrally located in the background may be of more importance in
determining a background color. In some embodiments, the center of
an image may have a relatively high weight, especially when the
image covers an entire portion of the GUI.
[0051] Once the hue and grayscale histograms are generated, the
histograms are condensed, breaking the colors down into less
granular color selections (blocks 202 and 204, respectively). Based
upon the condensed hue and grayscale histograms, dominant hues and
grayscales are selected (blocks 206 and 208, respectively). The
dominant hue or grayscale is chosen (block 210). If the dominant
hue is chosen instead of the grayscale value, the dominant color
will be determined based upon the weighted hue histogram from block
206, otherwise the dominant color will be based on the dominant
grayscale from block 208 (block 212). The color from block 212 or
206, based upon the determination from decision block 210, is
validated and possibly overridden or modified (block 214). Thus, in
the example provided in FIG. 6, the background color 216 is derived
from the input image 218.
[0052] Turning now to a discussion of selecting the dominant
colors, the dominant colors may be determined according to the
process 194. As discussed above, the image is resized (block 196).
Hue and grayscale histograms are generated based upon this resized
image (blocks 220 and 222, respectively). The hue and grayscale
histograms are condensed (blocks 224 and 226, respectively). Next,
the condensed hue and grayscale histograms are filtered (blocks 228
and 230, respectively). The filter hue and grayscale histogram data
is then sorted (blocks 232 and 234, respectively). Next, based upon
the sorted histogram data, the dominant colors for each hue are
determined (block 236) and a grayscale color is created (block
238). The hue and grayscale colors are then merged based on the
dominance of color in the image (block 240), thus defining the
dominant colors 242 of the input image 218.
[0053] Having now discussed the processes for determining
background and dominant colors for a given input image, the
discussion now turns to selecting layout colors from the dominant
colors derived in the process 194. FIG. 7 illustrates a process 260
for determining a primary text color based upon the dominant colors
derived from process 194 of FIG. 6. FIG. 8 illustrates a process
262 for determining a second text color based upon the dominant
colors derived from process 194 of FIG. 6, where the second text
color coordinates with the primary text color of FIG. 7.
[0054] The selection process 260 begins by iterating through the
dominant colors of the input image (e.g., the dominant colors
determined by process 194 of FIG. 6) (block 262). For each dominant
color, the computer may determine whether the dominant color
contrasts with the background color (decision block 264). If it
does, the color is chosen and may be artistically modified to
become an appropriate primary-text color (block 266). For example,
the brightness, hue, intensity, etc. of the color may be altered
based upon parameters defined by the algorithm implementing the
process 260.
[0055] When a dominant color is determined not to contrast with the
background, the next dominant color in the iteration is selected,
and the process 260 is repeated. If, after iterating through all of
the dominant colors, no contrasting colors are found, a fallback or
default color selection is chosen (block 268). The fallback color
may be artistically modified (block 266). For example, the
brightness, hue, intensity, etc. of the color may be altered based
upon certain parameters. After the dominant color or fallback color
is artistically modified, the artistically modified color is set as
the primary text color 270.
[0056] Process 290 of FIG. 8 may be used to select a secondary text
color. The selection process 290 begins by iterating through the
dominant colors of the input image (e.g., the dominant colors
determined by process 194 of FIG. 6) (block 292). For each dominant
color, the computer may determine whether the dominant color is
close to the primary text color (decision block 294). If the
dominant color is not close to the primary text color, the computer
may determine whether the color contrasts with the background color
(decision block 296). If the color does contrast with the
background color, the dominant color may be artistically modified
(block 298) and the artistically modified color may be selected as
the secondary text color 300. For example, the brightness, hue,
intensity, etc. of the color may be altered based upon parameters
defining quality attributes of the color. If, however, the color is
close to the primary text color or the color contrasts with the
background color, the next dominant color in the iteration of
dominant colors is selected, and the process 292 is repeated. If,
after iterating through all the dominant colors, there is no valid
color selection, the process may attempt a second pass at selecting
a secondary text color, iterating through the dominant colors
(block 302) to determine whether at least one of the colors
contrasts with the background color (decision block 304). If one of
the dominant colors does contrast with the background color, the
selection may be artistically modified (block 298) and selected as
the background color 300. For example, the brightness, hue,
intensity, etc. of the color may be altered based upon certain
parameters defining quality attributes of a color. If there is no
remaining dominant color that contrasts with the background, a
fallback color may be chosen (block 306). The fallback color may be
artistically modified (block 298) and may be selected as the
secondary color 300. For example, the brightness, hue, intensity,
etc. of the color may be altered based upon parameters defining
quality attributes of the color.
[0057] FIG. 9 illustrates the results of the process 90 of FIG. 3,
which is described in more detail in processes 192, 194, 260 and
290 of FIGS. 6-8. FIG. 9 represents a GUI view 320 presented in the
iTunes.RTM. application. As illustrated by the enlarged album icon
322 and the selector arrow 324, a user has selected an album in the
Albums view of iTunes.RTM.. Based upon this selection, the layout
generator may determine a primary text color and a secondary text
color for use in the GUI layout. For example, based upon the album
artwork (e.g., icon 322) the layout generator may determine that
the background color should be #efefef, that the primary text color
should be #921316, and that the secondary text color should be
#1f274e. The colors have been applied to the primary text 326,
secondary text 328, and background 330 of a portion of the GUI 332
describing specific elements of the selection (e.g. the album
indicated by selector arrow 324) made by the user. In addition, as
described above, the layout generator may add a sized version of
the image 334 to the portion 332. As discussed above, image 334 may
fade into the background 330, thus illustrating the importance of
edge weighting the image when determining the background 330 color.
Additionally, as described above, tertiary colors may also derived
and used in the layout. For example the "ratings" stars 334 and/or
the time length indicator 336 may use a version of the secondary
color #1f274e that is of a different brightness, transparency or
other altered characteristic.
[0058] While the provided example uses an album view in
iTunes.RTM., it is important to note that this example is not
intended to limit the scope of implementation of the dynamic layout
generator. For example, the layouts could be applied to movie
views, TV episode views, artist information views, etc. Each of
these views could use the colors defined in the layout in ways
specific to the view. For example, album views with contributing
artists could use a tertiary color to present the featured artist
in a separate color. Additionally, TV episodes could be listed in
the tertiary color, while the title and season are displayed in the
primary and secondary colors, respectively. Further, implementation
is not limited to iTunes.RTM. but could also be implemented in out
applications with a GUI as well. For example, the above-described
techniques could be used in applications such as OS X.RTM. or
GarageBand.RTM., by Apple, Inc.
[0059] As the input image 218 may be blended into the background of
a view of the GUI, it may be desirable to remove any borders
present in the input image. FIG. 10 is a process 350 for cropping
borders. The process 350 may occur during the background color
validation of process 192 of FIG. 6 (block 214 of FIG. 6). The
process 330 takes as inputs an input color 352 and the input image
218. Multiple locations along each edge of the input image 218 are
selected (block 354). For each location, the pixel color is
compared to the input color 352, moving from the edge pixels of the
locations inward (block 356). Further, for each location, it is
determined whether the pixel color contrasts with the input color
(decision block 358). If not, the location is once again moved
inward (block 356). If the color does contrast with the input color
352, the length of the border for each edge location is found
(block 360). Once these steps have been repeated for each location,
the locations are sorted based on the border length (block 362).
The longest and shortest border lengths for an edge are compared
(block 364). If the length difference is within a certain
percentage or the longest length is zero (decision block 366), the
edge is determined to be a border (block 368). Otherwise, the edge
is determined not to be a border (block 370). Blocks 354-370 are
repeated for each edge. Once each edge has been determined to be a
border or not a border, a determination is made as to whether 3 or
4 edges are a border (decision block 372). If 3 or 4 edges are a
border, the image is bordered (block 374). When the image is
bordered, a determination may be made that the input color is not
valid (e.g., because a border was present when the input color was
determined), the image may be cropped by the longest border length,
and the background color selection is repeated (block 376). When 3
or 4 edges are not a border, it may be determined that the image
218 is not bordered (block 378). Accordingly, the input color may
be determined to be valid (block 380).
[0060] As may be appreciated, by implementing the techniques
described herein, graphical user-interfaces used in electronic
devices may be greatly enhanced. For example, the background colors
and/or images as well as the text colors of the graphical
user-interface may complement a graphical image currently displayed
in the graphical user-interface. Through providing a complementary
layout, the layout may be more appealing to the user, thus,
creating an enhanced user experience.
[0061] The specific embodiments described above have been shown by
way of example, and it should be understood that these embodiments
may be susceptible to various modifications and alternative forms.
It should be further understood that the claims are not intended to
be limited to the particular forms disclosed, but rather to cover
all modifications, equivalents, and alternatives falling within the
spirit and scope of this disclosure.
* * * * *