Member Engagement Multi-objective Optimization In A Social Network

Abstract

This disclosure relates to systems and methods that include obtaining from a member activity database, with a processor, data indicative of past interactions by members of a social network with content items associated with the social network, the content items including sponsored content items and organic content items. A positioning condition for content items is determined for individual members, based on the data as obtained, including sponsored content items and organic content items, on a user interface displayed to the individual members that is estimated to meet a optimization criterion, the optimization criterion including engagement with the content items by members individually and by members overall. The user interface is provided to one of the members based, at least in part, on the positioning condition.

Description

PRIORITY

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application No. 61/972,662, "SYSTEM AND METHOD FOR SUBSET MULTI-OBJECTIVE OPTIMIZATION IN A SOCIAL NETWORK", which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

[0002] The subject matter disclosed herein generally relates to member engagement, both individually and overall, for general multi-objective optimization in a social network.

BACKGROUND

[0003] Providers and administrators of social networks may have various metrics and objectives by which the success of the social network is judged. Member engagement with the social network may be determined based on activities by the users within the social network, such as by clicking links, commenting on posts to the social network, and posting new content to the social network, among many potential activities in the social network. Revenue may also be a factor by which success is judged, as the providers of social networks may have expenses that may need to be met as well as a profit motive. In various circumstances, the steps that a social network provider may take to increase revenue, such as by displaying advertisements or sponsored content, may tend to reduce member engagement with the social network.

BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings.

[0005] FIG. 1 is a block diagram illustrating various components or functional modules of a social network, in an example embodiment.

[0006] FIGS. 2A and 2B are an abstract depiction of a news feed as displayed on a user interface and a tiled page display, respectively, each as generated by a user interface module, in example embodiments.

[0007] FIG. 3 is an abstract depiction of the information in the social graph database, in an example embodiment.

[0008] FIG. 4 illustrates a Pareto curve that may be generated by a multi-objective optimization module, in an example embodiment.

[0009] FIG. 5 is a flowchart for subset multi-objective optimization in a social network, in an example embodiment.

[0010] FIG. 6 is a block diagram illustrating components of a machine able to read instructions from a machine-readable medium.

DETAILED DESCRIPTION

[0011] Example methods and systems are directed to member engagement-sensitive multi-objective optimization. Examples merely typify possible variations. Unless explicitly stated otherwise, components and functions are optional and may be combined or subdivided, and operations may vary in sequence or be combined or subdivided. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of example embodiments. It will be evident to one skilled in the art, however, that the present subject matter may be practiced without these specific details.

[0012] Multi-objective optimization techniques have been developed that may provide for generating an positioning condition of sponsored and organic content on user interfaces of a social network to optimize the various constraints in the social network. Thus, in the above example, an optimization criterion may be to maximize revenue while maintaining member engagement of at least ninety-five (95) percent of a baseline. Such optimization techniques may, for instance, display various amounts of revenue-generating content to users until a maximum revenue-generating content amount is determined that still produces member engagement of at least ninety-five (95) percent of the baseline.

[0013] Considerations of member engagement factor in both engagement for members individually as well as for all or a subset of the membership of the social network as a whole. Unless both are considered concurrently in the provisioning of content items to various positions on a user interface, certain members may receive undesirable positioning of content items on their user interface. For instance, a positioning condition that is applied generally may cause a negative reaction and drive away or otherwise provide a negative experience for individual members. As such, a positioning condition is utilized that factors in positioning requirements both for the social network as a whole as well as for the individual members to whom the content items are displayed and that seeks to provide multi-objective optimization for member engagement.

[0014] FIG. 1 is a block diagram illustrating various components or functional modules of a social network 100, consistent with some examples. The social network 100 may be utilized to provide content items positioned in user interfaces. The provision of such content items may be conducted so as provide multi-objective optimization criteria including, but not limited to, member engagement with the content items.

[0015] A front end 101 consists of a user interface module (e.g., a web server) 102, which receives requests from various client-computing devices, and communicates appropriate responses to the requesting client devices. For example, the user interface module(s) 102 may receive requests in the form of Hypertext Transport Protocol (HTTP) requests, or other web-based, application programming interface (API) requests. An application logic layer 103 includes various application server modules 104, which, in conjunction with the user interface module(s) 102, may generate various user interfaces (e.g., web pages, applications, etc.) with data retrieved from various data sources in a data layer 105. In some examples, individual application server modules 104 may be used to implement the functionality associated with various services and features of the social network service. For instance, the ability of an organization to establish a presence in the social graph of the social network 100, including the ability to establish a customized web page on behalf of an organization, and to publish messages or status updates on behalf of an organization, may be services implemented in independent application server modules 104. Similarly, a variety of other applications or services that are made available to members of the social network service may be embodied in their own application server modules 104. Alternatively, various applications may be embodied in a single application server module 104.

[0016] In some examples, the social network 100 includes a content item publishing module 106, such as may be utilized to receive content, such as electronic messages, posts, links, images, videos, and the like, and publish the content to the social network. The one or more of the content item publishing module 106, the application server modules 104, or the social network 100 generally may include a multi-objective optimization module 108. As will be disclosed in detail herein, the multi-objective optimization module 108 may manage electronic messages and organize and display such electronic messages according to objectives determined by the administrators of the social network.

[0017] As illustrated, the data layer 105 includes, but is not necessarily limited to, several databases 110, 112, 114, such as a database 110 for storing profile data, including both member profile data as well as profile data for various organizations. Consistent with some examples, when a person initially registers to become a member of the social network service, the person may be prompted to provide some personal information, such as his or her name, age (e.g., birthdate), gender, interests, contact information, home town, address, the names of the member's spouse and/or family members, educational background (e.g., schools, majors, matriculation and/or graduation dates, etc.), employment history, skills, professional organizations, and so on. This information is stored, for example, in the database 110. Similarly, when a representative of an organization initially registers the organization with the social network service, the representative may be prompted to provide certain information about the organization. This information may be stored, for example, in the database 110, or another database (not shown). With some examples, the profile data may be processed (e.g., in the background or offline) to generate various derived profile data. For example, if a member has provided information about various job titles the member has held with the same or different companies, and for how long, this information can be used to infer or derive a member profile attribute indicating the member's overall seniority level, or seniority level within a particular company. With some examples, importing or otherwise accessing data from one or more externally hosted data sources may enhance profile data for both members and organizations. For instance, with companies in particular, financial data may be imported from one or more external data sources, and made part of a company's profile.

[0018] Once registered, a member may invite other members, or be invited by other members, to connect via the social network service. A "connection" may require a bi-lateral agreement by the members, such that both members acknowledge the establishment of the connection. Similarly, with some examples, a member may elect to "follow" another member. In contrast to establishing a connection, the concept of "following" another member typically is a unilateral operation, and at least with some examples, does not require acknowledgement or approval by the member that is being followed. When one member follows another, the member who is following may receive status updates or other messages published by the member being followed, or relating to various activities undertaken by the member being followed. Similarly, when a member follows an organization, the member becomes eligible to receive messages or status updates published on behalf of the organization. For instance, messages or status updates published on behalf of an organization that a member is following will appear in the member's personalized data feed or content stream. In any case, the various associations and relationships that the members establish with other members, or with other entities and objects, are stored and maintained within the social graph database 112.

[0019] The social network service may provide a broad range of other applications and services that allow members the opportunity to share and receive information, often customized to the interests of the member. For example, with some examples, the social network service may include a photo sharing application that allows members to upload and share photos with other members. With some examples, members may be able to self-organize into groups, or interest groups, organized around a subject matter or topic of interest. With some examples, the social network service may host various job listings providing details of job openings with various organizations.

[0020] As members interact with the various applications, services and content made available via the social network service, information concerning content items interacted with, such as by viewing, playing, and the like, etc.) may be monitored and information concerning the interaction may be stored, for example, as indicated in FIG. 1 by the database 114. Thus, previous interactions with a content item by various users of the social network 100 may be stored and utilized in determining, among other factors, how various types of content items, such as organic content items and sponsored content items, result in differences in engagement levels with the content items by users of the social network 100.

[0021] Although not shown, with some examples, the social network 100 provides an application programming interface (API) module via which third-party applications can access various services and data provided by the social network service. For example, using an API, a third-party application may provide a user interface and logic that enables an authorized representative of an organization to publish messages from a third-party application to various content streams maintained by the social network service. Such third-party applications may be browser-based applications, or may be operating system-specific. In particular, some third-party applications may reside and execute on one or more mobile devices (e.g., phone, or tablet computing devices) having a mobile operating system.

[0022] FIGS. 2A and 2B are an abstract depiction of a news feed 200 as displayed on a user interface 202 and a tiled page display 203, respectively, each as generated by the user interface module 102, in example embodiments. The news feed 200 includes multiple positions 204 in which individual content items 206 are displayed. While the positions 204 are illustrated as a top-down list, it is to be understood that the positions 204 may be any of a variety of ordered or orderable positions 204, such as tiles or a ribbon bar. The content items 206 are variously organic content items 206A generated based on content provided to the social network by members of the social network and sponsored content items 206B generated based on a fee paid to the administrators of the social network to have the sponsored content items 206B displayed on the news feed 200.

[0023] Various content items 206 include links or selectable menu items 208 with which a member viewing the news feed 200 may interact with content items 206. Selecting a link 208 may cause a related article to be displayed on the member interface 202, may allow the member to comment on, share, "like," or otherwise interact with the content item 206. Interaction with a link 208 by a member may be registered in the member activity and behavior database 114 as an engagement with the associated content item 206.

[0024] The tiled display 203 similarly includes positions 204 that, rather than being organized hierarchically in a scrollable, linear list, are displayed in individual pages 210 that may be switched sequentially via links 212. In contrast to the news feed 200, the positions 204 within each page 210 do not necessarily follow a linear progression in their relative value, as will be disclosed herein. Rather, a fifth position 204(5) may tend to produce more interactions than a fourth position 204(4) because the fifth position 204(5) is located in a better position on the second page 210(2) than the fourth position 204(4) is located on the first page 210(1).

[0025] The positions 204 may be ranked according to their relative order in the news feed 200 and/or corresponding to their relative visibility to or likelihood of being interacted with by a member on the news feed 200. For instance, in the news feed 200, a first position 204(1) may be the most likely position to be interacted with by a member followed in sequence by a second position 204(2), a third position 204(3), and so forth. As noted above, the tiled pages 203 may cause positions 204 to have values that do not follow sequentially. Each position 204 may be assigned a relative value indicative of the relative visibility or likelihood of interaction with a corresponding content item 206. Thus, in an abstract example, on the news feed 200 the first position 204(1) may be assigned a value of "10", the second position 204(2) may be assigned a value of "7", the third position 204(3) may be assigned a value of "5", and so forth. In a further abstract example, in the tiled pages 203, the third position 204(3) may be assigned a value of "7", the fourth position 204(4) may be assigned a value of "4", and the fifth position 204(5) may be assigned a value of "6".

[0026] Content items 206 may be placed in various positions 204 according to a positioning condition as managed by the multi-objective optimization module 108. The positioning condition includes two factors: positioning for a group of members of the social network, such as all of the members or a subset of the members; and positioning for each member individually. That is to say, the positioning condition may specify that for the wider group of members a certain ratio of sponsored content items 206B and organic content items 206A be placed in a given position 204 over time to the group of members (e.g., that the ratio of organic content items 206A to sponsored content items 206B for the third position 204(3) is four-to-one). The positioning condition may also concurrently specify that a news feed 200 as displayed on an individual member's user interface 202 have certain restrictions on the number and positional relationship of certain content items 206, such as sponsored content items 206B.

[0027] The restrictions on the positioning of content items 206 in a news feed 200 of an individual member may include both the number of sponsored content items 206B that may be displayed on the news feed 200 at one time. The number may be relative to the number of positions 204, e.g., may be a percentage of the total number of positions 204. Thus, in an example, the restriction may be that a given member's news feed 200 have no more than twenty-five (25) percent of the positions 204 filled with sponsored content items 206B.

[0028] The restriction may also be based on sponsored content item 206B density. Density may be defined over a window of positions 204. For instance, the density restriction may be that over any five (5) consecutive positions 204, not more than two (2) may be sponsored content items 206B. There may be multiple density restrictions in effect at once. Thus, in an example, the density restriction may further require that no more than two (2) out of three (3) consecutive positions 204 have sponsored content items 206B and that no more than three (3) out of six consecutive positions 204 have sponsored content items 206B.

[0029] Further, the density requirement may specify a minimum number of intervening positions 204 between certain content items 206. Thus, in an example, a density requirement may specify that at least two (2) positions 204 separate consecutive sponsored content items 206B. Such a requirement may be applied to any type of content item 206.

[0030] The positioning condition may factor in any quantifiable aspect of the relationship of content items 206 to positions 204 in the news feed 200 for the members of the group or subgroup and individually. Thus, for instance, the positioning condition may place restrictions on the total value of sponsored content items 206 that are displayed, on the source (e.g., individual members, groups, companies, content automatically generated by the social network, etc.) of the content items 206, and so forth. Additionally, restrictions may provide both an upper bound and a lower bound on the factor involved in the restriction. Thus, the density requirement may, e.g., require that at least one (1) of five (5) consecutive positions 204 be a sponsored content items 206B.

[0031] The positioning condition may further specify individual member requirements based on member profile data 116 and activity data 118 of the associated individual member. For instance, a density requirement may be adjusted to allow for more or less density of sponsored content items 206B on the news feed 200 of a member who is either particularly likely or unlikely to engage with sponsored content items 206B, respectively. The density requirement may be further adjusted to allow for more or less density of sponsored content items 206B on the news feed 200 of a member who is either particularly valuable or of relatively little interest to sponsoring entities of the sponsored content items 206B. Thus, if the profile data 116 of a member places the member in a valuable demographic then the density requirement may be adjusted to allow for more sponsored content items 206B. For instance, a corporate executive or a member located in a particular area may be of more value to a sponsoring entity than a low-level employee or a member who is located in an undesirable location. The, density requirement for members with such profile data 116 may be increased.

[0032] The density requirement, among other potential requirements as disclosed herein, may further be adjusted or otherwise selected or chosen based on a number of positions 204 visible on the user interface 202 at any given time. Thus, in an example, if relatively few positions 204 are visible on the user interface 202 then the density requirement may allow for fewer sponsored content items 206B so that the visible positions 204 are not unduly filled with sponsored content items 206B. It is to be recognized that this is for example and that particular adjustments to the density requirement based on the number of visible positions 204 may serve any purpose, including to increase the percentage of sponsored content items 206B, and so forth.

[0033] Additionally, factors such as an amount of time required to load the user interface 202 on the user device of the member may result in adjustments to the density requirement. For instance, the density requirement may require fewer sponsored content items 206B if the load time is relatively long. The density requirement may be adjusted proportionally to the length of time required to load the page. Further, it is to be recognized that the density requirement may be adjusted in any of a variety of ways based on page load times so as to compensate for an experience of the member that may be impacted by a varying load time.

[0034] The various adjustments to the positional condition for an individual member may be applied cumulatively. Thus, a member who is a corporate executive but located in an undesirable location may have offsetting adjustments to their positional condition. It is noted that the adjustments for individual members may be of fractional granularity. For instance, if one density requirement is not more than two (2) out of every five (5) positions 204, the density requirement may be adjusted up by 0.4 for every profile data 116 that is desirable. Thus, in the above example, a member who is a corporate executive in a desirable location may have an adjusted density requirement of not more than 2.8 out of every five (5) positions, which may be rounded to the nearest integer, or three (3) out of every five (5) positions.

[0035] Various types of sponsored content items 206B may be treated differently than others. Sponsored content items 206B may differ in the circumstances under which the sponsoring entity pays for the sponsored content item 206B to the social network administrators. In an example, a sponsored content items 206B may be charged based on an impression by a member, e.g., because the member either saw or may have seen the sponsored content item 206B. Such a sponsored content item 206B may provide for the sponsoring entity being charged each time the sponsored content item 206B is placed in a position 204 or each time the sponsored content item 206B is affirmatively displayed on a user interface 202.

[0036] Alternatively, the sponsored content item 206B may provide for the sponsoring entity being charged only when a member interacts with the sponsored content item 206B. As noted herein, interaction may include selection of a link, a comment, a "like", a share, and so forth. Thus, the social network administrators would receive payment if there is interaction with the sponsored content item 206B.

[0037] The different nature of the sponsored content items 206B may be utilized in determining their positions 204 in the news feed 200 and the members to whom the sponsored content items 206B are displayed. Thus, the positioning condition may specify, for instance, that three-quarters of the sponsored content items 206B that are positioned in the first position 204(1) require an interaction while three-quarters of the sponsored content items 206B that are in the fifth position 204(5) are charged based on impression.

[0038] Moreover, the positioning condition may direct interaction sponsored content items 206B to members that have activity data 118 that suggests that the member is particularly likely to interact with sponsored content items 206B. Thus, in an example, a member who interacts with content items 206 with a frequency one (1) standard deviation more than the average member may be provided with fifty (50) percent more interaction sponsored content items 206B than the average for all members or the members of a specified subgroup. Similarly, a member whose interactions with content items 206 are with a frequency of one (1) standard deviation less than the average member may be provided with fifty (50) percent fewer interaction sponsored content items 206B than the general average.

[0039] The various factors disclosed herein may be considered over any of a variety of timeframes. That is to say, considerations may be made for both short term and long term periods. Thus, an optimization criterion may establish a diversity requirement both a first time period, such as a single display of content items 206 on the user interface 202 or for a relatively short time period, such as one day, as well as second time period, such as one week to one month. The first and second time periods may specify different optimization criteria, and an individual implementation that meets first optimization criteria may violate the second optimization criteria and may thus not be implemented.

[0040] By way of example, an optimization criterion for a period of a month may specify that not more than one-quarter of the content items 206 be sponsored content items 206B. The optimization module 108 may determine for a particular display of content items 206 in the news feed 200 that a density requirement should allow for one (1) sponsored content items 206B for every two (2) positions 204, and that this would meet a one day optimization criterion of not more than half of the content items 206 being sponsored content items 206B. However, that high of a density of sponsored content items may violate the one month time period requirements if, over the one month time period, the number of sponsored content items 206B displayed are near the one-quarter threshold. Thus, the density requirement may be invalidated and a less-dense requirement put in place that would not violate requirements of the one month time period.

[0041] FIG. 3 is an abstract depiction of the information in the social graph database 112, in an example embodiment. Individual members 300 may access a news feed 200 on user devices 202, such as by logging in to the individual member's 300 account on the social network 100. The members 300 depicted represent variously all of the members of the social network 100 or a portion of the members of the social network 100 to which the multi-objective optimization techniques described herein may be applied.

[0042] The multi-objective optimization module 108 may select a subset of members 300A (denoted by squares in FIG. 3) of the total population of the members 300 (denoted by both circles and squares in FIG. 3) on which to perform multi-objective analysis. The members 300A in the subset may be randomly selected from the total population of members 300 or may be selected based in part on profile data 116 of the members 300. In an example, the members 300A of the subset may be selected so that profile data 116 of the members 300A of the subset broadly matches the profile data of the total population of members 300, i.e., so that the subset meets some standard of being representative of the whole population. Such a standard may include or otherwise produce a distribution of the profiles of the members 300A of the subset that is the same or substantially similar to that of the total population of members 300.

[0043] In various examples, a size of the subset of members 300A may be selected by the multi-objective optimization module 108 based on a number of objectives or constraints for which performance is being optimized and a size of the total population of members 300. Further, a proportionate size of the subset of members 300A to the total population of members 300 may increase as the total population of members 300 decreases. For instance, in an illustrative example, with two (2) objectives or constraints and a total population of members 300 of three hundred million (300,000,000), the multi-objective optimization module 108 may select approximately two thousand (2,000) members 300A to be part of the subset. With three (3) objectives or constraints the multi-objective optimization module 108 may select approximately three thousand (3,000) members 300A to be part of the subset, and so forth. With a total population of members 300 of one hundred and fifty million (150,000,000), the multi-objective optimization module 108 may select approximately one thousand four hundred (1,400) members 300A for the subset to optimize for two (2) objectives or constraints or approximately two thousand one hundred (2,100) members 300A for the subset to optimize for three (3) objectives or constraints.

[0044] The multi-objective optimization module 108 may further adjust the size of the subset of members 300A based on a desired statistical confidence in the ultimately-determined optimized configuration, as disclosed herein. For instance, in an illustrative example, if the above subset sizes produce a statistical likelihood that the optimization, when applied to the total population, will be accurate to plus-or-minus two (2) percent ninety-five (95) percent of the time, then increasing the subset size may produce better accuracy more reliably while decreasing the subset size may produce worse accuracy less reliably. However, increasing the subset size may increase a need for computational resources and time to determine the optimization while decreasing the subset size may lessen the need for computational resources and time.

[0045] In various examples, the multi-objective optimization module 108 may perform modeling based on the subset of members 300A of the effect of changing various constraints, such as the positional condition. As will be disclosed herein, additionally or alternatively the subset of members 300A may be utilized to test the constraints as optimized either concurrent with applying the constraints as optimized to the total population of members 300 or before applying the constraints as optimized to the total population of members 300. Such a test of the constraints as optimized may be utilized to update assumptions or the optimization techniques and/or, in the case of testing the constraints as optimized prior to applying the constraints to the total population of members 300, prevent constraints that do not produce the desired result from being applied to the total population of members 300. It is noted that while the test is described as applying to the same subset of members 300A as was used to model the constraints in the first instance, the subset of members on which the constraints as optimized may be applied may be different from the subset of members 300A to which the model was applied. Thus, the constraints as optimized may be determined based on a first subset of members 300A while the constraints as optimized may be tested on a second subset of members 300 that include some, all, or none of the first subset of members 300A.

[0046] As disclosed herein, the constraints may be: overall engagement by members 300 with content items 206; and revenue. As noted, engagement may be determined based on a member interaction with a content item 206, with overall engagement being determined based on a total number of interactions. In various illustrative examples, each interaction counts toward engagement the same as each other interaction; thus, a single member 300 reading a posted article, commenting on the article, and "liking" the article may produce three (3) equal engagement counts. Alternatively, a given member 300 may only receive one (1) engagement count regardless of the number of times the member 300 interacts with a single content item 206; thus, the above example would produce only one (1) engagement count. Further alternatively or additionally, certain interactions may be considered to create relatively more engagement than others; thus, reading the article may produce one (1) engagement count, commenting on the article may produce 1.5 engagement counts, and "liking" the article may produce 0.5 engagement counts.

[0047] Relative engagement levels for the various positions 204 may be determined based on relative differences in engagement counts for the positions 204. Thus, the first position 204(1) may have an engagement level of 1.0 while the second position 204(2), which may generate only eighty (80) percent of the engagement counts as the first position 204(1), may have an engagement level of 0.8. The value of positions 204 may be determined, at least in part, based on an the engagement level or an average number of engagement counts that content items 206 in the given position 204 tend to generate. Thus, the first position 204(1) may be considered the first position 204(1) because content items 206 placed in the first position 204(1) tend to generate the most engagement counts of all of the positions 204, the second position 204(2) may be considered the second position 204(2) because content items 206 placed in the second position 204(2) tend to generate the second most engagement counts, and so forth. Thus, the value of positions 204 may be determined without respect to a relative physical position of the positions 204. The individual values of the individual positions 204 may be determined based on an aggregate of the content items 206 placed in the positions or may be separately determined based on.

[0048] As the members 300 of the social network 100 utilize the social network 100 over time and their interactions with content items 206 is recorded in the member activity and behavior database 114, an impact on engagement of placing sponsored content items 206B in various position 204 instead of organic content items 206A may be determined. For instance, over time both organic content items 206A and sponsored content items 206B may be presented to members 300 in the first position 204(1). In an illustrative example, the multi-objective optimization module 108 may determine, based on member 300 interaction with the various content items 206 placed in the first position 204(1), that the first position 204(1) has a normalized engagement level of 1.0 when populated by organic content items 206A and an engagement level of 0.6 when populated by sponsored content items 206B, i.e., that the average member is forty (40) percent less likely to engage with a sponsored content item 206B in the first position 204(1) than an organic content item 206A.

[0049] Within each position 204, the ratio of organic content 206A to sponsored content 206B may be based on the position across the news feeds 200 as displayed to each of the members 300 individually, either of the total population following the multi-objective optimization or the subset while the multi-objective optimization module 108 is determining a potential optimal configuration. However, in various examples utilized herein, only one content item 206 may be in a given position 204 at a time. As such, the ratio applies to unique presentations of the news feed 200 to multiple members 300 over time. Thus, if a particular position 204 has a ratio of four (4) organic content items 206A in a given position for every one (1) sponsored content item 206B, that would imply that, as the news feed 200 is presented to multiple members 300 over time, organic content items 206A would tend to be placed in the position 204 four (4) times for every one (1) time a sponsored content item 206B is placed in the position 204.

[0050] Thus, considering the first position 204(1) in isolation for the purposes of this example, the multi-objective optimization module 108 may determine a positioning condition across the various members 300 or a subset of members 300A that if sponsored content items 206B were placed in the first position 204(1) all the time that engagement with content items 206 in the first position 204(1) would be expected to drop forty (40) percent compared with placing only organic content items 206B in the first position 204(1). Relatedly, if sponsored content items 206B were placed in the first position 204(1) half the time and organic content items 206A were placed in the first position 204(1) the other half then engagement with content items 206 in the first position 204(1) would be expected to be twenty (20) percent less than if only organic content items 206A were placed in the first position 204(1).

[0051] Moreover, the multi-objective optimization module may consider changes to engagement with content items 206 in a position 204 relative to a baseline rather than relative to only using organic content items 206A. Thus, for instance, the positioning condition may specify a baseline mix of organic content items 206A and sponsored content items 206B that may be normalized to an engagement value of 1.0 and changes to the ratio of organic content items 206A and sponsored content items 206B displayed in the position 204 may increase or decrease the engagement value relative to the 1.0 baseline. Increasing the ratio of organic content items 206A to sponsored content items 206B may tend to increase engagement while decreasing the ratio may end to decrease engagement.

[0052] In various examples, the principles described with respect to one position 204 may be applied to each of the positions 204 individually and a profile of the news feed 200 as a whole developed. Thus, the organic content item 206A to sponsored content item 206B ratio may be adjusted for each position 204 individually to arrive at a particular engagement level for the social network 100 as a whole. Alternatively, the principles described with respect to one position 204 may be applied to the news feed 200 as a whole without respect to individual positions 204. Thus, positions 204 may be ignored and the overall ratio of organic content items 206A and sponsored content items 206B may be considered for the news feed 200 as a whole. Such an approach may be computationally relatively simple but may be relatively less accurate than considering positions 204 individually.

[0053] As disclosed herein, the multi-objective optimization module 108 may also factor in revenue as a constraint. Revenue may be generated based on sponsored content items 206B and may be based on impressions, as disclosed herein simple presentation of sponsored content items 206B on the news feed 200 and/or based on member 300 interactions with the sponsored content item 206B, such as according to the mechanisms disclosed herein (also known as the "click-through rate"). The providers of sponsored content items 206B may associate a bid or a fee with the sponsored content item 206B that may be paid to the social network 100 provider upon the sponsored content item 206B being displayed in a news feed 200 or upon a member 300 engaging with the sponsored content item 206B, as appropriate. The bid or fee may be specific to a particular position 204 in which the sponsored content item 206B is displayed and may be higher or lower depending on the visibility of the position 204 and the likelihood of member 300 engagement with sponsored content items 206B in various positions.

[0054] Revenue from the placement of a sponsored content item 206B may thus be known prior to placement of the sponsored content item 206B in the news feed 200 if the revenue is based on impressions. Additionally or alternatively, revenue may be estimated prior to placement of the sponsored content item 206B in the news feed 200 based on the bid associated with the sponsored content item 206B multiplied by the click-through rate for the position 204 in which the sponsored content item 206B is to be placed. Thus, the first position 204(1) may be associated with higher revenue or higher expected revenue than the second position 204(2), which may be associated with higher revenue or expected revenue than the third position 204(3), and so forth.

[0055] FIG. 4 illustrates a Pareto curve 400 that may be generated by the multi-objective optimization module 108, in an example embodiment. The multi-objective optimization module 108 may analyze each position 204 on the impact to overall member engagement and revenue to various ratios of organic sponsored content items 206A and sponsored content items 206B. The multi-objective optimization module 108 may thus, for a number of positions X, perform 2.sup.X computations to create a Pareto curve with overall member engagement 402 on one axis and expected revenue 404 on the orthogonal axis. Given the potentially large number of computations then, utilizing a statistically significant subset of members 300A may provide significant computational savings over utilizing the total population of members 300. The Pareto curve 400 is a mechanism for visualizing the resultant data points 406 and it is to be understood that the multi-objective optimization module 108 may generate the data points 406 without necessary creating the curve 400 itself.

[0056] As noted here, the data for overall member engagement with content items 206 in different positions 204 and, by extension, estimated revenue may be based on data stored in the member activity and behavior database 114 for the subset of members 300A. Thus, the Pareto curve 400 may, in various examples, be generated without having to collect new data. The multi-objective optimization module 108 may utilize member activity and behavior data stretching back a predetermined period of time or may utilize a fixed amount of the most recent member activity and behavior data across the subset of members 300A.

[0057] Each data point 406 represents a mix of organic and sponsored content items 206A, B within various positions 204. Alternatively, each data point 406 may represent the mix of organic and sponsored content items 206A, B throughout the news feed 200 without respect to positions 204 as specified by the positioning condition. Each data point 406 thus anticipates revenue based on a given mix of content items 206 and, if implemented with respect to the social network, provides an operating condition for the mix of content items that is estimated to produce a certain amount of revenue.

[0058] As disclosed herein, the multi-objective optimization module 108 may include a particular optimization criterion that may be applied to select a particular data point to be implemented with the total population of members 300. For instance, as described above, the optimization criterion may be to maximize revenue while maintaining at least ninety-five (95) percent overall member engagement relative to a baseline. (This example is presented herein for consistency of illustration; it is to be emphasized that any suitable constraints may be utilized and any suitable criterion or criteria within those constraints may be utilized.) In the illustrated curve 400, then, while some data points 406A may have relatively high expected revenue they also would be expected to reduce overall member engagement by more than five (5) percent. As a result, the data point 406B that has the highest revenue and a overall member engagement of ninety-five (95) percent or better may be selected by the multi-objective optimization module 108 and the mix of organic and sponsored content items 206A, B applied as an operating condition to the total population of members 300.

[0059] The inclusion of a data point 406 may depend on the positioning condition also being attainable in view of the requirements for individual member engagement, as detailed above. That is to say, even though the curve 400 is based on overall member engagement, a positioning condition that produces that overall member engagement and revenue level may not be valid if the overall engagement and revenue is only attainable while violating individual member engagement requirements. In such a case, the data point 406 may not be included in the curve 400.

[0060] Similarly, a data point 406 may be attainable with a range of individual member engagement levels or statistics. The multi-objective optimization module 108 may further optimize the positioning condition to provide for desired individual member engagement. In an example, the multi-objective optimization module 108 may specify that, within a data point 406, the positioning condition should be set that tends to result in individual members 300 continuing to have levels of interactions consistent with their past interactions with content items 206 prior to implementation of the positioning condition.

[0061] The positioning condition may thus be set that provides to an individual member 300 any combination of content items 206 in specified positions 204 that is likely to maintain the member's 300 rate of interactions. That combination of content items 206 and positions 204 may be the same as or similar to the combination the member 300 has seen before or may depart significantly. For instance, a member 300 who has been presented with one-quarter sponsored content items 206B in the second position 204(2) and one-third sponsored content items 206B in the fifth position 204(5) may have the same individual member engagement with one-fifth sponsored content items 206B in the second position 204(2) and one-half sponsored content items 206B in the fifth position 204(5). The mix of interaction sponsored content items 206B and impression sponsored content items 206B may also be adjusted as appropriate.

[0062] The content item 206 mix specified by the data point 406B may be subjected to a preliminary or trial run, such as with the subset of members 300A or with a second subset of members 300 that include some, all, or none of the first subset of members 300A. Alternatively, the preliminary run may be with the total population 300. Based on the actual results of the preliminary run the multi-objective optimization module 108 may make adjustments to the mechanisms described herein to account for deviations from the expected result. For instance, if total engagement decreases by more than five (5) percent then the proportion of sponsored content items 206B displayed on the news feed may be reduced. Adjustments based on actual performance may be incorporated on an ad hoc basis for the current data point 406B. Alternatively, such adjustments may be maintained as permanent or semi-permanent adjustment factors to provide adjustments to future optimizations. Such adjustment factors may be aggregated over time and eventually discarded as they become out of date or are replaced by more recent adjustments.

[0063] The term "optimized" and related words should not be taken as meaning strictly optimized or maximized for the purposes of this disclosure. Thus, the optimization techniques do not necessarily require finding an absolutely optimal or maximized operating condition for the social network 100. Moreover, a maximally optimized operating condition that is determined may not necessarily be utilized and a less than maximally optimized operating condition may be selected instead, such as if, in the above example, a relatively small decrease in revenue may produce a relatively large increase in member engagement. The multi-objective optimization module 108 may be configured with logic that would allow for, for instance, selecting a revenue total that is 0.2 percent less than the maximum value for a two (2) percent relative increase in engagement.

Flowchart

[0064] FIG. 5 is a flowchart for member engagement multi-objective optimization in a social network, in an example embodiment. The flowchart may be performed with any of a variety of devices and systems disclosed herein, as well as with any suitable system known in the art.

[0065] At operation 500, data indicative of past interactions by members of a social network with content items associated with the social network are obtained from a member activity database with a processor, the content items including sponsored content items and organic content items.

[0066] At operation 502, a positioning condition for content items is determined for individual members with the processor based on the data as obtained, including sponsored content items and organic content items, on a user interface displayed to the individual members that is estimated to meet a optimization criterion, the optimization criterion including engagement with the content items by members individually and by members overall. In an example, the user interface includes hierarchical positions, individual content items being placed in individual positions, and wherein the positioning condition establishes a requirement for a distribution of sponsored content items in the positions. In an example, the distribution of sponsored content items is based on a density of sponsored content items in a predetermined window of consecutive positions. In an example, the distribution of sponsored content items is based on a number of intervening organic content items between sponsored content items in consecutive positions. In an example, the number of intervening organic content items is based, at least in part, on a number of positions that the processor is configured to cause to be displayed on the user interface at one time and an amount of time to cause the user interface to be displayed.

[0067] In an example, the optimization criterion further includes a revenue constraint of the social network. In an example, the optimization criterion is configured to provide for a maximized revenue while maintaining member engagement individually and overall within a percentage of a baseline. In an example, the organic content items are provided to the social network by the members and sponsored content items provided to the social network by sponsors of the sponsored content items, and wherein the operating condition is based, at least in part, on a percentage mix of organic content items and sponsored content items. In an example, the optimization criterion is based, at least in part, on engagement with the content items over a first time period and a second time period longer than the first time period.

[0068] At operation 504, the user interface is provided, with the processor, via a network interface to one of the members based, at least in part, on the positioning condition. In an example, providing the user interface includes causing the content items to be displayed on a user device associated with one member of the individual members, and wherein member engagement individually is based, at least in part, on interaction by the one member with the content items. In an example, providing the user interface includes causing the content items to be displayed on user devices individually associated with the individual members, and wherein member engagement overall is based, at least in part, on interaction by multiple members with the content items. In an example, the interaction with the content items includes at least one of accessing a link included in the content item, commenting on the content item, sharing the content item, and providing an indication of approval of the content item.

System

[0069] FIG. 6 is a block diagram illustrating components of a machine 600, according to some example embodiments, able to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically, FIG. 6 shows a diagrammatic representation of the machine 600 in the example form of a computer system and within which instructions 624 (e.g., software) for causing the machine 600 to perform any one or more of the methodologies discussed herein may be executed. In alternative embodiments, the machine 600 operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 600 may be a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a smartphone, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 624, sequentially or otherwise, that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 624 to perform any one or more of the methodologies discussed herein.

[0070] The machine 600 includes a processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), or any suitable combination thereof), a main memory 604, and a static memory 606, which are configured to communicate with each other via a bus 608. The machine 600 may further include a graphics display 610 (e.g., a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)). The machine 600 may also include an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit 616, a signal generation device 618 (e.g., a speaker), and a network interface device 620.

[0071] The storage unit 616 includes a machine-readable medium 622 on which is stored the instructions 624 (e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions 624 may also reside, completely or at least partially, within the main memory 604, within the processor 602 (e.g., within the processor's cache memory), or both, during execution thereof by the machine 600. Accordingly, the main memory 604 and the processor 602 may be considered as machine-readable media. The instructions 624 may be transmitted or received over a network 626 via the network interface device 620.

[0072] As used herein, the term "memory" refers to a machine-readable medium able to store data temporarily or permanently and may be taken to include, but not be limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. While the machine-readable medium 622 is shown in an example embodiment to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions. The term "machine-readable medium" shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., software) for execution by a machine (e.g., machine 600), such that the instructions, when executed by one or more processors of the machine (e.g., processor 602), cause the machine to perform any one or more of the methodologies described herein. Accordingly, a "machine-readable medium" refers to a single storage apparatus or device, as well as "cloud-based" storage systems or storage networks that include multiple storage apparatus or devices. The term "machine-readable medium" shall accordingly be taken to include, but not be limited to, one or more data repositories in the form of a solid-state memory, an optical medium, a magnetic medium, or any suitable combination thereof.

[0073] Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0074] Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied on a machine-readable medium or in a transmission signal) or hardware modules. A "hardware module" is a tangible unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.

[0075] In some embodiments, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module may be a special-purpose processor, such as a field programmable gate array (FPGA) or an ASIC. A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module may include software encompassed within a general-purpose processor or other programmable processor. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

[0076] Accordingly, the phrase "hardware module" should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, "hardware-implemented module" refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.

[0077] Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).

[0078] The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented module" refers to a hardware module implemented using one or more processors.

[0079] Similarly, the methods described herein may be at least partially processor-implemented, a processor being an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an application program interface (API)).

[0080] The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the one or more processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.

[0081] Some portions of this specification are presented in terms of algorithms or symbolic representations of operations on data stored as bits or binary digital signals within a machine memory (e.g., a computer memory). These algorithms or symbolic representations are examples of techniques used by those of ordinary skill in the data processing arts to convey the substance of their work to others skilled in the art. As used herein, an "algorithm" is a self-consistent sequence of operations or similar processing leading to a desired result. In this context, algorithms and operations involve physical manipulation of physical quantities. Typically, but not necessarily, such quantities may take the form of electrical, magnetic, or optical signals capable of being stored, accessed, transferred, combined, compared, or otherwise manipulated by a machine. It is convenient at times, principally for reasons of common usage, to refer to such signals using words such as "data," "content," "bits," "values," "elements," "symbols," "characters," "terms," "numbers," "numerals," or the like. These words, however, are merely convenient labels and are to be associated with appropriate physical quantities.

[0082] Unless specifically stated otherwise, discussions herein using words such as "processing," "computing," "calculating," "determining," "presenting," "displaying," or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or any suitable combination thereof), registers, or other machine components that receive, store, transmit, or display information. Furthermore, unless specifically stated otherwise, the terms "a" or "an" are herein used, as is common in patent documents, to include one or more than one instance. Finally, as used herein, the conjunction "or" refers to a non-exclusive "or," unless specifically stated otherwise.

Claims

1. A method, comprising: obtaining from a member activity database, with a processor, data indicative of past interactions by members of a social network with content items associated with the social network, the content items including sponsored content items and organic content items; determining, with the processor, for individual members, based on the data as obtained, a positioning condition for content items, including sponsored content items and organic content items, on a user interface displayed to the individual members that is estimated to meet a optimization criterion, the optimization criterion including engagement with the content items by members individually and by members overall; and providing, via a network interface, with the processor, the user interface to one of the members based, at least in part, on the positioning condition.

2. The method of claim 1, wherein the user interface includes hierarchical positions, individual content items being placed in individual positions, and wherein the positioning condition establishes a requirement for a distribution of sponsored content items in the positions.

3. The method of claim 2, wherein the distribution of sponsored content items is based on a density of sponsored content items in a predetermined window of consecutive positions.

4. The method of claim 2, wherein the distribution of sponsored content items is based on a number of intervening organic content items between sponsored content items in consecutive positions.

5. The method of claim 4, wherein the number of intervening organic content items is based, at least in part, on a number of positions that the processor is configured to cause to be displayed on the user interface at one time and an amount of time to cause the user interface to be displayed.

6. The method of claim 1, wherein the optimization criterion further includes a revenue constraint of the social network.

7. The method of claim 6, wherein the optimization criterion is selected to provide for a maximized revenue while maintaining member engagement individually and overall within a percentage of a baseline.

8. The method of claim 1, wherein providing the user interface includes causing the content items to be displayed on a user device associated with one member of the individual members, and wherein member engagement individually is based, at least in part, on interaction by the one member with the content items.

9. The method of claim 1, wherein providing the user interface includes causing the content items to be displayed on user devices individually associated with the individual members, and wherein member engagement overall is based, at least in part, on interaction by multiple members with the content items.

10. The method of claim 9, wherein the interaction with the content items includes at least one of accessing a link included in the content item, commenting on the content item, sharing the content item, and providing an indication of approval of the content item.

11. The method of claim 1, wherein the organic content items are provided to the social network by the members and sponsored content items provided to the social network by sponsors of the sponsored content items, and wherein the operating condition is based, at least in part, on a percentage mix of organic content items and sponsored content items.

12. The method of claim 1, wherein the optimization criterion is based, at least in part, on engagement with the content items over a first time period and a second time period longer than the first time period.

13. A system, comprising: an electronic data storage, including: a content item source including sponsored content items and organic content items of a social network; and a member activity database including data indicative of past interactions by members of the social network with the content items; a network interface; and a processor, configured to: obtain, from the member activity database, the data indicative of past interactions; determine, for individual members, based on the data as obtained, a positioning condition for content items, including the sponsored content items and organic content items, on a user interface displayed to the individual members that is estimated to meet a optimization criterion, the optimization criterion including engagement with the content items by members individually and by members overall; and provide, via a network interface, the user interface to one of the members based, at least in part, on the positioning condition.

14. The system of claim 13, wherein the user interface includes hierarchical positions, individual content items being placed in individual positions, and wherein the positioning condition establishes a requirement for a distribution of sponsored content items in the positions.

15. The system of claim 14, wherein the distribution of sponsored content items is based on a density of sponsored content items in a predetermined window of consecutive positions.

16. The system of claim 14, wherein the distribution of sponsored content items is based on a number of intervening organic content items between sponsored content items in consecutive positions.

17. The system of claim 16, wherein the processor is configured to display a number of positions on the user interface at one time, and wherein the number of intervening organic content items is based, at least in part, on the number of positions and an amount of time to cause the user interface to be displayed.

18. The system of claim 13, wherein the optimization criterion further includes a revenue constraint of the social network.

19. The system of claim 18, wherein the optimization criterion is selected to provide for a maximized revenue while maintaining member engagement individually and overall within a percentage of a baseline.

20. The system of claim 13, wherein the processor is configured to provide the user interface by causing the content items to be displayed on a user device associated with one member of the individual members, and wherein member engagement individually is based, at least in part, on interaction by the one member with the content items.

21. The system of claim 13, wherein the processor is configured to provide the user interface by causing the content items to be displayed on user devices individually associated with the individual members, and wherein member engagement overall is based, at least in part, on interaction by multiple members with the content items.

22. The system of claim 21, wherein the interaction with the content items includes at least one of accessing a link included in the content item, commenting on the content item, sharing the content item, and providing an indication of approval of the content item.

23. The system of claim 13, wherein the organic content items are provided to the social network by the members and sponsored content items provided to the social network by sponsors of the sponsored content items, and wherein the operating condition is based, at least in part, on a percentage mix of organic content items and sponsored content items.

24. The system of claim 13, wherein the optimization criterion is based, at least in part, on engagement with the content items over a first time period and a second time period longer than the first time period.