A Brief into Design Thinking on Advertising Strategic Products

A Brief into Design Thinking on Advertising Strategic Products

2021｜Type: Original｜Tag: Experience

In design, there are many design tools for sorting out problems, starting with the user, discovering and defining the problem, and finally delivering the solution. In the field of machine learning, based on specific scenarios, we see another way of solving problems - the function approach.

Before we start

We embraced a very large number of terminology when we first entered the advertising industry. Every adman should have gone through the journey of climbing the hill of terminology, Automatic Bidding, One-Click-Started-Delivering, ROAS lift-based Strategy, and so on. Every day I examine myself three times, what is this, how to take effect, and the effect is credible? It seems that strategy design is not a unique term in the field of experience design, but may also be strategy algorithm, strategy framework, and so on.

Strategies have different breakthroughs, different audiences, and different depths of conversion, so that different meanings of the strategies that people hear. In the process of learning advertising strategic products, we gradually discovered the touchpoints in which designers can actively participate. During the exploration of the next-Gen ad network, from the perspective of designers, we can sort out how the experience design of strategic products is different from that of functional products, what the design process is like, and what the common design patterns are. What are the breakthroughs and opportunities for design?

What the strategy is

In specific scenarios, data analysis, algorithms or AI capabilities provided by the technology team are used to achieve business goals, user growth or risk control.

Methodology

Data strategy: data analysis and modelling as the approach.
Algorithm strategy: enterprises with mature businesses expect to solve existing business problems through AI technology.

Goal

Business Strategy: Serving business goals such as revenue and profit, we maximize business goals through the application of common business models such as advertising, value-added services, commissions, etc., while taking into account the balance with user experience.
User Growth Strategy: Serving the goals of user growth, active user growth, etc., such as optimizing and reducing the cost of customer acquisition through advertising strategies, and designing more effective push strategies through EDM, SMS, and other channels to reach audiences. Often needs to be segmented according to certain audiences or source channels, and design different strategies to approach the optimal solution for growth as much as possible.
Risk Control Strategies: Reduce potential future losses to the business, including but not limited to capital risk and policy risk. Capital risk can be further divided into enterprise capital risk and user capital risk, the former such as preventing fraudulent loans and loopholes, and the latter such as preventing fraud and theft.

Scenario

Search strategy: Roughly divided into general search engines like Google, but also includes searches for vertical applications like Zhihu. Solve the following problems:

How does it return accurate search results based on a user's query, rank the results, and match the user's intent?
Is there a way to predict a user's intent of searching in advance? How to perform search hints, error correction, and disambiguation?
How can the quality of content, such as web pages, be judged?

Recommendation Strategy: Roughly divided into content recommendation and goods recommendation. The former includes the recommendation system for applications such as graphic information, short and long videos, and audio, and the latter includes the recommendation system for the home page and detail page of major platforms such as e-commerce. Solve the following problems:

How to solve the problem of the learning phase, including the learning phase from 0 to 1 without feedback data, as well as new users or new content or goods.
What user, content, or goods-related leads can be used to help improve recommendation results?
How to evaluate and test the advantages and disadvantages of multiple recommendation methods.

Advertising strategy: Much more complex than search and recommendations. The following problems can be solved depending on the role of the industry in which they are located:

For Advertisers: how to minimize ad delivery costs and maximize ROI, how to identify cheating and fake traffic.
For Publishers: how to maximize ad revenue.
For Ad Networks and Exchanges: how to design a good strategy or mechanism that can strike a balance between minimizing advertisers' placement costs, maximizing publishers' revenue, and user experience, or at least two of them.

Scheduling strategy: Scheduling and matching usually need to be considered simultaneously. Scheduling generally refers to the coordination of resources under time and space constraints, and matching generally refers to the matching of market supply and demand. The following problems can be solved according to the different roles of supply and demand in the market:

Unilateral market scheduling: the demand side has limited influence, in the case of a certain demand, on how to solve the problem of optimal scheduling under the constraints of effectiveness and limited resources. E-commerce supply chains and distribution enterprises are the main ones.
Bilateral market scheduling: the platform has certain influence and control over both supply and demand sides. Scheduling and matching solve the problem of optimal matching when there is an imbalance between supply and demand in the local environment. The main focus is on dating and recruitment.
Trilateral market scheduling: the most complex scenario. Typical scenarios such as takeaway. The platform provides services for users, merchants, and riders while solving the matching problem of user demand and merchant supply, the matching problem of takeaway orders and riders, and the scheduling problem of rider delivery.

The forms of an advertising strategic product

The form is what is confusing at first, it seems that you can hear strategy everywhere, but it's not the same strategy. This is brought about by the differences in the audiences for which the strategies are aimed. It is more appropriate to understand it according to the role of the industry in advertising:

For Advertisers

Advertisers can directly take action on two categories of delivery strategy and data strategy, which usually exist on web pages and APIs.

Ad Delivery Strategy

Single Goal oCPX: Provide advertisers with automatic optimisation of effects based on ad delivery goals and bidding price, continuously improving the efficiency and ROI of advertisers.
Dual Goals oCPX: Currently, the implementation of Dual Goals optimisation is different in the WeChat traffic and non-WeChat traffic, with the former being a two-stage optimisation and the latter being a learned-based deep optimisation strategy.
Automatic Bidding: According to the advertiser's demands and budget, it spends the budget reasonably and efficiently, and finds the bidding plan with the lowest conversion cost.
One-Click-Started-Delivering: Advertisers set an ad-started-delivering budget, and the system will spend the budget quickly within 6 hours to help ads explore aggressively and gain impressions, during which the conversion cost may be high.
Priority Delivery: The system will spend the budget for your ad as soon as possible and raise the bidding price when necessary. Your actual cost may slightly exceed the target bidding price.
Stable Delivery: The system will keep your actual cost as close to the target bid as possible while trying to ensure stable delivery.
Limit Cost: the system will try to get more conversions without exceeding your advert's target bid.

Ad Data Strategy

ROAS lift-based Strategy: Supports advertisers in adjusting eCPM based on the different values of different audiences.

For Publishers

The strategies of publishers are to obtain more budget, improve the Ad Fill Rate as the goal, to further enhance the commercial value of the media. In addition to this, some publishers have new media access, new form exploration, ecological construction among media, and so on.

For Ad Networks and Exchanges

Provide general and basic strategies for advertising systems, such as the recalled model, the rank model, the learning phase of machine learning, bidding adjustment, etc. Provide scalable customized APIs for industry and media optimization, etc.

Retrieval: original targeting, ANN branching, TAG branching
TopN Scoring: Selected Aggregated Traffic, Business-Oriented Filtering, Docwash, Diverse Sorting
Rerank: eCPM Sorting Mechanism, Reranking, Business-Oriented Filtering, etc.

For Industry

Responsible for digging deep into industry-specific strategies, such as strategies for in-game promotion, and e-commerce promotion.

The life cycle of an advertising strategic product

The formula for calculating eCPM in advertising is the foundation of the majority of advertising strategic products. Decomposing the most basic calculation formula introduces bidding factors, calibration factors, industry factors, traffic factors, and user experience factors, which will have an effect on eCPM in the form of multiplication. However, there are currently nearly 100 factors of eCPM, the meaning of which is not clear, and it is difficult to quickly attribute, forming Simpson's paradox. However, in addition to eCPM strategies, there are also non-eCPM class strategies, such as Selected XS Traffic, Audience Network, WeChat Contracts, and so on. The current sorting mechanism has too many factors and strategies. Some strategies take effect in a certain industry and even squeeze other industries, which is bound to be not a long-term solution.

The next-Gen ad network at this stage is exploring a new framework where eCPM calculations will be reshaped by the value of Traffic, Audience, and the Ecological Platform. In addition to this, the factors will be sorted out and the access mechanism will be established. Let the strategies evolve actively by eliminating the fittest and the best. By tightening the access, in addition to experiments that directly follow the regular release process, it is also possible to launch HoldBack experiments (5% of the control group's traffic and 95% of the experimental group's traffic) to verify the long-term effect, as well as launching Reverse experiments (95% of the control group's traffic and 5% of the experimental group's traffic) to eliminate the impact of a certain strategy for verification.

The design process of an advertising strategic product

Discover

Industry Background

Before 2020, the large volume of data and the commonality of different industries were the main features of the shallow optimization goal. Since 2020, advertisers’ needs have changed significantly, more and more in pursuit of deep optimization goals, but the problem is also obvious, data is sparse, and industry differences are significant. This has led to huge differences in the application of strategic products in every industry. The common practice is to run a test in one industry where the problem is found, receive an excellent performance showcase, and then to other industries. Different tracks in the same industry are also witness to different developing stages, variability of conversion paths, filliness of data, data sparsity, and other issues. What needs to be confirmed in this is the industry background of the strategic product.

Competitive Analysis

Problems and opportunities encountered in the market are common, facing the same opportunity, we may stand on the same starting line with our competitors, and trusting customers to establish a deeper partnership has become the core solution to this problem. There is also a chance that our competitors could have gotten in on the ground, it is more suggested to conduct rapid research on the successful practices and the effect of application, to fill the gap.

User Research

Strategic products pursue the global optimal solution. strategic products could be divided into sub-strategies and factors by the audience, the different effective sessions, the industries, and the traffic. The final assessment is the global optimal solution. Functional products focus on the demands of core user groups. strategic products need to consider the demands of each segment and plan different strategies.

Data Analytics

Strategic products require a certain understanding of data and algorithms. Functional products can stand from the user’s view to think about what users need, strategic products are more from the machine’s view to find patterns.

Define

Objective

When we think about the objectives of an ad strategic product, we can start with the audience segment it is aimed at, what benefits it can bring to advertisers, publishers, and advertising platforms, and what specific problems it can solve for which clients in which industries.

Solution

As can be seen from the objective, the strategy relies on a specific problem for a specific type of client in a specific industry. We expect that the ways of solving the problem can start small, rather than reach a global optimal solution at the very first beginning. Sorting out the forms of the strategies is often encountered in industrial support strategy. It supports its industry but crowds out the quotas of other industries. We expect that every strategy can balance between local and global optimization, to make it a universal technique, thus promoting the overall evolution of strategies.

Develop

The biggest difference between the strategic product and the functional product in the whole process is that the roles of algorithm engineers and data engineers are involved. There are a few more self-iteration in the process to meet the solution so that every role can support each other to make it feasible as a whole.

Product: Steering the ship of the market, attracting demands from industry specialists and clients, discovering opportunities through data analysis, writing product requirements documents, and describing logic in detail.
Algorithm: With modelling as the core, it is necessary to define the boundaries as well as the inputs and outputs of the business and describe the business in a model language such as functions. This part of the work determines the goals and conductions of the subsequent model research work, which includes the collection and preparation of data, feature extraction, training, and evaluation.
Design: Understand PRDs and business modelling, find the design objectives of strategic products, and deliverables such as UX/UI design.
Developer: Evaluation based on business, modelling, and experience design, including but not limited to workloads, feasibility and ROI.

Delivery

Testing and Release

Regression testing to ensure that new bugs are not introduced into the newly launched strategic product. Release acceptance at scale until full release according to the programme.

Evaluation

Strategic products are more quantitative. Functional products tend to look at time-series trends, whereas strategic products often use A/B testing experiments to look at the impact of a single variable on an effect, or data modelling to analyse the impact of each variable.

Design Patterns of Advertising Strategic Products

Strengthened Trust

01 Affirmation of PIPL and Data Protection Agreement

Description: From access to use, communication with users is required regarding data security. Many strategic products rely on user data for more precise audience targeting. the PIPL, which came into force on 1 November 2021, has raised even higher requirements. We recommend that platforms and products that involve user data immediately ask for permissions upon entry and make it easy and intuitive for users to set them up. If needed, the time after the user's selection can also be disclosed so that updates and shutdowns can be made easily. When prompting users to set or view their permissions, explain the purpose and reason for using their data and make sure the explanation is easy to understand.
Case: The design of the Data Protection Agreement. After the advertiser unsigned the Data Protection Agreement, the platform temporarily did not support any features that involved data from the first-party. Based on the agreement, sort out the disabled actions across the platform as a result, clearly inform advertisers of those actions that are temporarily unavailable and show the reasons.

02 Reserve knowhow of industry and traffic

Description: Conversion paths vary widely across industries, fully understand the industry and its data. Establishing collaboration with industry experts could reduce the unnecessary iteration of datasets later. Advantages and disadvantages vary across conversion paths, as do limitations. For strategic products on the ad network side, you need to weigh traffic allocation to boost overall revenue, for strategic products on the traffic side, you need to improve the Global Fill Rate. Finding the right breakthrough point for a strategic product is also an important means to significantly improve the efficiency of the strategic product.
Case: ROAS lift-based Strategy, the breakthrough point of ROAS lift-based Strategy cannot be separated from the industry characteristics. Real estate, home, automotive and other industries, have low frequency with high ATV and hard decision-making time two obvious characteristics, resulting in the ad delivery failure to meet expectations and the data sparse. Knowledge of the actual industry and traffic can help us find the opportunity point for advertising strategic products

03 Encourage user practice and feedback

Description: Provide users the hands-on instruction, feedback, and error correction. Make sure users have the option to give feedback when a strategic product is working in a way they don't expect or want. And, use that feedback to improve the model as often as possible. Once the user submits feedback, confirm that feedback and even let them know how the system will respond to it.
Case: Ads Diagnostics. Users can give direct feedback when they think the conclusions of an ad diagnostic don't match their judgements based on their own experience.

Accurate Data

01 Improve data preparation

Description: the better the process of data planning, preparation, collection, access, governance, and measurement, the better the quality of the final output. If sufficient attention is not paid to data quality from the beginning, potential risks lie in data applications, turning into DATA CASCADES, which are difficult to diagnose and detect until they are affected. A good governance of the data can help us avoid downstream problems. You can improve the data by creating some projects: Collaborate with industry experts, Bulkly collect the data, Timely the feedback, and Embrace the noise.
Case: The data governance of first-party data, where the data link has been refined with iterations. In version 1.0, you can only see the data of Client ✕ Link ✕ Node. In subsequent iterations, in addition to integrating the optimization goal, the key actions, the attributes, and the attribute values are described at a finer granularity, to make the dimension richer, and to further solidify the foundation for measurement and application.

02 Check data labelling

Description: For supervised learning, accurate data labelling is one of the key factors of outputs in machine learning. Labelling can be done through an automated process or by a specialist. Effective and efficient tools can make it more likely that the data will be labelled correctly. When data labelling encounters labels that are difficult to deal with, it is possible to check if it is a deeper issue.
Case: In both the original Data Labelling Platform and the User Feature Platform, actions are provided for platform users to modify labels for inaccuracy.

03 Accept noisy data

Description: Simulate the real world and avoid striving for perfection when preparing the training set. Consider what data you wish to obtain from the user and ensure that the data is represented correctly in the training set.
Case: ROAS lift-based Strategy, uses a data table approach to selecting samples in a machine-learning-samples-based way. Multiple data tables will be joined into one training set. Random dereplication will be performed when the same user appears repeatedly in multiple user fields within multiple data tables.

04 Maintain Data Sets

Description: Maintain the experience by maintaining data quality. Monitor datasets promptly to identify issues. Differences between training data and live data, keeping an eye on changes in the dataset and the impact.
Case: Modelling, sample calibration of data tables accessing. Data updates within the data table can cause the dataset not to meet the base conditions when used.

05 Weigh accuracy and recall

Description: Define whether the goal is more or higher quality based on strategic product positioning. Evaluate the trade-off between whether the advice given by the system is broad (prioritising recall) or precise (prioritising accuracy). This decision will have a significant impact on the user experience. Prioritise accuracy: modelling in high-risk domains (e.g. healthcare) and where the risk from errors is high. Models output recommendations more conservatively and only output reliable recommendations. Accuracy is prioritised in this scenario. Prioritise Recall: in lower-risk domains (e.g. feeds) and where sorting the list of recommendations is not an issue, the user will get more results, with the possibility of episodic surprises and irrelevance. Recall is prioritised in this scenario.
Case: Modelling to extract the audience file from the predicted outputs.

Good Strategy

01 Establish Appropriate Expectations for Strategic Products

Description: What the strategic product can and cannot do needs to be communicated to the user. Because strategic products are probabilistic, they may give unexpected or even incorrect outputs at some point. This makes it critical to help users calibrate the capabilities and outputs. It's common practice to be explicit about the capabilities and limitations of a good strategic product.
Case: One-Click-Started-Delivering. It can boost ads cold boot and quickly determine the potential. However, this strategic product does not guarantee that the cold boot will be successful, that the budget will be spent in 6 hours, or that the conversion cost will be guaranteed. Problems within the settings and creatives, and complex real-time changes could challenge it. It's important to establish the right expectations for the One-Click-Started-Delivering.

02 Articulate specific goals and benefits for the strategic product

Description: Help users understand what the strategic product can accomplish and how it makes the experience better or delivers new value, rather than explaining how the underlying technology makes it happen. Of course, the technical details needed to explain the documentation will vary depending on the users for whom the strategic product is intended.
Case: Potential-Ads. Stepped price increases are expected to increase ad impressions, disclosing the most price-performance ratio bidding point to the user.

03 Provide necessary explanations for strategic products

Description: Explanations are for understanding, not for completeness. Reduce the information to what is necessary, provide the information they need to make decisions and take action, and don't try to explain everything that happens in the system. This is because the fundamentals behind a strategic product are usually too complex to be summarised in a simple sentence. If it's critical to explain the theory behind a strategic product to the user, make sure it appears in a stable, convenient help centre.
Case: ROAS lift-based Strategy, which uses a data table to select samples. The launched feature needs to meet the requirement that based on the joint of data tables, the number of rows in which the required fields meet the minimum after deduplication, and anything less than that will fail the calculation. This is a limitation of the back-end verification, which cannot be verified in a real-time way on the front end. The goal of the page is to reduce failures of launching, and in addition to some of the basic front-end checks that can be done, it also needs to be very careful to make clear the information needed for the actions.

04 Provide clear constraints for strategic products

Description: Restrictions are strategic red lines, common restrictions are: Restrictions of laws: restrictions of the PIPL, the Law on the Protection of Minors, etc. Limits of user experience: the upper limit times of the daily advertisements in WeChat Moments, etc. Limits of project resources: machine resources, quota occupation, etc. Limits of traffic: selected traffic patterns, etc. Limits of the industry: such as the game industry, etc.
Case: ROAS lift-based Strategy with quota limits using it. To reduce the cost of resources, each account has a quota limit.

05 Provide detailed documentation for strategic products

Description: Besides the user flow of a strategic product, documents also help users better understand how it works. This can help users understand the fundamentals of it, and provide how-to-use-it to help users grasp the mechanics in a specific context. This allows for more reliable information to be collected in the user feedback. It is important to note that documents usually open in a new page which means the user has left the current page and the original user flow is interrupted.
Case: Tencent Ads Data Management Platform Document.

06 Specify all error types for strategic products

Description: Understand the types of errors users are likely to encounter and deliver for the solution. Prepare for the error predictions of the strategic product during the discovery phase. Specify all the possible errors and consequences for the strategic product. Mechanisms can be set under the given prediction errors as well as false alarms or missing alarms: Manage the expectations of users’ actions through interpretation. Provide manual controls in case of failed predictions. Improve customer service support, etc.
Case: Modelling, and sample calibration of data tables. Typical back-end calibration, failure may be due to: sample data volume does not meet the conditions, fields and values can not be identified, field values not in the specified range, the same date under the same ID can not be both positive and negative samples and so on. These are some common errors in a training set.

07 Provide reversible actions for strategic products

Description: Allowing users to try out the strategic product at a low cost is particularly appropriate for new users who are eager to get started, who do not have time to fully consider, and who are wary of it until they fully understand what the system offers.
Even after getting started, it is important to continue to make user actions and decisions as reversible as possible. Commonly this can be achieved by providing visible actions such as undo, edit, and delete.
Case: Modelling, sample calibration of data tables. Each time the user enters the page will refresh whether the data table is available or not, and unavailable scenarios provide the user with edit action promptly.

08 Choose Familiar Expressions for Strategic Products

Description: With some strategic products, you may want to convey the magic of the strategy through fantastic visual elements and enlightening copy. Unfamiliar elements may raise the learning curve. Avoid over-decorating and help users get started with a new strategy through a familiar, comfortable UI, which can help them focus on the core task at hand and build trust.
Case: In addition to the visual elements of the interface, there is also the risk of over-used word SMART. Often the most straightforward syntax, or imaginative metaphors, can be used to clarify things for users, while vague and ambiguous language is usually not sufficient and should be avoided as much as possible.

09 Provide a trusted frame of reference for strategic products

Description: A benchmark can increase the trust of the user. Users may not trust the output of the strategic product, we suggest giving the output of the strategic product along with some benchmarks. It can be from the third-party evaluation, the median of the core metrics, the successful industrial experience and so on.
Case: In the evolution of advertising diagnosis, we attempted to disclose to users the performance of the industry and competitors as benchmarks in the same market. Specific cases can refer to the competition.

10 Attempt to disclose confidence for a strategic product

Description: In some cases, you can help users gauge the level of trust in the output of a strategic product by disclosing model confidence, which explains the certainty of the strategic product as well as alternatives.
Case: Modelling, disclosure of AUC and Logloss in model training results. why not just use Accuracy? Because many machine learning models predict classification problems as probabilities. To calculate Accuracy, you need to convert probabilities into categories first, which requires manually setting a threshold. If the predicted probability of a sample is higher than this prediction, put this sample into one category, below this threshold, put it into another category. So this threshold greatly affects the calculation of Accuracy. Using AUC or Logloss can avoid converting the predicted probability into categories. In Advertising, what we want to measure is the ranking of ads for each user, and what we need to calculate is the binary classification result for each user, which is a more fine-grained binary classification. So the traditional AUC is not very applicable. So there is a GAUC (short for Group AUC), the actual calculation of the AUC of each user, through the weighted average to get the GAUC, to reduce the bad impact of comparing the ranking of ads for each user. Log loss reflects the average deviation of the samples and is often used as the loss function of the optimisation model. The smaller the log loss, the better. That is a measure of the degree of fit between the predicted CTR and the actual CTR.

Stable Control

01 Try to automate the low-risk strategic products

Description: When determining automation, weigh the stakes and how comfortable users are with automation. Recommended in the low-risk, well-established strategy products, such as content recommendation systems. Try to avoid using it in high-risk, or new products. Result in lower user trust or a greater likelihood of error.
Case: Automated products that reduce labour costs by establishing rules belong to low-risk and user-controllable products. In addition, the cost may drift when Automated-Bidding is taking effect in the started-delivering phase, the degree of aggressiveness is between the One-Click-Started-Delivering and manual bidding, after the started-delivering phase, it will explore the lowest conversion cost. This is also a reflection of the stable control in the automation.

02 Try to automate the strategic products in steps

Description: Critically consider the balance between automation and user control. Start with the lowest level of automation and increase the level of automation progressively. Ensure that users easily adjust the level of automation.
Opportunity: There are three versions of the Tencent Ads Data Management Platform, the Basic Edition, the Professional Edition and the Private Edition. In the future, based on the customer's usage and effects, we will evaluate the customer level based on core indicators and open up the features progressively based on it.

03 Try to supervise the Automation of Strategic Products

Description: Keeping users supervising the automation can offer users security, and users can fix the problems as they arise. At the same time, permissions could be set and reclaimed when necessary.
Opportunity: ROAS lift-based Strategy has a quota limit. Quotas can be automatically assigned based on the advertiser's usage and effects, and strategies that don't work well are automatically taken offline and replacement strategies are automatically brought online.

04 Return the control right to users when automation fails

Description: If the strategic product fails to complete the task as expected, malfunctions during execution, or provides poor-quality output in the execution result, the most direct way to clarify the reason for the disruption or error and provide users with actionable alternatives is to let the users take over, even if they need to be provided with additional explanatory documents or even manual support.
Case: Tencent Ads Data Management Platform, which establishes a timely chat between users and the R&D teams through WorkWeChat. When the user encounters a tricky problem, the R&D teams will be the first to define the problem and provide support.

Last but not least

The use of design thinking to explore the opportunity of advertising strategic products, and even all strategic products, requires designers to roughly grasp the core principles of their business, have a preliminary understanding of the characteristics of the industry and traffic, and have the ability to analyse basic data. In advertising, how is eCPM calculated, what are the characteristics of the industry's data link, what is the customer assessment, how long is the cycle of conversion and so on? This allows experience design to go deeper, into the logic and deliver the interface at shallow. Thinking about the user experience gap of business under the design process and design pattern may be the breakthrough for designing strategic products under mature business and massive data.

Reference

The Strategic Product Manager: Models and Methodologies
Zhou Xing: What is a good advertising strategy