A review of causality-based fairnessmachine learning

With the wide application of machine learning driven automated decisions (e.g., education, loan approval, and hiring) in daily life, it is critical to address the problem of discriminatory behavior toward certain individuals or groups. Early studies focused on defining the correlation/association-based notions, such as statistical parity, equalized odds, etc. However, recent studies reflect that it is necessary to use causality to address the problem of fairness. This review provides an exhaustive overview of notions and methods for detecting and eliminating algorithmic discrimination from a causality perspective. The review begins by introducing the common causality-based definitions and measures for fairness. We then review causality-based fairness-enhancing methods from the perspective of pre-processing, in-processing and post-processing mechanisms, and conduct a comprehensive analysis of the advantages, disadvantages, and applicability of thesemechanisms. In addition, this review also examines other domainswhere researchers have observed unfair outcomes and theways they have tried to address them. There are still many challenges that hinder the practical application of causality-based fairness notions, specifically the difficulty of acquiring causal graphs and identifiability of causal effects. One of the main purposes of this review is to spark more researchers to tackle these challenges in the near future.


INTRODUCTION
Artificial intelligence (AI) techniques are widely applied in various fields to assist people in decision-making, such as hiring [1,2] , loans [3,4] , education [5] , criminal risk assessment [6] , etc.The motivation for using machine learning models is that they can mine hidden laws and useful information from data with huge volumes and various structures more quickly and effectively than human beings.Most importantly, people often mix personal emotions when making decisions, making their decisions unfavorable to certain groups.It is canonically believed that the decisions made by automatic decision-making systems are more objective, and, thus, there will be no discrimination against specific groups or individuals.However, this assumption cannot always be met.Due to the biased training data and inherent bias of adopted models, machine learning models are not always as neutral as people expect.
Since many automated systems driven by AI techniques can significantly impact people' s lives, it is important to eliminate discrimination embedded in the AI models so that fair decisions are made with their assistance.Indeed, in recent years, fairness issues of AI models have been receiving wide attention.For example, automated resume screening systems often give biased evaluations based on traits beyond the control of candidates (e.g., gender and race), which may not only discriminate against job applicants with certain characteristics but also cost employers by missing out on good employees.Early research on achieving fairness of algorithms focused on statistical correlation and developed many correlation-based fairness notions (e.g., predictive parity [7] , statistical parity [8] , and equalized odds [9] ), which primarily focus on discovering the discrepancy of statistical metrics between individuals or sub-populations.However, correlation-based fairness notions fail to detect discrimination in algorithms in some cases and cannot explain the causes of discrimination, since they do not take into account the mechanism by which the data are generated.A classic example is Simpson' s paradox [10] , where the statistical conclusions are drawn from the sub-populations and the whole population can be different.On the other hand, discrimination claims usually require demonstrating causal relationships between sensitive attributes and questionable decisions, instead of the association or correlation between the sensitive attributes and decisions.
Consider the example of the graduate admissions at University of California, Berkeley in 1973, which confirms the importance of developing causal perspective admission to detect and eliminate discrimination.From the statistical results of historical data, roughly 44% of all men who applied were admitted, compared to 35% of women who applied.Then, a flawed conclusion may be drawn with the support of the difference in admission rates between males and females.That is, there exists discrimination towards women for their graduate admission.After an in-depth examination of this case, there is no wrongdoing by the educational institution, but a larger proportion of women applied to the most competitive departments, causing a lower admission rate than men.However, the question of discrimination is far from resolved, e.g., there is no way of knowing why women tended to apply to more competitive departments from the available data alone.Therefore, it is helpful to detect discrimination and interpret the sources of discrimination by understanding the data generating mechanism, namely the causality behind the problem of discrimination.In addition, causal models can be regarded as a mechanism to integrate scientific knowledge and exchange credible assumptions to draw credible conclusions.For this admission case, it seems that, due to women' s socialization and education, they tend to toward fields of studies that are generally crowded.Therefore, it is necessary to explore the causal structure of the problem.Fortunately, more and more researchers have paid attention to detecting and eliminating discrimination from the perspective of causality, and various fairness concepts and fairness-enhancing methods based on causality have been proposed.
Compared with the fairness notions based on correlation, causality-based fairness notions and methods take additional consideration of the knowledge that reflects the causal structure of the problem.This knowledge reveals the mechanism of data generation and is helpful for us to comprehend how the influence of sensitive attributes change spreads in the system, which is conducive to improving the interpretability of model decisions [11][12][13][14] .Therefore, causality-based fairness machine learning algorithms help to enhance fairness.However, causality-based fairness approaches still face many challenges, one of which is unidentifiable situations of causal effects [15] .In other words, the causal effect between two variables cannot be uniquely computed from the observational data without extra assumptions.
The previous survey articles offer a high-level summary of technology to eliminate algorithm discrimination, and there is no detailed discussion on specific subareas [16][17][18][19] .Wan et al. [20] provided an exhaustive review of the methods of using the in-process mechanism to solve the fairness issues.Makhlouf et al. [21] summarized the advantages and disadvantages of each fairness notion and their suitability, aiming to help us select the fairness notions that are most suitable for a particular scenario.Makhlouf et al. [22] only focused on reviewing the concept of fairness based on causality.Instead of eliminating discrimination from a technical perspective, Wu et al. [23] summarized human-oriented fairness-enhancing methods as a way to explore the role of human beings in addressing algorithmic fairness.
To complement the previous survey papers on fairness-enhancing machine learning, this survey thoroughly reviews the concept of fairness based on causality and summarizes the core ideology behind the causality-based fairness-enhancing approaches.This survey aims to stimulate future exploration of causality-based fairness technology, because of the importance of causal modeling in improving algorithmic fairness.In this survey, the review of the concept and technology of fairness based on causality is conducted in several phases.The survey first reviews causality-based definitions and measures of fairness and summarizes the suitability of these causality-based fairness notions.Next, it provides a comprehensive overview of state-of-the-art methods to achieve fairness based on these causality-based fairness notions.The survey also discusses the practical applications, beyond classification, that the causality-based fairness methods are expected to benefit greatly.Finally, this survey discusses the challenges of eliminating discrimination from a causal perspective, including the acquisition of causal graphs and identifiable issues.It also reviews the efforts for addressing these challenges and summarizes the remaining issues, which provides some assistance to solve these problems for future research.Figure 1 shows the organizational structure of this survey.
The rest of this survey is structured as follows.Section 2 introduces an example to interpret the importance of causal modeling for addressing fairness issues.Section 3 presents the background of the causal model.Section 4 introduces definitions and measures of causality-based fairness notions and discusses the suitability or applicability of them.Section 5 discusses fairness mechanisms and causality-based fairness approaches, and compares these mechanisms.Section 6 introduces several typical applications of causality-based fairness methods.Section 7 analyzes the challenges and the research trends for applying causality-based fairness-enhancing methods in practical scenarios.

THE IMPORTANCE FOR CAUSALITY TO DETECT DISCRIMINATION: AN EXAMPLE
The importance of applying causal analysis to discrimination discovery is explored in this section.Consider a simple example that is inspired by a legal dispute about religious discrimination in recruitment [24] .To keep the situation simple, assuming that a company takes the religious belief  ( = 1 if an applicant has a religious belief, otherwise  = 0) and the education level  ( = 1 denotes a candidate has received higher education, while  = 0 denotes the education level of am applicant is low) into account for deciding whether to hire applicant  ( = 1 represents the company decides to hire the applicant, otherwise  = 0).The religious belief of an individual can be influenced by several variables, but, in this example, assume that  is only influenced by the individual' s education level.For concreteness, we assume that, based on statistics on the collected data, we have the following conditional probabilities: Assume also that the proportions of applicants with faith are equal to those without religious beliefs, and the proportions of applicants with high education are the same as the one with low education, which means that ( = 1) = ( = 0) = 0.5 and ( = 1) = ( = 0) = 0.5.Then, the hiring decision made by the company is suspected of prejudice against applicants with religious beliefs because there are statistical differences in religious composition among employees (corresponding to statistical parity [8] ): From such probabilities mentioned above, the company is more likely to hire the applicants without religious beliefs than ones with faith, which indicates that its hiring decisions are unfair.However, this conclusion is wrong, since it only considers the association between religious beliefs and hiring decisions, instead of the causal relationship.In fact, understanding causal mechanisms behind hiring decisions can avoid drawing such wrong conclusions, since it exposes the mechanism of data generation.Through the causal analysis for three variables in the above example, the education level of the individuals is an observable confounder, that is, an individual' s educational background influences both his (or her) religious beliefs and hiring decisions made by the company.The higher the education level of individuals, the less willing they are to participate in religious activities.The causal relationships between these variables are shown in Figure 2(a).Based on the causal graph, further causal analysis of hiring decisions of this company is conducted to explore whether it is really discriminatory.In other words, intervention on  is performed to block the influence of  on  to evaluate the causal effect of  on  (more details of intervention can be seen in Section 3). Figure 2(b) shows the causal structure of such an example after intervening on .The hiring decisions made by the company are fair, if the hiring proportions when all applicants in the population have religious beliefs are the same as the hiring proportions when all applicants in the population have no religious beliefs, i.e., ( = 1|( = 1)) = ( = 1|( = 0)).Formally, these probabilities in this example are obtained as below: These values confirm that the hiring decisions made by the company do not discriminate against applicants with religious beliefs.Therefore, it is critical to conduct a causal analysis of the problem, since understanding the causal mechanisms behind the problem can not only help to detect discrimination but also help to interpret the sources of discrimination.

PRELIMINARIES AND NOTATION
In this review, an attribute is denoted by an uppercase letter, e.g., ; a subset of attributes is denoted by a bold uppercase letter, e.g., X; a domain value of attribute  is denoted by a lowercase letter, e.g., ; and the value assignment of subset attributes X is denoted by a bold lowercase letter, e.g., x.In particular,  represents the sensitive attribute (e.g., race) and  represents the predicted result of the AI model system (e.g., loans).
One of the most popular causal model frameworks is Pearl' s Structural Causal Model (SCM) [10] .A structural causal model M is represented by a quadruple ⟨U, (U), V, F⟩: 1. U denotes exogenous variables that cannot be observed but constitute the background knowledge behind the model.2. (U) represents the joint probability distribution of U. 3. V denotes endogenous variables that can be observed.4. F denotes a set of functions mapping from U ∪ V to V, which reflects the causal relationship between variables.For each  ∈ V, there is a mapping function   ∈ F from U ∪ (V \ ) to , i.e.,  =   ((),   ), where parent variables () ⊂ V \  are the endogenous variables that directly control the value of , and   is a set of exogenous variables that directly determine .
A causal model M is associated with a causal graph G = ⟨V, E⟩, a directed acyclic graph, where V is a set of nodes, each of which represents an endogenous variable of V, and each element in E indicates a directed edge →, pointing from a node  ∈ U ∪ V to another node  ∈ V if   uses values of  as input, which represents a causal relationship between the corresponding variables.The exogenous variables U can be either independent or dependent.If the exogenous variables U are mutually independent, the causal model is called Markovian model.In this case, exogenous variables typically are not represented in the causal diagram.In the case, the exogenous variables are mutually dependent (hidden confounders), the causal model is called the semi-Markovian model.In the semi-Markovian model, dashed bi-directed edges are used to represent the hidden confounders between two variables.pointing to variable .represents the population distribution of  if everyone in the population had their  value fixed at .This post-intervention distribution ( |()) is considered a counterfactual distribution since the intervention () forces  to take a certain value different from the one it would take in the actual world.For example, if  represents sex ( + , male;  − , female) and  represents the hiring decision ( + , hired;  − , not hired), (  − | =  + ,  =  + ) involves two worlds: a real world that a male applicant has been hired and a counterfactual world where the same applicant is female.Such expression means that, when a job-hunter whose gender is male has been observed to be hired, what is the probability that the same job-hunter would still be hired if this job-hunter were female.
Causality-based fairness notions aim to tell whether the outcome of a decision made by the AI decision model is discriminative, which are expressed by interventional and counterfactual probability distributions.The application of the causality-based fairness notions not only requires a dataset D as input but also relies on a causal graph G. Causal approaches aim to limit the causal effects of sensitive attributes on decisions, which are computed by interventional and counterfactual probabilities.However, since these probabilities cannot be observed, they fail to be uniquely assessed from D and G in some cases, which is called the unidentifiable issue.In other words, if two variables have different causal effect measures resulting from different causal models that all agree with the observational distribution, the causal effects are unidentifiable.

CAUSALITY-BASED FAIRNESS NOTIONS
Pearl defined the causality as three rungs: correlation, intervention, and counterfactual [25] .The first rung (correlation) reflects the ability of observation, which aims to discover the patterns in the environment.The second rung (intervention) reflects the ability of action, which refers to the prediction of the results of deliberate changes to the environment.The third rung (counterfactual) refers to the ability to imagine the counterfactual world and speculate on the causes of the observed phenomena.The second and third rungs aim to expose the root causes of the patterns that we observe.Thus far, many fairness metrics have been proposed and all of them can be placed in such causal rungs.Figure 4 presents categorization of fairness notions.Early definitions of fairness are based on statistical correlations, all of which can be found at the first rung.All causality-based fairness notions can be found at the second and third rungs, each of which considers the mechanism of data generation.Thus, causality-based fairness notions can better reveal the causes of discrimination than statisticsbased ones and have been attracting more and more attention.

Group
Total Effect [10] | The causal effects of the value change of the sensitive attribute  from  + to  − on decision  = , where the intervention is transmitted along all causal paths and is within the fair threshold  Effect of treatment on the treated [10] |  ( The difference between the distribution of  =  had  been  + and that of  =  had  been  − , given that  had been observed to be  − Path-specific fairness [26,27] | The causal effects of the value change of the sensitive attribute  from  + to  − on decision  =  along specific causal paths, is within the fair threshold  No unresolved discrimination [28] -It is satisfied when there is no directed path from sensitive attribute  to outcome  allowed, except through a resolving variable No proxy discrimination [28] If it is satisfied, there is no path from the sensitive attribute  to the outcome  blocked by a proxy variable

Individual
Counterfactual fairness [11] |  ( Individual direct discrimination [29]  (, It is based on situation testing where the causal reasoning is used to define the distance function  (,  ′ ) Equality of effort [30] Ψ+ () = Ψ− () where It detects discrimination by comparing the effort required to reach the same level of outcome of individuals from advantaged and disadvantaged groups who are similar to the target individual Hybrid PC-fairness [31] |

Individual -based
Based on similarity individuals

Total effects
Path-specific fairness In the real world, the focus of different machine learning tasks is different, and thus, various causality-based fairness notions are proposed to detect discrimination in different scenarios.This section introduces several representative causality-based fairness measurements that quantify fairness from the perspective of groups or individuals, respectively.Without loss of generality, assume that the sensitive attribute  and the outcome of the automated decision making  are binary variables where  =  + denotes the advantaged group (e.g., white men) and  =  − denotes the disadvantaged one (e.g., non-white men).Table 1 summarizes various causality-based fairness notions falling under different types.

Group causalitybased fairness notions
Group fairness notions aim to discover the difference in outcomes of AI decision models across different groups.
The value of an individual' s sensitive attribute reflects the group he (or she) belongs to.Considered an example of salary prediction where  + and  − represent male and female groups, respectively.Some representative group causality-based fairness notions are introduced as follows.

Total effect
Before defining total effect (TE) [10] , statistical parity (SP) is first introduced, since it is similar to TE but is fundamentally different from TE. SP is a common statistics-based fairness notion, which denotes similar individuals treated similarly regardless of their sensitive attributes.Statistical parity is satisfied if Intuitively, () measures the conditional distributions of  change of one' s sensitive attribute  from  + to  − , and it is considered to be fair if the difference between the conditional distributions is within the fair threshold .The main limitation of () is that () is unable to reflect the causal relationship between  and  .Total effect is the causal version of statistical parity, which additionally considered the generation mechanism of the data.Formally, total effect can be computed as follows: TE measures the difference between total causal effect of sensitive attribute  changing from  + to  − on decision  = .Intuitively, statistical parity represents the difference in probabilities of =  in the sampling population, while total effect represents the difference in probabilities of  =  in the entire population.
A more complex total effect considers the effect of changes in the sensitive attribute value on the outcome of automated decision making when we already observed the outcome for that individual, which is known as the effect of treatment on the treated (ETT) [10] .This typically involves a counterfactual situation which requires changing the sensitive attribute value of that individual at that time to examine whether the outcome changes or not.ETT can be mathematically formalized using counterfactual quantities as follows: where (  + | − ) represents the probability of  =  had  been  + , given  had been observed to be  − .(  − | − ) = (| − ) represents the conditional distributions of  =  when we observe  =  − .Such probability involves two worlds: one is an actual world where  =  − and the other is a counterfactual world where for the same individual  =  + .Notice that (  − | − ) = (| − ) for consistency.
Other fairness notions similar to TE are also proposed.For example, FACT (fair on average causal effect) [32] was proposed to detect discrimination of automated decision making, which is based on potential outcome framework [33,34] .It considers an outcome  is fair, if the average causal effect over all individuals in the population of the value changes of  from denotes the potential outcome of an individual  had  been  + .
TE and ETT both aim to eliminate the decision bias on all causal paths from  to  .However, they cannot distinguish between direct discrimination, indirect discrimination, and explainable bias.Two alternative graphs for the loan application system.(a) A causal graph of the loan application system, where Race is the sensitive attribute and Loan is the decision.(b) A causal graph of the system after removing unresolved discrimination.(c) A causal graph of the system that is free of proxy discrimination.

Path-specific fairness
The causal effect of sensitive attribute on the outcome can be divided into direct effect and indirect effect, and it can be deemed fair or discriminatory by an expert.Direct discrimination can be captured by the causal effects of  on  transmitted along the direct path from  to  , while indirect discrimination is measured using the causal effect of  on  along causal paths from  to  that pass through redlining/proxy attributes.

Figure 5(a) represents the causal graph of a simple example of a toy model of loan decision AI model, where
Race is treated as the sensitive attribute and Loan is treated as the decision.Since ZipCode can reflect the information of Race, ZipCode is a proxy for the sensitive attribute, that is to say, ZipCode is a redline attribute.Thus, the causal effects spreading along the path Race → Loan are then considered to be direct discrimination, and the causal effects spreading along the path Race → ZipCode → Loan are considered to be indirect discrimination.Note that the causal effects spreading along the path Race → Income → Loan are explainable bias since it is reasonable to deny a loan to an applicant if he (or she) has a low income.That is to say, the partial difference in loan issuance across different race groups can be explained by the fact that some racial groups in the collected data tend to be underpaid.
Path-specific effect [10] is a fine-grained assessment of causal effects, that is, it can evaluate the causal effect transmitted along certain paths.Thus, it is used to distinguish among direct discrimination, indirect discrimination, and explainable bias.For any set of paths , the -specific effect can be computed as below: where (  + |, − | π ) denotes the distribution of  =  where the intervention ( + ) (i.e., force  had  + ) is only transmitted along path  while the intervention ( − ) (i.e., actual world  =  − ) is transferred along the other paths (denoted by π).If  contains all direct edge from  to  ,   () measures the direct discrimination.If  contains all indirect paths from  to  that pass through redlining/proxy attributes,   () evaluates the indirect discrimination.If  contains all indirect paths from  to  that pass through explaining attributes,   (𝑦) assesses the explainable bias.

No unresolved/proxy discrimination
No unresolved discrimination [28] is a fairness notion which is based on Pearl' s structural causal model framework and aims to detect indirect discrimination.This criterion is satisfied if there is no directed path from the sensitive attribute  to the outcome which is not blocked by the resolving variables.A resolving variable is any variable in a causal graph that is influenced by the sensitive attribute to a certain degree but accepted by practitioners as nondiscriminatory, which is very similar to the use of explanatory attributes in the statistics-based fairness notion.For example, Figure 5 shows three causal graphs of a simple loan example.There exists such discrimination in the causal graph shown in Figure 5(a) since the effects of Race on Loan can be transmitted along the causal paths Race → Loan and Race → ZipCode → Loan, while there is no unresolved discrimination, since the effects of Race on Loan can only be transmitted through resolved attribute Income along Race → Income → Loan.
Similar to no unresolved discrimination, no proxy discrimination [28] also focuses on indirect discrimination.Given a causal graph, if this criterion is satisfied, the effects of the sensitive attribute  on the output  cannot be transmitted through any proxy variable  (which is also denoted as redlining variable).A proxy variable is a descendant of sensitive attribute  and the ancestor of decision attribute  .It is labeled as a proxy because it is exploited to capture the information of .The outcome of an automated decision making  exhibits no proxy discrimination if the equality of the following equation is valid for all potential proxies : In other words, this notion implies that changing the value of  should not have any impact on the prediction.A simple example is shown in Figure 5. ZipCode is a redlining variable due to it reflects the information of the sensitive attribute Race.There is no proxy discrimination in causal graph shown in Figure 5(c), since the causal path Race → ZipCode → Loan has been blocked by intervening ZipCode.
No unresolved discrimination is a flawed definition of fairness.Specifically, no unresolved discrimination criterion is unable to identify some counterfactual unfair scenarios where some attributes are deemed as the resolved attributes.On the other hand, policy makers and domain professionals should carefully examine the relevance between sensitive variables and other endogenous variables so as to discover all resolving attributes and potential proxies that may lead to discrimination spread.

Individual causalitybased fairness notions
Different from group fairness notions that measure the differences in the outcome of decision models between advantaged groups and disadvantaged ones, individual fairness notions aim to examine whether the outcome of decision models is fair to each individual in the population.Some representative group causality-based fairness notions are discussed here.

Counterfactual fairness
An outcome  achieves counterfactual fairness towards an individual  (i.e., O = o) if the probability of the outcome  =  for such individual  is the same as the probability of  =  for the same individual whose value of sensitive attribute changing to another one.Formally, counterfactual fairness can be expressed as follows for any O = o: where Counterfactual fairness was proposed by Kusner et al. [11] .They empirically tested whether the automated decision making systems are counterfactual fairness by generating the samples given the observed sensitive attribute value and their counterfactual sensitive value; then, they fitted decision models to both the original and counterfactual sampled data and examined the differences in the prediction distribution of predictor between the original and the counterfactual data.If an outcome  is fair, the predictor is expected that the predicted results of actual and counterfactual distributions lie exactly on top of each other.

Individual direct discrimination
Individual direct discrimination [29] is a situation testing-based technique [35] guided by the structural causal model for analyzing the discrimination at the individual level.Situation testing is a legally grounded technique to detect the discrimination against a target individual by comparing the outcome of the individuals similar to the target one from both the advantaged group and the disadvantaged one in the same decision process.In other words, for a target individual , select top- individuals most similar to  from the group  =  + (denoted as  + ) and top- individuals most similar to  from the group  =  − (denoted as  − ), and then perform oneto-one pairing according to the similarity ranking.The target individual is considered as discriminated if a significant difference is observed between the rate of positive decisions for all pairs from  + and  − (typically, higher than fair threshold ).The key issue for implementing situation testing is how to define a distance function  (,  ′ ) to measure similarity between individuals.
For the individual direct discrimination criterion, the distance function  (,  ′ ) is defined not only by adopting normalized Manhattan distance and overlap measurement but also by incorporating causal inference.Specifically, given a causal graph, only the variables that are direct parent nodes of the decision variable are considered to compute the similarity between individuals, which are denoted as X = ( ) \ {}.The formal definition of  (,  ′ ) is as follows: where  (  ,  ′  ) represents the causal effect of each of the selected variables (  ∈ X) on the outcome and   (  ,  ′  ) is a distance function proposed by Luong et al. [36] .Specifically, the normalized Manhattan distance is employed for ordinal/interval variables (i.e., , where  denotes the difference between the maximum and minimum of the variable   ) and the overlap measurement is employed for categorical variables (i.e.,   (  ,  ′  ) = 0 if   =  ′  , and   (  ,  ′  ) = 1 otherwise).
For each selected variable   , the definition of  (  ,  ′  ) is as follows: where (|(x)) is the effect of the intervention that forces X to take the set of values x, and (|( ′  , x \   )) is the effect of the intervention that forces   to take value  ′  and other variables in X to take the same values as x.

Equality of effort
Equality of effort fairness notion [30] detects bias by comparing the effort required to reach the same level of outcome of individuals from advantaged and disadvantaged groups who are similar to the target individual.That is to say, given a treatment variable , it quantifies how much this treatment variable  should change to make the individual achieve a certain outcome level in order to address the concerns of whether the efforts required to achieve the same level of outcome for individuals from the advantaged and disadvantaged groups are different.Following Rubin' s potential outcome model framework [37] , let    be the potential outcome for individual  had  been  and E[   ] be the expectation of outcome for individual .Then, for individual , the needed minimal value of treatment variable  to achieve -level of outcome is defined as follows: Unfortunately,    is not observable, which results in Ψ  () being uncomputable.Situation testing is then used to estimate it, where the distance function  (,  ′ ) of equality of effort is the same as individual direct discrimination mentioned in Section 4.2.2.Let  + and  − be the two sets of individuals with  =  + and  =  − that are similar to the target individual , respectively, and E[   + ] be the expected outcome under treatment  for the subgroup  + .The minimal effort required to achieve -level of outcome variable within the subgroup  + is computed as follows: Then, for a certain outcome level , individual -equal effort is satisfied for individual  if: Equality of effort can also be extended to identify discrimination at any sub-population level or system level, when  + is extended to the entire group with  =  + and  − denotes the entire group with  =  − .To distinguish individual -equal effort,  + is used to denote the first set, while  − denoted the second one.The -equal effort is satisfied for a sub-population if:

PC-Fairness
Path-specific Counterfactual Fairness (PC-fairness) [31] is used to denote a general fairness formalization for representing various causality-based fairness notions.Given a factual condition O = o where O ∈ V and a causal path set , a predictor Ŷ achieves the PC-fairness if it satisfies the following expression: where and  is a predefined fairness threshold (typically, 0.05).Intuitively,   ( ŷ − → + |o) denotes when the value of the sensitive attribute  changes from  + to  − , the causal effect of  on Ŷ through the causal path set  and given the factual observation o.
PC-fairness matches different causality-based fairness notions by tuning its parameters.For example, if the path set  contains all causal paths and O = , PC-fairness corresponds to the total effects in Equation (3).
Apart from that, it also includes new types of fairness that have not been studied yet in the past.For example, PC-fairness can detect individual indirect discrimination by letting O = V\ { } and the path set  containing all causal paths that pass through any redlining variables.

CAUSALITY-BASED FAIRNESS-ENHANCING METHODS
The need for causal models for detecting and eliminating discrimination is based on the intuition that the same individuals experience different outcomes due to innate or acquired characteristics outside of their control (e.g., gender).Therefore, causal models are useful for investigating which characteristics cannot be controlled by individuals and using the resulted understandings to identify and deal with discrimination.In other words, understanding the structure of root causes of the problem can assist in identifying unfairness and causes.Thus, there is a causal structure that must be considered rather than just the correlation between the sensitive attribute and outcome.Because of these advantages, many recent studies introduce fairness-enhancing approaches from the perspective of causality.According to the stages of training the machine learning algorithms, pre-processing, in-processing, and post-processing mechanisms can be used to intervene in the algorithm to achieve fairness.Therefore, causal-based methods can be divided into the above three categories.Figure 6 shows the general flow of different categorical causality-based approaches.This section provides an overview of studies for these categories, and then the advantages and disadvantages of these three types of mechanisms are summarized.

Preprocessing Causalitybased methods
Pre-processing methods update the training data before feeding them into a machine learning algorithm.Specifically, one idea is to change the labels of some instances or reweigh them before training to limit the causal effects of the sensitive attributes on the decision.As a result, the classifier can make a fairer prediction [36] .On the other hand, some studies propose to reconstruct the feature representations of the data to eliminate discrimination embedded in the data [38,39] .
For example, Zhang et al. [27] formalized the presence of discrimination as the presence of a certain path-specific effect, and then framed the problem as one maximizing the likelihood subject to constraints that restrict the magnitude of the .To deal with unidentifiable cases, they mathematically bound the .CFGAN [40] , which is based on Causal GAN [41] to learn the causal relationship between the attributes, adopts two generators to separately simulate the underlying causal model that generates the real data and the causal model after the intervention and two discriminators to produce a close to real distribution and to achieve total effect fairness, counterfactual fairness, and path-specific fairness.Salimi et al. [42,43] leveraged the dependencies between sensitive and other attributes, which is provided by the causal knowledge, to add or remove samples from the collected datasets in order to eliminate the discrimination.Nabi et al. [44] only considered mapping generative models for ( , V \ W|W) consisting of some attributes W to "'fair" versions of this distribution  * ( , V \ W|W) and ensured that  * (W) = (W).PSCF-VAE [45] achieves path-specific counterfactual fairness by modifying the observed values of the descendant attribute of the sensitive attribute on the unfair causal path during testing, leaving the underlying data-generation mechanism unaltered during training.

InProcessing Causalitybased methods
In-processing methods eliminate discrimination by adding constraints or regularization terms to machine learning models [46][47][48][49][50] .If it is allowed to change the learning procedure for a machine learning model, then in-processing can be used during the training of a model either by incorporating changes into the objective function or imposing a constraint.
For example, multi-world fairness algorithms [12] add constraints to the classification model that require satisfying the counterfactual fairness.To address the unidentifiable situation and alleviate the difficulty of determining causal models, it combines multiple possible causal models to make approximately fair predictions.A tuning parameter  is used to modulate the trade-off between fairness and accuracy.Hu et al. [51] proposed to learn multiple fair classifiers simultaneously from a static training dataset.Each classifier is considered to perform soft interventions on the decision, whose influence is inferred as the post-intervention distributions to formulate loss functions and fairness constraints.Garg et al. [52] proposed to penalize the discrepancy between real samples and their counterfactual samples by adding counterfactual logit pairing (CLP) to the loss function of the algorithm.Similarly, the authors of [53] proposed to add constraints terms during the training to eliminate the difference in outcome between two identical individuals, one from the real world and one from the counterfactual world that belongs to another sensitive group.Kim et al. [54] addressed the limitation that some causality-based methods cannot distinguish between information caused by the intervention (i.e., sensitive variables) and information related to the intervention by decomposing external uncertainty into intervention-independent variables and intervention-related ones.They proposed a method called DCEVAE, which can estimate the total effect and counterfactual effects in the absence of full causal maps.

Postprocessing Causalitybased methods
Post-processing methods modify the outcome of the decision model to make fairer decisions [55,56] .For example, Wu et al. [57] adopted the c-component factorization to decompose the counterfactual quantity, identified the sources unidentifiable terms, and developed the lower and upper bounds of counterfactual fairness in unidentifiable situations.In the post-processing stage, they reconstructed the trained decision model so as to achieve counterfactual fairness.The counterfactual privilege algorithm [58] maximizes the overall benefit while preventing an individual from obtaining beneficial effects exceeding the threshold due to the sensitive attributes, so as to make the classifier achieve counterfactual fairness.Mishler et al. [59] suggested using doubly robust estimators to post-process a trained binary predictor in order to achieve approximate counterfactual equalized odds.

Which mechanism to use
We discuss the various mechanisms for enhancing fairness above.Here, we further compare these mechanisms and discuss the advantages and disadvantages of them, respectively.This section provides insights into how to select suitable mechanisms for use in different scenarios based on the characteristics of these mechanisms.Every type of mechanism has its advantages and disadvantages.
The pre-processing mechanism can be flexibly adapted to the downstream tasks since it can be used with any classification algorithm.However, since the pre-processing mechanism is a general mechanism where the extracted features can be widely applicable for various algorithms, there is high indeterminacy regarding the accuracy of the trained decision models.
Similar to the pre-processing mechanism, the post-processing mechanism also can be flexibly used in any decision model.Post-processing mechanisms are easier to fully eliminate discrimination for the decision models, but the accuracy of the decision models depends on the performance they obtained in the training stage [60] .Furthermore, post-processing mechanisms require access to all information of individuals during testing, which may be unavailable because of reasons of privacy protection.
The in-processing mechanism is beneficial to enable a balance between accuracy and fairness of the decision model, which is achieved by explicitly modulating the trade-off parameter in the objective function.However, such mechanisms are tightly coupled with the machine learning algorithm itself and are difficult to optimize in the application.
Based on the above discussion and the studies that attempt to comprehend which mechanism is best to use in certain situations [61,62] , we can say that there is no single mechanism that outperforms the others in all cases, and the choice of suitable mechanisms depends on the availability of sensitive variables during testing, the characteristics of the dataset, and the desired fairness measure in the application.For example, when there exists evident selection bias in a dataset, it is better to select the pre-process mechanism for use than the inprocess one.Therefore, more research is needed to develop robust fairness mechanisms or to design suitable mechanisms for practical scenarios.

APPLICATIONS OF FAIR MACHINE LEARNING
This section enumerates different domains of machine learning and the work that has been produced by each domain to combat discrimination in their methods.

Data missing
One major challenge for fairness-enhancing algorithms is to deal with the biases inherent in the dataset that is caused by missing data.Selection biases are due to the distribution of collected data, which cannot reflect the real characteristics of disadvantaged groups.Martínez-Plumed et al. [63] learned that selection bias is mainly caused by individuals in disadvantaged groups being reluctant to disclose information, e.g., people with high incomes are more willing to share their earnings than people with low incomes, which results in bias inference that training in the training institution helps to raise earnings.To address this problem, Bareinboim et al. [64] and Spirtes et al. [65] studied how to deal with missing data and repair datasets that contain selection biases by causal reasoning, in order to improve fairness.
On the other hand, the collected data represent only one side of the reality, that is, these data do not contain any information about the population who were not selected.Biases may arise that decide which data are contained or not contained in the datasets.For example, there is a dataset that records the information of individuals whose loans were approved and the information about their ability to repay their loans.Although the automatic decision system that satisfies certain fairness requirements is constrained based on this dataset to predict whether they repay their loan on time, such a predictor may be discriminatory when it is used to assess the credit score of further applicants, since populations not approved for loans are not sufficiently representative in the training data.Goel et al. [66] used the causal graph-based framework to model the causal process of possible missing data for different settings by which different types of decisions are made in the past, and proved some data distributions can be inferred from incomplete available data based on the causal graph.Although the practical scenarios they discussed are not exhaustive, their work shows that the causal structure can be used for determining the recoverability of quantities of interest in any new scenario.
A promising solution for dealing with missing data can be found in causality-based methods.We see that causality can provide tools to improve fairness when the dataset suffers from discrimination caused by missing data.

Fair recommender Systems
Recommenders are recognized as the most effective way to alleviate information overloading.Nowadays, recommender systems have been widely used in variable applications, such as ecommerce platforms, advertisements, news articles, jobs , etc.They are not only used to analyze user behavior to infer users' preferences so as to provide them with personalized recommendations, but they also benefit content providers with more potential of making profits.Unfortunately, there exist fairness issues in recommender systems [67] , which are challenging to handle and may deteriorate the effectiveness of the recommendation.The discrimination embedded in the recommender systems is mainly caused by the following aspects.User behaviors in terms of the exposed items make the observational data confounded by the exposure mechanism of recommenders and the preference of the users.Another major cause of discrimination in recommender systems is that disadvantage items reflected in the observational data are not representative.That is to say, some items may be more popular than others and thus receive more user behavior.As a result, recommender systems tend to expose users to these popular items, which results in discrimination towards unpopular items and leads to the systems not providing sufficient opportunities for minority items.Finally, one characteristic of recommender systems is the feedback loop.That is, the systems exposes to the user for determining the user behavior, which is circled back as the training data for the recommender systems.Such a feedback loop not only creates biases but also intensifies biases over time, resulting in "the rich get richer" Matthew effect.
Due to the usefulness of causal modeling [10] , removing discrimination for recommender systems from a causal perspective has attracted increasing attention, where the cause graph is used for exposing potentially causal relationships from data.On the one hand, most discrimination can be understood with additional confounding factors in the causal graph and the effect of discrimination can also be inferred through the causal graph.On the other hand, recommendation can be considered as an intervention, which is similar to treating a patient with a specific drug, requiring counterfactual reasoning.What happens when recommending certain items to the users?The causal model has the potential to answer this question.For example, Wu et al. [68] focused on fairness-aware ranking and proposed to use path-specific effects to detect and remove the direct and indirect rank discrimination.Zhao et al. [69] and Zheng et al. [70] considered the effect of item popularity on user behavior and intervened in the item popularity to make fair recommendations.Zhang et al. [71] attributed popularity bias in the recommender systems to the undesirable causal effect of item popularity on items exposure and suggested intervening in the distribution of the exposed items to eliminate this causal effect.Wang et al. [72] leveraged counterfactual reasoning to eliminate the causal effect of exposure features on the prediction.Li et al. [73] proposed generating embedding vectors independent of sensitive attributes by adversarial learning to achieve counterfactual fairness.Huang et al. [74] regarded causal inference as bandits and performed -operator to simulate the arm selection strategy to achieve fairness towards individuals.
Nowadays, the explainability of recommender systems is increasingly important, which improves the persuasiveness and trustworthiness of recommendations.When addressing the fairness issues of recommender systems from the causal perspective, the explanation of recommendations can also be provided from the effects transmitted along the causal paths.Thus, we are confident that causal modeling will bring the recommendation research into a new frontier.

Fair natural language processing
Natural language processing (NLP) is an important technology for machines to understand and interpret human natural language text and realize human-computer interaction.With the development and evolution of human natural language, the natural language is characterized by a certain degree of gender, ethnicity, region, and culture.These characteristics are sensitive in certain situations, and inappropriate use can lead to prejudice and discrimination.For example, Zhao et al. [75] found that the datasets associated with multi-label object classification and visual semantic role labeling exhibit discrimination towards gender attribute, and, unfortunately, the model trained with these data would further amplify the disparity.Stanovsky et al. [76] provided multilingual quantitative evidence of gender bias in large-scale translation.They found that, among the eight target languages, all four business systems and two academic translation systems tend to translate according to stereotype rather than context.Huang et al. [77] used counterfactual evaluations to investigate whether and how language models are affected by sensitive attributes (e.g., country, occupation, and gender) to generate sentiment bias.Specifically, they used individual fairness metrics and group fairness metrics to measure counterfactual sentiment bias, conducted model training on news articles and Wikipedia corpus, and showcased the existence of sentiment bias.
Fair NLP is a kind of NLP without bias or discrimination with sensitive attributes.Shin et al. [78] proposed a counterfactual reasoning method for eliminating the gender bias of word embedding, which aims to disentangle a latent space of a given word embedding into two disjoint encoded latent spaces, namely the gender latent space and the semantic latent space, to achieve disentanglement of semantic and gender implicit descriptions.To this end, they used a gradient reversal layer to prohibit the inference about the gender latent information from semantic information.Then, they generated a counterfactual word embedding by converting the encoded gender into the opposite gender and used it to produce a gender-neutralized word embedding after geometric alignment regularization.As such, the word embedding generated by this method can strike a balance between gender debiasing and semantic information preserving.Yang and Feng [79] presented a causality-based post-processing approach for eliminating the gender bias in word embeddings.Specifically, their method was based on statistical correlation and half-sibling regression, which leverages the statistical dependency between gender-biased word vectors and gender-definition word vectors to learn the counterfactual gender information of an individual through causal inference.The learned spurious gender information is then subtracted from the gender-biased word vectors to remove the gender bias.Lu et al. [80] proposed a method called CDA to eliminate gender bias through counterfactual data augmentation.The main idea of CDA is to augment the corpus by exchanging gender word pairs in the corpus and constructing matching gender word pairs with causal interventions.As such, CDA breaks associations between gendered and gender-neutral words and alleviates the problem that gender bias increases as loss decreases when training with gradient descent.
There exists a certain degree of bias and fairness issues in word embedding, machine translation, sentiment analysis, language models, and dialog generation in NLP.At present, most studies only focus on a single bias (such as gender bias), and there is a lack of research results on other biases or eliminating multiple biases at the same time.Therefore, how should we analyze and evaluate the mechanism and impact of multi-bias in word embedding and machine learning algorithms?Establishing effective techniques for eliminating various biases in word embedding and machine learning algorithms requires further research which needs to be carried out for fair NLP.

Fair medical
Electronic health records (EHRe) contain large amounts of clinical information about heterogeneous patients and their responses to treatments.It is possible for machine learning techniques to efficiently leverage the full extent of EHRs to help physicians make predictions for patients, thus greatly improving the quality of care and reducing costs.However, because of discrimination implicitly embedded in EHRs, the automated systems may introduce or even aggravate the nursing gap between underrepresented groups and disadvantaged ones.The prior works on eliminating discrimination for clinical predictive models mostly focus on statistics-based fairness-enhancing approaches [81,82] .In addition, they do not provide an effective evaluation of fairness to individuals, and the fairness metrics they used are difficult to verify.Some recent studies focus on assessing fairness for clinical predictive models from a causal perspective [83,84] .For example, Pfohl et al. [83] proposed a counterfactual fairness notion to extend fairness to the individual level and leveraged variational autoencoder technology to eliminate discrimination against certain patients.

Causal analysis packages
This section introduces some representative packages or software for causal analysis, which are helpful for us to develop causality-based fairness-enhancing approaches.These packages can be roughly divided into two categories: one for discovering potential causal structure in data and the other for making causal inferences.Table 2 summarizes typical packages or software for causal analysis.
TETRAD [85] is a full-featured software for causal analysis after considerable development where it can be used to discover the causal structure behind the dataset, estimate the causal effects, simulate the causal models, etc. TETRAD can accept different types of data as input, e.g., discrete data, continuous data, time series data, etc.The users can choose the appropriate well-tested causal discovery algorithms it integrates to search causal structure, as well as input prior causal knowledge to limit the search.In addition, TETRAD can parameterize the causal model and simulate the data according to the existing causal diagram.Causal-learn package [86] is the Python version of TETRAD.It provides the implementation of the latest causal discovery methods ranging from constraint-based methods, score-based methods, and constrained functional causal models-based methods to permutation-based methods.In addition, there are many packages for causal discovery [87][88][89] .Tigramite [88] focuses on searching causal structure from observational time series data.In addition to providing classic causal discovery algorithms, gCastle [89] provides many gradient-based causal discovery approaches.
CausalML [90] is a Python package which encapsulates many causal learning and causal inference approaches.

Package name Program language Description
Causal discovery TETRAD [85] Java TETRAD is a full-featured software for causal analysis; after considerable development, it can be used to discover the causal structure behind the dataset, estimate the causal effects, simulate the causal models, etc Py-causal [87] Python Py-causal is a Python encapsulation of TETRAD, which can call the algorithms and related functions in TETRAD Causal-learn [86] Python Causal-learn is the Python version of TETRAD.It provides the implementation of the latest causal discovery methods ranging from constraint-based, score-based, and constrained functional causal models-based to permutation-based methods Tigramite [88] Python Tigramite focuses on searching causal structure from observational time series data.
gCastle [89] Python gCastle provides many gradient-based causal discovery approaches, as well as classic causal discovery algorithms

Causal effect and Inference
CausalML [90] Python CausalML encapsulates many causal learning and inference approaches.One highlight of this software package is uplift modeling, which is used to evaluate the conditional average treatment effect (CATE) Causaleffect [92] R Causaleffect is the implementation of ID algorithm DoWhy [93] Python DoWhy takes causal graphs as prior knowledge and uses Pearl's -calculus method to assess causal effects Mediation [91] R Mediation provides model-based method and design-based method to evaluate the potential causal mechanisms.It also provides approaches to deal with common problems in practice and random trials, that is, to handle multiple mediators and evaluate causal mechanisms in case of intervention non-compliance One highlight of this package is uplift modeling, which is used to evaluate the conditional average treatment effect (CATE), that is, to estimate the impact of a treatment on a specific individual' s behavior.
Mediation [91] is an R package which is used in causal mediation analysis.In other words, it provides modelbased methods and design-based methods to evaluate the potential causal mechanisms.It also provides approaches to deal with common problems in practice and random trials, that is, to handle multiple mediators and evaluate causal mechanisms in case of intervention non-compliance.
Causaleffect [92] is an R package which is the implementation of ID algorithm.ID algorithm is a complete identification of causal effects algorithm, which outputs the expression of causal effect when the causal effect is identifiable or fails to run when the causal effect is unidentifiable.DoWhy [93] , a Python package, also focuses on causal inference, that is, it takes causal graphs as prior knowledge and uses Pearl' s -calculus method to assess causal effects.
These packages used for causal analysis assist in developing causality-based fairness-enhancing methods, which are mainly reflected in exposing the causal relationship between variables and evaluating the causal effects of sensitive attributes on decision-making.However, they cannot be used directly to detect or eliminate discrimination.Although there are many software packages for detecting and eliminating discrimination, e.g., AI Fairness 360 Open Source Toolkit [94] , Microsoft Research Fairlearn [95] , etc, we are still lacking a package that integrates causality-based approaches.

CHALLENGES
Decision based on machine learning has gradually penetrated into all aspects of human society, and the fairness of its decision-making directly affects the daily life of individuals or groups, as well as users' trust and acceptance of machine learning application deployment.Recently, fair machine learning has received extensive attention, and researchers are gradually aware of the fact that relying solely on the observable data, with no additional causal information, is limited in removing discrimination, since the dataset only represents the selected population, without any information on the groups who were not selected, while such information can be achieved using knowledge of a causal graph or by a controlled experiment making use of interventions.
As such, causality-based fairness machine learning algorithms have attracted more and more attention and several causality-based fairness approaches have been proposed.Although causal fairness models can indeed help us overcome many of the challenges encountered with respect to fair prediction tasks, they still face many challenges, which are discussed in the following subsections.

Causal discovery
Causality-based fairness approaches require a causal graph as additional prior knowledge of input, where the causal graph describes the mechanism by which the data are generated, that is, it reveals the causal relationship between variables.However, in practice, it is difficult for us to obtain the correct causal graph.A basic way to discover the causal relationship between variables is to conduct randomized controlled trials.Randomized controlled trials consist of randomly assigning subjects (e.g.individuals) to treatments (e.g.gender), and then comparing the outcome of all treatment groups.Unfortunately, in many cases, it may not be possible to undertake such experiments due to prohibitive costs, ethical concerns, or they are physically impossible to carry out.For example, to understand the impact of smoking, it would be necessary to force different individuals to smoke or not smoke.As another example, to understand whether hiring decision models are gender-biased, it would be necessary to change the gender of a job applicant, which is an impracticality.Researchers are therefore often left with non-experimental, observational data, and they have developed numerous methods for uncovering causal relations, i.e., causal discovery.Causal discovery algorithms can be roughly classified into the following three categories: constraint-based, score-based, and those exploiting structural asymmetries.
Constraint-based approaches conduct numerous conditional independence tests to learn about the structure of the underlying causal graph that reflects these conditional independence.Constraint-based approaches have the advantage that they are generally applicable, but the disadvantages are that faithfulness is a strong assumption and that it may require very large sample sizes to get good conditional independence tests.Furthermore, the solution of this approach to causal discovery is usually not unique, and, in particular, it does not help determine the causal direction in the two-variable case, where no conditional independence relationship is available.
Score-based algorithms use the fact that each directed acyclic graph (DAG) can be scored in relation to the data, typically using a penalized likelihood score function.The algorithms then search for the DAG that yields the optimal score.Typical scoring functions include the Bayesian information criterion [96] , Bayesian-Gaussian equivalent score [96] , and minimum description length (as an approximation of Kolmogorov complexity) [97,98] .
Structural asymmetry-based algorithms take into account the setting that it is impossible to infer the causal direction from observations alone when the data distribution admits structural causal models indicating either of the structural directions   →   or   ←   .To address this problem, structural asymmetry-based algorithms make some additional assumptions about the function of the underlying true data-generating structure, so that they can exploit asymmetries to identify the direction of a structural relationship.These asymmetries manifest in various ways, such as non-independent errors, measures of complexity, etc. Existing methods that exploit such asymmetries are typically local solutions, as they are only able to test one edge at a time (pairwise/bivariate causal directionality), or a triple (with the third variable being an unobserved confounder) [99] .
In the absence of intervention and manipulation, observational data leave researchers facing a number of challenges: First, there may exist hidden confounders, which are sometimes termed the third variable problem.Second, observational data may exhibit selection bias.For example, younger patients may generally prefer surgery, while older patients may prefer medication.Third, most causal discovery algorithms are based on strong but often untestable assumptions, and applying these strong assumptions to structural or graphical models incites some harsh criticisms.

Identifiable issue
The identifiable issue is another main obstacle to the application of causal models in fair machine learning.The identifiability of causal effects, including total effect, path-specific effect, and counterfactual effect, has been extensively studied [10,[100][101][102][103] .Many causality-based fairness methods have been proposed to solve this problem so as to achieve fairness more effectively.This section summarizes the main identifiability conditions, the causality-based fairness methods that try to overcome the unidentifiable situations, and their limitations.

Identifiability
This review starts with the simplest identifiability condition of total effects, where the causal effect of a sensitive attribute  on decision  is computed by ( |( = )), which denotes the distribution of decision  after the intervention  = .Similar to causal discovery, randomized controlled trials can also be used to assess the total effect by performing an intervention on the sensitive attributes to completely avoid unidentifiable situations of the total effect.A randomized controlled trial may allow us not only to identify causal relationships but also to estimate the magnitude of these relationships.However, as mentioned in Section 7.1, it is impossible for us to undertake such experiments in many cases.The causal effect of the sensitive attribute  on decision  may not be uniquely assessed from observational data and causal graph alone, i.e., the unidentifiable situation.In Markovian models, the total effect is always identifiable, since ( |( = )) is always identifiable [10] .The simplest case is when there is no confounder between  and  (see Figure 7(a)).In this case, the causal effect ( |( = )) is consistent with the conditional probability (| = ).As such, the total effect can be computed as follows: If there exists an observational confounder between  and  (see Figure 7(b)), the total effect is also identifiable by summing the probabilities of all values  in the domain of the observable confounder : (16)   where the formula ∑  (|, )() in Equation ( 16) is also called the back-door formula.If there are no hidden confounders within the observable attributes, then the causal effect (|( = )) can also be computed by the following formula: (17)   where () represents the parent variables of .It is easy to see that the total effects computed by Equations ( 16) and ( 17) can produce the same result.For semi-Markovian models, the causal effect (|()) is not always identifiable, hence the total effect is not always identifiable.The causal effect (|()) is identifiable if and only if it can be reduced to a -free expression (i.e., turning the intervention operator () to observational probabilities) by -calculus [10] .calculus is composed of three inference rules: (i) insertion/deletion of observations, i.e., (  |z, ) = (  |z) provided that and  are dependence-separated at fixed  and Z after all arrows leading to  have been deleted in causal graph; (ii) action/observation exchange, i.e., (|(), z) = (|, z) if  and  are probabilistically conditionally independent at fixed Z after deleting all arrows starting from Z in causal graph; and (iii) deletion of actions, i.e., (|()) = () if there are no causal paths between  and  .
-calculus has been proven to be complete, that is, it is sufficient to derive all identifiable causal effects by calculus [100] .However, it is difficult to determine the correct order of application of these rules, and the wrong order may misjudge the identifiability of causal effects or produce a very complex expression.To address this issue, several studies attempt to give the explicit graphical criteria and map them to simple and concise -free expressions [101,104] .A simple case of the identifiability of the causal effect (|()) is when the sensitive attribute  is not influenced by any confounder [105] .In other words, the causal effect of  is identifiable, if all parents of  are observable.Graphically, there is no bi-directional edge connected to .Formally, the causal effect (|()) can be computed as follows: (|, ())(pa()) (18)   where () represents the values of parent variables of .
A complex case where the causal effect of  on V ′ = V \ {} is identifiable is that there may exist a bi-directed edge connected to the sensitive attribute , but there are no hidden confounders connected to any direct child of  [105] .Graphically, there is no bi-directional edge connected to any child of .If such criterion is satisfied, the causal effect (v ′ |()) is identifiable and is given by: where ℎ() denotes the set of 's children and pa(  ) denotes the set of values of   ' s parents.Equation ( 19) can be easily adapted to assess the effect of the sensitive attribute  on outcome  .
Tian et al. [105] also found that, although (v ′ |()) is not identifiable, (w|()) is still identifiable for some subsets W of V. Specifically, causal effect (w|()) is identifiable if there is no bi-directed path connecting  to any of its direct children in G  (W), where (W) denotes the union of a set W and the set of ancestors of the variables in W. G  (W) denotes the sub-graph of G composed only of variables in (W).
Moreover, the causal effect of sensitive attribute  on outcome  (|()) can also be computed by using the front-door criterion, if  (20)   These criteria can be generalized to the case where there is no bi-directed edges connection between the sensitive attribute and its direct children.Given the fact that the observational distribution (v) can be decomposed to a product of several factors, c-component factorization was proposed to decompose the identification problem into smaller problems (i.e., c-components, each of which is a set of observational variables that are connected by a common confounder in the causal graph) to evaluate the causal effect [15,105] .
Shpitser et al. [15] designed a sound and complete algorithm called ID to identify all identifiable causal effects where ID outputs the expression of the causal effect for the identifiable cases.In addition, they proved that all cases of unidentifiable causal effects (|()) can be completely attributed to a graphical structure called hedge.
As to the identifiability of counterfactual effect, if complete knowledge of the causal model is known (including structural functions, (u), etc.), any counterfactual quantity can be exactly performed using three steps: (i) abduction, update (u) by observation O = o to compute (u|o); (ii) action, modify causal model M by intervention () to obtain the post-intervention model M  ; and (iii) prediction, use post-intervention model M  and (u|o) to compute the counterfactual effect (  | ′ ).However, the above method is usually infeasible in practice due to the lack of complete knowledge of the causal model.In most cases, we only have the causal graph and observational data, which makes the counterfactual effect not always identifiable.The simplest unidentifiable case of counterfactual effect is due to the unidentifiability of ( = ,   =  ′ ).Graphically, there exists "w-graph" in the causal graph (see Figure 8(c)).
The analysis of the identifiability of counterfactual effect (  | ′ , o) concerns the connection between two causal models, M and M  ; thus, Shpitser et al. [15] proposed a make-cg algorithm to construct a counterfactual graph G ′ which depicts the independence relationship among all variables in M and M  .Specifically, make-cg first combines original causal graph and post-interventional causal graph by removing the same exogenous variables, and the duplicated endogenous variables that are not influenced by ().The resultant graph is the so-called counterfactual graph, which can be considered as a typical causal graph for a larger causal model.For example, Figure 8(a) shows an example causal graph, while its counterfactual graph of the counterfactual effect (  | ′ ) is shown in Figure 8(b).All the graphical criteria mentioned above for the identifiability of causal effects are applicable to the counterfactual graph and the c-component factorization of the counterfactual graph for performing the counterfactual inference [106] .Shpitser et al. [15] further developed ID* and IDC* algorithms to distinguish the identifiability of counterfactual effects and compute the counterfactual quantity.In addition, Pearl [10] further proved the results about the identifiability of counterfactual effects: if all the structural functions F are linear, any counterfactual quantity is identifiable whenever we have full knowledge about the causal model.Unfortunately, there is no single necessary and sufficient criterion for the counterfactual effects' identifiable issues in semi-Markovian models, even the linear causal model [107] .
For the identifiability of path-specific effect   (), it depends on whether (  + |, − | π ) is identifiable or not.Unfortunately, (  + |, − | π ) is not always identifiable, even in Markovian models.Avin et al. [107] gave the necessary and sufficient criterion for the identifiability of (  + |, − | π ) in Markovian models called recanting witness criterion.The recanting witness criterion is satisfied when there is a variable  along the causal path  connected to  through another causal path not in .Consider an example causal model whose causal graph is shown in Figure 8(d), when the causal path that we follow with interest  = { →  →  →  } with  as witness, then the recanting witness criterion is satisfied.The corresponding graph structure is called "kite" graph.When this criterion is satisfied, (  + |, − | π ) is not identifiable and   () is not identifiable also.Shpitser et al. [102] generalized this criterion to semi-Markovian models known as recanting district criterion.Specifically, if there exists district  that represents the set of variables not belonging to the set of sensitive attributes , but ancestral of decision  via a directed path which does not intersect Most previous approaches tend to make simplified or even unrealistic assumptions to avoid unidentifiable situations.For example, to avoid the unidentifiable issue of the counterfactual effect, Kusner et al. [11] adopted three different assumptions: (i) only using non-descendants of the sensitive attributes to build the classifier; (ii) postulating and inferring the non-deterministic sub-situations of the hidden variables based on domain knowledge; and (iii) postulating the complete causal model, treating it as the additive noise model, and then estimating the errors.Zhang et al. [26] evaded the unidentifiable issue of path-specific effect caused by satisfying recanting witness criterion via changing the causal model, i.e., cutting off all causal paths from sensitive variables to the decision that pass through the redline variables.However, such simplified assumptions modify the causal model equivalent to "redefining success".Although these methods made simplified assumptions to avoid identifiable issues, such assumptions may severely damage the performance of the decision model and impose uncertainty on these methods.Besides, such simplified assumptions may modify the causal model equivalent to "redefining success", while any kind of repair is not expected within a modified model, which results in fair inferences in the real world.
Recently, some workarounds for dealing with unidentifiable situations aim to stay within the true causal model, but they obtain the true unidentifiable causal effects by developing the upper and lower bounds of the causal effects.For example, Wu et al. [57] mathematically developed the upper and lower bounds of counterfactual fairness in unidentifiable situations and used a post-processing method for reconstructing trained classifiers to make counterfactual fairness.Zhang et al. [27] mathematically bound indirect discrimination as the pathspecific effect in unidentifiable cases and proposed a pre-processing method for reconstructing the observational data to remove the discrimination from the original dataset.Hu et al. [108] adopted implicit generative models and adversarial learning to estimate the upper and lower bound of average causal effect under unidentifiable cases.
One of the major reasons causal effects are not identifiable is the presence of hidden confounding.Most previous works [12,27,44,57] adopt the no hidden confounders assumption (i.e., Markovian model) to facilitate the assessment of the causal effects.However, in practical scenarios, the existence of hidden confounders is an inescapable fact, since measuring all possible confounders is impossible.For example, in many cases, we cannot measure variables such as personal preferences, most genetic factors, and environmental factors.In these cases, to deal with hidden confounders and identifiable issues, many studies adopt the potential outcome framework [33,34] and are devoted to finding so-called "proxy variables" that reflect the information of hidden confounders.For example, we cannot directly measure the socioeconomic status of patients, but patients' de-mographic attributes, such as their zip code, consumption ability, or employment status, can be the proxies for socioeconomic status.Variational autoencoder has been widely used to learn causal models with hidden confounders, especially for approximately inferring the complex relation between the observational variables and hidden confounders [109] .It is a computationally efficient algorithm for learning the joint distribution of the hidden confounders and the observed ones from observational data.An alternative way to eliminate the confounding bias in causal inference is to utilize the underlying network information that is attached to observational data (e.g., social networks) to infer the hidden confounders.For example, Guo et al. [110] proposed the network deconfounder to infer the influence of hidden confounders by mapping the features of observational data and auxiliary network information into the hidden space.Guo et al. [111] leveraged the network information to recognize the representation of hidden confounders.Veitch et al. [112] remarked that merely partial information that hidden confounders contain affects both the treatment and the outcome.That is to say, only a portion of confounders is actually used by the estimator to estimate the causal effects.Therefore, if a good predictive model for the treatment can be built, then one may only need to plug the outputs into a causal effect estimate directly, without any need to learn all the true confounders.Since experimental data do not suffer from hidden confounders, another method is to combine experimental and observational data together.For example, Kallus et al. [113] used limited experimental data to correct the hidden confounders in causal effect models trained on larger observational data, even if the observational data do not fully overlap with the experimental ones, which makes strictly weaker assumptions than existing approaches.
Overall, these potential outcome framework-based methods mostly rely on proxy variables.Before selecting proxy variables for hidden confounders, we need a thorough understanding of what a hidden confounder is supposed to represent, and whether there is any proxy variable that actually represents it.However, a sufficiently clear understanding may be impossible to attain in some cases.

Comprehensive definition of fairness
The sources of unfairness in machine learning algorithms are diverse and complex, and different biases have different degrees of impact on unfairness.Since most fairness notions, including causality-based fairness notions, quantify fairness in a single dimension, when comparing the capabilities of different fairness machine learning algorithms, using different fairness measures will often lead to different results.This means that, whether the algorithm is fair or not is relative, which depends not only on the model and data but also on the task requirements.There is a lack of complete and multi-dimensional causality-based fairness definition and evaluation system for fairness, and it is not possible to effectively quantify the fairness risk faced by machine learning algorithms.Therefore, we need to further explore comprehensive causal-based fairness notions and establish a comprehensive multi-dimensional evaluation system for the fairness of machine learning algorithms.In addition, the definition of fairness needs to be combined with the laws and the concept of social fairness of various countries to avoid narrow technical solutions.The proposition of PC-fairness and causalitybased fairness notion defined from both macro-level and individual-level [114] are useful explorations to solve this problem.

Achieving fairness in a dynamic environment
The existing works mainly focus on studying the fairness in machine learning in static, no feedback, short-term impact scenarios, without examining how these decisions affect fairness in future applications over time and failing to effectively adapt to evolutionary cycles.At present, the research on the fairness of machine learning shows a trend of dynamic evolution, which requires the definition of fairness and algorithms to consider the dynamic, feedback, and long-term consequences of decision-making systems.This is particularly evident in recommendation systems, loans, hiring, etc.Fortunately, some researchers are modeling the long-term dynamics of fairness in these areas [115][116][117][118][119] .D' Amour et al. [120] regarded dynamic long-term fair learning as a Markov decision process (MDP) and proposed simulation studies to model fairness-enhancing learning in a dynamic environment.They emphasized the importance of interaction between the decision system and the environ-ment.A complementary work [121] shows the importance of causal modeling in dynamic systems.However, due to the complexity of the real-world environment, it is impossible to model the real environment at a high level.Besides, current studies are carried out on low-dimensional data.Therefore, how to highlight important dynamics in simulations and effectively use collected data to ensure an appropriate balance between results and real-world applicability and how to adapt to high-dimensional data are current challenges.In addition, future causality-based fairness-enhancing studies can be combined with dynamic game theory for improving fairness in the confrontation environment and research the detection mechanism of dynamic fairness.

Other challenges
AI has become more and more mature after rapid development.Although most of the research on AI thus far has focused on weak AI, the design of strong AI (or human-level AI) will be increasingly vital and receive more and more attention in the near future.Weak AI only focuses on solving the given tasks input into the program, while strong AI or human-level AI (HLAI) means that its ability of thinking and action is comparable to that of a human.Therefore, developing HLAI will face more challenges.Saghiri et al. [122] comprehensively summarized the challenges of designing HLAI.As they said, unfairness issues are closely related to other challenges in AI.
There is still a gap between solving the unfairness problem in AI alone and building a trustworthy AI.Next, this review discusses the relationship between fairness and the other challenges in AI.

Fairness and robustness.
The robustness of the AI model is manifested in its outer generalization ability, that is, when the input data change abnormally, the performance of the AI model remains stable.AI models with poor robustness are prone to crash and, thus, fail to achieve fairness.In addition, the attacker may obtain private information about the training data and even the training data themselves, although the attacker has no illegal access to the data.However, the research on robustness is still in its infancy, and the theory and notions of robustness are still lacking currently.

Fairness and interpretability.
The explainability of discrimination is very important to improving users' understanding and trust in AI models, which is even required by law in many fields.On the other hand, interpretability can explain and judge whether the fairness of AI models is satisfied or not, which assists in improving the fairness of AI models.In some important areas, e.g., healthcare, this challenge becomes more serious because it requires that any type of decision-making must be fair and interpretable.
Causality-based methods are promising solutions to these challenges, as they can not only reveal the mechanisms by which data are generated but also enable a better understanding of the causes of discrimination.Of course, there are far more challenges faced by HLAI than those above, and more about the challenges of designing HLAI can be found in Saghiri et al.' s work [122] .

Future trends
More realistic application scenarios.Most of the early studies are carried out under some strong assumptions, e.g., the assumption that there are no hidden confounders between observational variables.However, these assumptions are difficult to satisfy in practical applications, which leads to erroneous evaluation.Therefore, the trained model cannot guarantee that it satisfies the fairness requirement.The current studies tend to relax these assumptions and address the unfairness issue of algorithms in more general scenarios.
Privacy protection.Due to legal requirements, sensitive attributes are often inaccessible in real applications.Fairness constraints require predictors to be in some way independent of the attributes of group members.Privacy preservation raises the same question: Is it possible to guarantee that even the strongest adversary cannot steal an individual' s private information through inference attacks?Causal modeling of the problem not only is helpful to solve the fairness issue but also enables stronger privacy-preserving than statistics-based methods [123] .Combining the existing fairness mechanism with differential privacy is a promising research direction in the near future.

Build a more complete ecosystem.
There is an interaction of fairness between applications in the real world.For example, in bank loans, there exists discrimination in loan quotas for groups of different genders, and this unfairness may be caused by the salary level of groups of different genders in the workplace.Therefore, we need to further explore achieving cross-domain, cross-institution collaborative fairness algorithms.

CONCLUSION
This review presents the relevant background, typical causality-based fairness notions, an inclusive overview of causality-based fairness methods, and their applications.The challenges of applying causality-based fairness notions in practical scenarios and future research trends on solving fairness problems in algorithms are also discussed.
Papers related to the topic of addressing fairness issues based on causality are mostly reviewed in this survey.
The statistics and analysis of these papers are also carried out in this survey and the results are presented in Figure 9. Figure 9(a) reports the proportion of papers published in reputable journals or conferences, while Figure 9(b) shows the number of publications from 2017 to 2021.The research community has only recently focused on defining, measuring, and mitigating discrimination in algorithms from a causal perspective, gradually realizing the importance of causal modeling of the problems to address fairness issues.
Therefore, we provide a relatively complete review of causality-based fairness-enhancing techniques to help researchers gain a deep understanding of this field, and we hope that more researchers will engage in this young but important field.On the one hand, discrimination detection and elimination from the causal perspective rather than statistics-based methods is more welcomed and trusted by the users of automated decision making systems, since causality-based fairness-enhancing methods consider how the data are generated and thus deeply understand the sources of discrimination.On the other hand, because of the completeness of the causal theory, it provides mathematical tools to discover discrimination when the dataset includes bias due to missing data.In addition, the main objective of this survey is to bridge the gap between the practical scenarios of discrimination elimination from the causal perspective and the ongoing theory problem.This is achieved by summing up causality-based fairness notions, approaches, and their limitations.Although the causal graph cannot be constructed without some untestable assumptions, it can still be used productively as well as serve as an auxiliary tool to incorporate scientific domain knowledge.In addition, causal graphs can exchange the causal statements that are under plausible assumptions but lack grounding in established scientific knowledge for inferring plausible conclusions.To conclude, causality-based fairness-enhancing approaches are promising solutions to reduce discrimination despite having challenges to overcome.

Figure 1 .
Figure 1.Organizational structure of this paper.

Figure 2 .
Figure 2. Two causal graphs for the hiring decision system.(a) A causal graph of the hiring decision system, where  (i.e., individual's religious belief) is the sensitive attribute and  (i.e., hiring decision) is the decision.(b) A causal graph of this system after intervention on .

Figure 7 Figure 3 .
Figure 3. (a) An example causal graph based on Markovian assumption; (b) a causal graph based on semi-Markovian assumption; and (c) a causal graph after performing an intervention on .

Figure 4 .
Figure 4.The categorization of fairness notions.

Figure 5 .
Figure5.Two alternative graphs for the loan application system.(a) A causal graph of the loan application system, where Race is the sensitive attribute and Loan is the decision.(b) A causal graph of the system after removing unresolved discrimination.(c) A causal graph of the system that is free of proxy discrimination.
the subset of endogenous variables except sensitive variables and decision variables.Any context O = o represents a certain sub-group of the population, specifically, when O = V \ {,  }, it represents a specific individual.According to Equation (7), the decision model achieves counterfactual fairness if, for every possible individual (O = o,  =  − ) of the entire population, the probability distribution of the outcome  is the same in both the actual ( =  − ) and counterfactual ( =  + ) worlds.

Figure 9 .
Figure 9. Statistical charts of references in this survey: (a) a pie chart that shows the proportion of publications in journals or conferences; and (b) a bar chart about the number of publications per year (from 2017 to 2021).

𝜏
An outcome  achieves counterfactual fairness towards an individual  (i.e., O = o) if the probability of  =  for such individual  is the same as the probability of  =  for the same individual, who belongs to a different sensitive group (|()) is identifiable.Before introducing the front-door criterion, the concepts of the back-door path and front-door path are first introduced, which are helpful for the understanding of the front-door criterion.A causal path connecting  and  which begins with an arrow leading to  is called a back-door path (e.g.,  ← ... ), while a path starting with an arrow pointing away from  is called a front-door path (e.g.,  → ... ).If the front-door criterion is satisfied, there exits a mediator variable  such that: (i) there are no backdoor paths from  to ; and (ii) all backdoor paths from  to  are blocked by .Formally, (|()) can be computed as follows: , and nodes   ,   ∈  (possibly   =   ), such that there is a causal path  →   → ... →  in  and a causal path  →   → ... →  not in , then the path-specific effect of  on  is not identifiable.7.2.2.Efforts for dealing with identifiable issuesIn Section 7.2.1, we show that the causal effects are not always identifiable only from observational data and causal graphs.Several causality-based fairness methods have been proposed from different perspectives to deal with identifiable issues.