TheScarab

The Chinese Government’s plans to introduce a “social credits” scheme to rate and rank the behaviour and conduct of its citizens far beyond their financial circumstances (the current focus of Western-created “credit scoring” systems) has predictably rattled observers.

One journalist summed up the situation starkly:

“The Communist Party’s plan is for every one of its 1.4 billion citizens to be at the whim of a dystopian social credit system, and it’s on track to be fully operational by the year 2020.”[i]

Many of the discussions have followed similar lines, focusing on the harrowing implications of such an intrusive state-run machine for individual freedoms and the right to privacy.

What has been less investigated is the essential structure of the social algorithms required to achieve the objectives of the Chinese government, and in particular the tensions between technical efficiency and political economy when mass surveillance is devolved to machine power and incorporated into social-behavioural systems in the presence of capitalist incentives.

Few treatments of the notion of “sovereign privacy” give it any respect. Yet, there are framings of the “state secrecy” question that goes beyond mere necessity (especially in such contexts as “law enforcement” and “national security”).

LSE’s Andrew Murray provides an interesting angle in his brief 2011 take on transparency:

“All bodies corporate (be they private or public) are in fact organisms made up of thousands, or even tens of thousands, of decision makers; individuals who collectively form the ‘brain’ of the organisation. The problem is that individuals need space to make decisions free from scrutiny, or else they are likely to make a rushed or panicked decision.”[ii]

When viewed as a “hive” of personnel insecurities, biases, errors, stereotypes, ambitions and proclivities, Central Governments emerge out of the monolithic pyramid we tend to envisage atop the panopticon of general surveillance and descend onto a more examinable stage, where their foibles and miscalculations and misdiagnoses can also receive useful attention.

Because the Communist Party’s 90 million members are an integral part of its overall structural integrity, its social management policies rely greatly on their ability to participate and contribute.[iii]

Many of the 20 million people who work in the 49000 plus state enterprises, especially from middle management and up, are fully paid-up members of the party. Some estimates put the percentage of the country’s 2 million press and online censors who belong to the party at 90%. Last year, the last barrier between the Party and command at all levels of state paramilitary and security institutions was removed, bringing an even larger number of non-career security commissars into both operational and oversight positions.

Such broad-based participation in the “social management strategy” might at first sight appear to favour the decentralised nature of social credit-based control. The only problem with that view is that the strategists behind the scheme see it in purgatory terms:

“The main problems that exist include: a credit investigation system that covers all of society has not yet been formed, credit records of the members of society are gravely flawed, incentive mechanisms to encourage keeping trust and punishments for breaking trust are incomplete, trust-keeping is insufficiently reward, the costs of breaking trust tend to be low; credit services markets are not developed, service systems are immature, there are no norms for service activities, the credibility of service bodies is insufficient, and the mechanisms to protect the rights and interests of credit information subjects are flawed; the social consciousness of sincerity and credit levels tend to be low, and a social atmosphere in which agreements are honoured and trust are honestly kept has not yet been shaped, especially grave production safety accidents, food and drug security incidents happen from time to time, commercial swindles, production and sales of counterfeit products, tax evasion, fraudulent financial claims, academic impropriety and other such phenomena cannot be stopped in spite of repeated bans, there is still a certain difference between the extent of sincerity in government affairs and judicial credibility, and the expectations of the popular masses.”[iv]

The goal is as much about moral self-policing as it is about social control. Self-policing inevitably induces low-intensity and highly diffuse factionalism and clique politics.

Chinese observers certainly understand the critical factor of power-play in these circumstances, as is obvious from the following passage by PhD student, Samantha Hoffman:

“The first is the struggle for power within the Party. The Party members in charge of day-to-day implementation of social management are also responsible to the Party.  As the systems were being enabled in the early 2000s, these agencies had a large amount of relatively unregulated power. The age-old problem of an authoritarian system is that security services require substantial power in order to secure the leadership’s authority. The same resources enabling management of the Party-society relationship can be abused by Party members and used against other within the Party (War on the Rocks, July 18, 2016). This appears to be the case with Zhou Yongkang, Bo Xilai, and others ahead of the 18th Party Congress. The problem will not disappear in a Leninist system, which not subject to external checks and balances. And it is why ensuring loyalty is a major part of the management of the party side of “state security”.[v]

But Hoffman and many like her misconstrue the implications of fragmented trust for social credit based control.

Complex social algorithms over time start to amplify signals that their makers do not fully understand and cannot control in advance. We have seen this many times with even much simpler systems like Facebook, Twitter and Instagram, whose operators have extremely narrow objectives: maximising attention retention to attract advertisers.

In a system designed to compel conformance to ideal criteria and yet dependent on large numbers of participants to shape that criteria, deviance can easily become more prominent when algorithms start to reinforce once latent patterns. Whether it is preening on Facebook or bullying on Twitter, there is a fundamental logic in all simple systems trying to mould complex behaviours, and this logic tends to accentuate deviancy because algorithms are signal-searching.

This is where the “sovereign privacy” point comes in. A state like China seeks inscrutability. It also seeks harmony of purpose. Social algorithms tend to want to surface hidden patterns and concentrate attention. A time-lag renders algorithm-tweaking for specified ends in advance highly unreliable. Very often, the operator is relying on surfaced trends to manage responses. The danger of rampant “leaking” of intention and officially inadmissible trends rise exponentially as the nodes in the system – financial, political, social, economic, psychological etc – increase. The “transparency” that results from the inadvertent disrobing of the intents of millions of Chinese state actors does not have to be the kind that simply forces the withdrawal of official propaganda positions. It can also be the kind that reveals which steps they are taking to regain control of the social management system.

The problem is somewhat philosophical. Right now, membership in the Communist Party and public conformance with the creed is non-revelatory. Integrating multiple “real behaviour” nodes together to compel “sincerity”, as is the official goal of the program could immediately endanger the status of tens of millions of until-that-moment perfectly loyal cadres and enforcers of moral loyalty. The proper political economy response, at least in the transition stages, is to flatten the sensitivity of the algorithms. Doing so however removes the efficiency, which alone makes the algorithms more effective than the current “manual” social conformity management system.

Unfortunately, such efficiency would render redundant large swathes of the current order. Which in turn means that lower levels of the control pyramid have very little incentive in providing complete data. The effect of highly clumpy data exacerbates algorithmic divergence from other aspects of social reality (in the same way that Twitter fuels political partisanship in America as oppose to merely report it) and prompts “re-interpretations” of the results churned out by the system. Over time, the system itself begins to need heavily manual policing. The super-elite start to distrust it. Paranoia about the actions of their technocratic underlings grow in tandem. Along with dark fears about a “Frankenstein revolt”.

At the core of all of this is the simply reality that any system that can realistically achieve mass deprivation of privacy will threaten sovereign privacy as well, and would thus not be allowed to attain that level of intrusion by the powers that be.

Notes:

[i] “China’s ‘social credit’ system is a real-life ‘Black Mirror’ nightmare”. Megan Palin. 19th September 2018

[ii] Andrew D Murray. 2011. “Transparency, Scrutiny and Responsiveness: Fashioning a Private Space within the Information Society”. The Political Quarterly.

[iii] See: Yanjie Bian, Xiaoling Shu and John R. Logan. 2001. “Communist Party Membership and Regime Dynamics in China.” Social Forces, Vol. 79, No. 3, pp. 805-841.

[iv] “State Council Notice concerning Issuance of the Planning Outline for the Construction of a Social Credit System (2014-2020)”. GF No. (2014)21.

[v] Samantha Hoffman. 2017. “Managing the State: Social Credit, Surveillance and the CCP’s Plan for China”. China Brief Volume, 17 Issue 11.