Society Technology

Orwell’s Past is China’s Future — and Maybe Ours

“If you want to keep a secret, you must also hide it from yourself.”
― George Orwell1984


Tucked among all the news stories lat month was a remarkable series of articles in the Wall Street Journal about China’s emerging “Social Credit Score.” This term refers to a system, currently under development, through which all Chinese citizens (and probably visitors) would be assigned a score that tracks how well they are or are not confirming to the social and legal norms of the State. As the Journal notes, the foundation for this remarkable evolution is already being put into place:

More than three dozen local governments across China are beginning to compile digital records of social and financial behavior to rate creditworthiness. A person can incur black marks for infractions such as fare cheating, jaywalking and violating family-planning rules. The effort echoes the dang’an, a system of dossiers the Communist party keeps on urban workers’ behavior.

In time, Beijing expects to draw on bigger, combined data pools, including a person’s internet activity, according to interviews with some architects of the system and a review of government documents. Algorithms would use a range of data to calculate a citizen’s rating, which would then be used to determine all manner of activities, such as who gets loans, or faster treatment at government offices or access to luxury hotels.

As the Journal reported, the system is being designed to gather all kinds of pertinent data and assemble them into a profile of every citizen’s behavior. As the figure below notes, the vision is for a system that would be all-encompassing and would be used to control access to everything from mass transit to education to banking services:


Of course, it was a U.S. company (Fair, Isaac, and Company) that invented the credit score idea back run 1989. Its goal was to help banks determine and manage default risk, and we should see this new effort in much the same light. For despite the bureaucrats’ claims that this is a system to incentive good behavior, it is really a mechanism for assessing the risk of individual “default” from the State’s prescribed social model:

Blacklists will expose offenders and restrict them from certain activities, while well-behaved citizens will earn access to “green lanes” that provide faster government services, the blueprint said. Citizens in jobs deemed sensitive—lawyers, accountants, teachers, journalists—will be subject to enhanced scrutiny, it said.

Some Chinese cities such as Shanghai already maintain a “red list” of model citizens and a “grey list” of bad actors. No doubt, these lists the early stage of what, in a decade or so, will become a complete modeling of individual, and company, adherence to rules set by the state. Of course, repressive regimes have long used network mechanisms, typically human agents, to monitor citizens and determine risk, but China’s vision, enabled by technology, takes this concept to a whole new level.

As I read the Journal articles, I was reminded of my experience working in China and teaching Chinese students here in the U.S. I have long noted that most Chinese students I speak with have experienced what I call “democratic fission,” which is my term for the splitting of democracy into two parts: social and personal. For Westerners, social and personal democracy have been intertwined since antiquity. For most of my Chinese students,  personal democracy  being able to work where they want or marry who they want  is the goal. Social democracy  chasing one’s leaders or laws — is not a major focus at this time. In such a state of mind, the idea of a system that rewards and punishes social interaction may seem a very natural evolution, completely in line with a country that places restrictions on information access and dissent.  Indeed, as the Journal also notes:

Zan Aizong, a Hangzhou human-rights activist, sees the system, once it’s fully operational, as an Orwellian exercise to keep closer tabs on a populace already lacking basic liberties such as freedom of speech. “Tracking everyone that way,” Mr. Zan said, “it’s just like ‘1984.’ ”

Sitting here in the U.S., it’s easy to read these stories and shake our heads in disapproval, wondering how the Chinese would ever allow such a system. But we have to ask: are we so different? Already, we allow the government and private companies to collect an endless amount of data about us and our lives. Moreover, a recent SCOTUS decision only expanded that reach, which will surely be put to quick use by the new security team coming to Washington, as Bloomberg recently noted:

Trump’s first two choices to head law enforcement and intelligence agencies — Republican Senator Jeff Sessions for attorney general and Republican Representative Mike Pompeo for director of the Central Intelligence Agency — are leading advocates for domestic government spying at levels not seen since the aftermath of the Sept. 11, 2001, terrorist attacks [Emphasis mine].

How far-fetched is it really to imagine a U.S. “terror risk score” system that assigns each of us a score based on how likely we are to support or commit an anti-government act. Family ties to Arab nations, for example, would lower your score, as would reading articles about IS or Islam. Going to a Christian church every week would probably raise it, as might joining a citizen militia or border patrol. Can we really be sure that what the Chinese are building is impossible in the U.S. or other countries? Is unthinkable that our “No Fly” list wasn’t the inspiration for China’s vision. I think not, and anyone concerned with liberty anywhere should pay close attention to what is happening in China, for that nation may soon become an exporter of the technology that it’s creating for its social credit score system.

I have written in the past about what I call “the Linn Effect,” which is when a new technology creates (at its inception) more problems than it solves. I am coming to believe that the Linn Effect is universal, and that the Internet is no exception. China’s new system is just the latest example of this phenomenon, and we would be wise to consider just how close we are to following in China’s path. Indeed, in 1984 Orwell wrote the following:

By comparison with that existing today, all the tyrannies of the past were half-hearted and inefficient. The ruling groups were always infected to some extent by liberal ideas, and were content to leave loose ends everywhere, to regard only the overt act and to be uninterested in what their subjects were thinking. Even the Catholic Church of the Middle Ages was tolerant by modern standards. Part of the reason for this was that in the past no government had the power to keep its citizens under constant surveillance. The invention of print, however, made it easier to manipulate public opinion, and the film and the radio carried the process further. With the development of television, and the technical advance which made it possible to receive and transmit simultaneously on the same instrument, private life came to an end. Every citizen, or at least every citizen important enough to be worth watching, could be kept for twenty-four hours a day under the eyes of the police and in the sound of official propaganda, with all other channels of communication closed. The possibility of enforcing not only complete obedience to the will of the State, but complete uniformity of opinion on all subjects, now existed for the first time.

Ominously, the Journal highlights that the social-credit system’s aim is “to allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.”  It’s a catchy slogan, to be sure, and one that Orwell himself would have recognized all too well.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: