The malignant side of social analytics :-(

A few weeks ago my good friend Susan Etlinger shared with me a scary article, called China’s ‘Social Credit’ System: A Frightening Use of Big Data and written by Bernard Marr, where he described new plans from the Chinese government to build a “social credit” system that is meant to rate each citizen’s trustworthiness (whatever that means). It would collect and analyze data from a large variety of channels (internet, social network, mobile applications, telecom networks, …) to give every citizen a “social score”. As with most social network analysis algorithms, we can only assume that an individual’s score would be influenced by both local and global context, that it will look at both what they do and who they do it with, and will take into consideration The Network Effect. Your score will be influenced by the scores of the people around you, so that those in your network with lower scores will be collectively dragging you down. And if, as seems to be indicated, these scores were used to reward or punish citizens (maybe effecting tax rates or access to housing, jobs, healthcare, …) then these scores could become a very strong and visible incentive for people to stop interacting with those who are “untrustworthy”, or at least have lower scores then them. The cost of being friendly with someone would be easily quantifiable. And considering the potential breadth of data sources and communication channels that could be available for this analysis, there would be nowhere to hide. This could become a world class tool for wide-spread citizen control and manipulation.

Now you could argue that citizens have been manipulated and controlled by governments since the beginning of time through the use of propaganda, misinformation, or even legislation. For me the big difference is the depth of the manipulation, where governments (or organizations) have the potential to make the manipulation much more personal in nature, targeting individuals instead of demographics. The insights it uses to design these control points are much more invasive and intimate in nature. It’s all pervasive, as more and more of our lives are captured across digital channels. And it’s completely invisible to most people, where they don’t even know they are being targeted. For example; few people think about the fact that their favorite search engine and social network controls most of what they see on a daily basis; it chooses their window on the world. This may be totally benign or malignant, only time will tell.

Today, or in the not too distant future, governments and corporations will have access to insights that would historically only have been known to yourself or a few close friends or family. They will be in a position to use these insights to manipulate and control you; however unlike with a family member, governments are definitely not thinking of your interests as an individual, at best you are part of a greater collective they need to manage. You are a means to an end; a talking wallet for the corporation or a willing (and ideally passive) tax payer for the government.

And on that slightly depressing note, I’m off for a Friday evening after work cocktail :-) Have a great weekend folks!

3 Responses to “The malignant side of social analytics :-(”

  1. Very powerful cautionary tale. Certainly the things that science fiction addresses – Minority Report, 1984, Cloud Atlas, etc.
    One thing that should not be forgotten is that analytics only affects our online, social media world. Of course, for massive majorities, the online world is no different than “the real world”. While manipulation and censorship are big and real risks, what we really think and who we talk to can also rely on the wet analog world. Brains and handshakes and talking in coffee shops, etc – I’m sitting in one now. I have a socialist talking about his worldview two tables to the left and a Trump supporter berating someone 2 tables to the right :)

    Liked by 1 person

    • More of the physical stuff is definitely a good thing. I was relaxing with a colleague in front of a chilli melon cocktail when your comment came in last night :-) You can’t beat a face to face. Sadly though I tend to spend more time on digital interactions than physical these days. Geography being a big part of my problem where most of my friends and colleagues live thousands of miles away. You being case in point :-)



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: