[US] Kenneth Lipartito, Translated by Liu Xin: Surveillance Capitalism: Origins, History, and Impact
The concept of "surveillance capitalism" has existed for about a decade, but it truly gained public attention thanks to Shoshana Zuboff’s 2019 bestseller, The Age of Surveillance Capitalism. While different scholars emphasize various aspects of the term, they all revolve around a central theme: the rise of new digital technologies that do not merely serve us, but surveil and even seek ways to manipulate our behavior to make us surrender our information. These technologies harvest vast amounts of data regarding our behaviors, preferences, and social interactions, which are then sold, packaged, and repurposed for profit.
What gives surveillance capitalism its distinctively capitalist character is the systematic dispossession of our personal data by private entities—primarily large corporations in the tech, information, and social media sectors—who then claim ownership over that data. As we move through the digital spaces of the internet—and increasingly through the physical world as facial recognition and other tracking technologies proliferate—fragments of our lives are captured, analyzed, and commodified. Although we might think this information brings transparency, transparency is like a two-way mirror where, in theory, both parties can see each other equally. Surveillance, however, highlights inequality: one side can see the other, while the other remains blind.
This data collection occurs on a massive scale, creating the conditions for powerful algorithms and machine learning systems to predict our behavior—such as what we will buy, where we will go, and for whom we will vote. These predictions are highly exploitable; the companies that generate them (such as Amazon and Facebook) can use them to target consumers with personalized marketing. Their true value, however, lies in being sold to third parties—the companies and institutions participating in data auctions—who use this data to push advertisements, political messages, financial products, and anything else that can be bought or sold.
Both the ingenuity and the danger of this system lie in its market logic. The more our actual behavior conforms to these data-driven predictions, the more valuable the data becomes. This creates a feedback loop in which the system constantly refines itself, making us more transparent and predictable, which in turn further enhances its economic value. In her book The Age of Surveillance Capitalism, Zuboff warns of the broad consequences of this process: the erosion of individual choice and free will. As our lives are increasingly managed by algorithms, our autonomy is undermined.
While the ethical and moral implications of this loss of autonomy are significant, I wish to focus more on its economic consequences—namely, the potential loss of creativity, innovation, and entrepreneurship. In a system where behavior is regulated and prediction is dominant, the space for creative disruption shrinks. When everything is designed to conform to expectations derived from data, the opportunities for the "accidental breakthroughs" that drive true innovation are reduced. Therefore, surveillance capitalism not only controls individuals but may also stifle the entrepreneurial spirit and creative innovation that have traditionally driven economic growth.
II. The Social Origins of Surveillance
One irony of living in a surveillance capitalist society is that, regardless of the dangers and ill effects, surveillance continues to spread because it also makes positive contributions. For example, we have seen how information technology empowers consumers and citizens by providing more information and giving people greater control over their lives. It is precisely due to the development of information forms like credit scores that financial tools such as credit cards have become available to many groups—particularly women and ethnic minorities who were previously excluded due to insufficient risk assessment information. Similarly, consumers can now instantly access goods and services through online retailers. This exerts pressure on traditional retailers, prompting them to lower prices or expand their online presence. We see similar positive effects from online travel platforms, which both facilitate travel arrangements and allow consumers to quickly and easily select the most cost-effective products. This is also true in politics: the increase in channels for discussion, debate, and investigation has driven governments to take responsibility more actively and strengthened citizen oversight of bureaucracies. To be sure, negative counter-examples can be found for every instance, but the existence of these positive factors makes it difficult to simply reject all surveillance activities.
Despite its potential drawbacks, surveillance is deeply integrated into our lives for the most pervasive reason: we live in complex and geographically vast societies. The term "surveillance" appeared around the time of the French Revolution. In a world bound by tradition, strict social hierarchies, and fixed social roles, there was no need for people to be "known," studied, identified, and recorded. In a world composed of free citizens, people lost these fixed identities. If capitalism is, in a sense, a process of dissolving the closed traditional structures and bonds of the past—those restricted by ethnicity, geography, and economy—or letting them "vanish into thin air," then it also requires us to live in what Georg Simmel called a "society of strangers." We are free, but we do not know one another. In such a world, there is a need for methods and mechanisms to establish trust and connect people who otherwise have no fixed relationship or shared history. By collecting information on individuals and verifying identities, surveillance serves this function.
If surveillance were entirely harmful, abandoning it would be easy. Unfortunately, there seems to be no simple way to separate the positive contributions of surveillance from its negative consequences. In the digital age, once information is released, it spreads rapidly and is nearly impossible to delete entirely. Even stricter privacy laws cannot protect us. Although I might be able to protect my own information, what I do and who I am can be inferred from those I am connected to and parsed out from massive datasets. Surveillance is not just an individual-level problem; it acts like an externality [1] that spills over onto us from others.
Many major decisions with profound impacts on our lives are made at a more macro level through algorithms. Surveillance mechanisms sort individuals into different categories; almost any category of identity or behavior can be relevant. In the economic sphere, categories of risk, consumption habits, and demand, or—for workers—skills, training history, personality traits, and identity categories, are frequently and widely employed. In these ways, surveillance helps decision-makers understand differences between individuals in a seemingly neutral manner and make decisions accordingly. However, it is difficult for this differentiation to remain neutral, as the data it relies on originates from historical records, which may perpetuate past inequalities and biases into the present. For instance, if an employer wants to target recruitment advertisements at the group of candidates most likely to apply, they will use algorithms that extract information from the profiles of existing employees in that industry. When these positions are primarily held by men, recruitment ads and notifications on social media will simply bypass women. For victims of past employment discrimination, algorithms may end up "automating inequality" rather than eliminating it. New surveillance technologies can also be affected by specific forms of inequality. For example, facial recognition technology is calibrated based on Caucasian facial features and thus frequently misidentifies African Americans and other people of different skin tones.
Some countries have begun attempting to curb these discriminatory effects. In 2018, the European Union implemented a comprehensive privacy and data protection framework. In contrast, the United States has almost no federal laws to restrict data collection or protect groups affected by it, such as workers. Individual states like California, Virginia, and Utah have enacted fairly comprehensive privacy and data protection regulations, while the relevant laws in other states, such as New York, Michigan, and Nevada, have a relatively narrower scope. Generally, these state laws follow a historical pattern first established in legislation related to consumer credit and credit reporting. These laws aim to grant people certain rights, such as the right to know what data of theirs is being collected, the right to opt out of certain data collection activities, and the right to correct or challenge inaccuracies in that data. To a large extent, this patchwork system of legislation—some of which is more favorable to businesses than to consumers or workers—does almost nothing to limit surveillance in the workplace or the market. Even the transnational laws of the EU aim not only to protect people from the intrusion of data collection but also to harmonize the relevant rules among member states and facilitate internal data exchange. In many ways, the emphasis on accuracy and the choice to "opt in" or "opt out" of surveillance activities may ultimately result in surveillance becoming more accurate and seemingly consensual, thereby actually strengthening the practice of surveillance itself.
Whether used for good or ill, surveillance spreads according to a powerful logic, an "iron law": the solution to the problem of surveillance is to install more surveillance. That is to say, the remedy for bias or injustice in surveillance is to obtain more information and implement even more rigorous monitoring. For example, if facial recognition technology misidentifies certain faces or skin tones, then we must develop better, more precise facial recognition technology.
This logic existed at the very earliest stages of the development of surveillance systems. One of the earliest instances of data collection in a capitalist economy was credit reporting, which dates back to the 19th century. Credit agencies collected and evaluated people's personal information and then sold the data and evaluations to lenders and creditors for profit. From the beginning, they were accused of engaging in "monitoring and prying," while simultaneously being criticized for not possessing enough—or sufficiently accurate—information about the subjects of their investigations, or for using inaccurate or biased data to defame those being surveilled. After World War II, the Federal Housing Authority (FHA), which insured home mortgages, required a credit report to be attached to every mortgage application. This reinforced the racial segregation brought about by "redlining" (where financial institutions refuse to provide loans or services to specific areas), making some groups (especially African Americans) ineligible for mortgages due to lower wealth levels, low-wage jobs, and associated employment histories resulting from racial discrimination. In the late 20th century, credit agencies were also accused of bias in their information collection and evaluation methods. Women, minorities, and anyone living a non-traditional lifestyle were subjected to intense scrutiny because their personal behavior did not conform to the expected white male standard.
Due to concerns that credit agencies were making arbitrary and discriminatory assessments based on limited data collected through non-standardized methods, the United States passed fair credit laws in the 1970s to regulate the types of data used to evaluate individuals. These laws both encouraged increasingly systematic data collection and allowed new types of credit scores based on statistical data to replace previous subjective descriptions of creditworthiness. Unlike the discarded descriptive evaluation methods, where a lender would look at a person’s life history and then decide whether to grant a loan, statistical methods do not possess that inherent subjectivity, which was disadvantageous to anyone who did not fit a narrow standard. Credit agency files transformed from a jumbled pile of public documents, private investigator findings, and neighborhood rumors into seemingly objective information, such as an individual's past credit habits. Eventually, this information was collected in machine-readable form, and through the application of statistical techniques such as correlation analysis and prediction, it gave rise to modern credit scores, pioneered by Fair Isaac—the FICO Score.
Although credit reporting appears more impartial and egalitarian today, the result has been to turn the FICO credit score into a passport for entry into numerous spheres of life, such as credit, mortgages, rental housing, and even employment. At the same time, contextual data unrelated to finance—such as census tracts and employment history—has become increasingly accessible; all of this data can be used to locate and track individuals. Credit scoring relies on correlations between different variables related to income, employment, and so on. However, these correlations can correspond with contextual data, thereby re-establishing links with race, gender, ethnicity, or other identity categories. This allows bias to seep in through a hidden backdoor. Similarly, even seemingly objective credit histories cannot escape the influence of structural inequalities, as they are ultimately formed based on past behavior. These records fail to account for historical factors, such as the long-term lack of wealth, employment opportunities, and housing for certain groups, which leaves their financial standing far inferior to that of others.
In short, decision-making algorithms are embedded in current social relations; they reflect existing data without considering the source or history of that data. While we can see the roots of this problem in the history of credit reporting and other assessment systems for services like insurance, a key difference today is that more sophisticated decision-making algorithms have been handed over to private institutions that are not required to maintain transparency or disclose their internal mechanics. This makes it even more difficult for people to detect the presence of bias, inequality, or the effects of racial discrimination within these decision-making algorithms.
III. The Surveillance Infrastructure
The examples above demonstrate that the same data and technology can be used for multiple purposes, which may exceed the original intentions. Even if surveillance is not viewed as "malice," we can see it evolving in unexpected ways and generating unpredictable consequences. It is better not to understand surveillance as a single activity, but rather as an infrastructure that has been gradually built up over the past century or more.
Infrastructures are more than just physical entities. They embody laws, rules, norms, labor, and political elements—that is, a series of institutional factors. These factors acquire a certain momentum and spread by virtue of economies of scale and scope, thereby becoming ubiquitous in our lives. Because of this, they shape and even determine certain activities. For instance, the infrastructure of automobile transport includes cars, roads, traffic signs, gas stations, garages, and traffic laws. All these elements interact to make the car an efficient, practical, and ubiquitous tool of daily life. In fact, if we did not have a well-managed system and support mechanism making cars easy to use, people would hardly want them on the road. However, such an infrastructure makes other modes of transportation either unlikely to be realized or much more difficult to achieve. In a car-centric city, people may choose to ride bicycles, but doing so will be difficult. Road and pavement conditions, traffic rules, and people's expectations are all designed for cars. Similarly, the possibility of public transit development decreases because car-owning voters see no reason to pay taxes for public transport. That is to say, they want better roads, not buses and trains.
Similarly, it is increasingly difficult to "opt out" of the surveillance infrastructure. We must be recorded and identified by surveillance systems to participate in social activities, such as applying for a home loan, visiting a hospital, or opening a bank account. Of course, provided there is sufficient political will, the momentum of any infrastructure can be curbed. If voters exert enough countervailing force, driving need not be the primary mode of travel in every city. However, improving a well-established infrastructure is difficult and costly, especially as it continues to evolve, growing more powerful in scale and function and more efficient. Engineers and entrepreneurs solve the thorny problems that limit the system's efficient operation, overcoming what Thomas Hughes called "reverse salients" [2]. The development of surveillance infrastructure attracts more investors, employs more workers, and takes on more functions. Our reliance on it promotes the development of the division of labor. We do not need to write our own code or create our own decision-making algorithms; others do this for us. Meanwhile, using Virtual Private Networks (VPNs), non-tracking search engines, and ad or cookie blockers to protect privacy requires effort and constant attention. Infrastructure is supposed to free us from these chores, but as more work and functions are handed over to the system, we create a "black box." In this way, design and operation become invisible to us. Consequently, we lose the ability to control or even question these operations.
One reason we may misunderstand the consequences of infrastructure reliance is that we often draw conclusions solely from the design level, ignoring emergent or unintended properties. Taking the internet as an example, it was endowed with characteristics of openness and flexibility at its inception. In the 1970s, the Internet Protocol/Transmission Control Protocol (IP/TCP) designed by engineers Vinton Cerf and Robert Kahn in a knowledge-sharing environment was decentralized, highly adaptive, and capable of handling many different networks. In contrast, traditional telephone operators pushed the X.25 standard, which envisioned several public data carrier networks. By virtue of its ability to handle multiple users and varying network designs, the IP protocol won out, and thus the internet, rather than public data networks, became the primary pathway for data transmission. This open design, in turn, allowed numerous other actors to develop programs and applications, eventually giving birth to Hypertext Markup Language (HTML) for the World Wide Web and triggering a massive transformation in data and communications via a global internet with vast and diverse websites.
To some extent, freedom seemed to triumph over control, but it also made the collection of data from internet users possible, leading to the emergence of today's surveillance capitalism. Google did not fully realize the profit potential inherent in its search data until around 2001. At that time, Google concluded that utilizing and selling this data was far more lucrative than simply selling access to the search engine. The established information infrastructure generated potential surveillance possibilities that were unforeseen at the time of its initial design.
IV. Information, Risk, and Uncertainty
Although this surveillance infrastructure was not centrally planned and often not intentional, it contains a powerful logic that is part of the grand history of information development, particularly reflected in the relationship between information, risk, and uncertainty. Naturally, uncertainty implies a lack of information. If we possessed perfect information, uncertainty would not exist. Nor would we have risk, as risk consists of probability calculations based on partial but incomplete information. Partial information is a resource that transforms the completely unknown—uncertainty—into calculable risk. With risk, various economic activities can be carried out, such as trade, investment, and innovation, to name but three.
Uncertainty exists because the future cannot be fully predicted. We simply cannot be certain about things that have not yet occurred. We might possess certain clues and forecasts (which are the partial bits of information that fill in the unknown blanks), allowing us to take risks—that is, to perform probability calculations on the future. But as the time horizon expands, the probabilities we can calculate decrease, and uncertainty increases. Is uncertainty, then, an inevitable factor in our lives?
Kenneth Arrow held a similar view. Although he once expounded the general equilibrium conditions for achieving economic stability in complex mathematical terms, he also conducted pioneering research into information asymmetry and uncertainty. He believed that uncertainty is an objective feature of life that may make a fair, Pareto-optimal equilibrium state unstable or even impossible to achieve. Arrow pointed out that markets for future goods and their prices do not exist. A capitalist economy relies on price information, yet price information regarding the future is missing. Economic actors might form theories or conjectures based on the information currently at hand, but Arrow argued there is no reason to believe these predictions will converge in a way that facilitates economic stability.
Therefore, uncertainty brings adverse consequences to the economy. As Jens Beckert has noted, under capitalism, investment and innovation can only be realized if there is some mechanism to overcome uncertainty and calculate the potential future returns of currently invested time and resources. Short of that, given that uncertainty is unavoidable, how do innovation and future-oriented investment happen under capitalism? Beckert maintains that the answer lies in combining information with analysis to make rational, probability-based predictions, but this also involves factors that exceed the scope of pure rationality. Here, Beckert proposes that narratives about the future fill the gap between the known and the unknown of what has yet to happen. On this point, he follows the views of John Maynard Keynes. Keynes feared that uncertainty would hinder investment and plunge the economy into depression, calling for reliance on the "animal spirits" of entrepreneurs to pull us out of this predicament. Beckert looks to narratives that tell us what to do and how to proceed. Such narratives might trigger irrational exuberance, or they might construct trajectories for economic or technological development—a paradigm that guides further investment in research and development. We can see examples of this in the trajectory of electric vehicle development or space exploration. Both have benefited from powerful narratives of the future that stimulated corresponding actions. In other cases, such narratives are more culturally specific. American firms seem particularly susceptible to the assumption that "innovation should lead to machines replacing labor," to the point where, when justifying investments in robotics and AI systems, they almost require calculating the amount of labor saved through layoffs.
Although narratives are not perfect predictions of the future, and in many cases are not even very reliable, they are still able to help shape the future by coordinating action, thereby mitigating the inhibitory effects of uncertainty. As Beckert argues, narratives alone do not guarantee the success of an innovation; they must be adjusted and regulated through information and analysis. At the same time, narratives can "reduce" information requirements by focusing on a single meaning and eliminating multiple different interpretations of raw data. Uncertainty makes the acquisition of information crucial, and it also makes finding methods to analyze and interpret that information—and give it a clear meaning—very important. This simplification is a key solution to the fundamental problem of informational uncertainty, though its results are not always satisfactory.
V. Information Inequality and Surveillance
With the rapid development of information technology, some enthusiasts in the information field believe it may be possible to overcome the limitations of uncertainty pointed out by skeptics. At present, perhaps we can sufficiently resist the tides of ignorance to move toward a true Pareto-optimal economy. If so, then the role identified by Beckert—where narrative fills the void left by information—would no longer be necessary. We would be able to view the future with clear, rational eyes. In 2002, Hal Varian became the Chief Economist at Google. He joined the company exactly at the turning point identified by Zuboff, when Google shifted to a strategy of collecting and commodifying information. Varian argued that we have now acquired enough information to formulate "perfect contracts." The technologies that facilitate such contracts are diverse; generally, any technology capable of closely monitoring behavior and collecting data on human activities will suffice. For instance, with abundant real-time data, car rental agreements can be enforced through in-car tracking technology. This technology can reveal a driver's style and whether they are abusing the vehicle or creating unsafe conditions. The same technology has already been applied to insurance underwriting, where premiums are linked not only to mileage but also to how the car is used. If a driver frequently accelerates or brakes sharply, insurance rates increase because the company's algorithms predict a higher risk of accidents for such drivers. For Varian, this close monitoring (which he does not refer to by the more pejorative term "surveillance") benefits all parties because it reduces transaction costs, decreases the need for organizations and hierarchies, and allows all transactions to become market transactions. Every trade, as well as every wage and salary, can be specifically customized according to the concrete circumstances of each unique customer, borrower, or employee.
The world of perfect contracts envisioned by Varian is also viewed as a world of information equality. From the perspective of classical liberalism, a contract is a voluntary agreement reached by two parties possessing equal autonomy and rights. Information makes both parties informed; the contract reflects their voluntary choice and self-interest. The underlying assumption here is that information acts like a beam of light, illuminating the originally dark and obscure corners of human existence. This light brings transparency, allowing both parties to understand each other as clearly as they are seen by the other. However, what this beautiful vision fails to consider is that the technologies which could be used to achieve transparency and information symmetry can just as easily produce the opposite effect. Varian argues that, theoretically, perfect price discrimination combined with free market entry might reduce profits to zero, which is considered a boon for consumers. But he overlooks the possibility that the same technology could be used to establish barriers to entry and allow perfect price discrimination to eliminate consumer surplus. Similarly, he believes that well-informed consumers would be in a stronger position when negotiating with producers. Yet, he again ignores the scenario where one party to the negotiation might accumulate a vast amount of information, making it difficult to reach a profound and precise consensus. The same applies to labor markets. Monitoring truck drivers can improve route planning and logistical efficiency. Theoretically, workers might benefit from these efficiency gains, but this depends largely on how much control workers exercise over that information.
This is why surveillance is more useful than neutral information or egalitarian transparency. Surveillance is a strategic act of information gathering aimed at reinforcing information asymmetry rather than eliminating it. Even as information collection technologies become increasingly powerful and ubiquitous, not everyone has equal access to the relevant tools, mastery of the technology, or access to the data. Those better able to utilize "surveillance power" are more capable of seeking gain at the expense of those who cannot. There are the "monitored"—those whose information is collected—and there are the "monitors," who remain hidden or even imperceptible to the monitored.
Of course, the landscape of surveillance is not static; there also exists "counter-surveillance." In this context, people can use the same digital tools to track those in power, hold politicians and corporations accountable, secure more favorable economic terms for themselves, and access diverse news sources. This increase in options helps marginalized groups fight back. However, there is no reason to assume that technology moves only in one direction at any given moment. Information infrastructures can support many different activities moving in opposite directions. Information tools can be used to convene a rally against a dictator, but they can also be used to incite a mob to attack elected legislators. Ever-broadening channels of information and communication can trigger democratic debate and spread useful knowledge, yet simultaneously, they accelerate the spread of disinformation and misinformation. Where will the world of surveillance capitalism lead us? Toward greater equality and democratic control, or toward stronger centralized control and inequality of power? The future is full of uncertainty, but answers may perhaps be found in the historical development of information and surveillance.
VI. Information, Entropy, and Uncertainty
Over 70 years ago, the founders of modern information theory—Claude Shannon, Warren Weaver, and Norbert Wiener—defined the concept of "information entropy." To them, information means learning something unexpected. Tossing a fair coin involves uncertainty—before the toss, the probability of heads or tails is equal (1/2). But tossing a biased coin that always lands heads provides no information because the result is known (probability of 1). Therefore, for any set of data, the higher the uncertainty, the greater the information load and the higher the entropy value. Information entropy is a measure of the uncertainty in a message. Shannon defined the smallest unit of information as a binary unit, the "bit." Heads or tails? One bit generated by a coin toss is enough to provide the answer. Higher uncertainty means higher entropy, necessitating more information. Once uncertainty disappears, there are no more "surprises," and both information and entropy drop to zero.
There is an irony here. What we seek is entirely new information that was previously unknown or unpredicted—that is, non-redundant information. More information brings more possibilities and more potential meanings and interpretations of data. Contrary to popular misconception, "pure data" does not yield clear, unambiguous results. Data must be sorted through classification mechanisms. They require mechanisms that simplify and bestow meaning. People face pressure to eliminate uncertainty and reduce information entropy, but the only way to do so is to exclude various possible meanings and reduce them to a single meaning devoid of surprises. Regarding surveillance, the goal is to collect personal information while simultaneously eliminating the uncertainty within that information. One method of achieving this is to make people follow behavioral predictions derived from that information. For example, people today are expected to self-monitor by constantly checking their credit scores, taking cues and suggestions from those scores to change their behavior and improve their rating. This makes their behavior more predictable and less risky. Undoubtedly, the same applies to drivers under insurance surveillance mechanisms. To an extent, the more we conform to expectations and predictions, the more accurate and thus more valuable the data collected about us becomes. But this process also carries social costs.
Algorithms and Artificial Intelligence (AI) embody the potential losses incurred in the pursuit of certainty. An algorithm is essentially code—that is, encoded information. All codes, including language itself, simplify information and fix its meanings. This is, of course, necessary; otherwise, we would be drowned in a heap of disordered symbols that would become mutually unintelligible, and we would fall into endless efforts to seek their meaning. Although fixed codes reduce uncertainty, there is—at least in natural language—always some deviation between words and the real world. Despite the criticism leveled against postmodernists, their views on linguistic indeterminacy are actually valuable—they remind us that there is no fixed, invariant way to describe the world through code. A map of the world can never be a 1:1 precise replication. New interpretations, new meanings, and new learnings are always possible. But under the powerful influence of today's information systems, there exists the risk of eliminating this critical deviation. Ultimately, we replace the interesting and creative uncertainty of life with a false certainty. In doing so, we eliminate the various conditions and possibilities from which action, creativity, and innovation arise.
VII. Conclusion
The pursuit of total transparency and predictability has not only transformed our economy but also our self-awareness and capacity for autonomous action. The bits and bytes of digital data represent a new form of control, in which algorithms shape what we can do rather than merely reacting to our behavior. Just as printed text once reshaped our perception of space and time, algorithmic surveillance reshapes our self-perception and the various possibilities available to us. This reduces the rich diversity of human experience to quantifiable metrics, thereby depriving us of the capacity for natural innovation rooted in human creativity and improvisation.
The digital environment, once brimming with potential, now faces the risk of becoming a closed loop that reinforces behavior and preset outcomes. The pursuit of absolute certainty reflects the scientific dream of conquering nature from days gone by, but this pursuit may lead to a state of stagnation. In such a world, everything is already known, and nothing new ever happens. As Beckert argued, it is precisely in the space of uncertainty that various conceptions of the future flourish, and where we find the impetus for the creative acts that drive capitalist development. Although over-reliance on unrealistic and groundless plans and visions—untested by data and analysis—can easily lead us astray, we now seem to have moved to the other extreme, blindly engaging in constant prediction, analysis, and auditing, leaving no room for creative uncertainty. Complexity is replaced by simplification; individual difference is replaced by equalization and categorization. If taken to the extreme, the result would be a completely closed world—a world of virtual reality. For some reason, many Silicon Valley entrepreneurs seem to believe this is exactly what people want and what society needs. But it is the tangible contact with what Philip Agre called the "radical strangeness" of the real world that is the source of our creativity and the source of our humanity.
Both total surveillance and closed traditional communities are ways of restricting and controlling human behavior and disciplining non-conformity, at the cost of creativity and innovation. Surveillance was once intended to liberate us from the many constraints of the limited and homogeneous communities of the past, allowing us to live in a society of strangers. But today, it may be imprisoning us within a cage of bits and bytes. In the novel The Crying of Lot 49, Thomas Pynchon worried that regardless of how much information we accumulate, the increase of entropy would prevail. We would be like the protagonist, Oedipa Maas, suspended in a matrix of countless zeros and ones, unable to access truth or meaning. Today, it appears that our endless struggle against entropy is the greater danger. Dwelling within a "black box," we are blind and ignorant, no longer clear on how things happen, unable to understand the algorithms that drive us, and no longer aware that we are exercising subjective agency with moral attributes. This leaves us vulnerable to rare "Black Swan events" like financial crises. What threatens this freedom is not the centralized state planning that conservative liberals once feared, but corporate algorithms within surveillance capitalism. These algorithms are constructed by those who hold a nearly utopian faith in the benefits of information technology.
In this regard, the surveillance character of the digital world also foreshadows the potential risks and harms inherent in new forms of digital information extraction carried out through artificial intelligence and Large Language Models (LLMs). This information possesses the same potential to "colonize" our social lives and render human autonomy and cognitive capacities subordinate to the data collection requirements of these systems. Indeed, they are so effective at prediction—especially in the form of generative AI—that concerns now exist regarding their potential to propagate and "pollute" real-life data, ultimately leading to a loss of contact with the profoundly unfamiliar elements of the physical world. Due to the recursive training on languages and information of their own generation, they may collapse under the very success of their data proliferation.
Of course, these future risks are not inevitable. History is replete with technologies that once promised utopian visions only to ultimately fail. Examples include AT&T’s heavily touted "Picturephone," which debuted at the 1964 New York World's Fair but failed completely less than a decade later; and the once widely acclaimed "One Laptop per Child" program, which aimed to achieve global educational equity but was later submerged in the torrent of history. In retrospect, surveillance capitalism may appear somewhat overstated. Its true risk lies in the vulnerabilities it introduces into our highly integrated information systems, leaving them susceptible to hackers, data leaks, and cybercriminals. These examples remind us that while infrastructure may gain powerful momentum, it is not unstoppable. Through concerted political efforts to design a different future centered on alternative goals and values, it is possible to overcome these risks and challenges.
(Author’s affiliation: Department of History, Florida International University, USA; Translator’s affiliation: School of Marxism, Nankai University) Source: Foreign Theoretical Trends [7], Issue 2, 2025 Web Editor: Zhang Jian