Administrative Decision-Making after the Big Data Revolution

For millennia organizations have collected information about facts and people in order to have a better understanding of the present and forecast future trends and behaviours. Until recently, however, gathering data was time-consuming and costly, and the ability to analyse it was limited. Furthermore, there was always a time lag between the collection of data and its analysis, and thus only past data was available for research. Within just a few years, the rise of the Internet and the development of digital technologies have consistently reduced such constraints. Nowadays, nearly every activity leaves “digital traces”, automatically gathered by computers whose storage capacity is almost unlimited. In addition, the large amount of data increasingly generated from multiple sources can be processed in real time to extract information and valuable knowledge.  The term “big data” refers to these features of the digital economy, a “fuzzy notion” that in recent time “has invaded  (also) the legal debate”. It is well known that several definitions of big data have been proposed. According to the first and most cited, big data  “is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision-making, insight discovery and process optimization.” The so-called “three Vs” (volume, velocity, variety) have been largely employed in subsequent literature to describe the main features of big data. Volume refers to the unprecedented quantity of data that can be gathered and processed, also thanks to the declining costs of collection and storage. Variety relates to the different sources (Internet, smartphones, digital cameras, sensors, etcetera) from which data is generated as well as to the heterogeneity of its format (text, images, geo-location, mobility data, etcetera). Velocity indicates that data is accumulated in real time, and also rapidly processed. The term “big data” is currently used not only to describe large datasets with those particular features (size, heterogeneity and speed), but also to refer to the analysis of such data and/or to the techniques that allow it (also named big data analytics). Big data has been, for instance, defined as “the practice of combining huge amounts of data of diversely sourced information and analyzing them, using more sophisticated algorithms to inform decisions”; as “a technique for converting data flows into a particular, highly data intensive, type of knowledge”; or, more broadly, as “the ability of society to harness information in novel ways to produce useful insights or goods and services of significant value.” This shift in definition (from data to its analysis) may be easily explained. The huge amounts of data collected and stored by private and public organizations would be useless - simply noise - if there was a lack of powerful computational tools able to handle and analyse them. The very value of big data lies in analytics and in its ability to quickly extract previously unknown, and potentially useful information from large datasets generated from different sources. Data mining techniques, in particular, as a subset of big data analytics, use sophisticated algorithms to find unexpected correlations and patterns in data, with the main purpose of anticipating future trends and forecasting facts, behaviours or processes.  This “seemingly oracular ability” to manage information and extract knowledge from attributes in data is now a key competitive advantage. Google, for instance, has exploited the data collected in its main activities to develop new services (voice recognition, translation, and spam filters), new products (self-driving cars) and even predict the spread of the flu. Likewise, Netflix and Amazon use big data analysis to predict consumers’ preferences and recommend to clients films, books and generally products. Social networks too adopt a similar model for suggesting friends, censuring images that have been posted, or deciding which updates to show the users. As has been noted, together those systems “play an increasing important role in selecting what information is considered most relevant to us, a crucial feature of our participation in public life.” Tech companies are not the only entities to rely on big data analytics. In the financial sector, for example, data mining is employed to predict the value of investments and financial instruments. And it plays an increasingly significant role in medical, pharmacological, sociological and other fields of research as well.  More importantly, big data analytics underpins a society which, according to Jack M. Balkin, is rapidly turning into an “Algorithmic Society”, i.e. “a society organized around social and economic decision making by algorithms, robots, and AI agents; who not only make decisions, but, in some cases, also carry them out”. From credit scoring to recruitment, the rating of universities to the evaluation of workers’ performance, from the approval of financial transactions to the diagnosis of diseases and the choice of the therapy, judgements and decisions once entrusted to human beings are now increasingly performed by computer systems by means of data mining techniques. The paper addresses the so-called big data revolution and its possible impact on administrative decision-making, with the main aim of understanding the challenges that the use of advanced analytics to support or to take administrative decisions would pose to the core principles of administrative law… (segue)