Over a million developers have joined DZone.

Data News: How Profiling Citizens with Big Data Constrains Freedom, and More

DZone's Guide to

Data News: How Profiling Citizens with Big Data Constrains Freedom, and More

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

Long time no see. Here are some writings worth reading (in case you missed them)

Detroit was allowed to declare bankruptcy last year. This began a protracted process by which the municipality renegotiated with its creditors, including government worker pensions. This illustrates several characteristics about the American “system”. First, it has de-stigmatized bankruptcies, as we saw with GM and Chrysler. Second, bankruptcy is not failure; it is a process by which the creditors are held at bay while restructuring or reorganization can take place. While economists often focus on the barriers to entry, by removing the stigma from bankruptcy and easing the process, it makes lower the barriers to exit. Bankruptcy does not necessarily mean liquidation, but when it does, it allows, the recycling of people and capital. It is an important, even if under-appreciated, part of the flexibility of the United States. This stands in stark contrast with Europe. Consider Rome. Florence Mayor Renzi has become the new Prime Minister of Italy and one of his first acts was to withdraw the proposal from former Prime Minister Letta to bailout Rome. Letta was offering a program of about 850 mln euros, which required parliamentary approval by the end of February. The Northern League and the 5-Star Movement mounted a filibuster to oppose. [to be continued...]

If you’re a faculty member, do yourself a favor: Google the phrase “professor under fire.” Many if not most of the thousands of hits have something to do with social media—in particular, with Twitter, Facebook, YouTube, or blogging. Social media constitutes the classic Catch-22 for academics: We can’t ignore it or avoid it, nor do most of us want to, yet it gets us into more trouble than anything else. As born pedants and guides, we find the opportunity to reach new audiences and interact with people, including students, irresistible. But there are times when we should resist that impulse. And there are other times when, even if we don’t put our words out there for public consumption, there’s a fair chance one of our students will do it for us. A series of recent incidents involving professors and social media has focused our attention on the politically fraught nature of that relationship. Or at least it should have. Instead, each new episode seems to catch us by surprise, leaving us more troubled and outraged. We appear to have adopted the stance that social “media” is just that—merely an alternative medium—and our words are no different whether spoken in a faculty meeting or tweeted online. We assume the great institutions that have historically afforded us protection on campus—tenure and academic freedom—will likewise protect us when we venture outside the academy onto websites and blogs. That has turned out not to be true in several high-profile cases. Professors have been censured, suspended, and even fired for things they have tweeted, blogged, or posted on Facebook—or things they said that other people have posted. [to be continued...]

Democracy is going through a difficult time. Where autocrats have been driven out of office, their opponents have mostly failed to create viable democratic regimes. Even in established democracies, flaws in the system have become worryingly visible and disillusion with politics is rife. Yet just a few years ago democracy looked as though it would dominate the world. In the second half of the 20th century, democracies had taken root in the most difficult circumstances possible—in Germany, which had been traumatised by Nazism, in India, which had the world’s largest population of poor people, and, in the 1990s, in South Africa, which had been disfigured by apartheid. Decolonialisation created a host of new democracies in Africa and Asia, and autocratic regimes gave way to democracy in Greece (1974), Spain (1975), Argentina (1983), Brazil (1985) and Chile (1989). The collapse of the Soviet Union created many fledgling democracies in central Europe. By 2000 Freedom House, an American think-tank, classified 120 countries, or 63% of the world total, as democracies. Representatives of more than 100 countries gathered at the World Forum on Democracy in Warsaw that year to proclaim that “the will of the people” was “the basis of the authority of government”. A report issued by America’s State Department declared that having seen off “failed experiments” with authoritarian and totalitarian forms of government, “it seems that now, at long last, democracy is triumphant.” Such hubris was surely understandable after such a run of successes. But stand farther back and the triumph of democracy looks rather less inevitable. After the fall of Athens, where it was first developed, the political model had lain dormant until the Enlightenment more than 2,000 years later. In the 18th century only the American revolution produced a sustainable democracy. During the 19th century monarchists fought a prolonged rearguard action against democratic forces. In the first half of the 20th century nascent democracies collapsed in Germany, Spain and Italy. By 1941 there were only 11 democracies left, and Franklin Roosevelt worried that it might not be possible to shield “the great flame of democracy from the blackout of barbarism”. [to be continued...]

Will Hutton poses the not unreasonable question, why is it necessary for a CEO to earn 190 times the average wage? Of course it’s not necessary, by any conceivable economic or psychological rationality. The logic of incentives doesn’t cut it here, and has been heavily undermined by work in hedonic psychology and behavioural economics showing that relative changes in income are what matters, rather than absolute levels. And unless one subscribes to a ‘war for talent’ ideology, it’s difficult to see how this is an efficient labour market, even if orthodox economics struggles to identify the precise ways in which it isn’t. So we have to view it as political, that is, CEOs are able to exploit organisational opportunities for rent extraction. That is scarcely very surprising. But I would go further, and suggest that it also has certain metaphysical-political elements to it, in which particular anthropological claims (namely that a minority of individuals have a freakish, inexplicable talent, making them incomparable to the general mass) and economic practices (salaries of 190 times those of the general mass) are constantly propping each other up. Both of these have to be viewed as equally foundational. If rampant salary inflation were explained purely in economic terms, there is the grave risk (from the beneficiaries perspective) that it could be falsified using economic evidence, namely that these rewards don’t translate into performance, as it is well-understood that they don’t. It needs, therefore, a substrate which suggests that these people are not ordinary, culturally or anthropologically, and can therefore not be measured using the yardsticks familiar to the rest of us. They are exempt from ordinary modes of evaluation and criticism, indeed their leadership status makes them the authors of new modes of evaluation and criticism. Moreover, seemingly unjustifiable levels of income are then profferred as examples of how unusual these people are. The more they are paid, the harder it is to subject them to any public measure of evaluation, because the less they appear to be normal members of society, and the more they seem like an eruption of genius or glorious violence. They don’t claim to have ‘earned’ their money, any more than Roger Federer claims to have ‘earned’ his back-hand; their money is one manifest symptom of how different they are from the rest of us, and must be tended accordingly. In my forthcoming book, I suggest that Schumpeter is the key figure here. Through his emphasis on entrepreneurship, Schumpeter implicitly offers an anthropology of difference, that is, his economic theory rests on the assumption that there are certain individuals who do not or will not operate according to the same rules as everyone else. They are exceptional and amoral, transcending the norms and standards which the rest of us allow to constrain us. They produce the institutions of capitalism, while the rest of us inhabit those institutions. It stands to reason that only a very small minority can be classified in this exotic way. [to be continued...]

We always find stereotyping plausible. That is why it is treacherous. But there are three big holes. The first is selection bias. If Indian doctors and engineers are given US visas, and they and their children are successful, is that because of Indian culture, or because the children of doctors and engineers also do well? The second is social networks. Chua and Rubenfeld may not emphasise psychological research but their worldview is ultimately about psychology rather than the contacts an immigrant network might bring. Most fundamentally, they fail to justify their basic premise: that the way to understand success is as a phenomenon that applies to religious or ethnic groups. They say that if we couldn’t generalise about Jews or Chinese Americans, “we wouldn’t be able to understand the world we live in”. Treating people as individuals is, apparently, just too much like hard work.

Corporations and governments are using information about us in a new—and newly insidious—way. Employing massive data files, much of the information taken from the Internet, they profile us, predict our good or bad character, credit worthiness, behavior, tastes, and spending habits—and take actions accordingly. As a result, millions of Americans are now virtually incarcerated in algorithmic prisons. Some can no longer get loans or cash checks. Others are being offered only usurious credit-card interest rates. Many have trouble finding employment because of their Internet profiles. Others may have trouble purchasing property, life, and automobile insurancebecause of algorithmic predictions. Algorithms may select some people for government audits, while leaving others to find themselves undergoing gratuitous and degrading airport screening. An estimated 500 Americans have their names on no-fly lists. Thousands more aretargeted for enhanced screening by the Automated Targeting System algorithm used by the Transportation Security Administration. By using data including “tax identification number, past travel itineraries, property records, physical characteristics, and law enforcement or intelligence information” the algorithm is expected to predict how likely a passenger is to be dangerous. [to be continued...]

Setting the quality of these studies aside for a moment, what is the Washington Post doing reprinting press releases? The Post’s website groups the Health & Science Section under national news. Is the Post so strapped that it can’t report its own national news, but instead must give over its pages to universities trying to promote themselves? Would the Post, with its history as an aggressive government watchdog, turn over its pages to press releases from the government? And maybe we shouldn’t be so quick to set aside the questionable quality of the two stories I’ve mentioned. A bit of reporting might have persuaded the Post’s editors that these stories should not have been done–or it might have uncovered reasons why they were more important than the press releases make them seem. Other recent press releases published in the post include “Why baby talk is good for your baby” from the University of Washington; “Men have a harder time remembering things than women do” from the Norwegian University of Science and Technology; “Despite warnings, about 24,000 kids are hurt annually in shopping car accidents,” from Nationwide Children’s Hospital; and “Too many men take testosterone when they don’t need it,” from the Endocrine Society, which represents endocrinologists and reports, on its website, that it got coverage of its testosterone release in The Washington Post! These press releases appear in print and online, under the rubric “Study Hall.” At the top of each story online, the Post’s editors write, “Study Hall presents recent studies as described by researchers and their institutions.” And they identify the institution that issued the release. [read more]

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}