The Longest Now

Psych statistics wars: new methods are shattering old-guard assumptions
Thursday October 20th 2016, 12:51 pm
Filed under: %a la mod,chain-gang,citation needed,Glory, glory, glory,knowledge,meta,metrics

Recently, statistician Andrew Gelman has been brilliantly breaking down the transformation of psychology (and social psych in particular) through its adoption of and creative use of statistical methods, leading to an improved understanding of how statistics can be abused in any field, and of how empirical observations can be [unwittingly and unintentionally] flawed. This led to the concept of p-hacking and other methodological fallacies which can be observed in careless uses of statistics throughout scientific and public analyses. And, as these new tools were used to better understand psychology and improve its methods, existing paradigms and accepted truths have been rapidly changed over the past 5 years. This shocks and anguishes researchers who are true believers in”hypotheses vague enough to support any evidence thrown at them“, and have built careers around work supporting those hypotheses.

Here is Gelman’s timeline of transformations in psychology and in statistics, from Paul Meehl’s argument in the 1960s that results in experimental psych may have no predictive power, to PubPeer, Brian Nosek’s reprodicibility project, and the current sense that “the emperor has no clothes”.

Here is a beautiful discussion a week later, from Gelman, about how researchers respond to statistical errors or other disproofs of part of their work.  In particular, how co-authors handle such new discoveries, either together or separately.

At the end, one of its examples turns up a striking example of someone taking these sorts of discoveries and updates to their work seriously: Dana Carney‘s public CV includes inline notes next to each paper wherever significant methodological or statistical concerns were raised, or significant replications failed.

Carney makes an appearance in his examples because of her most controversially popular research, with Cuddy an Yap, on power posing.  A non-obvious result (that holding certain open physical poses leads to feeling and acting more powerfully) became extremely popular in the popular media, and has generated a small following of dozens of related extensions and replication studies — which starting in 2015 started to be done with large samples and at high power, at which point the effects disappeared.  Interest within social psychology in the phenomenon, as an outlier of “a popular but possibly imaginary effect”, is so great that the journal Comprehensive Results in Social Psychology has an entire issue devoted to power posing coming out this Fall.
Perhaps motivated by Gelman’s blog post, perhaps by knowledge of the results that will be coming out in this dedicated journal issue [which she suggests are negative], she put out a full two-page summary of her changing views on her own work over time, from conceiving of the experiment, to running it with the funds and time available, to now deciding there was no meaningful effect.  My hat is off to her.  We need this sort of relationship to data, analysis, and error to make sense of the world. But it is a pity that she had to publish such a letter alone, and that her co-authors didn’t feel they could sign onto it.

Update: Nosek also wrote a lovely paper in 2012 on Restructuring incentives to promote truth over publishability [with input from the estimable Victoria Stodden] that describes many points at which researchers have incentives to stop research and publish preliminary results as soon as they have something they could convince a journal to accept.

Archiving Web links: Building global layers of caches and mirrors
Sunday June 12th 2016, 4:23 pm
Filed under: international,knowledge,meta,metrics,popular demand,wikipedia

The Web is highly distributed and in flux; the people using it, even moreso.  Many projects exist to optimize its use, including:

  1. Reducing storage and bandwidth:  compressing parts of the web; deduplicating files that exist in many places, replacing many with pointers to a single copy of the file [Many browsers & servers, *Box]
  2. Reducing latency and long-distance bandwidth:  caching popular parts of the web locally around the world [CDNs, clouds, &c]
  3. Increasing robustness & permanence of links: caching linked pages (with timestamps or snapshots, for dynamic pages) [Memento, Wayback Machine, perma, amber]
  4. Increasing interoperability of naming schemes for describing or pointing to things on the Web, so that it’s easier to cluster similar things and find copies or versions of them [HvdS’s 15-year overview of advancing interop]

This week I was thinking about the 3rd point. What would a comprehensively backed-up Web of links look like?  How resilient can we make references to all of the failure modes we’ve seen and imagined?  Some threads for a map:

  1. Links should include timestamps, important ones should request archival permalinks.
    • When creating a reference, sites should notify each of the major cache-networks, asking them to store a copy.
    • Robust links can embed information about where to find a cache in the a tag that generates the link (and possibly a fuzzy content hash?).
    • Permalinks can use an identifier system that allows searching for the page across any of the nodes of the local network, and across the different cache-networks. (Browsers can know how to attempt to find a copy.)
  2. Sites should have a coat of amber: a local cached snapshot of anything linked from that site, stored on their host or a nearby supernode.  So as long as that site is available, snapshots of what it links to are, too.
    • We can comprehensively track whether sites have signalled they have an amber layer.  If a site isn’t yet caching what they link to, readers can encourage them to do so or connect them to a supernode.
    • Libraries should host amber supernodes: caches for sites that can’t host those snapshots on their host machine.
  3. Snapshots of entire websites should be archived regularly
    • Both public snapshots for search engines and private ones for long-term archives.
  4. A global network of mirrors (a la [C]LOCKSS) should maintain copies of permalink and snapshot databases
    • Consortia of libraries, archives, and publishers should commit to a broad geographic distribution of mirrors.
      • mirrors should be available within any country that has expensive interconnects with the rest of the world;
      • prioritization should lead to a kernel of the cached web that is stored in ‘seed bank‘ style archives, in the most secure vaults and other venues
  5. There should be a clear way to scan for fuzzy matches for a broken link. Especially handy for anyone updating a large archive of broken links.
    • Is the base directory there? Is the base URL known to have moved?
    • Are distant-timestamped versions of the file available?  [some robustlink implementations do this already]
    • Are there exact matches elsewhere in the web for a [rare] filename?  Can you find other documents with the same content hash? [if a hash was included in the link]
    • Are there known ways to contact the original owner of the file/directory/site?

Related questions: What other aspects of robustness need consideration? How are people making progress at each layer?  What more is needed to have a mesh of archived links at every scale? For instance, WordPress supports a chunk of the Web; top CDNs cache more than that. What other players can make this happen?  What is needed for them to support this?

Soft, distributed review of public spaces: Making Twitter safe
Monday October 27th 2014, 2:56 pm
Filed under: %a la mod,ideonomy,knowledge,popular demand,wikipedia

Successful communities have learned a few things about how to maintain healthy public spaces. We could use a handbook for community designers gathering effective practices. It is a mark of the youth of interpublic spaces that spaces such as Twitter and Instagram [not to mention niche spaces like Wikipedia, and platforms like WordPress] rarely have architects dedicated to designing and refining this aspect of their structure, toolchains, and workflows.

Some say that ‘overly’ public spaces enable widespread abuse and harassment. But the “publicness” of large digital spaces can help make them more welcoming in ways than physical ones – where it is harder to remove graffiti or eggs from homes or buildings – and niche ones – where clique formation and systemic bias can dominate. For instance, here are a few ‘soft’ (reversible, auditable, post-hoc) tools that let a mixed ecosystem review and maintain their own areas in a broad public space:

Allow participants to change the visibility of comments:  Let each control what they see, and promote or flag it for others.

  • Allow blacklists and whitelists, in a way that lets people block out harassers or keywords entirely if they wish. Make it easy to see what has been hidden.
  • Rating (both average and variance) and tags for abuse or controversy can allow for locally flexible display.  Some simple models make this hard to game.
  • Allow things to be incrementally hidden from view.  Group feedback is more useful when the result is a spectrum.

Increase the efficiency ratio of moderation and distribute it: automate review, filter and slow down abuse.

  • Tag contributors by their level of community investment. Many who spam or harass try to cloak in new or fake identities.
  • Maintain automated tools to catch and limit abusive input. There’s a spectrum of response: from letting only the poster and moderators see the input (cocooning), to tagging and not showing by default (thresholding), to simply tagging as suspect (flagging).
  • Make these and other tags available to the community to use in their own preferences and review tools
  • For dedicated abuse: hook into penalties that make it more costly for those committed to spoofing the system.

You can’t make everyone safe all of the time, but can dial down behavior that is socially unwelcome (by any significant subgroup) by a couple of magnitudes.  Of course these ideas are simple and only work so far.  For instance, in a society at civil war, where each half are literally threatened by the sober political and practical discussions of the other half, public speech may simply not be safe.

Lila Tretikov named as Wikimedia’s upcoming ED
Thursday May 01st 2014, 5:49 pm
Filed under: fly-by-wire,ideonomy,knowledge,popular demand,wikipedia

And there was much rejoicing. Welcome, Lila!

Digital rights groups in Europe are gaining ground: a model to watch
Friday April 04th 2014, 1:13 pm
Filed under: Blogroll,chain-gang,international,knowledge

The recent historic wins for net neutrality in the EU demonstrate an organized and informed advocacy network that is still not echoed in the US or in many other parts of the world. We should celebrate and learn from their work.

Thanks to Axel Arnbak for his thorough and delightful writeup of this.

Aaron Swartz hackfests this weekend around the world: honoring his work
Friday November 08th 2013, 7:04 pm
Filed under: Aasw,Glory, glory, glory,international,knowledge,meta,metrics,popular demand,wikipedia

Help continue projects Aaron believed in, in person or online.
I’ll be at the Cambridge event and aftermath throughout the long weekend.

Related project summaries:

Socrates Jones: Wow. “There are no limits on the extent of smiting!”
Thursday August 22nd 2013, 2:53 am
Filed under: chain-gang,Glory, glory, glory,knowledge,metrics,poetic justice

Noted game designer Chief Wakamakamu writes:

So, guys. I’m pretty sure that, whenever you played Phoenix Wright, you thought to yourself “Man, this game would be so much better if it was about moral philosophy instead of high-stake courtroom arguments.”  

Well, I have come to make all your dreams come true. I’m currently looking for play-testers for Socrates Jones: Pro Philosopher, so that we can make it as awesome as it could possibly be before we unleash it on … a starved market.

Sate your hunger.  Interrogate antiquity’s moral philosophers for yourself.


Plumpy’Nut Patent – Has their “patentleft” option seen wide use so far?
Monday July 15th 2013, 10:31 am
Filed under: citation needed,ideonomy,knowledge,metrics

In 1996, two French food scientists, André Briend and Michel Lescanne, developed a nut-based food formulation to serve as an emergency food relief product in famine-stricken areas.  The goal was to have a high-density balanced food with a long and robust shelf life – one which, unlike the previous standard of milk-based therapeutic food, could be taken at home rather than in a hospital.

They soon formed the company Nutriset to further develop and commercialize the idea.  Their most popular product, Plumpy’Nut, has shipped millions of units and currently makes up roughly 90% of UNICEF’s stocks of ready-to-use therapeutic foods [RUTFs] for famine relief.

In forming their company, they captured their idea in the form of a patent (a standard way to declare ownership of and investment) and went on to build a production chain around it.  This included tweaked formulas and a family of products; production and packaging factories; and grant-writing and research to get certification + field-feedback + approval from various UN bodies.  This involved few years of up-front investment and reputation-building, and then ramping up mass production of millions of pounds of Plumpy’Nut and its derivatives. They later set up a novel “patentleft” process allowing companies in developing countries to use the patent commercially, and make derivatives from it, at no cost — after a brief online registration. This is something which has received surprisingly little attention since, considering how simple and elegant their solution. Read on for details! (more…)

Kenya’s laptop dream: reaching for the firmament, and rote naysaying
Saturday July 13th 2013, 11:51 am
Filed under: Blogroll,ideonomy,international,knowledge

Over at ZeroGeography, Mark Graham shares a prepub version of an essay he wrote for the Guardian, about the new Kenyan drive to provide laptops to its primary students. Firstly, thank you to the author for posting your thoughts on his blog as well.

The argument that “this [money] could be better spent“, however, is a bit stale.  I don’t generally go in for critical theory and analysis (despite the obvious rightness of tvtropes!), but sometimes patterns show up so strongly in someone’s writing or argument that they are clearly part of a larger social norm and can be understood as such.

This essay is one part imperialist critique of developing countries investing in new tools, one part assumption of bad implementation, and one part missed context.

Graham worries that Kenya’s new e-learning plan — which extends recent efforts to make laptops available to older students, to all primary schools — is not part of a larger strategy; though the budget speech he cites describes such a strategy.  He makes assumptions about how much of the national budget goes to different basic needs which don’t seem to be accurate.

The central argument is one I hear often about why underdeveloped regions should slow down technological & educational change.  It runs something like this:

A) don’t introduce new things, fix old things first.
B) come up with a strategy addressing all possible issues before including modern tech.
C) if your country is poorer than mine, there must be something basic and low-tech you need more.
D) technology amplifies existing skills.  it is wasteful to subsidize it for the less privileged, who can’t use it properly anyway.

These arguments don’t stand up to a second look.  Sure, it would be ideal to fix “all the things” — various underlying inequalities, inadequacies of the existing system of experts and mentors and teachers, gaps in the quality of textbooks and in local job opportunities for better-educated youth. But no single effort will do all of that.  If you are lacking many things, your primary long-term bottleneck is often your ability to develop new solutions: you need more seed corn, not more ugali.   Outside of an immediate crisis, you need knowledge, tools, factories, and other local capacity, so you can go on to invest in your own community while resolving other problems, basic and complex.

And lastly, the idea of not offering a powerful opportunity to those less privileged, because it might take them some time to make the best use of it… that sort of argument is not even wrong.  In the short term, any opportunity would be used ‘more thoroughly’ by the already-privileged.  But they have usually had that opportunity to begin with; government programs simply subsidize it for those without.  Comparing who could “use it better” is a fallacy.

By definition, when you start bootstrapping you don’t have a lot; you get there step by step.  And every individual and community deserves access to bootstrapping tools: Blackboards, electricity, glasses, phones, bikes, computers, and other technology.  Not necessarily for free, sometimes requiring sweat and barnraising by the community, but as part of a civil campaign to make this part of society everywhere.  These are all generative technologies, catalysing other new work, returning far more than their cost in what they enable.  This is true three times over for computers: they are communication devices, creative tools for making and sharing, and factories for new tools. So the results of a community learning to use them includes trying and discovering new things not currently imagined.

Graham does make the following excellent point in his essay:

There is a long history of people and states framing information and communication technologies as a solution to economic, social, political, and even environmental problems.

So there is.  Kenya should be clear that having tools, capacity, knowledge, stronger social networks, and access to more markets and jobs is not the same as solving specific problems.  This will make it easier to solve some problems; it will create others; it will add to the general standard of living and also the expectations that come with it.  It will empower people to do both good and bad things. It will be a boon to gamers and activists and gambling and muckraking and cottage industry and artists and pornography and transparency.

But it will surely prepare the country’s youth to be an active part of the internetworked world in which we live, and to help design its future.

Annotation Notes from a recent discussion with this year’s Berkterns
Thursday June 13th 2013, 10:18 pm
Filed under: citation needed,knowledge,meta,popular demand,wikipedia

Anno-notes.  (thanks, piratepad)

Wiki AuditCom : Yearly call for volunteers to help vet our books
Friday April 26th 2013, 3:28 pm
Filed under: international,knowledge,metrics,wikipedia

As noted last week by AuditCom chair Stu West.

The Wikidata Revolution: enabling structured data love
Wednesday April 24th 2013, 4:00 pm
Filed under: international,knowledge,popular demand,Uncategorized,wikipedia

A year after its announcement as the first new Wikimedia project since 2006, Wikidata has now begun to serve the over 280 language versions of Wikipedia as a common source of structured data that can be used in more than 25 million articles of the free encyclopedia.

By providing Wikipedia editors with a central venue for their efforts to collect and vet such data, Wikidata leads to a higher level of consistency and quality in Wikipedia articles across the many language editions of the encyclopedia. Beyond Wikipedia, Wikidata’s universal, machine-readable knowledge database will be freely reusable by anyone, enabling numerous external applications.

Wikidata is a powerful tool for keeping information in Wikipedia current across all language versions. Before Wikidata, Wikipedians needed to manually update hundreds of Wikipedia language versions every time a famous person died or a country’s leader changed. With Wikidata, such new information, entered once, will automatically appear across all Wikipedia language versions. That makes life easier for editors and makes it easier for Wikipedia to stay current.” – Sue Gardner

The development of Wikidata began in March 2012, led by Wikimedia Deutschland, the German chapter of the Wikimedia movement. Since went live on October 30, a growing community of around 3,000 active contributors started building its database of ‘items’ (e.g. things, people or concepts), first by collecting topics that are already the subject of Wikipedia articles in several languages. An item’s central page on Wikidata replaces the complex web of language links which previously connected these articles about the same topic in different Wikipedia versions. Wikidata’s collection of these items now numbers over 10 million. The community also began to enrich Wikidata’s database with factual statements about these topics (data like the mayor of a city, the ISBN of a book, the languages spoken in a country, etc.). This information has now become available for use on Wikipedia itself.

It is the goal of Wikidata to collect the world’s complex knowledge in a structured manner so that anybody can benefit from it.  Whether that’s readers of Wikipedia who are able to be up to date about certain facts or engineers who can use this data to create new products that improve the way we access knowledge.” – Denny Vrandečić, Wikidata project lead

The next phase of Wikidata will allow for the automatic creation of lists and charts based on the data in Wikidata. Wikimedia Deutschland will continue to support the project with an engineering team that is dedicated to Wikidata’s second year of development and maintenance.

Wikidata is operated by the Wikimedia Foundation and its fact database is published under a Creative Commons 0 public domain dedication. Funding of Wikidata’s initial development was provided by the Allen Institute for Artificial Intelligence [AI]², the Gordon and Betty Moore Foundation, and Google, Inc.

More information available here:

Volunteers can get involved with Wikidata in many ways.  Some of the first applications demonstrating the potential of Wikidata applications, and as a platform:

  • The simia “tree of life” drawn from relations among biological species in Wikidata’s database
  • “GeneaWiki” generates a graph showing a person’s family relations as recorded in Wikidata.  See for example: the Bach family

One Weird Kernel Trick: from Zero to Stats Hero in only Twelve Days
Tuesday April 09th 2013, 7:35 pm
Filed under: Glory, glory, glory,knowledge,meta,metrics,poetic justice

From the “too good to be true (but it is)” dept:


Open physics questions foreshadow future insight yet elude answers
Friday April 05th 2013, 6:16 pm
Filed under: ideonomy,knowledge,Uncategorized

See John Baez’s Open Questions in Physics.

Wikiphilia trumps party identity, says new PLoS study
Thursday April 04th 2013, 5:02 pm
Filed under: knowledge,Uncategorized,wikipedia

We haven’t resolved systemic biases yet, but this is one sign of the value of focusing on neutrality and a common goal:

Being ‘Wikipedian’ trumps party affiliation, study finds

The study results were discussed among researchers back in November.
the L.A.Times

The Unscrupulous Institute for Cultural Diplomacy Fails
Friday March 29th 2013, 7:39 pm
Filed under: international,knowledge,wikipedia

I had never heard of the Institute for Cultural Diplomacy before they started harrassing and legally threatening my friend Mako Hill. But they are clearly an organization that understands neither cross-cultural communication nor diplomacy.

There is also no possible outcome of this dispute that is in their favor. The Streisand effect can’t be reversed by filing law suits. Filing law suits will never make a diplomacy institute look good. And Wikipedia policy isn’t structured in such a way that it could ever have an article about them after this without mentioning the drama in the first place – even if enough editors one day changed their minds about notability.

Update: Cory D. was thoughtful enough to put up a note about the case on BoingBoing.

Annotation Hacks: Hypothesis XXX begins to converge
Thursday March 28th 2013, 1:12 pm
Filed under: international,knowledge,popular demand,Uncategorized,wikipedia

The various threads around, the Open Annotation spec, and the campus-wide annotation projects at MIT, Yale, and Harvard are starting to converge. It’s nice to see a future pillar of the global web take shape – with no less friction but a more diverse audience than gathered to create the early Internet specs.

I’m at the Convergence Workshop at Harvard on the topic today, and will be at the iAnnotate workshop in San Francisco in 3 weeks. Consolidating notes on a “Hypothesis XXX” hackpad. [Btw: We dearly need a fully open hackpad equivalent with more reliable uptime than piratepad et al.! I default to HP when I have a doc that needs to sustain heavy editing and be guaranteed available during a narrow window of time at a conference… but I would much rather use a Wikimedia or similarly hosted service, with a more explicit guarantee of ongoing availability, at no cost ever.]

Bad Behavior has blocked 901 access attempts in the last 7 days.