On Trust, Part II

In part I I gave the basic grounding for my take on the issues of trust, reputation, and policing. Now I want to dig into some of the deeper issues there.

The relay problemThus far, I have referenced two different broad models of trust: the communitarian model and the appeal to authority. Looking broadly at the two models, what we see is this:

• The communitarian model relies on mutual enforcement, on peer pressure. The expectation of future interactions causes individuals to make an essentially selfish gesture — that of trading goodwill now for future goodwill from others. You can graph this is a tight group as a highly connected network.
• The appeal to authority model is based on mediated trust — parties both have to place their trust and thus their well-being, in the hands of an allegedly disinterested third party. Again, the motives for this are essentially self-serving; by having a fair system, in the long run the entire community benefits. You can graph this as a triad: two participants in a transaction, A and B, with actor C acting as a mediator and guarantor of trust.

There’s a couple of interesting points to make about the authority model, however. Consider an extended diagram which illustrates the nature of hierarchical authority. What we see here is that most authority structures are not flat. Above your closest guarantor is another guarantor, and so on up the chain. What’s more, your greatest influence in a peer-pressure sense is limited to the close guarantor. Since trust is defined by an expectation of repeated interaction, You can only develop true trust with authorities who are “local” to you. Above that, what you really get is what I would term faith.

This can be best illustrated by the ways in which governments tend to work. The government you can actually influence is your local one. You can vote for your alderman, city council member, school board member, and so on. You can more easily make use of your social network to influence opinions of this person. In a well-organized or small enough community, you may even (gasp!) know them. The same holds true for the individuals used by the community to police behaviors: policemen, judges, district attorneys, and so on. Your “vote” is commensurately larger as a proportion of that individual’s total reputation (which in a democratic system is formalized at elections, just as on the Internet it is formalized when you do a Google search and get back a PageRanked result).

However, any legal appeals process may wander through the courts up through the chain. Your “share” of control over your district judge is minimal in a peer pressure sense. Your “say” in any given election is lessened in a strict mathematical sense, given the fact that every vote counts once, and there’s simply more people involved. Influencing the choices of the federal government in the U.S. is a very difficult proposition for a typical individual.

Worse, in the real world, we don’t even have that kind of flat voting system. In practice, because of how society is structured, you start getting the notion of unequal influence; subnetworks that can more effectively get out the vote, or subnetworks that can supply more donations for the campaign fund, and so on. The phrase “influence peddling” carries a very literal meaning; what is being traded is effectively the weight of reputation, a bias in arbitration. And we must keep in mind that every act of funding by a politician is effectively an act of arbitration.

In the end, the authority model is still reliant on communitarian principles; a bad supplier loses whole markets, not individual customers. A politician that snubs a constituent is damaging his relationship with that constituent’s network. A game that snubs one GLBT guildmember is damaging their relationship with all GLBT guilds and their members. And so on.

The perspective on a given case of arbitration differs dramatically depending on whether you are within the network of one of the actors or not. This is what is at the root of the phenomenon of classism in politics; the idea that arbitrators will tend to decide in favor of the party who is like them, such as the one they see at the country club. Consider the diagram where B and C are both in the same peer group, whereas A is not. Clearly, B will have more influence over the actions of C than A will, because B and C can expect to have future interactions, whereas further interactions between A and C are less likely. The question is whether C can be impartial in this case; for this sort of situation we have the concept of recusal.

Alas, recusal generally requires that C have the integrity to step back themselves; verification of the propriety of recusal lies above C, with someone we can call C’. And again, it’s far more likely that all the C‘s will be hanging out together in the future, so they’re less likely to accuse each other of wrongdoing…

There are many variant graphs that can be constructed. Consider the “scalar issue” — there may be a relationship between A and the eventual arbitrator of a dispute, but it may be a closer relationship on B‘s part, diluting the influence that A has on arbitration.

Even more interesting is the question that arises when C truly is independent, but has completely separate agendas informing their decision. For example, credit card companies act as guarantors of identity when we buy things. They are also involved in selling identity. There’s a conflict right there. The many cases of interlocking corporate boards serve as another example.

The issues with authorityHere is where we must turn to the notion of betrayal. It doesn’t mean “he done me wrong.” Specifically, betrayal can be defined as a high trust relationship that is broken.. We tend to feel it keenly in the case of interpersonal relationships — the more reinforced the relationship has been, the worse we will feel the betrayal.

In the communitarian model, betrayal is easily resolved; we can exile the betraying member (in fact, almost by definition, the act of betrayal is the act of departing the relationship web). In the authority model, however, the trust relationship shifts to the relations between A and C. When B betrays A it isn’t just A who feels the sting; since C is actuing as guarantor, A loses confidence in them as well. In effect, the loss of confidence attacks the foundation of the government itself.

This is the subtle erosion seen when you notice that a game operator isn’t cleaning up the supposedly impermissible racism or homophobia on global chat. It’s the more blatant erosion seen when an arbitration decision is decided wrongly, or when a nerf is made that favors one class over another.

Given our instinctive understanding of human tribal hierarchies, we tend to ascribe many of these actions to bias, when it’s likely many of them are simply ignorance or ineptitude. A thresholding phenomenon probably contributes here; there’s a distance (probably in the more distant suburbs of the monkeysphere) beyond which all entities are “them” and it’s very easy for us to say that THEY are the betrayers. At that point, we start to claim vast conspiracies when in fact incompetence is a far more likely explanation under Occam’s Razor. This is an extremely common pattern observable in both real-life politics (some of the more fanciful allegations made by Michael Moore come to mind) and in virtual world administration (they hate all <<insert group names here>>).

Now, the nature of the Internet is fundamentally communitarian at its basic technical level. Much of the Internet’s basic architecture was designed with the specific intent of not having a centralized authority (although of course there are some that have coalesced, such as the guarantors of identity that verify the validity of domain names). Resorting to an appeal to authority tends to fail when there is no authority.

This places a particular burden on all the regulatory initiatives that seek to tame the Net. You can only regulate a communitarian model by destroying it and converting it to an authority model. And the more distant the authority is, the higher its burden of proof, because humans distrust facelessness.

Humans seem torn between their desires for both models. We do crave the communitarian model; everything from novels like Cory Doctorow’s Eastern Standard Tribe to the multiplicity of social networking services out there serve as notice that we do in fact have an emotional hunger for the trustworthy peer group, a hunger that is perhaps not serviced well by an impersonal world. The drive to create guilds even in game that offered no explicit support for it (as happened in the early days of Ultima Online is a good signal that there are currents here that run deep.

In today’s world, the frictions of locality and status and culture have been lubricated, and these new networks are somewhat different in character than they used to be. Whereas once your peer group could be relied on to be homogenous in certain axes, and heterogeneous in others, based on the factors shaping it, today it is apt to be homogenous on those axes that are most likely to be disruptive or controversial. It is no accident that historical events like the Enlightenment, the American Revolution, Protestantism, the rise of robber barons, the Red Scare, large-scale terrorism, and the appearance of EFF can be linked with the development of and widespread acceptance of media. In previous times, the people who had an affinity with these philosophies and worldviews were widely separated; with the advent of better communications and travel, they were able to coalesce into new networks ever more tightly, and thus start to achieve a sort of “bloc voting” power on the world’s affairs.

How very different from the world in which say, a single sympathizer of Enlightenment ideals, might live, were there no means of communicating with others like them! They would likely be attacked as going against social norms, and would be unsupported by a peer group.

This sort of homogenous network has long been a major driver of human progress; consider the early forms of banks, as they existed in the Jewish and Muslim worlds, with networks of merchants serving as guarantors of wealth. Letters of introduction served a truly enormous purpose in permitting large-scale commerce. The network of trust that existed between distant merchants permitted ventures that were not otherwise possible, as a rug merchant in Turkey allowed withdrawals of assets that were originally deposited in Egypt.

Then again, the weakness of communitarian models is also in their homogeny, which has many vulnerabilities. A glance at the medical sciences will each us that homogenous populations are at great risk from outside attackers, for example. A homogenous community is less likely to effectively assess transgressions against outside parties. It is more resistant to ideas in many ways.

• Can we architect communitarian models that resist homogeny?
• Should we see the move towards this new sort of tribalism as a net positive or negative?
• Should we encourage instead the current real-world model of authority?
• How do we solve the disenfranchisement issues if we pursue the authority model?

This last one becomes more urgent as disenfranchisement rises as the authority chain grows longer. The bigger the infrastructure of a hierarchy, the more likely it is that poor little A at the bottom will not have his needs served by C””’ way at the top.

We might also ask ourselves whether the communitarian model is in fact a polite fiction we sell ourselves; Tim Burke touched on this in his comments on Part I. Is there such as thing as true bottom-up authority? After all, communitarian ideals are driven by consensus, but in practice consensus, particularly in small groups, is driven by strongmen, by natural leaders, by persuasive techniques. This is precisely why the development of things like secret balloting was such a core driver of human socil development.

This line of thought leads us to ask whether trust itself is in fact fundamentally authoritarian. The 3rd party mediator structre is to a degree a reaction to the inherently authoritarian nature of humanity, particularly of small groups. It is an attempt to create arbitration via the “rule of law” as opposed to the rule of the mob (which is in practice usually the rule of the mob’s leader).

And yet… the hardwiring in our brains is to risk the lack of trust. We’re hardwired to give an untrustworthy situation a shot. At that point, our understanding of mediation changes; it’s not about providing trust via proxy, it’s about healing the network after a failed transaction, and about chunking the good and bad transactions that are possible into classifiable groups.

Effectively, the trust chain looks like this:

Identity/reputation -> accreditation (moral authority) -> enforcementIn the end, the modern fluidity of information, and the modern ability to exit a group, greatly affect the viability of the authority model. Today we have the cute picture of authority structures asking, “what is our recourse against word of mouth?” This is a fascinating question, because it starts conjuring up entirely new models of trust. There is no valid graph for a large authority structure that is also highly trusted. At best it runs on faith.

Why this mattersConsider the insurance industry. Insurance is all about assuming no discontinuities in risk. They lay bets against average risk levels, and they plan on paying out slightly less than they take in. When the overall level of risk rises suddenly, the insurance industry suffers. This is why it’s the insurance companies that fund forward-thinking fields like bioscience (pandemics = risk), research against global warming, and so on; and also fund efforts that mitigate rapid changes to existing industries, such as DRM. Rapid shifts in cultural standards are a serious problem for the risk industry, and that means that fluidity of information flow may actually work against insurers, just as rumors spreading of a cash shortage causes a run on a bank.

This is also the movie industry’s problem with cell phones — when you can call your friends to warn them off of Gigli before the movie has even ended, you’re undermining the traditional patterns of risk. It’s possibly also why the Spanish government fell after the Madrid bombing — so much information, so much of it likely wrong, flew across the cell phone channels that the political landscape changed in Spain almost overnight.

What arises is the question of whether the (generally considered by all, including me, as benign) democratization of information can paradoxically result in undermining our freedom of thought. It takes time to sort through large quantities of information, and “bloc voting” in the reputation game can throw the wrong websites to the top of Google, or possibly throw misinformation about whether the U.S. government warned all the Jews in the Twin Towers in advance, or whether the Spanish government was to blame for the deaths of everyone on those trains.

The saving grace is that the bottom-up model has many channels in it, homogenous and stampeding though they may be. Network studies of Amazon bestsellers in the politics category show a distressing bipolarity to the books read by each “tribe” — there’s left-wing books and right-wing books, but few people cross the divide to read the other side’s arguments. But that’s still an improvement over the number of channels present in the authority model: just one.

Why is any of this a problem? Because of the culture clash that results in applying the solutions of one of the models — authority or communitarian — to the other problem domain. The inevitable area of conflict will be whenever dispute resolution or transactional costs force an interaction of the two worlds. An example of this today is the justification given by Blizzard for not permitting recruitment of GLBT guilds on their General Chat channel, a purely transactional decision based primarily on costs to the authority running afoul of communitarian notions of enforcement.

In the next segment, whenever I get around to it:

• The realm of politics and trust
• Finally, LambdaMOO.
• Does altruism have a place?
• Confidence, recourse, and autonomy of response: the ingredients of the models