Humility is not a Virtue, But Caution Can Be

I don’t have a breakthrough ethical system to present. I cut my teeth on Kant and Mills, and I haven’t come much further since. I prefer utilitarianism when working out difficult moral quandaries, and I try to make sound ethical choices in my everyday life. In one respect, however, I have been neglectful, and that has been in my behavior toward myself.

I am a sentient being who feels pleasure and pain; my happiness should carry as much moral weight as anyone else’s. Unfortunately, my moral intuition tells me something very different- that it is virtuous to treat myself badly.

It is altruistic to deprive yourself of pleasure to ease the pain of others, or to suffer pain in someone else’s stead. However, my moral intuitions do not seem capable of honestly judging when my pain or lack of pleasure will really help others. It feels as though, by always putting myself last, everyone around me should benefit, but that is far from the case.

Imagine living your whole life like this.

To show where my intuition is failing, consider the following scenario- Bridgett decides to go on a drive through her happy town one sunny day. She stops at a four-way stop, and there is a truck that rolls to a stop there as well. Bridgett knows she has the right-of-way, but she is feeling humble and deferential, and decides the nice thing to do would be to wave the truck on. She doesn’t lose much time that way, and the driver of the truck may be in a hurry.

Unfortunately, Bridgett failed to check her rear-view mirror, and misses the car that has stopped behind her. This car will also be inconvenienced by her act. In addition, the rules of the road have been muddied a little, and the more Bridgett adopts this behavior- the more everyone adopts this behavior- the more awkward pause-and-wave exchanges will happen at four-way stops.

Bridgett and the car behind her were mildly inconvenienced in this scenario, but let us move her off of the sleepy exurban street and onto a freeway. Bridgett usually avoids freeways, especially in cities, because she does terribly on them. In the dark corners of her mind, she has acted in a deferential manner so often that the state of Texas handbook might as well state “Bridgett never gets the right-of-way.” Bridgett knows that she has to get onto the ramp, speed up, and merge with freeway traffic, but her ingrained instincts are screaming at her to slow down and let everyone else go ahead.

Today there is a lot of traffic, and a zipper merge is required. Bridgett wants to let everyone ahead, but it seems there is a wall of cars without end. Bridgett hesitates on the gas- the merge lane will soon end, and she isn’t up to speed. Cars are whizzing past. The situation has become dangerous, and traffic is piling up behind her. This can result in a traffic jam in the best case, and an accident in the worst case.

There are unending situations in everyday life in which deferential habits can cause more harm than good. It’s a popular fact that depriving oneself may make one unable to help others at all- “put on your own oxygen mask first.” But in addition to this, when you lower yourself on the social hierarchy, you create a greater disparity that the unscrupulous are tempted to exploit. In a sense, you unbalance a social order that requires a good deal of fairness to operate. Zipper merges prevent accidents and traffic jams only if you are willing to go ahead when it is your turn.




Truth is something that exists outside of our notions of social hierarchy. Fire burns, and it will burn the hand of king and peasant alike, even if the king decrees that he alone is allowed to touch the flame.

Unfortunately, my intuition is unwilling to consider this in its reckonings. I have an unfortunate habit of apologizing for being right when I am shown to be correct in a factual disagreement, and sometimes I will even stay silent when I have knowledge that I should share.

My spouse is an uncommonly intelligent person who has the ability to notice and remember far more details than I can. I space out, and my memory is inconsistent. However, after a lot of introspection I’ve finally concluded that, though my brain may be quirky, I’m not actually stupid. I can keep up with my spouse on an intellectual level at least enough to carry on very interesting conversations, to banter point and counterpoint with him on complex topics, and to be an interesting and entertaining companion to him.

Even so, I still have a hard time stepping out of my self-appointed deferential role. To give a fairly mundane example- not too long ago my spouse was returning a product to the manufacturer, and we went to the UPS store to get a box and label for shipping. The manufacturer requested that the product code be written on the box, so my spouse wrote it on the side in sharpie. He was filling out the label when I noticed a missing digit on the product code he had written.

It took me a few moments to work up the courage to point it out, and even then, I did so in a little faltering voice- “um, I think the five is missing?” I had no reason to add the question mark- I knew the five was missing- but I added the question mark to the end of my sentence anyway.

My spouse, being the reasonable human he is, checked the number and corrected it. I apologized for correcting him as we left the UPS store, but he insisted that he was of glad I had pointed out the error, and thanked me for my help.

Had he not been so understanding- if instead of being a nice person he’d been an arrogant jerk- he might have sneered at my correction. He might have insisted he was right, and pointed out both my astigmatism and my short attention span as evidence that I must be wrong. In fact- he might have just insisted he was smarter than me, and therefore I should shut my mouth. After all, why would a smart person deign to check the number on the box per the mere word of a bespectacled goofball?

If he had acted so, would his arrogance have been the opposite of the mistake I almost made when I hesitated to point out the product code error? In other words, if the jerk version of my spouse had felt less sure of himself, would be have been more willing to check the number? Or would he have fought to maintain the status differential between us, and been less willing to check? From what I’ve seen in arguments between people who view themselves that differently, I believe the latter- a person of very high status, and who finds their high status to be important, is less willing to check an error pointed out by someone of a very low social status.

Conversely, if I had been the one to write the number on the box, and someone else had pointed out an error to me, I would have instantly re-written the number without checking what I’d written. I assume I am wrong, and do not seek to justify myself before altering my behavior to suit others.

I propose that the evil version of my spouse and the current version of me are two sides of the same coin- that we are actually making two versions of the same mistake instead of different mistakes altogether. We are using our sense of relative status as the standard to determine the reliability of information instead of making an analysis of the information itself.

You might argue that relative status can be used as a heuristic to analyze the source of information, but such a heuristic can actually blind one to making an honest examination of the information source- and an excuse to ignore it altogether. If the evil version of my spouse cared more about knowledge than status, he might have considered the fact that, while I do have an astigmatism, my glasses were clean and my prescription up-to-date. He might have considered that, while I usually walk around in a fog, the very fact that I pointed out the error was an indication that I was paying more attention than usual. If the evil version of my spouse had thought about the matter further, he might have realized that product codes are generally long and confusing strings of characters, and that it would be helpful to have a general policy of double-checking them, just in case.

In other words, status hierarchies can be somewhat useful, but are inadequate and outdated technology for analyzing anything truly complicated. It is more useful to ulitise caution instead of humility- to check for mistakes in one’s self regularly not because you think that you are stupid, but because you know that you are a buggy system who deals with complex information.

If I am a buggy system, does that necessarily make me a bad system? If I am, does thinking I am ‘bad’ tell me where I am going wrong? Will calling myself a fool fix my errors? No, it won’t. In fact, this kind of self-flagellation tends to increase my errors. Feeling humble and deferential has held me back, kept me quiet, and made me hesitate to contribute to a society I could otherwise help. At the very least, it has caused me to neglect my own moral worth. Reminding myself that automatic deference will not help anyone is an effective way of countering it. I imagine that thinking overconfidence will not help one’s self may be similarly helpful in overcoming that error.


Santa Claus and Modest Epistemology


There are four kinds of children. The first kind never believe in Santa Claus, the second kind believe at first and then work out the truth about Santa Claus for themselves, and the third kind continue believing until someone else reveals the truth.

The fourth kind of child makes an honest attempt to question the Santa Claus paradigm, utterly fail in their reasoning, and never understand where they went wrong until they are 37 and Eliezer Yudkowsky writes a book called “Inadequate Equilibria.

I think I may be the only person in the last category. If not, I hope you find this enlightening. If so, I hope you at least find this entertaining.




When I was five or six years old, I began to question the Santa Claus paradigm for the usual reasons. I had a rough idea of the size of the world- namely that is was really big– and I understood the difficulty with a single man visiting every house in the world in a single night. My own father was a pilot, and I was familiar with how long it would take for him just to go to Little Rock and back in a learjet. Even considering the fact that many people don’t have children, and many children don’t celebrate christmas, it seemed like an impossible task. My parents and my sister explained that Santa solved this problem using magic.

My second problem was a petty one, but it was still a problem. I knew many children at school who were not only naughty- they were huge jerks, and all of them got what they wanted for Christmas. In fact, many of them would brag very loudly about what they got for Christmas, and why it was better than what everyone else got. In the back of my mind, I excused this as their parents being nice enough to take the coal out of the jerk’s stocking and replacing it with a present that they’d bought before the kid woke up.

More troubling, the adults around me would often tell me about the importance of charity and of remembering those who were less fortunate during Christmas. But if Santa Claus was real and had magic powers to enable his generosity, why didn’t he bring poor children all of the clothes, food, money, and even toys they would need for the entire year?

(I’m not the only one who was troubled by this. Watch this- it is the best Christmas movie ever made.)


Soon, even the magical explanation for Santa’s impossible Christmas journey began to crumble. My own experiments with magic had not yielded any fruit. I was unable levitate objects by snapping my fingers, twitching my nose, or concentrating very hard. I wasn’t able to curse people no matter how angry I got. I poured salt in my bath water, but this did not give me a mermaid tail. (I was a big fan of the movie Splash.)

Anyone reading this may think that, at this point, I was one step away from rejecting the Santa hypothesis. However, when I sat down and thought about the problem, my brain argued that there were a lot of big, smart grownups who endorsed Santa Claus, and not just my parents. There were grownups who made movies about why we should believe in Santa Claus (like Miracle on 34th street.) There were grownups who used satellites to track the progress of Santa’s sleigh on TV every Christmas Eve. Santa Claus himself was in the Macy’s parade every year, and then he went on a tour of the nations malls to gather intelligence about all the children. Sure, there was the odd grownup who did not seem to believe, but my parents explained that these grownups were merely “grumps” and “grinches” who were unhappy, and therefore wanted to ruin Christmas for everyone else.

In the end, I decided that even though Santa Claus seemed unlikely, the idea would require a worldwide conspiracy to perpetuate (I didn’t have this vocuabulary at Five. What I actually thought was ‘everyone in the whole world would have to be in on it.’) Really- a worldwide conspiracy, just to trick me into believing in a jolly man who brought presents? Yeah, right. Even if I, a little five year old, didn’t understand magic, all of those big smart grownups probably did.

I will usually joke, at this point in the story, that when I found out about Santa Claus I almost became a conspiracy theorist. Becoming paranoid was obviously not the correct lesson to draw from Santa Claus, but if not, where did five year old me go wrong?




My First mistake was that I didn’t question the system that produced the Santa Claus story. I assumed that the only reason for creating the worldwide conspiracy would be just to trick me, and I didn’t consider that it was me and every other 2-10 year old (roughly.) I didn’t consider that one of the reasons why everyone might go along with this is that they were incentivized to tell children about Santa Claus. It is really fun to tell young children fantastic stories and have them believe it. In retrospect, I should have realized the incentive to trick gullible kids for fun- I did have an older sister, after all. I also failed to consider how many people follow traditions without ever questioning why, but I give myself a pass for this one. I had no idea grownups did that at even seven years old (when I was told the awful truth.)

My second mistake was not assigning a low enough prior probability on magic. I assumed magic a priori, and left my experiments to disprove the magical hypothesis. This made me accept very weak evidence for magic, such as rudolph’s teeth marks in the carrot we’d left beside the cookies and egg nog for Santa. The teeth marks seemed too large to belong to my Dad, but if I’d examined them more closely I’d have seen they were the same size. Even if the tooth marks had been too big, it would have been easy to fake. Finding a mechanism for magic, however, was impossible. Magic is, by definition, that which has no mechanism. Because of this, the improbability of magic should have had a much larger weight in my mind, and not only were my failed magical experiments enough- they were more than sufficient to invalidate the hypothesis.

My final mistake was that I was terrified to test the null hypothesis. I wasn’t just afraid of getting no presents, I had grandparents after all, but I was afraid of being naughty. I conceived of myself as a nice girl- not the kind of girl who dares to question nice men who give you presents every year and only ask that you mind your parents and not bother him. The type of people who question Santa are mean grinches. I did not want to be a mean grinch.

Even though the final mistake is the most difficult for me to correct, even now, I don’t give my younger self a pass. This mistake is a reflection of my greatest fear, but it’s the one I long most to break free from. The desire to be “good” has made me hold myself back for no reason time and time again. Now that I’m a little older, I’ve learned that the truth doesn’t mind being examined, and to do so doesn’t make you a mean grinch. In fact, examining the truth only makes it stronger.*

The last gift Santa gives to children who believe is the gift of skeptical inquiry. Whether the children discover the truth for themselves, whether they are told, or whether they discover it by accident, they come out on the other side with a +1 to WIS- an increased ability to question the  world around them. It’s the first grown-up step into a world that is much bigger than the world of magic.


Merry Christmas

*I’m keeping this line because it sounds nice and feels right, and because I’m going to tear it apart in a future post.

My Inadequate Epistemological Mess

I’ve been reading Eliezer Yudkowsky’s new book, Inadequate Equilibria, on LesserWrong as it’s been released, chapter by chapter. I have too many thoughts to politely fit into a comments section, so I will mull them over here. I urge you to read the original work before you continue. 

Yudkowsky’s reasoning cuts like a knife, presenting a compelling argument for why many systems fail to fulfil their stated purpose, and why it isn’t hubris for you or I to question the products of those systems or their experts. He introduces a concept of “modest epistemology,” in which intelligent people fail to grasp at low-hanging fruit because surely, if it were so easy someone already would have.

This work has had a profound impact on me personally. Line by line, the work severed mental ties that have bound me my whole life.

I’ve been juggling the arguments for and against this type of modesty for a long time, and I realized yesterday that this has been reflected in my work. Take, for example, the following two segments from my web serial/book series The Coven. In these sections, two different characters discuss the main character, Grace, who created a heliocentric model of the solar system and was subsequently reprimanded by a church authority.


The first scene occurs when Grace attempts to join a secret society of scientists:


“I believe I’ve worked out your character, Lady. You discovered an interesting puzzle and happily solved it, like any intelligent child would. Then, like a child, you ran off to tell someone, expecting praise. When you informed a learned cleric about your discovery, and he told you it was an error, you assumed he was right and you were wrong- even though you’d seen the evidence with your own eyes. Am I correct?”


I could feel my face burn with shame. “Yes, you are correct.”

Sir Silas continued. “You’re  not kind to your inferiors because you value them as equals, but because you don’t have the confidence to acknowledge your own worth. You may be sweet, and you may possess a unique genius, but you lack the strength to fight the forces of oppression.”

The next scene takes place during a later conversation about the same event:


“… In your presentation, you made it clear that you built your theory, and then tested your theory against Sir Boromir’s observations to see if everything fit your model . You made some mistakes along the way, but you arrived at the correct conclusion. If I’d judged you on your work alone, you would be a member of the guild today.”


“Then why-”

Sister Jubilee held up her hand. “Please allow me to continue. Sir Silas objected to your initiation because he felt you lacked courage, but my reason was quite the opposite. You are naturally timid, but I believe the caution this afforded you was the reason you were so methodical in your approach to science. Unfortunately, your timidity also denied you the chance to learn what the bold learn early in life- the necessity of resisting strong impulses.

Both Sir Silas and Sister Jubilee were correct in their own way. You must have humility to test alternatives to your own hypotheses and to learn from the work others have done- it’s a way to correct innate biases. Unfortunately, if you aren’t willing to question established authority, then you can never hope to do any better.

Discovering when it is appropriate to question established authority has always been a delicate matter. Yudkowsky presents a way to discern where inefficiencies in established systems exist, so we can do just that.




When I was in Junior High, I was  anti-empiricism. I was very fond of the ancient Greek philosophers, having recently discovered Plato, and I believed that anyone could arrive at the correct answer to any question if they used the correct system of reasoning. If someone ever arrived at an incorrect conclusion, the issue was never incomplete observational evidence; the issue was flawed logic.

For example, in the seventh grade I was forced to partake in the coming-of-age ritual we know as frog dissection. I was saddened by the prospect of cutting open an innocent animal, and, like many others, wondered why it was necessary. I understood that our teachers wanted us to have a hands-on experience with anatomy, but I thought a model might be just as effective as an actual frog.

As it turns out, I wasn’t able to examine the frog’s anatomy very closely, because my lab partners decided it was far more amusing to throw frog guts in my hair than do the assignment.  Instead, I waited until my lab partners grew bored of flinging frog guts, copied the frog-anatomy diagram from the textbook onto my lab sheet, and got an A on the assignment, anyway.

This incident illustrated two points to me: 1. Not only is frog dissection inhumane to the frogs, but Junior High is inhumane to children. 2. Students don’t dissect frogs so that they will discover something new, but to prove that they can find what the book tells them. The textbook is the ultimate authority.

I wasn’t surprised that Junior High was just as inhumane as Elementary School had been, and I wasn’t surprised that the science textbook was supposed to be the ultimate arbiter of scientific knowledge, above observation. In my mind, grownups already knew everything, and all that was left for a student to do was to put the information together in the proper way. When  grownups seemingly made mistakes, like allowing a small girl to be routinely bullied by her classmates, it was for mysterious reasons beyond my ken, such as “tough love.”

If grownups already know everything, why bother with empiricism? Just build models based on what they already know, and hope you will gain wisdom about things like “tough love” in time.

The entire frog exercise seemed like a utilitarian 0. However, the assignment was given by the authorities, so in the end I decided that I must have had messed up the assignment in some way- nevermind that the authorities gave me an A. Some small part of my mind questioned the necessity of using real frogs, but I didn’t question any other aspect of the situation.





I told the frog story to illustrate how deeply modest epistemology can be ingrained. My thinking has matured a great deal since I was in the seventh grade. I understand now that there are still gaps in humanity’s knowledge, and that empiricism is a vital tool in making discoveries and testing the models we build. However, my habit of deferring to authority is so deeply ingrained that I have a tough time believing that I can do anything useful.

My deference to authority may have started out as an innate sense of my place in the social order, which Yudkowsky discusses in the chapter Status Regulation and Anxious Underconfidence. Or I may have been trained to be modest by the school system- the function of which makes a lot more sense after reading the chapter Moloch’s Toolbox. Heck, a lot of my anxious underconfidence may be related to a lifetime of frog guts in my hair.

Regardless, Yudkowsky’s work has left me with a task- to overcome my habitual deference, so I can focus on analyzing when I can hope to do better than a system that allows bullying, dead babies, and a billion or so other daily tragedies. His final chapters give a rough set of ideas one can use to accomplish this, and a warning to those who would become overconfident instead of underconfident.

Yudkowsky’s best advice on how to calibrate one’s ability to predict when you can do better is, sadly, advice I cannot use: “Bet on everything where you can or will find out the answer.” One of my most annoying habits is apologizing to my friends for being right in an argument, and winning any zero-sum game makes me feel even worse than losing. I’ve gotten very good at knocking myself down the social ladder so that others won’t.

Instead, I am going to have to be methodical in my approach, to chip my way around my bad mental habits and to find ways to reward myself that don’t make me feel guilty. Most importantly, I need to find the strength within myself to look up at the giants around me and say, “you might be wrong,” and to do this without experiencing a “slapdown” that will discourage further growth. 

To this end, I think it will be useful to look at the scenario as though I am a third-party observer and ask the following questions.

  1. Have I diligently researched the claim I’m questioning, as well as the system that produced it? This includes performing any feasable real-world test.
  2. Do I assume I can’t do better because I actually can’t, or because I fear the social consequences of my presumption?
  3. Is there a way I can improve things for myself or others and still avoid adverse social consequences?
  4. If not, are the potential rewards worth the social risk?

(As someone who flinches away from betting, I assume #4 is very similar to questions all betters ask.)

I have the mental tools I need to verify what works; I will simply need to make periodic checks to make sure I’m using them. My mental process is so automatic that I rarely notice it, so I will have to make a mantra to remind myself that it’s ok to win. Building new habits is difficult.

I see this book as a call to action- not just to me, but to the rationality community. You have tools- now use them to start winning. I have been observing the online rationality community for some time, and I see some winning already occurring, but I wonder how many observers there are out there like me- timid armchair enthusiasts. If we all get out of our armchairs, what will happen?