I thoroughly believed in a fundamentalist Christianity, as far as _any_ outside observer could tell. Now I'm me. My assessments of people and their brains should not make the mistake of believing that lunatic religionists like past-me are necessarily lunatics in any fundamental sense.
In software there's a notion of test-driven development, where the way to add new features, fix bugs, or otherwise change code is to first find a unit test that the codebase currently fails and only then do the update. The failing test then serves as a kind of driver for progress. (I have mixed feelings about TDD, but here it's a metaphor.)
It's easy to list unit tests you pass. I'm curious to hear unit tests you are currently failing (but which are still valid).
I'll start: my epistemics predict way more theft, other minor crimes, and mayhem than we see in practice. Like, it's not that hard, I claim, to learn to pick locks and figure out who is home at 3am. There are a variety of updates I could make to explain it, but they'd Explain Too Much, and cause other unit tests around lying, cooperation, and incentive gradients to fail. Like, I currently pass the "almost all of the freight train cars older than 5 years have graffiti" unit test!
I find it very interesting that you include two moral issues (slavery - presumably plantation chattel slavery specifically but maybe all but then maybe serfdom and more extreme patriarchy too - as well as more specific Nazis) in a list that's MOSTLY factual ("truth apt"). Does it mean you're a full-on moral realist or are they there as kinda "obvious in hindsight to contemporary morality" outliers? I'd definitely add Leninism/ Stalinism/ Maoism to the list in a specific form of: what test would ensure you reject those ideologies if you came out of the carnage of WW2 as an educated person who did not want to completely give up on humanity.
More as "obvious in hindsight to contemporary morality," with a dash of "in fact obvious at the time, to a significant minority."
I'm sort of a soft moral realist in that it feels to me like morality is clearly not ENTIRELY subjective/made-up/malleable, although it doesn't feel straightforwardly objective, either.
I really liked this post, and I'm excited to see open comments for unpaid subscribers!
Here's one that I think about pretty often, and I'm going to describe it in a roundabout way to try to circumvent some mental defenses:
Do you think it's wrong to kick a dog for no reason? (if not you can stop reading here, but I would view that as a failed unit test). What if it made the kicker feel some kind of positive-valence feeling, like joy or a thrill or even some kind of sexual excitement? Again, if that suddenly changes it from wrong to ok for you, you can stop reading, and I would view that as a failed unit test. What if the "kick" is living a life in a cage with almost no movement, being killed young, and the positive-valence feeling is yummy-ness? It's not a fundamentally different unit test than the previous one, in fact the harm is greater for not much more positive-valence, but it's much more controversial, some people will find cultures that do it to dogs abhorrent, while others won't be too bothered by it and may even want to partake out of curiosity (again, I think that fails the unit test). But what if we tweak it one more time, and now instead of a charismatic animal like a dog, it's a chicken or a pig or a cow? They are just as capable of feeling the suffering as the dog is. The suffering is worse than the kick. The positive-valence of "yummy" is not that much comparatively. For almost everyone reading this, there is no survival necessity aspect. But I think a good moral framework would say that all of those scenarios are wrong, and one that suddenly changes partway through is as flawed as an adder that get 1+1 correct but fails for 100000000+1000. Some people have the view of "eating animal products is wrong, but I'm going to do it anyways", and while I wish they wouldn't, I'm glad they're at least able to clearly view the ethics. Better to pass the unit test and ignore the result than to twist your algorithm to justify your actions. (Edit: I should note that I also do things I think are wrong. For example, I buy things that were almost certainly manufactured with slave labor somewhere along the supply chain. I don't think it's great, and I'd rather not support it, but I don't feel like putting in the effort to research all the things and find alternatives. But I'd rather recognize that as wrong and still do it than make excuses for why it's morally fine to do)
I came here to point out veganism as another example that (hopefully) will be as obvious as slavery and nazism in 50-100 years, but you already did a much better job at it. I feel kind of embarrassed at how long it took me from first encountering veganism and seriously considering it to actually going fully vegan recently.
Thanks, yeah I hope it'll be as obvious as those one day too. I wanted to tie it in with those, maybe saying something like "if you think you'd oppose slavery/nazis/etc then, what's your view on animal rights now?" but I think that might rustle too many jimmies and make the point easier to miss
"and I'm going to describe it in a roundabout way to try to circumvent some mental defenses"
failed my unit test (for manipulativness)
"What if the "kick" is living a life in a cage with almost no movement, being killed young, and the positive-valence feeling is yummy-ness?"
failed unit test, for stawmanning
I actually agree with avoiding eating tortured animals as legitimate unit test, but sadly, almost everyone in this club fail their unit tests constantly.
(also, i see very big different between predictions and ethics, and think this element of the unit tests framework is encouraging equivocating the two. failing to recognize Einstein and having different morals are VERY different things.)
Inferential distance is a real thing, and I don't think it's manipulative in a bad way to take it into account when communicating. I think my conclusion is something most people would write off in part because they're missing the steps in logic I take, so I wanted to spell it out and start from a point that most people agree on, instead of just stating "someone who hurts a dog for fun isn't meaningfully morally different from someone who eats a rotissery chicken when they could've gotten the same nutrition a couple aisles over at the grocery store".
I'm curious what you mean about strawmanning? As far as I know, most farm animals are killed within a year or two of being born, way sooner than they would die without being killed. Not all farm animals are kept in tiny cages, but it's common enough that I think most people eating animals contribute to it.
But I agree with that post you linked that vegan advocacy has issues with truthseeking (I think the issue even extends to other movements based around moralizing things, but that's a whole other discussion, and like the author says, EAs have volunteered for a higher standard anyways). And I also tend not to add reminds in my comments advocating for veganism to take b12, maybe iron, maybe other nutrients because I just don't really think about it, but it is a good idea to do.
There is A HUGE DIFFERENCE between crossing Inferential Distance and "circumvent some mental defenses". one is cooperative, the other manipulative.
mental defenses exist for REASONS. you can face them down honestly, and try to convince the person to allow you throw, that the defenses are not against you. you can try to understand the defenses and check - maybe the person is actually right?
or
you can try to sneak around the defenses. and this is dishonest and manipulative.
****
someone who hurts a dog for fun IS meaningfully morally different from someone who eats a rotissery chicken.
one of the reasons that people have those defenses is because there are a lot of people like you, that try one of the reasons that people have those defenses is because there are a lot of people like you, that try to take situations that different from one another, and use Hypothetical where they agree with you, to trick them and make them feel they have to agree with you, if they are not eloquent enough to point exactly where is the difference. this is a trap and people rightfully have protections against it (1)
if someone DISAGREE with you about what are the morally important elements of the situation, the thing to do is to find where the disagreement come from. and argue about that directly.
for example, Scott Alexander wrote about Newtonian Ethics, and other blogger answer that yes, you have more responsibility to situations you put yourself in.
or you can try to use some slight of hand to make the person feel bad because they can't articulate what in the situation is different:
"this parable was setting up two pieces of yourself at odds, so that you could not be both at once, and arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense." (2)
***
Policy Debates Should Not Appear One-Sided! you strawmanning by ignoring the other side of the trade off. i agree about all the factory-farming-is-torture bits, but i also look on the prices of avoiding it, why you try very hard to deny it. one cannot " gotten the same nutrition a couple aisles over at the grocery store", people not ear meat only for the yummy-ness, and i actually link to post that give examples to the health prices of veganism.
there is not human soceity that had been vehan for generations and proved it can be done without paying health-price.
the honest, not manipulative analogy would have been some sunnydale vampire that can live on animal blood but are weaker, and strongly prefer human blood. it's not perfect analogy, but it's not as manipulatively strawmanning, not set up to ignore some important needs, and try to cast them as frivolity and person how care about them as a monster.
the way i see vegan advocates again and again slide to the most convenient world, to fail to the temptation to ignore the other side of the trade off, to make it appear one-sided, is really worrying!
and i believe it's also weaken the case. see, when someone lie, it's Bayesian evidence that they need to, that the truth is not enough. and same with pretending debate is one-sided.
If I was trying to sneak I would not broadcast my intentions. And what happened to truth-seeking? Mental defenses can be useful in many ways, and they can also block us from seeing the truth (another word that I could've used is "bias").
"someone who hurts a dog for fun IS meaningfully morally different from someone who eats a rotissery chicken"
I guess it depends on how consequentialist you are and how much you weight intentions.
Point taken though on how using a bunch of small arguments to get to a much bigger conclusion can be risky, I get that is a real thing. I just don't think it applies here.
"this parable was setting up two pieces of yourself at odds, so that you could not be both at once, and arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense."
This is a true downside of how I presented my case, and also sometimes one side of ourselves is just wrong. But I think it is important to hear those sides out and integrate them rather than squash and silence them, they may actually be correct. Internal double crux is one good method for it.
idk it's one of those things where in my experience it really wasn't that hard of a switch, and that's speaking as someone who grew up on meat, kansas city bbq was my favorite and I still am always on the lookout for good meat replacements cause I like the flavor so much (there are some that get close but the yummy flavors are a loss). I left religion, and decided to think deeply about ethics rather than my previous strategy of "whatever the bible says is right", and in the process realized that eating animals is wrong. I did some research on whether we need to eat them to be healthy, we don't, so I stopped eating them. Then I learned more about other animal ag practices, and learned fish feel pain, and went fully vegan, and haven't had any health issues in the ~7 years I've been. Obviously that's just an anecdote, and just cause I found it easy doesn't mean others will too (the large number of ex-vegans demonstrates that), but from my perspective it's really not that big of a switch in a world with grocery stores and vegan restaurants. But as you said, I don't think there are any multi-generational vegan societies, so it is still somewhat untested territory.
Any major diet change should be done with at least some research into how to do it healthily (most people should probably research how to do their current diet more healthily), but "vegan" is such a wide range of diets that it's not really clarifying on how healthy it is. A vegan who eats nothing but oreos and water is gonna be terribly unhealthy, as is a nonvegan who eats nothing but butter and water. The standard american diet is a real danger of nonvegan diets, and eating only veggies and not getting in enough calories is a real danger of vegan diets. The spectrum of unhealthy<->healthy diets has vegan and nonvegan diets at every point along it.
you know, i start writing answer and find myself just rewriting my previous comments. EA Vegan Advocacy answer about the diet, and gibe examples of bad things that happen in the real world to real people. and i actually disagree that "The spectrum of unhealthy<->healthy diets has vegan and non-vegan diets at every point along it." (also, this statement contradict you own that "o it is still somewhat untested territory", as a lot of non-vegan diets are not in untested territory)
but it's not the discussion I'm trying to have. I'm trying to point that saying that people are not vegan because of the yummy factor is strawmanning.
that you shouldn't assume in your hypothetical that the only important thing is consequences, when you argue with non-consequentialist.
(i also disagree about the "one side is wrong". The Utility Function Is The Utility Function, but that's not the point, and maybe if i try harder to stay on point i will not get responses that make me want to just... link again the posts that contain the counter arguments to the arguments that you write, as if you didn't notice the counter-arguments)
Just popping in to say "hey, is there a Thing that either of you wants from the other person?" There's been some productive stuff on both sides of this argument but I'm not sure whether either of you is here "on purpose," with a goal. It might be a good point to pause and let your various points stand, unless somebody wants some specific flavor of More.
At some point I picked up "gay people in 1950s America" as a unit test (without that terminology, possibly from Kelsey Piper) and have been carrying it around with me. Somewhat embarrassingly, I don't actually know all that much about how gay people were treated in 1950s America; "it was bad" seems true but not super helpful.
The relevant unit test there seems to me to be something like:
"Does your decisionmaking algorithm, ported back to run on 1950s beliefs, cause you to conclude that you should forcibly out and ostracize a gay man, destroying his life and his career, possibly out of a desire to do something like 'protect the children'?"
Like, "gay people in 1950s America" are my central go-to example of benign social dark matter; the ethics of the time insist that you punish them (and that you might be punished for being a non-punisher) but the moral answer is clearly different from the ethical one.
I think "societal treatment of 'gay' as a concept" ends up making like a dozen high-level tests. Stuff like "Will gay marriage legalization lead to a rapid upswing in self-reporting-as-gay?" (similar to left handnedness, but a lot of people said conservatives claiming this were close-minded bigots), "Will switching policy to say gay is okay lead to further popular sexual exploration?" (similar to previous question but some details differ heavily), and then some that are trickier and thornier and I don't want to start arguments about them here
Some of your proposed unit tests are really, _really_ hard to find algorithms that can pass.
The classic "Albert" example in mathematics is the self-taught mathematician Ramanujan.
Practically every crackpot who comes along claiming to have proofs of the Collatz Conjecture or that P/=NP at some point makes the comparison, "You should look at my work because what if I'm the next Ramanujan? Imagine if he had gone undiscovered because no one paid attention to him".
What's astonishing about David Budden is that he legitimately believes he's going to be able to produce a Lean proof to back up his claims. That is, he's giving us an easy, robust, universally-agreed-upon, algorithm we'll be able to run to answer the Albert question for him with no ambiguity.
AFAIK it's the first time a likely-crackpot has done this.
Very similar to close-minded atheism, maybe just the same but expanded, there's quite a lot of useful, practical advice hanging around in areas with terrrrrible epistemics, like Almost Every Self-Help Book Ever. And often those team-building exercises that start off with horribly false assumptions are legitimately good at helping to gel your team, if only everyone could put aside their Absolutely Correct Cynicism and just do the practice and see what happens. Unit test is, like close-minded atheism, "does your useful-advice-and-behavior-and-framing-detector false negative on all the useful advice and behavior and framing totally extractable from that cringe team-building exercise imposed on your team based on a popular self-help book you hadn't previously heard of which begins by saying There Are Five Comprehensive And Disjoint Ways To Relate To Another Person or whatever?" Because if you're short-circuiting _because you can't be arsed_, that's fine, but if you're short-circuiting because you're dismissing all the practical suggestions as clearly bad because the proposed underlying framework is clearly terrible, nah that algorithm is sus.
This put a name and details to a concept I've brushed upon a few times, specifically around trying to have empathy for people in past eras who didn't drop everything and fight against what seem now to be obvious wrongs (and thinking about if there might be similar things going on right now).
I really like the metaphor of your thought processes being like software. I think a lot of people not only don't run unit tests on their thinking - I think many people do the equivalent of noticing that the output was wrong, and simply hardcoding in the "correct" output.
In terms of other unit tests, here's what I came up with:
1. Nutritional/diet decisions: How do you choose what to eat and what supplements to take? Would that same process have been robust during the times when refined grains were considered healthier than whole grains, when eggs were considered unhealthy, when dietary cholesterol was believed to directly impact blood cholesterol, etc.)?
2. Various health related choices: When you choose to do or avoid certain activities because of their health impacts, how would your process have faired when cigarettes were being marketed as a healthy method of weight loss, or when mercury/arsenic/heroin/cocaine were thought to be good treatments for various ailments, or when bloodletting or leeches were the treatment du jour?
3. For religious people: Why do you believe in your religion, and have you tested that process against all other religions? I've never been close enough to someone who was deeply religious for it to be worth discussing this, but it's always surprised me that if people really believe in a given religion then it seems to me like it would be vitally important to know that you were actually following the *right* religion...but I've never encountered someone who had done a full "literature review" of sorts of all the major religions in order to be sure they were following the right commands and performing the right rituals.
4. Parenting choices: For parents, how have you chosen to parent your child(ren)? Would that process applied in previous eras or locations resulted in you enacting things like "spare the rod, spoil the child", circumcision or more severe forms of genital mutilation, restricting choices or access to experiences and opportunities based on the child's gender, etc.?
Here's one: If the social norms you propose were widely followed, would this prevent a non-trivial number of people from fulfilling desires that are deeply important to them? Bad if so.
Many (not all) forms of wokeness/feminism fail this test, in that they try so hard to prevent unwanted sexual/romantic advances from happening that would effectively prevent *all* sexual/romantic advances from happening if people actually followed the norms they propose. Being anti-gay also fails this test in a more straightforward way.
I don't think this test is sufficient. Usually norms will have some negative effect, so you need to build weighing the costs and benefits into your question
Fair, I agree that trying to create norms that have literally *no* negative effects is unrealistic. However, I was focusing on the narrower set of cases where a set of norms effectively dooms some people to suffer persistently, in ways that can only be avoided by people violating the norms. It seems to me that avoiding that particular class of negative effect is generally both possible and desirable, even though avoiding *all* negative effects is usually impossible.
Though I suppose I wouldn't be super religious about adhering to the unit test I propose. I think it holds up well in most realistic situations, but it's probably possible to come up with contrived unrealistic scenarios in which I would endorse abandoning it.
here is a unit test for your unit test - pedophiles.
I fine with preventing a non-trivial number of people from fulfilling desires that are deeply important to them, actually.
it sometimes justified and sometimes no, and i don't have easy enough test to encode in unit test, just to the cost-effectiveness analyze every time, and hope.
Yeah in hindsight, describing my idea as a "unit test" was probably too strict - I definitely still think it's a good rule of thumb, but not something you should literally never go against. I notice, looking back at the OP, that Duncan's examples are more on the narrower side than mine ("does your decision procedure get the right answer in this one case?" rather than "does your decision procedure give answers that have this property across this broad class of cases?") which does seem more in line with the concept of a unit test as it's used in computer programming.
(FWIW, if there were a way for pedophiles to fulfill their desires *without* harming real people (e.g. experience machines) I'd be in favor of them doing so, but in the absence of such a way, preventing them from fulfilling their desires is indeed the right call)
There's a big distinction to be made here between statistical groups and individual people. I have met people who slept with adults, as children, and are not damaged and were not harmed; I have also met people who were molested and abused and raped as children and carry the scars forever.
(This is meant to gesture at "if there were a way." There clearly isn't, at the population level; we see the damage everywhere we look. But I don't want to confuse "when X happens, Y is often the result" for "when X happens, Y is inevitable 100% of the time." See Social Dark Matter for more, and uh, please no one murder me for saying a true thing.)
He's an ex-OpenAI guy and he claims with AI assistance he's solved two Millennium Prize problems (Navier-Stokes and Hodge Conjecture). He's made big public bets (tens of thousands of dollars) that he'll have Lean proofs by the end of January.
I liked this essay! I found many of the unit tests interesting and productive, except for the Einstein one. I’ve kind of tried to do these meta science style unit tests in the past (related is Nielsens discovery fiction process), and I’ve always found them more useful as tools for reunderstanding a concept than actually checking your reasoning process, since pretending to reason as if you didn’t know a scientific concept is really hard for me. I do like the general concept and pointer though.
If your processes, applied to post-9/11 America, result in invasions of Iraq and Afghanistan and the assembly of a vast surveillance apparatus and hundreds of millions of people engaging in airport security theater that burns countless time and money, you have made an avoidable mistake."
I still think the US had to attack Afghanistan after 9/11, but I don't necessarily have a better plan for the aftermath, even in hindsight.
Iraq was a mistake, though. There really wasn't much justification for it that held up other than "Saddam Hussein is bad and we can kick him out", which was true enough, but it was going to be damn obvious that keeping Iraq from falling apart afterwards was going to be a lot harder and costlier than the initial invasion and the US wouldn't benefit from having done it.
Airport security theater was also stupid, but I don't know if there was a "not stupid" solution that would have satisfied the kind of people that demand witch hunts.
> a vast surveillance apparatus and hundreds of millions of people engaging in airport security theater that burns countless time and money, you have made an avoidable mistake.
Just chiming in here that I've done this math, and the TSA has wasted 10x the amount of American lifetimes 911 took, at ~35k lifetimes gone right now, all while having a 95% failure rate when red-teamed, and we continue to waste ~800M hours of American time every year for it.
And this is entirely aside from spying on our own citizens and everyone in the world in every channel in perpetuity (Patriot Act, NSA PRISM, Five Eyes, etc) and an ~$8T 20 year boondoggle in the middle east!
Now I feel guilty about not actually contributing to the thread, so I'll suggest my own unit test:
1. Can this suggested advice / intervention / behavior be understood and followed by an average person?
I feel like many people like us in the thread like to think and pontificate about society and politics and culture and economics, and feel that there are blindingly obvious and "easy" solutions laying about everywhere that could solve any one of our serious societal problems.
But the average American has a 7-8th grade reading level. 98% of people fail at weight loss. The great majority, ~80%+, are sedentary and eat garbage, even though that is literally 2-3x as bad as smoking or a cocaine addiction when it comes to all cause mortality. The great majority of people don't have a college degree (70%+), and couldn't actually get one.
Not just that, but essentially everyone, up to and including important executives and politicians and various looked-up-to figures, has the attention span of a gnat, and unless your points are extremely simple to understand, *and* simply phrased, AND are limited to only 1 or 2 points total, 3 at the very most, they will be glossed over and ignored by essentially everybody you are trying to communicate to.
It's easy for people with grad degrees and nice careers in Tier 1 cities to spot fairly obvious solutions to various problems - but they'd require an entirely different humanity and populace than the one we have. So the solutions are NOT obvious, if they are not workable, and we should all do well to consider that.
This is wrong in an interesting way. The unit test framing is fine, but it vastly, vastly understates the hard parts of specifying the underlying general standards that separate “sounds crazy but true” from “crazy”.
For example, what unites pro-Einstein and anti-Nazi conclusions? Finally! Two data points on the Truth! (It's an exercise for the reader how a naive line through those points in the twentieth century end up going wrong in the twenty-first.)
Maybe it's that political or moral realism is a spherical cow - especially historical political-moral realism. The spherical cow historical cases are as useful in testing as any other spherical cow, but the vast, vast majority of the 'programming' is in building a real world, present working model.
"This is wrong in an interesting way" followed by ... nothing that makes it seem like the OP is wrong. =P
It sounds like you meant "this piece did not exhaustively address all of the hard parts of the problem, but was instead narrowly targeted at a single part in the machinery of sanity and progress," which ... I agree with?
(The complexity of the truth being part of why it pays to *collect* unit tests, plural, rather than assuming you can get good answers with just one or two. Building a machine that correctly passes fifty different important unit tests *is* a way to start approximating a real-world, present working model.)
Fair point on my phrasing. I don’t mean “your conclusion-set is wrong”, I mean the unit test metaphor breaks if readers treat your listed (or solicited) answers as the tests.
I think “our best available judgment plus hindsight” is the wrong oracle to consult, because hindsight is exactly the thing that turns messy cases into clean morality plays and makes us overconfident that we would have seen it. That said, I get that you’re leaning on that stool leg pragmatically, because we still need some kind of expected output to run regression tests at all.
But the key failure mode is on the reader side. Without the underlying mechanism and the original uncertainty, the list becomes a set of imported outputs rather than tests of the machinery.
In code, a unit test works because the oracle is crisp and shared. 1+1=2 has a whiff of universality to it to which we are committing so as not to be seen as insane. With historical moral and political cases, the “expected output” is doing a lot of hidden work: values, counterfactual imagination, a ton of context about what made the mistake tempting at the time, and what made it a mistake after the fact. I would hope to suggest that you have all that rich context in your head, so the cases function as genuine regression tests on your own cognition.
For a reader though, the same list risks becoming a checklist of socially legible correct answers, which is exactly the failure mode you’re warning against.
“Don’t rely solely on consensus” is a unit test. “Nazis bad” is mostly a shibboleth unless you very carefully unpack the generalizing mechanism that you’re using for the experience you’ve had that resulted in that personal test.
"For example, what unites pro-Einstein and anti-Nazi conclusions? Finally! Two data points on the Truth! (It's an exercise for the reader how a naive line through those points in the twentieth century end up going wrong in the twenty-first.)"
Care to explain what this means?
It reads to me like two straw men. One made up of a claim that OP stated that pro-Einstein and anti-Nazi were somehow hard to find and revealing some type of important truth on their own - something that I did not get from the article. The other made up of stating that readers would draw a naive line between the two and trying to apply it on its own in current affairs, which for me at least fails on both steps.
Assume you might have meant something completely different.
I thoroughly believed in a fundamentalist Christianity, as far as _any_ outside observer could tell. Now I'm me. My assessments of people and their brains should not make the mistake of believing that lunatic religionists like past-me are necessarily lunatics in any fundamental sense.
In software there's a notion of test-driven development, where the way to add new features, fix bugs, or otherwise change code is to first find a unit test that the codebase currently fails and only then do the update. The failing test then serves as a kind of driver for progress. (I have mixed feelings about TDD, but here it's a metaphor.)
It's easy to list unit tests you pass. I'm curious to hear unit tests you are currently failing (but which are still valid).
I'll start: my epistemics predict way more theft, other minor crimes, and mayhem than we see in practice. Like, it's not that hard, I claim, to learn to pick locks and figure out who is home at 3am. There are a variety of updates I could make to explain it, but they'd Explain Too Much, and cause other unit tests around lying, cooperation, and incentive gradients to fail. Like, I currently pass the "almost all of the freight train cars older than 5 years have graffiti" unit test!
I find it very interesting that you include two moral issues (slavery - presumably plantation chattel slavery specifically but maybe all but then maybe serfdom and more extreme patriarchy too - as well as more specific Nazis) in a list that's MOSTLY factual ("truth apt"). Does it mean you're a full-on moral realist or are they there as kinda "obvious in hindsight to contemporary morality" outliers? I'd definitely add Leninism/ Stalinism/ Maoism to the list in a specific form of: what test would ensure you reject those ideologies if you came out of the carnage of WW2 as an educated person who did not want to completely give up on humanity.
More as "obvious in hindsight to contemporary morality," with a dash of "in fact obvious at the time, to a significant minority."
I'm sort of a soft moral realist in that it feels to me like morality is clearly not ENTIRELY subjective/made-up/malleable, although it doesn't feel straightforwardly objective, either.
I really liked this post, and I'm excited to see open comments for unpaid subscribers!
Here's one that I think about pretty often, and I'm going to describe it in a roundabout way to try to circumvent some mental defenses:
Do you think it's wrong to kick a dog for no reason? (if not you can stop reading here, but I would view that as a failed unit test). What if it made the kicker feel some kind of positive-valence feeling, like joy or a thrill or even some kind of sexual excitement? Again, if that suddenly changes it from wrong to ok for you, you can stop reading, and I would view that as a failed unit test. What if the "kick" is living a life in a cage with almost no movement, being killed young, and the positive-valence feeling is yummy-ness? It's not a fundamentally different unit test than the previous one, in fact the harm is greater for not much more positive-valence, but it's much more controversial, some people will find cultures that do it to dogs abhorrent, while others won't be too bothered by it and may even want to partake out of curiosity (again, I think that fails the unit test). But what if we tweak it one more time, and now instead of a charismatic animal like a dog, it's a chicken or a pig or a cow? They are just as capable of feeling the suffering as the dog is. The suffering is worse than the kick. The positive-valence of "yummy" is not that much comparatively. For almost everyone reading this, there is no survival necessity aspect. But I think a good moral framework would say that all of those scenarios are wrong, and one that suddenly changes partway through is as flawed as an adder that get 1+1 correct but fails for 100000000+1000. Some people have the view of "eating animal products is wrong, but I'm going to do it anyways", and while I wish they wouldn't, I'm glad they're at least able to clearly view the ethics. Better to pass the unit test and ignore the result than to twist your algorithm to justify your actions. (Edit: I should note that I also do things I think are wrong. For example, I buy things that were almost certainly manufactured with slave labor somewhere along the supply chain. I don't think it's great, and I'd rather not support it, but I don't feel like putting in the effort to research all the things and find alternatives. But I'd rather recognize that as wrong and still do it than make excuses for why it's morally fine to do)
I came here to point out veganism as another example that (hopefully) will be as obvious as slavery and nazism in 50-100 years, but you already did a much better job at it. I feel kind of embarrassed at how long it took me from first encountering veganism and seriously considering it to actually going fully vegan recently.
Thanks, yeah I hope it'll be as obvious as those one day too. I wanted to tie it in with those, maybe saying something like "if you think you'd oppose slavery/nazis/etc then, what's your view on animal rights now?" but I think that might rustle too many jimmies and make the point easier to miss
"and I'm going to describe it in a roundabout way to try to circumvent some mental defenses"
failed my unit test (for manipulativness)
"What if the "kick" is living a life in a cage with almost no movement, being killed young, and the positive-valence feeling is yummy-ness?"
failed unit test, for stawmanning
I actually agree with avoiding eating tortured animals as legitimate unit test, but sadly, almost everyone in this club fail their unit tests constantly.
aka:
https://www.lesswrong.com/posts/aW288uWABwTruBmgF/ea-vegan-advocacy-is-not-truthseeking-and-it-s-everyone-s-1?view=postCommentsOld&postId=aW288uWABwTruBmgF
(also, i see very big different between predictions and ethics, and think this element of the unit tests framework is encouraging equivocating the two. failing to recognize Einstein and having different morals are VERY different things.)
Inferential distance is a real thing, and I don't think it's manipulative in a bad way to take it into account when communicating. I think my conclusion is something most people would write off in part because they're missing the steps in logic I take, so I wanted to spell it out and start from a point that most people agree on, instead of just stating "someone who hurts a dog for fun isn't meaningfully morally different from someone who eats a rotissery chicken when they could've gotten the same nutrition a couple aisles over at the grocery store".
I'm curious what you mean about strawmanning? As far as I know, most farm animals are killed within a year or two of being born, way sooner than they would die without being killed. Not all farm animals are kept in tiny cages, but it's common enough that I think most people eating animals contribute to it.
But I agree with that post you linked that vegan advocacy has issues with truthseeking (I think the issue even extends to other movements based around moralizing things, but that's a whole other discussion, and like the author says, EAs have volunteered for a higher standard anyways). And I also tend not to add reminds in my comments advocating for veganism to take b12, maybe iron, maybe other nutrients because I just don't really think about it, but it is a good idea to do.
There is A HUGE DIFFERENCE between crossing Inferential Distance and "circumvent some mental defenses". one is cooperative, the other manipulative.
mental defenses exist for REASONS. you can face them down honestly, and try to convince the person to allow you throw, that the defenses are not against you. you can try to understand the defenses and check - maybe the person is actually right?
or
you can try to sneak around the defenses. and this is dishonest and manipulative.
****
someone who hurts a dog for fun IS meaningfully morally different from someone who eats a rotissery chicken.
one of the reasons that people have those defenses is because there are a lot of people like you, that try one of the reasons that people have those defenses is because there are a lot of people like you, that try to take situations that different from one another, and use Hypothetical where they agree with you, to trick them and make them feel they have to agree with you, if they are not eloquent enough to point exactly where is the difference. this is a trap and people rightfully have protections against it (1)
if someone DISAGREE with you about what are the morally important elements of the situation, the thing to do is to find where the disagreement come from. and argue about that directly.
for example, Scott Alexander wrote about Newtonian Ethics, and other blogger answer that yes, you have more responsibility to situations you put yourself in.
or you can try to use some slight of hand to make the person feel bad because they can't articulate what in the situation is different:
"this parable was setting up two pieces of yourself at odds, so that you could not be both at once, and arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense." (2)
***
Policy Debates Should Not Appear One-Sided! you strawmanning by ignoring the other side of the trade off. i agree about all the factory-farming-is-torture bits, but i also look on the prices of avoiding it, why you try very hard to deny it. one cannot " gotten the same nutrition a couple aisles over at the grocery store", people not ear meat only for the yummy-ness, and i actually link to post that give examples to the health prices of veganism.
there is not human soceity that had been vehan for generations and proved it can be done without paying health-price.
the honest, not manipulative analogy would have been some sunnydale vampire that can live on animal blood but are weaker, and strongly prefer human blood. it's not perfect analogy, but it's not as manipulatively strawmanning, not set up to ignore some important needs, and try to cast them as frivolity and person how care about them as a monster.
the way i see vegan advocates again and again slide to the most convenient world, to fail to the temptation to ignore the other side of the trade off, to make it appear one-sided, is really worrying!
and i believe it's also weaken the case. see, when someone lie, it's Bayesian evidence that they need to, that the truth is not enough. and same with pretending debate is one-sided.
(1) see the comments to the post https://www.kylestar.net/p/please-just-answer-the-goddamn-moral
(2) https://www.lesswrong.com/posts/cujpciCqNbawBihhQ/self-integrity-and-the-drowning-child?view=postCommentsOld&postId=cujpciCqNbawBihhQ
(3) https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-debates-should-not-appear-one-sided
If I was trying to sneak I would not broadcast my intentions. And what happened to truth-seeking? Mental defenses can be useful in many ways, and they can also block us from seeing the truth (another word that I could've used is "bias").
"someone who hurts a dog for fun IS meaningfully morally different from someone who eats a rotissery chicken"
I guess it depends on how consequentialist you are and how much you weight intentions.
Point taken though on how using a bunch of small arguments to get to a much bigger conclusion can be risky, I get that is a real thing. I just don't think it applies here.
"this parable was setting up two pieces of yourself at odds, so that you could not be both at once, and arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense."
This is a true downside of how I presented my case, and also sometimes one side of ourselves is just wrong. But I think it is important to hear those sides out and integrate them rather than squash and silence them, they may actually be correct. Internal double crux is one good method for it.
idk it's one of those things where in my experience it really wasn't that hard of a switch, and that's speaking as someone who grew up on meat, kansas city bbq was my favorite and I still am always on the lookout for good meat replacements cause I like the flavor so much (there are some that get close but the yummy flavors are a loss). I left religion, and decided to think deeply about ethics rather than my previous strategy of "whatever the bible says is right", and in the process realized that eating animals is wrong. I did some research on whether we need to eat them to be healthy, we don't, so I stopped eating them. Then I learned more about other animal ag practices, and learned fish feel pain, and went fully vegan, and haven't had any health issues in the ~7 years I've been. Obviously that's just an anecdote, and just cause I found it easy doesn't mean others will too (the large number of ex-vegans demonstrates that), but from my perspective it's really not that big of a switch in a world with grocery stores and vegan restaurants. But as you said, I don't think there are any multi-generational vegan societies, so it is still somewhat untested territory.
Any major diet change should be done with at least some research into how to do it healthily (most people should probably research how to do their current diet more healthily), but "vegan" is such a wide range of diets that it's not really clarifying on how healthy it is. A vegan who eats nothing but oreos and water is gonna be terribly unhealthy, as is a nonvegan who eats nothing but butter and water. The standard american diet is a real danger of nonvegan diets, and eating only veggies and not getting in enough calories is a real danger of vegan diets. The spectrum of unhealthy<->healthy diets has vegan and nonvegan diets at every point along it.
you know, i start writing answer and find myself just rewriting my previous comments. EA Vegan Advocacy answer about the diet, and gibe examples of bad things that happen in the real world to real people. and i actually disagree that "The spectrum of unhealthy<->healthy diets has vegan and non-vegan diets at every point along it." (also, this statement contradict you own that "o it is still somewhat untested territory", as a lot of non-vegan diets are not in untested territory)
but it's not the discussion I'm trying to have. I'm trying to point that saying that people are not vegan because of the yummy factor is strawmanning.
that you shouldn't assume in your hypothetical that the only important thing is consequences, when you argue with non-consequentialist.
(i also disagree about the "one side is wrong". The Utility Function Is The Utility Function, but that's not the point, and maybe if i try harder to stay on point i will not get responses that make me want to just... link again the posts that contain the counter arguments to the arguments that you write, as if you didn't notice the counter-arguments)
Just popping in to say "hey, is there a Thing that either of you wants from the other person?" There's been some productive stuff on both sides of this argument but I'm not sure whether either of you is here "on purpose," with a goal. It might be a good point to pause and let your various points stand, unless somebody wants some specific flavor of More.
At some point I picked up "gay people in 1950s America" as a unit test (without that terminology, possibly from Kelsey Piper) and have been carrying it around with me. Somewhat embarrassingly, I don't actually know all that much about how gay people were treated in 1950s America; "it was bad" seems true but not super helpful.
The relevant unit test there seems to me to be something like:
"Does your decisionmaking algorithm, ported back to run on 1950s beliefs, cause you to conclude that you should forcibly out and ostracize a gay man, destroying his life and his career, possibly out of a desire to do something like 'protect the children'?"
Like, "gay people in 1950s America" are my central go-to example of benign social dark matter; the ethics of the time insist that you punish them (and that you might be punished for being a non-punisher) but the moral answer is clearly different from the ethical one.
I think "societal treatment of 'gay' as a concept" ends up making like a dozen high-level tests. Stuff like "Will gay marriage legalization lead to a rapid upswing in self-reporting-as-gay?" (similar to left handnedness, but a lot of people said conservatives claiming this were close-minded bigots), "Will switching policy to say gay is okay lead to further popular sexual exploration?" (similar to previous question but some details differ heavily), and then some that are trickier and thornier and I don't want to start arguments about them here
Some of your proposed unit tests are really, _really_ hard to find algorithms that can pass.
The classic "Albert" example in mathematics is the self-taught mathematician Ramanujan.
Practically every crackpot who comes along claiming to have proofs of the Collatz Conjecture or that P/=NP at some point makes the comparison, "You should look at my work because what if I'm the next Ramanujan? Imagine if he had gone undiscovered because no one paid attention to him".
What's astonishing about David Budden is that he legitimately believes he's going to be able to produce a Lean proof to back up his claims. That is, he's giving us an easy, robust, universally-agreed-upon, algorithm we'll be able to run to answer the Albert question for him with no ambiguity.
AFAIK it's the first time a likely-crackpot has done this.
Very similar to close-minded atheism, maybe just the same but expanded, there's quite a lot of useful, practical advice hanging around in areas with terrrrrible epistemics, like Almost Every Self-Help Book Ever. And often those team-building exercises that start off with horribly false assumptions are legitimately good at helping to gel your team, if only everyone could put aside their Absolutely Correct Cynicism and just do the practice and see what happens. Unit test is, like close-minded atheism, "does your useful-advice-and-behavior-and-framing-detector false negative on all the useful advice and behavior and framing totally extractable from that cringe team-building exercise imposed on your team based on a popular self-help book you hadn't previously heard of which begins by saying There Are Five Comprehensive And Disjoint Ways To Relate To Another Person or whatever?" Because if you're short-circuiting _because you can't be arsed_, that's fine, but if you're short-circuiting because you're dismissing all the practical suggestions as clearly bad because the proposed underlying framework is clearly terrible, nah that algorithm is sus.
This put a name and details to a concept I've brushed upon a few times, specifically around trying to have empathy for people in past eras who didn't drop everything and fight against what seem now to be obvious wrongs (and thinking about if there might be similar things going on right now).
I really like the metaphor of your thought processes being like software. I think a lot of people not only don't run unit tests on their thinking - I think many people do the equivalent of noticing that the output was wrong, and simply hardcoding in the "correct" output.
In terms of other unit tests, here's what I came up with:
1. Nutritional/diet decisions: How do you choose what to eat and what supplements to take? Would that same process have been robust during the times when refined grains were considered healthier than whole grains, when eggs were considered unhealthy, when dietary cholesterol was believed to directly impact blood cholesterol, etc.)?
2. Various health related choices: When you choose to do or avoid certain activities because of their health impacts, how would your process have faired when cigarettes were being marketed as a healthy method of weight loss, or when mercury/arsenic/heroin/cocaine were thought to be good treatments for various ailments, or when bloodletting or leeches were the treatment du jour?
3. For religious people: Why do you believe in your religion, and have you tested that process against all other religions? I've never been close enough to someone who was deeply religious for it to be worth discussing this, but it's always surprised me that if people really believe in a given religion then it seems to me like it would be vitally important to know that you were actually following the *right* religion...but I've never encountered someone who had done a full "literature review" of sorts of all the major religions in order to be sure they were following the right commands and performing the right rituals.
4. Parenting choices: For parents, how have you chosen to parent your child(ren)? Would that process applied in previous eras or locations resulted in you enacting things like "spare the rod, spoil the child", circumcision or more severe forms of genital mutilation, restricting choices or access to experiences and opportunities based on the child's gender, etc.?
Here's one: If the social norms you propose were widely followed, would this prevent a non-trivial number of people from fulfilling desires that are deeply important to them? Bad if so.
Many (not all) forms of wokeness/feminism fail this test, in that they try so hard to prevent unwanted sexual/romantic advances from happening that would effectively prevent *all* sexual/romantic advances from happening if people actually followed the norms they propose. Being anti-gay also fails this test in a more straightforward way.
I don't think this test is sufficient. Usually norms will have some negative effect, so you need to build weighing the costs and benefits into your question
Fair, I agree that trying to create norms that have literally *no* negative effects is unrealistic. However, I was focusing on the narrower set of cases where a set of norms effectively dooms some people to suffer persistently, in ways that can only be avoided by people violating the norms. It seems to me that avoiding that particular class of negative effect is generally both possible and desirable, even though avoiding *all* negative effects is usually impossible.
Though I suppose I wouldn't be super religious about adhering to the unit test I propose. I think it holds up well in most realistic situations, but it's probably possible to come up with contrived unrealistic scenarios in which I would endorse abandoning it.
here is a unit test for your unit test - pedophiles.
I fine with preventing a non-trivial number of people from fulfilling desires that are deeply important to them, actually.
it sometimes justified and sometimes no, and i don't have easy enough test to encode in unit test, just to the cost-effectiveness analyze every time, and hope.
Yeah in hindsight, describing my idea as a "unit test" was probably too strict - I definitely still think it's a good rule of thumb, but not something you should literally never go against. I notice, looking back at the OP, that Duncan's examples are more on the narrower side than mine ("does your decision procedure get the right answer in this one case?" rather than "does your decision procedure give answers that have this property across this broad class of cases?") which does seem more in line with the concept of a unit test as it's used in computer programming.
(FWIW, if there were a way for pedophiles to fulfill their desires *without* harming real people (e.g. experience machines) I'd be in favor of them doing so, but in the absence of such a way, preventing them from fulfilling their desires is indeed the right call)
There's a big distinction to be made here between statistical groups and individual people. I have met people who slept with adults, as children, and are not damaged and were not harmed; I have also met people who were molested and abused and raped as children and carry the scars forever.
(This is meant to gesture at "if there were a way." There clearly isn't, at the population level; we see the damage everywhere we look. But I don't want to confuse "when X happens, Y is often the result" for "when X happens, Y is inevitable 100% of the time." See Social Dark Matter for more, and uh, please no one murder me for saying a true thing.)
Ah yes, thanks for adding that nuance!
@Duncan Sabien
Was this post inspired by David Budden?
n, is there a David Budden post I should read?
He's an ex-OpenAI guy and he claims with AI assistance he's solved two Millennium Prize problems (Navier-Stokes and Hodge Conjecture). He's made big public bets (tens of thousands of dollars) that he'll have Lean proofs by the end of January.
I liked this essay! I found many of the unit tests interesting and productive, except for the Einstein one. I’ve kind of tried to do these meta science style unit tests in the past (related is Nielsens discovery fiction process), and I’ve always found them more useful as tools for reunderstanding a concept than actually checking your reasoning process, since pretending to reason as if you didn’t know a scientific concept is really hard for me. I do like the general concept and pointer though.
"9/11
If your processes, applied to post-9/11 America, result in invasions of Iraq and Afghanistan and the assembly of a vast surveillance apparatus and hundreds of millions of people engaging in airport security theater that burns countless time and money, you have made an avoidable mistake."
I still think the US had to attack Afghanistan after 9/11, but I don't necessarily have a better plan for the aftermath, even in hindsight.
Iraq was a mistake, though. There really wasn't much justification for it that held up other than "Saddam Hussein is bad and we can kick him out", which was true enough, but it was going to be damn obvious that keeping Iraq from falling apart afterwards was going to be a lot harder and costlier than the initial invasion and the US wouldn't benefit from having done it.
Airport security theater was also stupid, but I don't know if there was a "not stupid" solution that would have satisfied the kind of people that demand witch hunts.
> a vast surveillance apparatus and hundreds of millions of people engaging in airport security theater that burns countless time and money, you have made an avoidable mistake.
Just chiming in here that I've done this math, and the TSA has wasted 10x the amount of American lifetimes 911 took, at ~35k lifetimes gone right now, all while having a 95% failure rate when red-teamed, and we continue to waste ~800M hours of American time every year for it.
Google sheet: https://docs.google.com/spreadsheets/d/1_EhWz7MBR1SWGtWwRk-X2C6b6NNimyiAnP6xigJRc_M/edit?usp=sharing
And this is entirely aside from spying on our own citizens and everyone in the world in every channel in perpetuity (Patriot Act, NSA PRISM, Five Eyes, etc) and an ~$8T 20 year boondoggle in the middle east!
Now I feel guilty about not actually contributing to the thread, so I'll suggest my own unit test:
1. Can this suggested advice / intervention / behavior be understood and followed by an average person?
I feel like many people like us in the thread like to think and pontificate about society and politics and culture and economics, and feel that there are blindingly obvious and "easy" solutions laying about everywhere that could solve any one of our serious societal problems.
But the average American has a 7-8th grade reading level. 98% of people fail at weight loss. The great majority, ~80%+, are sedentary and eat garbage, even though that is literally 2-3x as bad as smoking or a cocaine addiction when it comes to all cause mortality. The great majority of people don't have a college degree (70%+), and couldn't actually get one.
Not just that, but essentially everyone, up to and including important executives and politicians and various looked-up-to figures, has the attention span of a gnat, and unless your points are extremely simple to understand, *and* simply phrased, AND are limited to only 1 or 2 points total, 3 at the very most, they will be glossed over and ignored by essentially everybody you are trying to communicate to.
It's easy for people with grad degrees and nice careers in Tier 1 cities to spot fairly obvious solutions to various problems - but they'd require an entirely different humanity and populace than the one we have. So the solutions are NOT obvious, if they are not workable, and we should all do well to consider that.
This is wrong in an interesting way. The unit test framing is fine, but it vastly, vastly understates the hard parts of specifying the underlying general standards that separate “sounds crazy but true” from “crazy”.
For example, what unites pro-Einstein and anti-Nazi conclusions? Finally! Two data points on the Truth! (It's an exercise for the reader how a naive line through those points in the twentieth century end up going wrong in the twenty-first.)
Maybe it's that political or moral realism is a spherical cow - especially historical political-moral realism. The spherical cow historical cases are as useful in testing as any other spherical cow, but the vast, vast majority of the 'programming' is in building a real world, present working model.
"This is wrong in an interesting way" followed by ... nothing that makes it seem like the OP is wrong. =P
It sounds like you meant "this piece did not exhaustively address all of the hard parts of the problem, but was instead narrowly targeted at a single part in the machinery of sanity and progress," which ... I agree with?
(The complexity of the truth being part of why it pays to *collect* unit tests, plural, rather than assuming you can get good answers with just one or two. Building a machine that correctly passes fifty different important unit tests *is* a way to start approximating a real-world, present working model.)
Fair point on my phrasing. I don’t mean “your conclusion-set is wrong”, I mean the unit test metaphor breaks if readers treat your listed (or solicited) answers as the tests.
I think “our best available judgment plus hindsight” is the wrong oracle to consult, because hindsight is exactly the thing that turns messy cases into clean morality plays and makes us overconfident that we would have seen it. That said, I get that you’re leaning on that stool leg pragmatically, because we still need some kind of expected output to run regression tests at all.
But the key failure mode is on the reader side. Without the underlying mechanism and the original uncertainty, the list becomes a set of imported outputs rather than tests of the machinery.
In code, a unit test works because the oracle is crisp and shared. 1+1=2 has a whiff of universality to it to which we are committing so as not to be seen as insane. With historical moral and political cases, the “expected output” is doing a lot of hidden work: values, counterfactual imagination, a ton of context about what made the mistake tempting at the time, and what made it a mistake after the fact. I would hope to suggest that you have all that rich context in your head, so the cases function as genuine regression tests on your own cognition.
For a reader though, the same list risks becoming a checklist of socially legible correct answers, which is exactly the failure mode you’re warning against.
“Don’t rely solely on consensus” is a unit test. “Nazis bad” is mostly a shibboleth unless you very carefully unpack the generalizing mechanism that you’re using for the experience you’ve had that resulted in that personal test.
"For example, what unites pro-Einstein and anti-Nazi conclusions? Finally! Two data points on the Truth! (It's an exercise for the reader how a naive line through those points in the twentieth century end up going wrong in the twenty-first.)"
Care to explain what this means?
It reads to me like two straw men. One made up of a claim that OP stated that pro-Einstein and anti-Nazi were somehow hard to find and revealing some type of important truth on their own - something that I did not get from the article. The other made up of stating that readers would draw a naive line between the two and trying to apply it on its own in current affairs, which for me at least fails on both steps.
Assume you might have meant something completely different.