We can probably influence the far future
by paulfchristiano
I believe that the (very) far future is likely to be very important, and that my decisions today probably have (very) long-lasting effects. This suggests that I ought to be mindful of the effects of my actions on the very far future, and indeed I am most interested in activities that I think will have positive long-term effects. In contrast, many or most thoughtful people consider it unproductive to think more than a few decades ahead, to say nothing of millions of years.
When I express this view, I often encounter intense skepticism about the possibility of having any predictable influence on the far future. Even if our decisions have some effect, can we reason about it any useful way? I believe that the answer is yes, and that we can in fact have a fairly detailed and specific effect on the very far future. In this post I want to make a very simple argument for this conclusion: if we think there is a good chance that we will ever have an opportunity to have a predictable long-term influence, as I suspect we should, then interventions which improve our own capacity (such as investment) will have an indirect, significant, and predictable long-term effect.
The basic argument
It is easy to see that in principle there are opportunities for interventions to have very long-last effects. In the most extreme case, we can imagine the emergence of a pressing existential risk, an event which threatens to permanently cut short or curtail human history. If possible, averting such a risk would have a massive effect; it could be the difference between an essentially barren future and one full of rich experiences and valuable lives.
The practical question is: are these opportunities actually available? Surely there is some probability with which we might avert an existential risk over the coming few decades (say), but this probability might be very small. Many people would balk at the suggestion that they prioritize a one in a billion reduction in extinction risk over a very significant improvement in the quality of life of 1% of the existing population. Although I am unusually sympathetic to the aggregative utilitarian position, I would also find such an extreme tradeoff problematic even on purely moral grounds.
There is some debate about this question today, of whether there are currently good opportunities to reduce existential risk. The general consensus appears to be that serious extinction risks are much more likely to exist in the future, and it is ambiguous whether we can do anything productive about them today.
However, there does appears to be a reasonable chance that such opportunities will exist in the future, with significant rather than tiny impacts. Even if we don’t do any work to identify them, the technological and social situation will change in unpredictable ways. Even foreseeable technological developments over the coming centuries present plausible extinction risks. If nothing else, there seems to be a good chance that the existence of machine intelligence will provide compelling opportunities to have a long-term impact unrelated to the usual conception of existential risk (this will be the topic of a future post).
If we believe this argument, then we can simply save money (and build other forms of capacity) until such an opportunity arises. If nothing looks promising and we have waited long enough, we may eventually fall back to what we would have done anyway–my own view is that this wouldn’t be so bad–or we can reevaluate the situation and perhaps hold out for an opportunity even further in the future.
Implications
I don’t think this is necessarily (or even probably) the best thing to do with resources today. I want to clearly make the point that we can have a predictable long-term impact.
Given that we can have a predictable long-term impact, I think that in order to justify any other intervention on aggregate welfare grounds we must argue that it has an even better long-term impact—in the same way that supporters of an anti-poverty intervention arguably have an obligation to argue that it compares favorably to an unconditional cash transfer. I do believe that many interventions can meet this bar, but I think that the case is not often made (even for many interventions which are explicitly targeted at long-term effects).
Viewed in light of this obligation, I think that the (aggregative utilitarian) case for many contemporary philanthropic projects is much more speculative than it at first appears, resting on a tentative chain of hypotheses about the relationship between “generally good-seeming” stuff today and very long-term outcomes. For many of these common-sensical projects, I am skeptical that they meet the bar I’ve described here. They might be justified on other (also morally compelling) grounds, but are unlikely to be cost-effective when considered exclusively in terms of long run aggregate welfare.
Will the opportunities be taken?
In addition to the existence of opportunities, we’d like to know whether those opportunities will have any “room for more funding”: if they are such compelling opportunities for a long-term impact, will other people take them regardless of what we do?
In this respect, the situation seems similar to more familiar philanthropic projects. For example, many people today are interested in mitigating the negative impacts of poverty, improving education, or accelerating technological progress. But these are large projects, and so a funder interested in them today can have a significant marginal impact even though they already receive substantial attention. I don’t see any reason to expect far-future-influencing to be fundamentally different from more conventional objectives, except perhaps by being somewhat more unusual and consequently somewhat more neglected.
It is sometimes argued that in general the future will be richer and more capable, so that the low-hanging fruit will be more reliably taken and the remaining philanthropic opportunities will be less good. I find this argument plausible, but am skeptical about the magnitude of the effect. First, I would observe that insofar as the world is getting richer, a philanthropist who saves should expect to get richer as well (in fact slightly faster). And I think it is unlikely that philanthropic spending (as a fraction of the world economy) will rapidly increase over the coming generations. So the effect is being driven primarily by overall improvements in the quality of decision-making around the world. But the sign of this increased decision-making-quality effect is not entirely clear: similar gains might also accrue to the individual philanthropist, who certainly benefits from the wisdom of the world around her. Again, on balance I expect that such improvements will tend to reduce the impact of an individual thoughtful philanthropist, but I am not convinced this is a large effect.
Concretely, looking at the historical record I don’t believe that a philanthropist 100 years ago would be expected to have a much larger effect on the world (using the same fraction of of the world’s resources) than a similarly-positioned philanthropist today, and I am highly skeptical that we are at the unique moment of maximally-impactful-philanthropy.
[…] If we believe this argument, then it suggests that the arrival of machine intelligence may lead to a substantial crystallization of influence. By its nature, this would be an event with long-lasting consequences. Incidentally, it would also provide the kind of opportunity for influence I was discussing in my last post. […]
Casually, it looks to me like the major philanthropic foundations of a century ago had an enormously larger impact on the probable long-term future of the world than the major philanthropic institutions of today have.
I don’t strongly disagree with this claim. But what predictable long-term influence do you think they have had?
I’m happy to see more posts on this blog! Having a simple and robust baseline against which to compare long-term philanthropy seems valuable, but we also need techniques for actually making those comparisons. How would you go about making comparisons to this baseline?
Correct me if I’m wrong, but it seems to me that your plan for affecting the far future in a predictable way essentially amounts to:
1) Invest resources
2) Wait until there is an opportunity to use accumulated resources to affect the far future in a predictable way
3) Take advantage of said opportunity
This doesn’t seem like a convincing argument to me. If we are unable to identify such opportunities today, what makes you think that we will be able to find one in our lifetimes?
Often people discuss the state of existing opportunities, with respect to say AI risk or synthetic biology risk, and reach a conclusion such as “It is too early to have a meaningful impact on these things now.” If you believe this, you may expect that it won’t always be too early to have an impact on these things. I think this is the strongest reason to be optimistic: we can see plausible ways to intervene on the future, which may or may not pan out, and the strongest argument against them is “it’s too early to intervene meaningfully.” Even setting aside these particular areas, it does seem casually like people alive 100 years ago had much worse prospects of directly intervening on any existential risks, and so we might expect to have better prospects still 100 years hence.
Moreover, to the extent that you have some pool X of possible activities, the pool over the next 100 years is simply a lot bigger (and more varied) than the pool available at the moment. I think it’s reasonably likely there are opportunities available today, and somewhat more likely that there will be opportunities available tomorrow. It’s clear that there aren’t any obvious opportunities available today, and it seems plausible there will be obvious opportunities available tomorrow.
In general, my point is that even a decisive argument against existing opportunities for far-future-influencing is not going to be too persuasive, unless it also implies that there will be no better opportunities in the future. This may be a very straightforward point, but it doesn’t seem to be widely recognized or reflected in discourse.
I admit that this is plausible. On the other hand, it is also plausible that there are good opportunities today that we haven’t figured out yet. Yet you seem to be claiming that opportunities for preventing existential risk are likely to get better with time. I have two problems with this claim:
1) I am not convinced that there is evidence for it
I grant that 100 years ago there wasn’t much that could predictably be done to prevent existential risk. But 50 years ago, one might have tried to mitigate the possibility of extinction via thermonuclear war. Today (or maybe a decade ago) would be (have been) a good time to deal with global warming (which, admittedly, is likely non-existential but potentially catastrophic). It is certainly plausible that better opportunities will show up with regard to other fields in the future, but this is uncertain. On the other hand, I don’t see the effective altruist community discussing interventions to mitigate global warming today. Why should I expect them to deal with the risk from synthetic viruses when that becomes relevant? The idea that there will be better opportunities in the future is a great way to rationalize never doing anything.
2) If it is in fact the case that opportunities for preventing existential risk consistently improve over time, this would be very bad. It would imply that existential risk continues to be a problem (and probably gets worse) as time goes on. If humanity has a 99.99% chance of going extinct in the next 1000 years regardless of what we do, it becomes very difficult to influence the far future.
I think EA’s do discuss global warming and nuclear proliferation, e.g. I have looked into both of these ideas and discussed them with other EA’s at sufficient length to be reasonably confident I won’t find them promising upon further inspection, and GiveWell has (more usefully) looked into it as well. There has also been fairly extensive discussion of synthetic biology and so on, and certainly in that area the main sticking points with the general public and practitioners concern whether we can do anything useful so far in advance.
It’s definitely plausible that there are good opportunities around today, as I point out. This argument is typically made in response to people who are skeptical about the goodness of any available interventions. If you think that no available interventions are good, and that’s why we can’t easily find good interventions, then I would expect you to be much more optimistic about there being good interventions at some future point.
We could have intervened on nuclear war earlier in this century, but I believe the actual destructive capabilities available have been mostly increasing and should be expected to increase further (and are still not yet great enough that I would consider extinction a very plausible outcome). There is random variation in the political situation that will make some moments (like the cold war) particularly good times to intervene, but personally I’m more concerned about future wars (and I would be more concerned about the cold war than WWII, and about WWII than WWI, and about WWI than any preceding war, and overall it seems like the trend is pretty clear).
I agree that the far-future influencing plan involves the implicit belief that there are some “goalpost,” or some point where the trouble is basically behind us. I’ve written a bit about this and hope to write more in the future; I think it’s quite plausible, but that it’s not plausible this will happen soon (for example until our understanding of intelligence is relatively stable or there are huge changes in social organization) and I think that as we approach the goal posts the situation will look different than it currently does (e.g. it won’t look like there are a bunch of upcoming risks that we don’t understand and aren’t equipped to handle).
You still seem to be making several arguments that I find uncomfortable:
1) That there are useful interventions today that we merely do not know about:
If there were an intervention that could *reliably* affect the far future in a positive way, wouldn’t someone in the EA community have found it and then publicized it widely? Why have we not heard about this intervention? This at very least suggests that if such opportunities exist that the EA community is ineffective at identifying such opportunities, which seems like it is roughly as bad as them not existing.
2) That such opportunities will be more abundant in the future:
I don’t think that you’ve given a compelling argument for why this is the case. I suppose that the chance of existential risk is going to (generally) increase as we come up with more technologies that could conceivably lead to human extinction. Then again, the chances will drop dramatically if we ever colonize another planet.
3) That the time between now and the end of our lives is an exceptional period in human history in terms of providing opportunities for existential risk:
You seem to argue that:
(a) In the past/present such opportunities have been relatively rare
(b) In the near future there will be such opportunities
(c) In the not-all-that-distant-future we will have reached a goalpost and these opportunities will be small again
I think that there is always good reason to be skeptical of arguments claiming that we are special. Do you really imagine people 1000 years in the future or 1000 years in the past not making similar claims?
1) I think finding good interventions is extremely difficult, even where they exist.
2) Yes, I think that the risk of extinction generally goes up as we increase our ability to influence and reshape our environment (those capabilities are not yet strong enough to present a really credible challenge to human survival, but it seems quite likely they eventually will be). It also seems likely to increase in light of predictable technological developments, including a transition to machine intelligence.
But most of all, I want to stress that this is an argument that we can influence the far future at all. If you think that today there are no such opportunities (a position I find implausible and often argue against), then simply by regression to the mean you should be more optimistic about them existing in the future.
3) a) I don’t think such opportunities are (too) rare today; I think many things we can do predictably influence the far future.
b) Over the next few hundred years human society will move through a substantial fraction of the doublings it has ever moved to; this is not the blink of an eye. Again, we can also see fairly predictable developments of this form.
c) I agree completely that we ought to be skeptical of such arguments. I give one reason in my next post for why a transition to machine intelligence might be quite special. I think the strongest reason to suspect that risk will eventually decline is that eventually the flow of novel technological capabilities seems very likely to decline. I also think that we can see particular technologies, such as machine intelligence, that would radically “harden” human society. And finally, I would reserve some significant probability for such “hardening” to happen eventually anyway, even if I could not see any particular reason it would happen soon.
I don’t think anyone 1000 years ago had a plausible case that existential risk might loom large in the next 100 years. I think that from about 1700 on people could plausibly have made the case that the next 100 years would be much more dangerous than any time before, and I would find that case compelling. I also think it’s not a terrible conclusion (each of 1700-1800, 1800-1900, and 1900-2000 seems to have a very good chance ex ante of being more dangerous than the any earlier time in history). The trickier part seems to be believing that things will be OK eventually. I may write about this in a future post.
1) If there are actions that if we took would affect the far future, but we cannot feasibly determine what these actions are, then we cannot affect the far future. This is true in the same sense that we cannot prove the Riemann Hypothesis.
2) I’m not sure how you think regression to the mean is supposed to apply here unless you think the default is being able to. This seems strange. It is practically a theorem that most people (taken over all people to ever exist) do not have the capacity (or at least will not use said capacity) to strongly influence large numbers of future people.
3) a) OK. Name one.
b) What do you mean by “doublings”? I doubt that we are going to square our population in the next century.
c)
* I don’t think that we can reasonably predict the effects of a machine intelligence that is produced via unknown mechanisms. Some implementations would undoubtedly have very different impacts than others and I do not think we have a clear enough picture of which implementation we are going to get to work first or how it will behave to make specific predictions about the impact.
* I also think that the claim that we are running out of novel technologies is credible. This has been predicted before incorrectly.
* I agree that nobody 1000 years ago had a plausible case that humanity was going to go extinct in the next century. This is irrelevant. I merely claimed that people 1000 years ago predicted such. 1000 years from now, our reasons for thinking extinction is plausible may seem unfounded. Furthermore, 1000 years ago, there may have been fewer people caring about human extinction, and probably plenty of the people who worried about their culture being annihilated in the next century were correct.
1) Sure, but it’s reasonable to spend resources trying to prove the RH (if you cared enough). I think there are ways to improving
2) OK, that’s a different argument than “Nanotech looks far off, and interventions to influence it seem too early to matter,” which is the sort of discussion which typically occurs. I agree you need some evidence that you are at a special point in history. If you just run the calculations, though “The world is currently small, and looks likely to be very large in the future” is basically ample evidence on a utilitarian perspective; I think we have quite a lot more evidence than that.
3) a) Improving accuracy of consensus regarding impacts of AI, amongst researchers and policy-makers.
b) According to Brad DeLong (via wikipedia) we’ve had < 25 doublings of economic might over the last million years. Trend-extrapolating we might expect another 5 over the next century, which is a good fraction of all doublings ever.
c) Looking back, it now seems clear that people 50 years ago *did* have reason to fear for the continuity of civilization, though in fact things went fine and probably even if they hadn't the human race would have continued. And people in 1800 didn't have much reason to worry, but I think they still seem to have had more reason to worry than anyone before them (since the dawn of technological civilization).
Independent of the trend, it just seems clear that sophisticated biotech, nanotech, or AI could result in an existential risk eventually, and I think most practitioners in any of these areas would agree that "eventually" has a good chance of appearing this century. One may be optimistic about policy responses, but there really haven't been corresponding policy challenges in the past.
I agree that most points in history don't get much influence, and that seems like a strong argument. Do you also find the doomsday argument compelling? It already seems very likely from our observations that we live at an exceptional time just based on the ratio between the size of civilization and the plausible future size of civilization. I don't think this licenses confidence that extinction risk will never fall to low levels, whether from exhausting the list of most dangerous tech developments, stronger coordination, or hardening due to technologies like machine intelligence. We are pretty confident it's not going to in the very near term, but there just doesn't seem to be much evidence that it won't over the next few hundred years.
Also, note the strategy under discussion here might work fine 1000 years ago. I.e., if someone 1000 years ago says "I don't see a clear way to influence the far future, but maybe there will be one soon" and then saves for 100 years, and then in 1100 does the same thing, and so on, then the question would just be whether they could have reliably done this 10 times without having the returns appropriated. In the modern world, it looks like there is a pretty good chance you could pull that off, i.e. you could have your share of the world grow in expectation. The main argument against is that no one did anything much like that, and it's hard to tell how many tried but presumably it is more than 0. This is closely related to the next post.
1) Granted. But don’t say “we can do X” when you really mean that we are physically capable of doing X but have no idea how to actually accomplish it.
2) I think I was more complaining more that I don’t understand why you think regression to the mean implies that we are going to have more opportunities in the future. As a side note, human population does seem to be showing signs of leveling off (given population growth rates of wealthy countries).
3) a) OK…. What are the predictable significant outcomes of this? Note that I do not believe that AI risk is nearly as significant a threat as you seem to believe.
b) Fine. Why does economic doubling imply such opportunities? We’ve also seen several doublings in the last century, but as far as I can tell did not see any particular predictable way to significantly influence the future then.
c)
i) Do you really think that someone investing money 1000 years ago could have reliably managed for it to still exist today and be used for anything remotely resembling its intended purpose? This sounds ridiculously implausible to me.
ii) It’s not quite that I find the doomsday argument compelling or that I find the argument that these existential risks are not as bad as they might be compelling, but the combination of these arguments seems somewhat convincing. I suppose hardening could happen if society gets significantly restructured to deal better with long-term threats, but given the response to global warming, I’m skeptical. Other than that, I find it somewhat implausible that the threats just on the horizon are serious, but that the ones just beyond them won’t also be. Either, we have not much to worry about with biotech/nanotech/AI, or we’re doomed by the minefield we’re about to go through.
3) a) This isn’t a good place to have a longer argument on this point, though my most recent post makes one argument for the link between AI and long-term impacts.
b) I was claiming that while “the next few hundred years” is in some sense the “near” future, it’s actually quite a long time in terms of “amount of stuff that happens.” So it’s not that surprising if an unprecedented opportunity happens to arise in that period, or if there are big changes over that period.
c)
i) I think it’s not implausible that I could find a successor who shares my values well enough that in expectation it’s 1/2 as good as me having the money myself (or whatever is necessary for the value drift to be outrun by capital accumulation).
ii) See (b) above. Also, note that big enough problems may precipitate either hardening or doom.
3)b) The same could be said about the last century, and yet I do not see any clear opportunities that existed then.
c) i) Can you even name an investment vehicle that would have maintained its value from 1000 years ago until today (much less doubled every generation)? Also, ignoring changing the goal, how many of the things that people 1000 years ago might have been aiming for are even sensible objectives today?
b) The last century did have reasonable-looking opportunities in the form of averting nuclear war. But my point was that if you draw “nothing, nothing, nothing, nothing, lame” from a barrel, then you shouldn’t be too confident that the next item will be nothing/lame, since you just don’t have that much evidence (even neglecting the positive trend). And once you see some promising-looking indications for the next item being good, that should quickly overcome the inductive record.
i) If you are willing to manage things actively, it seems like making loans is fine, as is buying property, or any number of other investment approaches. Is this question distinct from “were returns to capital large?”. The main problem other than picking successors to manage the endowment seems to be weak property rights over the last 1000 years, which in fairness looks like it will be predictably better over the next 100 years.
In terms of goals, “more human happiness” sounds fine as a goal, as does “more Christian values,” or “advancing human knowledge,” all of which might have been picked 1000 years ago (as far as I know). I think you can do better if you want to specifically try to state clear goals that will stand the test of time, by specifying procedures for picking goals that can adapt to better info, rather than picking goals directly.
Picking successors is a tough one, though something I expect to get much easier once machine intelligence is available and we can preserve a copy of an adjudicator or design a manager from scratch.
b) OK. Granted. I cannot by general arguments exclude the possibility with high confidence.
i) I think finding reasonable investment instruments is exactly the problem. In addition to most of your potential investments failing to have significantly accrued value over the last 1000 years, how many of them would have survived all the wars that took place? How many would have survived governments falling? How many families have actually managed to hold onto wealth from back then?
I mean sure a lot of this is better now, but we should be worrying about the things that we cannot predict. What happens if we hit a post-scarcity society and accumulated capitol becomes irrelevant? What happens if influence becomes a more useful measure of wealth?
Also, sure machine intelligences might plausibly make it easier to maintain a consistent goal across generations, but it doesn’t seem like something that we can reliably count on.
There is a trade-off which has to be carefully accessed by rational altruists. As more information becomes available about particular actions which may intensely affect the far future, more people are going to seize the opportunity and invest on them. So as epistemic uncertainty decreases, marginal utility for investing in that particular cause does also.
The early investment in the right X-risk will be the one that makes the most difference in the end, but only retrospectively we will know which was the right x-risk to be worried about. Prima facie, this reasoning seems to me to imply we should spend resources on different X-risks prior to gaining personal access to the necessary epistemic level o certainty.
In particular, if other smart people already feel secure to think that a particular X-risk is worth acting against, I would take that as good reason to invest against it, while building up my knowledge of it.
– Diego Caleiro
[…] In the search for the most cost-effective interventions, one major area of debate is whether the evidence favours focussing on human or animal welfare. Some have argued for estimates showing animal welfare interventions to be much more cost-effective per unit of suffering averted, with an implication that animal welfare should perhaps therefore be prioritised. However this reasoning ignores a critical difference between the two types of intervention: the long-term impact. There are good arguments that the long-term future has a large importance, and that we can expect to be able to influence it. […]
Mr. Christiano: “I was claiming that while “the next few hundred years” is in some sense the ‘near’ future, it’s actually quite a long time in terms of ‘amount of stuff that happens.'”
This is a great point. In many contexts I think it’s best not to talk about years but instead GDP doublings. We might also talk about the “far” future, or within ~5 doublings, and the “very far” future or beyond this.
My feeling is that while we can likely help the very far future, we can’t do it except by addressing nearer-term goals. That would include with philanthropy. E.g.,
http://lesswrong.com/lw/h3q/a_rational_altruist_punch_in_the_stomach/8oq7
To be clear, do you think that full human-level AI is plausible in the far but not very far future?
[…] In the search for the most cost-effective interventions, one major area of debate is whether the evidence favours focussing on human or animal welfare. Some have argued for estimates showing animal welfare interventions to be much more cost-effective per unit of suffering averted, with an implication that animal welfare should perhaps therefore be prioritised. However this reasoning ignores a critical difference between the two types of intervention: the long-term impact. There are good arguments that the long-term future has a large importance, and that we can expect to be able to influence it. […]
[…] can probably influence the the world in the near future, as well as in the far future, and both near and far seem to […]
[…] Christiano (2014). We can probably influence the far future. Rational […]