i am looking at ways of understanding the repugnant conclusion. here is one way that may get out of it.

we have a seed AI that has to make guesses about the consequences of its actions and what consequences are desirable. I say the desirable consequences are the preferred state of the world if all minds (with preferences, goals, desires) that do exist or could exist given the options available were taken into consideration. I want the seed AI to makes guesses (guesses are all any mind can make about the future) about what the total set of minds (acting as if they were one) would make. The combined mind would understand the perspective of all minds involved. Of course this is not possible but with better data about the world the seed AI would be able to make better guesses about the minds and and in turn better choices.

here is an example of how this relates to population ethics.

1. One person with one mind has to decide if they want a small, slightly annoying pin prick on the left hand that lasts one second or a repeated hammers smash for 10 hours on the right hand that is really painfull. the person (seed AI) has to make a guess (granted a very educated guess in this case) about the world in the future and what would be a preferred outcome. assuming they are only looking at the pain (and think no other positive or negative consequences will come out of either outcome and assuming the person does not desire pain) they would choose the pin prick. the left hand ends up loosing as it gets a pin prick but overall the person (or a seed AI) guesses that this makes the world better overall, even if the left hand looses. The mind can put both hands into consideration as it choose what to do.

 

2. We could take this same idea but use two different people. one getting the slight pin prick and the other getting the hammer smashed. this time the seed AI is not one of the brains but is making a guess (granted slightly less as sure) of what the two brains, if they booth understood the goals, preferences, desires of the other and them selves, how much they each desired not to be in pain (in this case they have identical desires not to have pain, or that is what the seed ai is guessing). they are each self motivated not to be in pain themselves but the seed AI is not self motivated. It just tries to make guesses about what will bring about more goals, preferences, desire satisfaction and in this case it would be the pin prick on one person and not the hammer in the other.

 

3. we could do the same but this time have 3 people, and two pin pricks and one hammer. again the seed AI makes guesses about all three minds. is it better to do a pin prick on person 1, person 2 or a hammer on person three. What about two pin pricks (one on each person 1 and 2). It does not know for sure but it can take into account (make guesses about the goals, preferences, desires) all three of the minds.

 

4. It can do the same but with a pin prick to a person and a hammer to person in a coma with an estimated 10 seconds to live (not pictured), or a hammer to a cow foot. In each cases to makes guesses about the brain states of the person, the coma person or the cow. It makes guesses about, what a mind with full information of the goals, preferences, desires of each were, would want.

 

5. and the same for any population ethics questions. The seed AI has a single goal of maximizing more of what matters to those that matter (to the degree of each) and to do this it needs more data about who matters, what matters to them, what the consequences of its actions will be and so on. It has to do its best to take all minds that could exist in the future into account. It has to look at all minds that do exist (or could exist) as one and make best guesses from here.

All it can make is guesses as no mind will have all of the information, especially when makeing guesses about the futuer. (i am using the word guess as makeing choices when we do not have full information. We never have full informations and so we always make guesses. that is not how it is generally used in day to day converstaion)