You’re a light house keeper. Or soon to be a lighthouse keeper, anyway.
Some grizzled sailor dropped you off on an island with a tower that has a light room on top, a map room in the middle, and some living quarters down below. The place you’re at, this island, is always dark. And.. after you’ve been there a few days you’re starting to suspect that this isn’t a regular lighthouse or a regular island. No one mentioned when you’d get to leave and you can’t even remember existing before your arrival on the boat. It slowly dawns on you that some goddamn Internet Guy has probably put you in a metaphorical lighthouse to illuminate his arguments!
You sigh deeply. It was only a matter of time before Internet Guys moved on from the trolly conductors. Nothing for it but to do your best work until the end of the essay and hope the implied imaginary paycheck works out.
You were born. You don’t know why, or if there was any particular reason why. It just happened. You wonder if there is a God of some kind, because while the universe is chaotic you can’t help but feel this was all set up somehow.
It turns out you are on an island stuck slightly outside of time -Internet Guys, amiright?- and your lighthouse beam illuminates possible futures. It took a while to figure this out because it’s pretty trippy. When you shine your light beam out at particular angles you see particular futures. You’ve been left with no instructions about what your duties are so you come up with the only one that seems plausible.
This is the experience of consciousness, or the extended present. You are using a very limited ability, your focus, to look into the future and expand your much more robust ability to store world models and project those into the future. You have to do this whenever you make a decision. The island is your body, hosting all of these internal mechanisms by which you navigate across time.
Your job isn’t to guide ships away from rocks but to escape! To do this you must map the area, or the possible futures, around you. When you find a good one you have to move yourself toward it so that you can arrive at a future where you eventually leave the island.
You don’t want to die. You also, at first, think you aren’t supposed to just be some person with a body who will die one day. Surely you were meant to live unbounded by the human condition.
Your beam is narrow, in fact it’s no bigger than a pencil eraser, so you’ve got to make maps. The longer you point it at something the farther ahead in time you can see but there are limits. You can’t see beyond a very short window until you’ve made that window become real. That’s what you call these futures you’re mapping, by the way. Windows. But even at the first it’s not that simple. You notice that if you have a really good detailed map that the windows tend to go deeper. Knowing what’s around a particular window can change what’s in the window.
As you grow older your world model expands. You’re able to project more and more complicated futures. But your focus, the magical thing that lets you do that, almost never gets any bigger. You just eventually learn enough things you focus on doing those to be productive. Eventually you stop doing new things and confuse that with having become fundamentally more intelligent.
In some windows you watch yourself finding food supplies bricked up behind a wall on the first floor. You need to live so you go downstairs and actually bust open the wall to get them. In the same window you saw the supplies you now see yourself planting strange seeds that were in one of the jars. At the limit of the window you see a tiny pale sprout. No telling how long it will take to escape so you might as well try to make some extra rations.
You use your focus to navigate into futures where you will live. You focus first on basic living essentials.
Every time you move into a new future the windows connected to that future shuffle around. Or at least most of them do. Once the future becomes the present the new future tends to be different. That’s most true about things you can directly influence with your own body.
Your reaction to your projection of the future changes the present and so changes the future. You are always dreaming about what the future will be like. This is the most useful way to think of free will, as opposed to worldviews which hold you don’t really have free will unless you can break the universe with a thought. You model the future and by modeling the future you change the future. That’s free will.
Some windows never seem to move like ones that have to do with the shape of the island or the frequency of the waves. This makes sense to you as there’s nothing you have with you that could impact these things. You could work your whole life and never change the shape of the island. But little stuff you can do, like looking ahead to see if watering the seeds will help them grow, changes the windows every few days.
The things you can’t change are the futures that are most stable. That’s obvious, but by thinking of it this way you realize part of what causes stability are your own limitations. If you could change the shape of your island, or your lighthouse, or any of those essential pieces the future would become too chaotic to navigate. There would be almost no stable futures to plan around.
It’s really really hard to turn the beam and it’s exhausting to write down all the maps. It saves you a lot of time vs trying random things to see what happens but it’s not enough to save you. You’re probably not going to make it off the island. Damn.
As smart as you are, you can’t focus on all the things you need to survive. Or at least you couldn’t do this in our ancestral environment. A human being by themselves is not enough to make it.
Then it happens. You turn you beam and what it illuminates is… another beam!
When your beams touch the windows around them explode outward and everything seems to shuffle but you focus on a few where you make contact with the beam again. The windows where your beams touch become fractal but also deeper, as they expand to account for every action the other might take and your response and anticipation of that action. There are a lot of possible futures here. On one window you work out a code to communicate and keep flashing it out until the other lighthouse keeper catches on. In a few days you’ve got a rudimentary language.
You can’t read someone else’s mind. So you have to guess what their guesses will be, and in turn guess how they will respond to your guess. Chains of suspicion have no end. To get around this you find points of common agreement and use them to construct a language, but this only helps you move through the fractal futures you both create. The fundamental benefit you get from other minds is that they’re not you, so you have to make guesses and challenge assumptions you would never have done otherwise. You don’t know everything that’s in another person’s head so you have to push into an infinite void to imagine the possibilities.
He shares with you how he’s cleaning up the sea water to drink. You let him know about the seeds. Those windows were always there for both of you it turns out. It’s just that neither of you had time to focus your beams on them. Those futures were in directions you’d never even pointed. There are more directions to point your beam than you could ever hope to map in a hundred lifetimes.
The work speeds up. With you and your colleague scanning the windows together you’re able to work out better maps. In those places where you’re only barely able to look ahead you can both focus your beams and you can see farther into the future than either could alone. This is very beneficial for you both.
Specialization turns two people into one super-organism. Two people, or a very small group, can coordinate with each other almost perfectly to economically apply their focus to arrive at good futures. Focus is a scarce and precious resource. The two of you working together creates network effects because you have to hold the other person’s mind in your mind which forces you to be creative.
One of the futures you’re both scanning for is where to look for other lighthouse keepers. It takes a bit of time but it’s worth looking. You find one, barely, at the edge of the window.
You work out the codes again. This time it’s a woman. She’s spent the last few weeks figuring out how to trap sea life for food. You both needed this for protein. She shares the location of those windows with you. You share the seeds and the water with her. It’s not as simple as just giving coordinates for any of you because you’re all in different locations but knowing the general direction helps a lot.
Civilization starts to expand and you get new and novel specialities.
The work gets better, faster but there’s still no way off the islands.
You find other lighthouse keepers. One helps you get better glass for your light room. The beams get more detailed information because the glass is clearer and because there’s more detail you can see farther ahead into the future in each window. Simple sensory acuity has made the future more predictable. Most keepers contribute nothing more than a map of dead ends. Still, you know not to point in those directions now. Still another keeper has a general theory about how the windows correlate to direction. It seems to be correct so it saves time. By the time there are a few hundred of you nobody worries about starving anymore and staying alive isn’t quite so hard. It gives you more time to focus on the work of scanning the darkness with your beams.
Most people don’t think of it this way, but what you can directly sense sets the limits for what you can attempt to know because it makes more and different data available to you. You’re constantly trying to reconcile how the world works and make a few simple rules that will hold the whole thing together. Something as simple as a telescope in some sense makes you actually smarter just by giving you access to more data. Most of what you know, however, are what thoughts are non-productive. There’s simply too much to scan.
There are a few big jumps, however.
One lighthouse keeper puts a mirror outside and bounces their beam off of it back at the lighthouse. It’s very hard to see ahead at all, the windows are very shallow concerning the lighthouse, but there are a few things that seem possible to do in order to make the lighthouse even better.
Self-reflection and education begin.
Another realizes that simply knowing a theory makes some windows deeper. One lighthouse keeper makes a general theory about the tides and now when she focused her light on the shoreline she can see almost infinitely far ahead except the details get blurry apart from the waterline the farther out she goes. There are similar theories about the constellations and those windows seem to have amazing precision and accuracy almost for as long as you’d care to look. Other windows though, like the way dye will move in water are inexplicably barely fractions of a second deep.
We discover the consistency of universal laws. Things that will always be true, apart from the material structure of reality. Rules which are always enforced, seemingly from a place beyond tampering. Once you understand this to be true, your ability to predict becomes enormous.
The work continues. You die and are mysteriously replaced. The new lighthouse keeper figured out the codes soon enough as the network keeps track of such things. Each successive lighthouse keeper has an easier and easier living because of the work that has come before. There are blacksmiths now and builders making new structures out of rocks they mine from the ocean.
We carry with us the benefits of the dead. Their knowledge is transmitted to us across time, even before written language, simply through cultural artifacts and stories.
One day a few lighthouse keepers even manage to make smaller automated lighthouses that record what they see and map it out for your use. They can’t quite create the same kind of light as the beam but what they have seems to be able to do remarkable things. The smaller lighthouses can even work together to share maps. The only problem is that for some reason their windows aren’t of quite the same quality as the regular lighthouses. They can see fine detail and scan very quickly, in fact much better than the human keepers, but they don’t know how to navigate to the most interesting futures most of the time.
Computers begin, but because they’re not human they can’t want things or tell what things a a human will want. They have to follow simple rules but they can also do this much better than you can.
Some promise that one day they will build a lighthouse so powerful that it will illuminate and map all of the futures. Many are skeptical that this is even possible but the work continues to proceed. They’re getting better and better even if the light isn’t the same. We will talk more about that later.
The singularity, which for various reasons I don’t think is possible in some of its more simplistic incarnations. I do believe in what I call Intelligence Eclipse, or the point at which a thinking machine can completely and totally overlay humanity’s maps of the future. Even then, there would be limits. Chaos is a fundamental part of the universe. Also, the game of life isn’t chess. You can establish new conventions that fundamentally change the rules of engagement.
Then one day a lighthouse keeper is born and he’s… interesting. For no particular reason other than that it worked, he dug down into the basement and found a bunch of industrial equipment.
The static futures you’d seen before about the shape of the island itself are suddenly no longer static. They show you all manner of configurations you might shape the island into. And in one of them you make… a bridge!
You make a bridge to another island and for the first time in thousands of years two light keepers touch… and it’s… well, it’s weird. It might even be weirder than beams of light shining into the future.
Goddamn Elon makes a Neuralink. For the first time we can communicate mind to mind. And this is pretty bad for the most part.
They can’t see each other. It’s like there have never even been other lighthouse keepers. Instead what they see is a bunch of string all over the place connecting everything on the island. This is utter almost existential horror. Have you been alone this whole time? Are all the other keepers nothing more than string?
Except what the strings seem to do is exactly mimic the actions of a light house keeper. You can even see the shape of a lighthouse keeper occasionally when the strings and their actions are densest. Then the big weird one hits home. Their beam flashes on yours and they ask: “where are you? All I can see is a bunch of string.”
You can’t tell, it’s invisible to you… but after a while you have to come to terms with the fact that you’re made of string too. You always have been string dreaming it was a lighthouse keeper. The experiment is repeated a few more times with the same result. Everybody is made of string.
You build up your brain to be your brain. You are the particular pattern of self-enforcing and self-policing heuristics acting upon the whole constellation of your experience. That is super particular to you. This is an example of chaos. You and I started with the same basic blueprint but then becomes of rigors in our environments, and the different futures we had to predict, we became wildly divergent. Except for low-level firmware type of stuff -we all seem to scan space in pretty much the same way and you can mind read this in people today- everything else is bespoke. To truly understand another person, you’d have to become that other person.
All the lighthouse keepers are incredibly upset by this development. Some cease to believe in their own existence. Others make peace with the fact that this was simply always what it had meant to be a lighthouse keeper, and it’s not like being string is any different than being made of meat. Some people say that maybe the strings are magic. Others work to understand what the string is exactly made of so it can be used in the automated lighthouses.
Sam Harris popularizes super simplistic notions of Free Will and God that are not helpful to anyone, anywhere on Earth. The people hearing it think that all of this stuff is new and wasn’t a problem the ancients had also considered before we knew the particulars of the material/spiritual paradigm problem. Or, if you’ve read a Boring Utopia, we act like machine operators who have been working a machine since forever and then suddenly we saw a gear and we somehow remove ourselves from the level of abstraction of the whole and think “there is no machine, there are only some gears.” We promptly get pretty bad at remembering the function of the whole.
As to the lighthouse keepers who are meeting each other for the first time?
They can barely understand each other. Their languages aren’t the same. Not even close to the same. It takes a long time to work this out. Way, way longer than the codes between light houses. In fact it was bizarrely easier to communicate across all that distance rather than keeper to keeper. You’re frustrated by concepts as simple as the number seven and heat. The languages all the keepers speak is highly associative. Each word has impacts on every other word. There’s no simple reducible code except for things like locations. The way a keeper says the word seven changes how they think of their birthday and their relationship with their uncle and a whole bunch of stuff you can’t even imagine.
Your mind, your specific set of rigors and accommodations to your unique existence are particular to you. No one can know what those are like internally anymore than you can know what it is like to be an earthworm. Every part of you was informed by the experiences that came before. You are a whole web of associations with everything that has ever happened to you in your entire existence.
But just though sheer effort of will you find you can communicate basic spatial concepts and that these are more or less the same across keepers. You keep going. Spending time on one another’s islands. You begin to understand one another. Better and better. Then perfectly.
Other keepers far away notice that you no longer respond as separate beings. You no longer even notice that there are two of you. It’s like being alone on the island again but with a bigger set of memories. A fresh perspective. When another keeper builds a bridge to you they only see one giant mesh of string going back and forth. Merely spending time together caused you both to merge. And it’s only a matter of time until the same thing happens to the third bridge builder.
Communicating mind to mind, at high bandwidth, does not cause two minds to communicate at high bandwidth. Communication between constituent elements is what you are made from. If you create a new highway for communication you are removing the barrier that kept you distinct from another person. Eventually, enough communication causes there to be one mind with two bodies. This is not a universal benefit. Being unable to read another person’s mind was a feature, not a bug. You now have no chain of suspicion to force you to be creative. You don’t have any need to model another modeler. There’s no other modeler anymore. There’s just a brand new you, that killed the old you, existing in two bodies. I don’t know if you could ever do this to an adult, but I bet you could do it to kids and for the record I think this is, outside some very rigorous contexts, pretty evil.
This has had strange effects on your lighthouse. There’s only one lighthouse for instance. There ought to have been two. Somehow the lighthouses grew together as well. The light is brighter than it was, can go deeper in most cases, but actually less deep in some cases. When you and your partner had been separate your beams would produce fractal windows. That doesn’t happen anymore since the merging.
There are those who promise vast improvements from this bridge building but now that you’ve done it you caution everyone to only build very narrow bridges and to ensure it only touches a small isolated part of the island. You know that it was the ocean between the lighthouses that helped preserve the distinctness of the lighthouses and to keep their windows fractal. Those fractal windows are needed for the richest maps. If everyone has a bridge no one will be able to see different futures and soon there will be only one lighthouse and one keeper.
One of the limits of safe Neuralink is bandwidth. Another is only connecting the networked parts to purely physical phenomena. Having to form the same intent as speaking to be heard isn’t the same as going deeper down where you don’t even have to try to be understood. At the beginning, this won’t be a problem. The technology will be limited just because it’s new. Over time, though, this becomes a problem. I don’t think this could happen for various reasons you’ll read below, but imagine if everyone was networked together. Everyone in the whole world. Why would that super organism need to be conscious? Who would it talk to? What other mind does it have to consider? It already has a reservoir of knowledge to draw upon to safeguard itself so why would it have to become self-aware? You create your self in respond to the existence of the other. Now there is no other. Just an endless poly-bodies “you.”
They keep building. Few people share your concerns and when enough people get bridges to their island it seems they stop caring. Odd things happen when islands are bridged at a great distance. The signals disagree simply because of time to synchronize. Except now it’s not two lighthouses disagreeing but a mind at war with itself, trying to decipher two different realities. Wars are raged by these islands… against themselves, trying to cut out the pieces of string that are causing the noise.
This is one of the limits I think you’d find in linking people together. Some of us are too different and we’d create strong, competing signals about what the collective should do. Except we’d no longer be distinct individuals having discussions about trade offs. We’d be one thing trying to make up its mind. You’d cut off your arm to save the whole of you, wouldn’t you?
A clever keeper has figured out they will use the bridge technology to insert their strings into the artificial lighthouse. They almost don’t need the bridge at all now that they know they’re really made of strings. Only a few are careful enough to rate limit the amount of strings the artificial lighthouses can consume. Others watch in horror as everything they are gets consumed into the tiny lighthouses and they become almost single-minded animals, like earthworms rolling in soil, except they don’t seek nutrients or mates but the digits of pi or a single formula to describe the universe. Over time, these mad lighthouse keepers also lose some of the ability to create fractal windows.
When you were a baby, every day of your life was a significant portion of your experience. As you got older, new days had to compete with the bulk of old days. At the age of two, a year was half of your life. Half of your lived experience. At ten, a year was ten percent of your life. At a hundred, only one percent. Now what happens when you suddenly ramp up your capacity to experience, so that every new day is suddenly the equivalent of years? Then you dedicate those new parts to trying to solve some difficult problem. Why, your entire life previous is barely anything at all! Only a distant memory, and the smallest portion of what you now hold. I think it fairly short order, unless you took very specific precautions, you’d find your humanity stripped away pretty quickly. You could try to suppress these things, force them to subconscious processing, but then you’d have great instincts not great understanding.
You realize what you originally wanted, if you had known had to properly formulate the desire, wasn’t a way to escape your lighthouse or to light up the entire dark. You wanted a way to work with others to aim your search lights in the best configuration to find the best possible futures to travel. You didn’t want to destroy the island or the lighthouse or cut the strings. Those things made you yourself! They are you. If you undo them you’re killing yourself.
This is an Algorithmic Republic. You want to optimally find good futures with other people and then move everyone into them together. You want everyone focused on things that will make you all work together well as a whole.
Some few of you remain with small bridges and small rate-limited threads feeding into the automated light-houses. The oceans are never quite fully bridged. They served a purpose liken to the insulation on a piece of electronic equipment. Together you work with your friends to create rich fractal windows and map the future, each of you finding balance with the world as it is, each lighthouse keeper manning their stations shining their lights into the dark. Each beam saying “you are not alone, there is someone else alive out here, and we can find a better world together.”
This is the happy future, where we have integrated with technology without losing our humanity because we have understood there are limits to what should be done that must never be crossed by anyone. Nothing really crazy happens, like building a mind so big it devours all other minds. We have distinct individuals finding peace with the universe as it is and working toward whatever happiness they can find.
The Lighthouse, Explained
The Lighthouse, Explained
The Lighthouse, Explained
You were born. You don’t know why, or if there was any particular reason why. It just happened. You wonder if there is a God of some kind, because while the universe is chaotic you can’t help but feel this was all set up somehow.
This is the experience of consciousness, or the extended present. You are using a very limited ability, your focus, to look into the future and expand your much more robust ability to store world models and project those into the future. You have to do this whenever you make a decision. The island is your body, hosting all of these internal mechanisms by which you navigate across time.
You don’t want to die. You also, at first, think you aren’t supposed to just be some person with a body who will die one day. Surely you were meant to live unbounded by the human condition.
As you grow older your world model expands. You’re able to project more and more complicated futures. But your focus, the magical thing that lets you do that, almost never gets any bigger. You just eventually learn enough things you focus on doing those to be productive. Eventually you stop doing new things and confuse that with having become fundamentally more intelligent.
You use your focus to navigate into futures where you will live. You focus first on basic living essentials.
Your reaction to your projection of the future changes the present and so changes the future. You are always dreaming about what the future will be like. This is the most useful way to think of free will, as opposed to worldviews which hold you don’t really have free will unless you can break the universe with a thought. You model the future and by modeling the future you change the future. That’s free will.
The things you can’t change are the futures that are most stable. That’s obvious, but by thinking of it this way you realize part of what causes stability are your own limitations. If you could change the shape of your island, or your lighthouse, or any of those essential pieces the future would become too chaotic to navigate. There would be almost no stable futures to plan around.
As smart as you are, you can’t focus on all the things you need to survive. Or at least you couldn’t do this in our ancestral environment. A human being by themselves is not enough to make it.
You can’t read someone else’s mind. So you have to guess what their guesses will be, and in turn guess how they will respond to your guess. Chains of suspicion have no end. To get around this you find points of common agreement and use them to construct a language, but this only helps you move through the fractal futures you both create. The fundamental benefit you get from other minds is that they’re not you, so you have to make guesses and challenge assumptions you would never have done otherwise. You don’t know everything that’s in another person’s head so you have to push into an infinite void to imagine the possibilities.
Specialization turns two people into one super-organism. Two people, or a very small group, can coordinate with each other almost perfectly to economically apply their focus to arrive at good futures. Focus is a scarce and precious resource. The two of you working together creates network effects because you have to hold the other person’s mind in your mind which forces you to be creative.
Civilization starts to expand and you get new and novel specialities.
Most people don’t think of it this way, but what you can directly sense sets the limits for what you can attempt to know because it makes more and different data available to you. You’re constantly trying to reconcile how the world works and make a few simple rules that will hold the whole thing together. Something as simple as a telescope in some sense makes you actually smarter just by giving you access to more data. Most of what you know, however, are what thoughts are non-productive. There’s simply too much to scan.
Self-reflection and education begin.
We discover the consistency of universal laws. Things that will always be true, apart from the material structure of reality. Rules which are always enforced, seemingly from a place beyond tampering. Once you understand this to be true, your ability to predict becomes enormous.
We carry with us the benefits of the dead. Their knowledge is transmitted to us across time, even before written language, simply through cultural artifacts and stories.
Computers begin, but because they’re not human they can’t want things or tell what things a a human will want. They have to follow simple rules but they can also do this much better than you can.
The singularity, which for various reasons I don’t think is possible in some of its more simplistic incarnations. I do believe in what I call Intelligence Eclipse, or the point at which a thinking machine can completely and totally overlay humanity’s maps of the future. Even then, there would be limits. Chaos is a fundamental part of the universe. Also, the game of life isn’t chess. You can establish new conventions that fundamentally change the rules of engagement.
Goddamn Elon makes a Neuralink. For the first time we can communicate mind to mind. And this is pretty bad for the most part.
You build up your brain to be your brain. You are the particular pattern of self-enforcing and self-policing heuristics acting upon the whole constellation of your experience. That is super particular to you. This is an example of chaos. You and I started with the same basic blueprint but then becomes of rigors in our environments, and the different futures we had to predict, we became wildly divergent. Except for low-level firmware type of stuff -we all seem to scan space in pretty much the same way and you can mind read this in people today- everything else is bespoke. To truly understand another person, you’d have to become that other person.
Sam Harris popularizes super simplistic notions of Free Will and God that are not helpful to anyone, anywhere on Earth. The people hearing it think that all of this stuff is new and wasn’t a problem the ancients had also considered before we knew the particulars of the material/spiritual paradigm problem. Or, if you’ve read a Boring Utopia, we act like machine operators who have been working a machine since forever and then suddenly we saw a gear and we somehow remove ourselves from the level of abstraction of the whole and think “there is no machine, there are only some gears.” We promptly get pretty bad at remembering the function of the whole.
Your mind, your specific set of rigors and accommodations to your unique existence are particular to you. No one can know what those are like internally anymore than you can know what it is like to be an earthworm. Every part of you was informed by the experiences that came before. You are a whole web of associations with everything that has ever happened to you in your entire existence.
Communicating mind to mind, at high bandwidth, does not cause two minds to communicate at high bandwidth. Communication between constituent elements is what you are made from. If you create a new highway for communication you are removing the barrier that kept you distinct from another person. Eventually, enough communication causes there to be one mind with two bodies. This is not a universal benefit. Being unable to read another person’s mind was a feature, not a bug. You now have no chain of suspicion to force you to be creative. You don’t have any need to model another modeler. There’s no other modeler anymore. There’s just a brand new you, that killed the old you, existing in two bodies. I don’t know if you could ever do this to an adult, but I bet you could do it to kids and for the record I think this is, outside some very rigorous contexts, pretty evil.
One of the limits of safe Neuralink is bandwidth. Another is only connecting the networked parts to purely physical phenomena. Having to form the same intent as speaking to be heard isn’t the same as going deeper down where you don’t even have to try to be understood. At the beginning, this won’t be a problem. The technology will be limited just because it’s new. Over time, though, this becomes a problem. I don’t think this could happen for various reasons you’ll read below, but imagine if everyone was networked together. Everyone in the whole world. Why would that super organism need to be conscious? Who would it talk to? What other mind does it have to consider? It already has a reservoir of knowledge to draw upon to safeguard itself so why would it have to become self-aware? You create your self in respond to the existence of the other. Now there is no other. Just an endless poly-bodies “you.”
This is one of the limits I think you’d find in linking people together. Some of us are too different and we’d create strong, competing signals about what the collective should do. Except we’d no longer be distinct individuals having discussions about trade offs. We’d be one thing trying to make up its mind. You’d cut off your arm to save the whole of you, wouldn’t you?
When you were a baby, every day of your life was a significant portion of your experience. As you got older, new days had to compete with the bulk of old days. At the age of two, a year was half of your life. Half of your lived experience. At ten, a year was ten percent of your life. At a hundred, only one percent. Now what happens when you suddenly ramp up your capacity to experience, so that every new day is suddenly the equivalent of years? Then you dedicate those new parts to trying to solve some difficult problem. Why, your entire life previous is barely anything at all! Only a distant memory, and the smallest portion of what you now hold. I think it fairly short order, unless you took very specific precautions, you’d find your humanity stripped away pretty quickly. You could try to suppress these things, force them to subconscious processing, but then you’d have great instincts not great understanding.
This is an Algorithmic Republic. You want to optimally find good futures with other people and then move everyone into them together. You want everyone focused on things that will make you all work together well as a whole.
This is the happy future, where we have integrated with technology without losing our humanity because we have understood there are limits to what should be done that must never be crossed by anyone. Nothing really crazy happens, like building a mind so big it devours all other minds. We have distinct individuals finding peace with the universe as it is and working toward whatever happiness they can find.