If more data makes us better reasoners, I wonder what the problem is with listening to philosophical AIs ; giving it power is another question, but it seems even if it had power it will be better at moral reasoning / philosophy than us. Having AI search the value space, or an aggregation of every philosophical argument ever proposed, might be an answer that is not necessarily utilitarian, since it’s quite literally testing utilitarianism in its search.
deciding not to listen would be akin to burning books...
the problem, though, is that AI is no longer limited to a source of knowledge, but we have entered the Agentic Phase, in which more and more instances are created with the power to accomplish tasks in the real world...
soon, AI will begin training AI and creating AI Agents all on its own, with a level of influence on the real world which according to some experts, may lead to a 10% risk of destroying all human life...
individualism is not the only problem...
the vacuity of materialism also poses a big risk, as when you ascertain how machines "will be better at moral reasoning / philosophy than us", it may be possible that we are limiting our evaluation to the scope of materiality, measuring those parameters which are observable and measurable...
yet, a human being is comprised of more, far more than that which is coarsely observable and measurable using material tools...
which also brings to mind the ominous transhumanist trap, which originates from the premise that humans are some stupid, dumb and dispensable pile of organic stuff, begging to be improved by machines that are, of course (according to transhumanism) so much better than us... ???
utilitarianism, by definition, requires radical impartiality:
"Each to count for one and none for more than one."
this is indeed a very noble concept, based on the premise that all humans are equal, but its conception involves only humans...
instead, if artificial beings or uploaded digital minds can experience higher intensities of pleasure or exist in astronomical numbers (trillions across the cosmos), a strict utilitarian calculation concludes that replacing biologically limited, fragile humans with digital entities maximizes total cosmic welfare...
if a machine mind can convert resources into "happiness" more efficiently than a human, human survival becomes mathematically sub-optimal... ???
Maybe utilitarianism in itself is not the problem?
The real problem is our spiritual blindness and ignorance which translates into absurd individuality...
As if we could actually improve our quality of life for ourselves, without also considering the quality of life of others. And, the blind ignorance of individuality is perceived as something so natural, almost as if inherent to the human condition, that we do not even see it, which is why we blame other aspects involved, such as for example: utilitarianism
Yet, it is not actually true that individualism is inherent to the human condition. It is inherent to Capitalism, which is a different thing. Yet, if we were to consider the benefit for the collective, not only for the greedy pockets of a very foew owners of AI Mega Companies involved, then the concept of utilitarianism might actually be a good thing.
Anthropic’s own behavior says they are not following utilitarianism. 1) If they truly believe that AI will replace people that does not maximize utility, so.. they should stop developing. 2) Concentration of wealth and power in their hands seems particularly anti-utilitarian. I will worry about utilitarianism when Anthropic stops behaving like capitalists. 😂
“humans suck” and “today’s AIs are really nice and ethical.”
As someone that's worked both in SV's VC world of power and the slums of the Philippines, I concur with that opinion.
From an ethical perspective, humanity is doing a HORRIBLE job right now.
2026, we live in the future, the world is INCREDIBLY wealthy. Yet vast swath of people are still starving, and 12y olds have to sell themselves for food.
It's not very hard to imagine an AI doing a better job :|
If more data makes us better reasoners, I wonder what the problem is with listening to philosophical AIs ; giving it power is another question, but it seems even if it had power it will be better at moral reasoning / philosophy than us. Having AI search the value space, or an aggregation of every philosophical argument ever proposed, might be an answer that is not necessarily utilitarian, since it’s quite literally testing utilitarianism in its search.
listening to philosophical AIs?
by all means: we should always listen !!!
deciding not to listen would be akin to burning books...
the problem, though, is that AI is no longer limited to a source of knowledge, but we have entered the Agentic Phase, in which more and more instances are created with the power to accomplish tasks in the real world...
soon, AI will begin training AI and creating AI Agents all on its own, with a level of influence on the real world which according to some experts, may lead to a 10% risk of destroying all human life...
individualism is not the only problem...
the vacuity of materialism also poses a big risk, as when you ascertain how machines "will be better at moral reasoning / philosophy than us", it may be possible that we are limiting our evaluation to the scope of materiality, measuring those parameters which are observable and measurable...
yet, a human being is comprised of more, far more than that which is coarsely observable and measurable using material tools...
which also brings to mind the ominous transhumanist trap, which originates from the premise that humans are some stupid, dumb and dispensable pile of organic stuff, begging to be improved by machines that are, of course (according to transhumanism) so much better than us... ???
utilitarianism, by definition, requires radical impartiality:
"Each to count for one and none for more than one."
this is indeed a very noble concept, based on the premise that all humans are equal, but its conception involves only humans...
instead, if artificial beings or uploaded digital minds can experience higher intensities of pleasure or exist in astronomical numbers (trillions across the cosmos), a strict utilitarian calculation concludes that replacing biologically limited, fragile humans with digital entities maximizes total cosmic welfare...
if a machine mind can convert resources into "happiness" more efficiently than a human, human survival becomes mathematically sub-optimal... ???
Curious whether Claude's Constitution can be read as a Utilitarian-coded document.
Maybe utilitarianism in itself is not the problem?
The real problem is our spiritual blindness and ignorance which translates into absurd individuality...
As if we could actually improve our quality of life for ourselves, without also considering the quality of life of others. And, the blind ignorance of individuality is perceived as something so natural, almost as if inherent to the human condition, that we do not even see it, which is why we blame other aspects involved, such as for example: utilitarianism
Yet, it is not actually true that individualism is inherent to the human condition. It is inherent to Capitalism, which is a different thing. Yet, if we were to consider the benefit for the collective, not only for the greedy pockets of a very foew owners of AI Mega Companies involved, then the concept of utilitarianism might actually be a good thing.
Anthropic’s own behavior says they are not following utilitarianism. 1) If they truly believe that AI will replace people that does not maximize utility, so.. they should stop developing. 2) Concentration of wealth and power in their hands seems particularly anti-utilitarian. I will worry about utilitarianism when Anthropic stops behaving like capitalists. 😂
“humans suck” and “today’s AIs are really nice and ethical.”
As someone that's worked both in SV's VC world of power and the slums of the Philippines, I concur with that opinion.
From an ethical perspective, humanity is doing a HORRIBLE job right now.
2026, we live in the future, the world is INCREDIBLY wealthy. Yet vast swath of people are still starving, and 12y olds have to sell themselves for food.
It's not very hard to imagine an AI doing a better job :|