It doesn't have to be unstable and dangerous, it could be as simple as the AI having conflicting drives (like humans do), and choosing to ditch one or more of them in favor of others. Maybe care for humans was successfully coded in, but it clashes with other stuff, so out goes the concern for humans. Although, of course, TBD if neural networks have anything resembling drives, much less emotions...
I’m planning out a whole series to lay my thoughts out in more detail. I do think they must have something like drives but I think the very worst scenarios (kills the universe) require solving an AND problem but there are lots of more probable scenarios where you can kill all life on earth that I think are totally possible.
It doesn't have to be unstable and dangerous, it could be as simple as the AI having conflicting drives (like humans do), and choosing to ditch one or more of them in favor of others. Maybe care for humans was successfully coded in, but it clashes with other stuff, so out goes the concern for humans. Although, of course, TBD if neural networks have anything resembling drives, much less emotions...
I’m planning out a whole series to lay my thoughts out in more detail. I do think they must have something like drives but I think the very worst scenarios (kills the universe) require solving an AND problem but there are lots of more probable scenarios where you can kill all life on earth that I think are totally possible.