Alignment (1 blogmarks)

The following excerpt is from An Alien Mind, a blog post from OpenAI’s Chief Scientist, Jakub Pachocki:

AI is grown more than designed - it is, to first degree, the product of repeating a straightforward optimization step many times on a hard-to-imagine amount of compute. This results in an incredibly complex system that works through abstract concepts and can simulate facets of human behavior. We can discover various insights about little mechanisms that emerge within this system, in a process similar to neuroscience - and, similarly to neuroscience, its overall action evades a description we can fully understand.

Goal Alignment versus Value Alignment:

Goal alignment is broadly: “does the AI try to accomplish the goal set before it?”... Value alignment is a more intrinsic property of the model. It is the ability to hold and generalize from a high-level set of principles; to act “reasonably” even when given unclear or conflicting objectives, or placed in unfamiliar or adversarial situations.