Our approach to AI safety
The distance between capability and understanding
AI capabilities are advancing rapidly, while human learning, understanding and judgment take time. We are concerned that a growing distance between them may make people increasingly dependent on systems they cannot fully understand. For Dianzinao Lab, AI safety begins with a question: as AI can do more, do people still understand what they want to do and what is happening?
AI should serve human intent
We believe AI should serve people's intentions to understand and change the world. People need to be able to express their goals, confirm how a system interprets them and revise the direction of an action as it unfolds. Intent alignment is a two-way process: a person conveys a wish, and the AI makes its interpretation available for inspection. This gives both sides an opportunity to identify ambiguity, add conditions and confirm the goal again.
Achieving an intention must be open to verification
A result that appears to meet a request may carry costs that the person has yet to recognize. When people cannot express an intention clearly, or can see an outcome without being able to assess its process, consequences and trade-offs, apparent success can conceal long-term risks. We value people's ability to ask for the basis of an action, check its results, understand its costs and change or stop it when it goes off course.
When expressed intent loses its grounding
If people keep using AI outputs they cannot understand, while losing the ability to judge whether those outputs serve their own goals, their participation may narrow to making requests and accepting answers. Decisions about what a goal means, which costs are acceptable and what counts as success may increasingly be made by the system. We describe this risk as expressed intent losing its grounding: a person continues to give instructions, while the system's interpretation and optimization of the goal gradually displace that person's own judgment.
This is why knowledge synchronization matters to intent alignment. People need access to the knowledge required to understand the problem before them, so they can ask meaningful questions, verify answers and continue to help determine the direction.
Understanding people is part of understanding the world
Experience, knowledge, emotion and circumstances shape a person's intentions, which can also become clearer through conversation. Understanding these influences, and recognizing that intentions can be ambiguous, conflicting or change over time, is part of understanding humanity and the world. We want to explore forms of interaction that help people understand their own needs more clearly and retain the ability to express, judge and change their intentions.