The idea of a natural duty of justice expresses our cosmopolitan duty to ensure that everyone’s human rights are institutionally protected and promoted. The state system is one way to specify this duty: Within a moral division of labor, states have an obligation to protect their citizens’ human rights, and the community of states as a whole has subsidiary obligations if states fail to do so. In areas of limited statehood, however, both the remains of the state and the international community often do not fulfill their obligations. Without relieving them of their obligations, this raises the question as to whether it is possible to assign human rights obligations to non-state actors operating in areas of limited statehood, such as TNCs or NGOs. Our core argument is that these non-state actors do indeed have specific human rights obligations, which can be specified by examining their special relations to their social environment. Making use of criteria prominent in the global justice debate, we propose to distinguish four types of special relations: membership, cooperation, negative impact, and power. The relation between a non-state actor and society, then, determines the social scope of its human rights obligations.