Pervasive computing aims to saturate ambient environments with sensors and processors; affective computing aims to infer emotive and intentional states from physiological readings and physical actions. The convergence of pervasive and affective computing offers genuine promise for creating ambient environments which (re-)configure themselves according to user’s emotive states. On the down-side, there is a real risk of privacy invasion if emotions, behaviours, and even intentions are recorded and subject to the same content-access rules as telephone calls, IP logs, and so on. Based on an experiment to enhance Quality of Experience (QoE) in a visit to a public collection enhanced with pervasive and affective computing, chapter 11 discuss the subtle interactions and requirements of enhanced service provision vis-à-vis privacy rights. The outcome will contribute to the discussion on ensuring an effective relationship between technologists and application developers on the one hand, and those concerned with privacy rights, ethical computing and formulation of social policy on the other, to promote and protect the rights and interests of citizens.