Since 50 years, the Common Agricultural Policy (CAP) has impacted the evolution of agricultural landscapes by driving changes in land use and farming practices in Europe. This viewpoint paper discusses to what extent the CAP does contribute to the management of EU agricultural landscapes. Agricultural landscapes are described as a multi-scale public good. We propose a typology characterizing the scales relevant for landscape management We argue that agricultural landscape planning requires actions on three scales: (1) the management of landscape elements at farm level, (2) the integration of fields and farms in the agricultural landscape at landscape level, and finally (3) the conservation of the diversity of agricultural landscapes in EU as a global public good. We provide evidence that the CAP has until now mainly focused on the first scale. We show how policy instruments could be refined for the CAP to integrate the two other scales In particular, we propose potential mechanisms to coordinate farm landscape management actions for a better integration of scale 1 and 2 (e.g agglomeration bonus or the setting of environmental cooperatives). We also describe how the signature of the European Landscape Convention by the European Commission could promote trans-frontier landscape cooperation in order, not only to conserve high-quality rural landscapes, but also to insure the conservation of EU landscapes diversity. This paper provides a knowledge base to support an effective CAP policy design in the direction of improved landscape management, as important component of the EU project towards a more sustainable agriculture.