Spintronics and Nanomagnetism offer a new path for neuromorphic computing at the interface between physics and engineering. In the brain, memory (synapses) is provided in tremendous amounts and totally entangled with processing (provided by neurons). Thanks to the intrinsic non-volatility of nanomagnets, and their compatibility with CMOS processes, spintronics can bring massive embedded memory to unconventional circuits, thus escaping the Von-Neumann bottleneck. In addition, spintronics is a playground to investigate and generate novel physical effects that can be leveraged for brain-inspired computing, such as tunable fast non-linear dynamics, controlled stochasticity and multifunctionality. We can then envisage assembling large networks of interacting spintronic nano-objects and tuning their interactions to induce complex dynamics such as convergence to multiple metastable states, synchronization, chaos, soliton diffusions, phase transitions and criticality. By studying theoretically and experimentally the dynamical response of these networks, it can be adjusted to generate specific local and global dynamics when a specific input is presented, allowing instantaneous pattern recognition and classification. Such interdisciplinary studies will demonstrate that the impact of nanomagnetism and spintronics extends beyond the traditional memory markets to embedded low power chips that can learn to perform complex cognitive tasks with key applications to medicine, unmanned vehicles, robotics and big data in general.