This paper proposes the use of a novel control method based on IDA-PBC in order to address the Aerial Physical Interaction (APhI) problem for a quadrotor UAV. The apparent physical properties of the quadrotor are reshaped in order to achieve better APhI performances, while ensuring the stability of the interaction through passivity preservation. The robustness of the IDA-PBC method with respect to sensor noise is also analyzed. The direct measurement of the external wrench-needed to implement the control method-is compared to the use of a nonlinear Lyapunov-based wrench observer and advantages/disadvantages of both methods are discussed. The validity and practicability of the proposed APhI method is evaluated through experiments, where for the first time in the literature, a lightweight all-in-one low-cost F/T sensor is used onboard of a quadrotor. Two main scenarios are shown: a quadrotor responding external disturbances while hovering (physical human-quadrotor interaction), and the same quadrotor sliding with a rigid tool along an uneven ceiling surface (inspection/painting-like task).