Laser feedback-based self-mixing interferometry (SMI) is a promising technique for displacement sensing. However, commercial deployment of such sensors is being held back due to reduced performance in case of variable optical feedback which invariably happens due to optical speckle encountered when sensing the motion of non-cooperative remote target surfaces. In this work, deep neural networks have been trained under variable optical feedback conditions so that interferometric fringe detection and corresponding displacement measurement can be achieved. We have also proposed a method for automatic labelling of SMI fringes under variable optical feedback to facilitate the generation of a large training dataset. Specifically, we have trained two deep neural network models, namely Yolov5 and EfficientDet, and analysed the performance of these networks on various experimental SMI signals acquired by using different laser-diode-based sensors operating under different noise and speckle conditions. The performance has been quantified in terms of fringe detection accuracy, signal to noise ratio, depth of modulation, and execution time parameters. The impact of network architecture on real-time sensing is also discussed.