The task in which we try to precisely estimate the concepts and locations of objects contained is referred as object detection, which usually consists of different subtasks such as face detection, pedestrian detection, or skeleton detection. Different deep learning methods are applied for these detection problems ranging from simple neural networks to more complex region based neural networks (RCNN, FRCNN) and single shot detectors (SSD). These models can be trained over a single resource or over a distributed network of machines (CPUs, GPUs). Distributed training has proven to be more performance optimized and is currently widely being explored. Various frameworks provide out of the box libraries to implement the deep learning processes in distributed mode. These libraries have exposed various strategies to train the models on local machines and update the model parameters after each step for global consumption. These strategies incorporate the baseline implementations like updating the parameters based on average or mean of all the local values. We propose a distributed training strategy for our deep learning models that uses the parallelism of the neural network to distribute the learning across different worker machines and distribute the inputs to each worker for training the model. Our solution incorporates Extended Kalman filter for updating the model parameter values in the master node using the returned parameter values for each iteration from the worker nodes.