To compress the deep neural networks, knowledge distillation is the form where knowledge is transferred to the smaller network (student) by training it on a transfer set produced by large network (teacher) [Hinton 2015]. The performance of teacher-student methodology was further improved by using multiple students instead of using single student network where each student tries to mimic the dense representation learnt by teacher network and tries to minimize the reconstruction loss [Teacher-Class Network 2020]. In the proposed methodology we used Generative Adversarial Network framework (GANs 2014.) which are very good at minimizing reconstruction loss, for transferring knowledge to multiple student networks. GANs simultaneously train two models: generator network G that trains on training data and mimic the dense representation produced by teacher network T, and discriminator model D that estimates the probability that a sample came from the real data distribution rather than G. In our case generator model G is the student network (S) and is penalized by discriminator model D to learn real distribution. We evaluated on several datasets i.e., MNIST, Fashion MNIST and Cifar10. Our approach has improved the accuracy and reduced the computation by compressing the network parameters.