Answer
As a background, please read our articles on “Who were the sons of God in Genesis 6:1-4?”, and “Who were the Nephilim?” With the understanding that the sons of God were the fallen angels, and that the Nephilim were the hybrid offspring of the union between the fallen angels and human women, the question then arises: What happened to the spirits of the Nephilim after they were killed, whether by the flood, or in the case of the possible post-flood Nephilim (Genesis 6:4; Numbers 13:33), after the flood?
Some speculate that the disembodied spirits of the Nephilim remained on the earth and became what we now refer to as demons. The presumption is that, as angelic-human hybrids, the spirits of the Nephilim would have been different from the human soul-spirit, having the ability to remain present in this world despite no longer having a physical body. This would possibly explain the desire the demons have to possess human beings, thus gaining control over a physical body. This would also make some sense from the perspective of the fallen angels, who are outnumbered 2-1 by the holy angels (see Revelation 12:4), giving them a good reason to seek to increase their ranks.
The Nephilim explanation for the origin of the demons is partly the result of a misunderstanding of who exactly are the “spirits in prison” in 1 Peter 3:19 (see also Jude 6). Many misunderstand the “spirits in prison” to be all of the fallen angels who rebelled against God. If all of the fallen angels are imprisoned, then there must be an alternate explanation for the existence of demons; thus, the need for the Nephilim explanation. However, clearly, not all of the fallen angels are imprisoned. Satan, the leader of the angelic rebellion against God, is not imprisoned. Why would God allow the rebel leader to remain free but then confine the angels who followed Satan in the rebellion? No, it makes more sense to understand the “spirits in prison” as the fallen angels who participated in an additional rebellion, viz., the sons-of-God/daughters–of-men incident. The fallen angels who mated with human females are the ones who are imprisoned. There is no solid biblical reason to reject the idea that the demons are the same beings as the fallen angels.
The idea that the demons are the disembodied spirits of the Nephilim is also drawn from the book of Enoch, which goes into great detail regarding the Nephilim. We have to remember that, while the book of Enoch contains some truth (Jude 14), it is not the inspired, inerrant, and authoritative Word of God. We should never base a belief exclusively, or even primarily, on extra-biblical literature. So, with no need to explain the existence of demons outside of the fallen angels, and with no clear evidence in Scripture for the spirits of the Nephilim continuing on Earth, there is no solid basis on which to identify the demons with the spirits of the Nephilim. While the idea is possible, it cannot be derived explicitly from Scripture, and therefore should not be considered the best explanation of the origin of the demons.
