Developing a systematic understanding of the attack surface of emergent networks, such as software defined networks (SDNs), is necessary and arguably the starting point toward making it more secure. Prior studies have largely relied on ad-hoc empirical methods to evaluate the security of various SDN elements from different perspectives. However, they have stopped short of converging on a systematic methodology or developing automated systems to rigorously test for security flaws in SDNs. Thus conducting security assessment of new SDN software remains a non-replicable and unregimented process. This paper makes the case for automating and standardizing the vulnerability identification process in SDNs. As a first step, we develop a penetration testing tool, POSEIDON, that reinstantiates published SDN attacks in diverse test environments. Furthermore, we enhance our tool with a fuzzing module to potentially detect other unknown vulnerabilities. In our evaluation, POSEIDON successfully reproduced 20 known attack scenarios, across diverse SDN controller environments, and also discovered 7 novel SDN application mislead attacks.