While working on my wireless node module, I had to determine how to tell if a network is a WEP, WPA, or WPA2 encrypted network based on the results of a iwlist wlan0 scan command. It turns out the hardest part of this was procuring a wireless network with WEP encryption enabled.
Basically, if the network has encryption enabled, we assume that the network uses WEP. Then, we look for a certain line which contains the WPA2 signature, and if that doesn't exist, we look for a line containing the WPA signature. Technically it is a little bit more complex than this; some networks have WEP, WPA, WPA2 & WPA, or just WPA2, but I don't need that much detail (yet).
Check out the relevant code in the GitHub Commit.