Colin Percival had this to say on security:
Schneier suggests that this “particular way of looking at the world” is very difficult to train — far more difficult than the domain expertise relevant to security. I respectfully differ: In my opinion, this mindset is not particular to security professionals; and universities have been successfully training people to hold this mindset for centuries.
…
the entire mid-term examination consisted of writing proofs; and a proof isn’t correct unless it considers all possible cases. Forgot to prove that a limit exists before computing what it must be? Your proof is wrong. Assumed that your continuous function was uniformly continuous? Your proof is wrong. Jumped from having proven that a function is continuous to assuming that it is differentiable? Your proof is wrong. Made even the slightest unwarranted assumption, even if what you ended up thinking that you had proved was true? Sorry, your proof is wrong.
…
More importantly than this, however, is that the sort of edge cases which mathematicians are trained to think about in writing a proof are exactly the sort which cause most security issues
Colin’s view is that the security mindset is one where you should be thinking with a analytical (in a mathematical sense) mindset when truly considering security. This is not something that is that simple to do on a continuous basis.
My understanding that I took away from the article is that one should expend a lot of effort upfront building up a proof that the security of your system is secure and implement it following the proof. This is definitely an ideal that one should work towards when trying to apply security. However, I feel some things in the ‘real world’ are so fuzzy that this can’t apply to all facets of security evenly. What to do then?
When it comes to writing software this might be more tractable. However, at the systems level where it usually means integrating a set of systems together how does that work? I guess choose completely secure components and make sure they are configured to work in a secure manner when communicating with one another when you have the choice is a good one. How about when you don’t have that choice?
