The great communication mistake of the Security Software industry

In the last year I have had a chance to see things from the other side of the coin. Instead of a vendor centric world I have seen a bunch of things from the reseller and customer perspective. I have seen two sins, of which I was guilty of myself in a big way, that vendors commit s regular as clockwork.

1)      Renaming everything... again. Or brand management gone mad

Stop renaming everything! And then re-renaming it. Customers are not fooled when you rename a piece or crap, nor is a loyalty cemented after an acquisition (like say Big Red or Big Yellow swallowing up a smaller security company) by removing the traditional names. We as the audience are just aggravated that we need to research what the hell it is that you have just made harder for us to figure out. Not just the big security iconic companies are guilty of this, it is everywhere.

2)      We have heard this before. Here is a sin I have been guilty of myself (in spades). Every vendor likes to tell the story that they are the industry thought-leader with a feature that no one else has.  In the reseller and buyer community we hear everybody’s version of the pitch. Not only are the vendors not differentiating their product offering they are doing damage to their credibility by telling the same lies everybody else is.  Here are some recent examples:

a.       Blue Coat, Web Sense, McAfee, and Symantec all have huge splash pages on how they have the biggest sensor network in the world and can protect you best from zero day threats because they are so HUGE

b.      Mobile Iron, Symantec, McAfee and Good are all at the limits of the mobile device operating systems (Apple IOS and Android mostly) and can only do the same limited things in mobile device security.  All have the same limits and the same roadmaps.  No amount of motion-graphics on a power point changes this.

c.       I won’t name names on this one because there is some insider knowledge to it. Vendors move a product forward to versions like 9 or 10. They put a pretty new logo and marketing campaign around it. This does not fix antiquated Databases, nightmarish installers or shortcomings in performance. Only fixing the product does that.

Now I won’t say that this is all bad. Marketing pushes the boundaries of truth by its very nature. But when the most likely circumstance is the audience KNOWS the degree to which you are pushing the truth boundaries, this is failed marketing.

What we did back when I was in vendor-land was a formula like:

A.      Cost or repackaging and remarketing.

B.      Cost of actually bringing a product to the features it claims to do

C.      Revenue gained from sales

A is always cheaper and Quicker than B. If C is higher than A then continue with A and never go to B. The answer is never actually fix the issues.

What about factor D?

D.      Company’s reputation and customer satisfaction.

What if we looked at the post sales satisfaction of customers? What if we factor in which resellers will never sell the product again or will do business with another vendor because of the pig-in-lipstick they bought?

Here is a novel approach brothers and sisters! Our customers and partners are intelligent mature human beings and are just looking to know what is being sold to them. They will remember who lied last time and will tell their friends. Sales by actually selling what you have to sell!

When upgrading to the next version is a downgrade or a train wreck

I know, with that title everyone thinks of Windows Vista. Although that is a good example of one of the points I want to make I don't think it is even the most severe. I think that vendors make two consistent mistakes in the continual cycle of upgrades. We as customers have come off the manic edge of enabling both issues but a lot of us are still bad about upgrading for upgrades sake.

Vendors lose sight of what their clients are looking for.

Yes this is the Vista one.  The same mistake was also made by Goldmine, Doctor Solomon’s Software McAfee, Adobe and nearly every other vendor at one time or another.  Maybe product management has a vision and has not done research. Maybe marketing did their research with myopic vision. Sometimes a vendor is trying to be a thought leader by adding or redesigning features and we as a consumers chose not to follow.

This variety of version change that degrades (or confuses) functionality is excusable if not laudable.  I think the quintessential explanation for this is the 1958 Edsel.  This was a good car despite some of the reputation it got. It was just not what consumers wanted. When, as a society, we were on the cusp of the 60s and everyone wanted their first VW beetle. Ford put out a big roomy monster with Tires the size of a bathtub.  And when the buying public chose to not buy the big Ford (here is the big Vista comparison) set out in an ad campaign to let us know how misinformed we were and our expectations were askew, that Edsel was a great car! We just needed to wake up to its greatness.

In the same vein Doctor Solomon’s Software made a similar mistake in 1994. The Product Management Team decided that Microsoft Exchange and Lotus notes are just a fad and mail clients like that will never catch on. SMTP anti-virus is what the world will want. The then they spent the next year trying to talk the software consuming world that they don’t want MS Exchange! Doctor Solomon’s tried to upgrade their security suite by driving an upgrade product where no one wants to go.

The severe "bad upgrade" however is when a vendor knows they are pushing a dead horse a mile farther. 

Sometimes there are  game changer technology changes like Wifi or web-apps. Sometimes hardware has improved to such a degree a products traditional architecture is absurd and surreal. (I used the NEW version of an app literally in the last 3 weeks that requires separate Data base server, management server, web server, communication server, data collector and load balancer for even the 1^st blush of functionality all physical servers!). Sometimes a vendor’s crown jewel functionality has just gone past their useful time. How would you like to own Qmem or Gramatic 5?

In the same way I can understand  vendors trying to lead or being misled by bad marketing I find those that are driving a product too far after the KNOW it is past its useful maximum I don’t.  It often crosses the line between being a bit delusional OR badly informed and plain unvarnished greed.

This leads into the sticky discussion of brand loyalties.  If a product comes out from a favored vendor it must be good, right? Off the top of my head I can’t think of any  vendor that hasn’t put out a pig in a poke at least once in their run.

So if we avoid blind loyalty, and are skeptical of suite pricing, maybe we can avoid both types of bad upgrades.

Suite packaged security can force feed us bad ideas and awful distractions

Once upon a time we picked products that were best of breed. We bought Word from MS and Lotus 123 because each individual point product was the best at what it did. That made sense. This was followed by the paradigm shift to best of suite. The idea of best of suite was we paid less for an aggregate from one vendor the all the best pieces from the best point products. With best of suite we need to just make sure that each piece was not necessarily the best but “good enough”.

This makes sense in a lot of areas. Especially in operations like the productivity example above. We have long since gotten into buying security by best of suite and by vendor relationship. This leads to a question: how much product fat, crap ware, and vendor agenda do we have built in with this deal?

So minus the vitriol I have looked at a number of these suites and I wonder if we are doing ourselves harm

Free can be very expensive. Both at security vendors I have worked with and at competitors out in the industry we have seen the Ronco strategy. Do you remember Ronco? When you buy the salad shooter we will throw in a set of steak knives, a vegetable peeler and this (99.99 value) picture of the founder!

Each time our security vendors and consultants bring in ‘value add’ products and functions, even for no cost, they are a distraction. A bad vulnerability scanner is worse than no vulnerability scanner. It causes us to stop looking for flaws.  In Security, very differently from productivity software, Mediocre does harm. Time spent in near-futile efforts of DLP and NAC causes us to use our best resources in a futile fashion. Time lost and productivity lost in a manner very similar to a penetrations  in cost. We can spend forever just trying to get where we started. We disrupt the user environment, at least minimally, every time we add a new security tool.

Perhaps more insidious than the distraction of suites with pointless bells and whistles is the vendor agenda of the suites. We have previously  discussed here why signature AV (and IPS if you really get down to it) are nearing the end of their useful life. Firewalls as we once knew them are near obsolete. Each vendor is presenting not only their products in the best possible light. They are projecting their ideals of the best way to secure our environments. What happens when vendor’s ideas get obsolete? Without slamming the security vendors of yester year we can all think of products that became bad investments. When we are buying Suites of security products we by necessary corollary buying the concepts and methods that go with them. It is not unreasonable to think a vendor would push their core ideology past its sell-by date as that strategy is their core money maker.

We have been adding of successive layers of security for a generation. Is it time to throw some of them away? Maybe the distractions and man hours of worn out tools are doing us more harm than good? Maybe, just maybe we are at the point where taking a deep breath and redesigning our data security environment and it will be easier to use and less extensive than patching the cludge we have yet again?

 Is our best of suite data security model the analogy of the 1990 ford Winstar that we have been driving for years and is finally prohibitively expensive to repair one more time and it is time to finally by a new car?

I don’t know… but if it were my mission critical data I would be sure to find out.

Custom requirements: Making vendors do things our way. Is it doing is harm?

Here is one of the great truisms of data security. It is like the weather joke “you know what they say about the weather here in (insert city name here) if you don’t like it wait 5 min and it will change!” They say it in every city I have ever been to. There is the equal social narcissism of “Here at (insert company name) we can break anything! We have the most complex software environment I have ever seen”

This perspective is kicking our tails

Back in the old days we often needed to push vendors and their products to meet our Business needs. In 1994 I worked at a company called Dr. Solomon’s Software. We had a heck of a time finding a CRM software. It needed to work well with our systems, could capture the data we needed, and we had to be able to get the data out again in a form we could use. After selecting one (no I won’t say which one) we had a DBA on staff for months after tuning and tweaking it. There were times when inputting data (I was tier II support in those days) it would freeze or fields would be un-selectable.

I think any of us who came through the 90s had moments like that with one technology or another. We had to use (and sometimes even pay for) software that felt terribly beta. I admit to working for a Company President at a slightly later company that said often: Ship it as is! F#$% it! We can fix it in a service pack. We need to get this out there.

As is the theme here, times have changed. Products and developers have been given the pest part of two decades to get it right. The metaphorical service pack has long sense shipped.

 I was with a friend at a major airline where she worked in the IT department. They were looking at a number of upgrades, migrations and a few places that they wanted to change vendors. It was real day-in-the-life of the IT department stuff. We came then to the part about custom requirements. There was an ocean of tiny custom needs that the airline had. Each software vendor had to meet them all make final consideration.  Those site specific things proved to be the bulk of the evaluation time and would be a major amount of the implementation costs

 We had a long look at that list of custom requirements. When we took a deep breath and considered most of them against the truly refined nature of the modern software offering they just did not matter. Small changes in process ( they are sometimes politically difficult I grant) could be made that would actually make the software run better, cost us less, and make our jobs easier

This led to a long conversation on the evolution of custom requirements. What methods did we decide on in the early 2000s that have become superseded by better software or methodology like VM and cloud computing?  What methods to we need to adjust our culture to using better?

To take this argument to data security from more general IT I have seen many clients that had an incident between 5 and 10 years ago they are still over reacting to. I have had to talk down 2^nd and 3^rd generation security directors from their institutionalized avoidance strategy of threats that haven’t been viable for years.  To make things worse the Captain Ahab like obsession with the “last war” often leaves us vulnerable to threats that escape our notice due to over-focus.

The Vendor and solution provider community have gotten pretty darn good. I am not saying to blindly trust that vendors have a better way. That would be equally insane to ignoring them.  Vendors are as self-serving as the rest of us. Often they have barely seen the client’s environment and don’t want to work harder than they need to. I am saying any time we as users of IT technology, especially security technology, see the majority of solution providers disagreeing with our doctrine we need to reconsider our doctrine.

Stopping Change Control from becoming Change Prevention.

It seems to be some kind of rule of human nature. It is fear of change meets the Iron Law of Bureaucracy (Every task will always expand to fill the time available) Our Change Control efforts that were once required so the light-speed changes in IT technology didn’t run over our business. How many times has it evolved to become a millstone around our collective necks? How much have we emotionally invested in this control?

On this particular topic I think we need to start with a definition. Taking that from Wikipedia:

Change Control is a formal process used to ensure that changes to a product or system are introduced in a controlled and coordinated manner. It reduces the possibility that unnecessary changes will be introduced to a system without forethought, introducing faults into the system or undoing changes made by other users of software. The goals of a change control procedure usually include minimal disruption to services, reduction in back-out activities, and cost-effective utilization of resources involved in implementing change.

It seems to me that there are two warring factors in terms of change control (aside from the Iron Law of Bureaucracy VS. high speed change argument). For purposes of argument I will call it the “Works fine” vs. “Really cool” argument.

If proof was needed proof that the pace of Traditional IT change has slowed we only need to look at Windows XP. Its successor, Vista, was largely rejected by industry and as a generality we use our old OS’s for 7 years. Because it worked fine. There was no compelling reason to upgrade until Win 7 came along and even then we were stately about the upgrade.

 We could look at our mail systems, they work just fine.  Everything is incremental organizational improvements. The calendar, messaging system and all the rest have been good enough for years. CRM, ERP, GRC, a lot of the traditional systems work fine.

Our users are human beings though. We are constantly being marketed to by really cool new toys.  The new form factors (smart phones and tablets) and new media Twitter, Google +, Facebook and IM (New Communication methods) are in saturation mode in their push on us.  Tethering off the Android and the IPhone makes every user at every desk their Own ISP, going right past our expensive firewalls and IPS. And all these new technologies are so cool!

So the Change Control problem is this: We have developed an organization and an industry that is oriented at looking at better ways to do our traditional activities. It works at a stately speed to do a thorough job. It is perfectly designed and organized to look at when to change the web browser version a year or two after they come out, but not at all on how to control data across smart phones.

For data security managers (or military officers) there is a great rule I learned in there early days as a restaurant manager: never give an order you know won’t be followed. If we tell our users they can’t have mail on their Android they will do it anyway. If we try and implement draconian controls we will make enemies of our users and they will end run around us.  By doing this we have made ourselves ineffective. Our user will work around our normal and reasonable controls as well as our knee-jerk slowness to adapt. As soon as our employees decide to that the IT department is the enemy everything we do is for nothing.

Change control

Data Security and Change Control.  Maybe we need to go back a bit to our 1995 way of doing things and unclench our fists on some things that have become our purview. I would like to offer a revolutionary thought:

As long as, as an enterprise, we control the data nothing else matters. By use of PKI, VDI, Whitelisting, or the similar if we hold control over proprietary company data (and not try to control an employee’s posting to their kids on FB). We don’t need to care at all about the methods that are used to handle that data, as employees treat that data with respect. Data Security of our precious assets does not need to be about controlling the user’s whole data life

Change control as a minimalist concept base on the use and control of data. Not applications, not operating systems.  This can be accomplished in a way that allows the cool, secures the data, and doesn’t break the bank. There are those that will say that unless we manage for every vulnerability in the system we can’t control the data. In 2011 this is no longer true.

·         Application whitelisting ( hell yes)

·         VDI. All  work data saved to the SAN

·         Network IPS

·         Simple forms of DLP like string matching (because if an employee wants to circumvent even really good DLP they will)

Control the data not the asset

We can (and do) argue the technology. The boil down is this:  by backing off our emotional investment in hard control of method and mechanism we can have a better work environment for less money and effort.

We need to be a partner not a slave to our auditors.

Compliance drives a large portion of our IT spend and our IT efforts. In 2011 it is just a fact of life. We were all scared to death back during the Enron/Anderson thing and by the tide of regulations that followed. Serious enough SOX violations can mean going to jail. Beyond that the PCI fines that can be levied on a company for violations can (in theory) reach 250,000 per occurrence.  We need to take all these regulations deadly serious… but here is where we run into the disconnect.

I have met very few security or administration professionals that have actually read the regulations they are complying with, I mean really read them.  Most overworked and under resourced security administrators just read the auditors requirements.

The Entire PCI DSS requirements doc is under 25 pages and they are low density pages. GLBA is less than a page.  Granted some of the others (Notably FISMA) are a lot more comprehensive. Most of what the large body of regulatory requirements asks for is near identical to what an average CISSP would design anyway. In retrospect I think the fusillade of regulations caused us to panic. No one wanted to be seen as not addressing the problem so we over reacted badly. This over reaction gave us two significant problems.

1)      Compliance drives Security, not the other way around.

2)      In our confusion and frustration we hire an auditor and believe everything they say

Neither of these seems to be particularly bad at first glance, A bit concerning maybe but not awful. After digging in a bit it grows more problematic.

Regulations will always be trailing edge. We can’t regulate something until we know about it. We don’t regulate something until there is a problem with it. So by the time a threat vector becomes subject to regulation it is already full blown and pervasive. By letting compliance drive security we are in a perpetual game of catch up with and always letting the malefactors have a head start.

The decisions to follow out auditors recommendations should fix this. They are IT Security professionals and have the time and intensity to spend on creating a proactive threat mitigation program. In a perfect world it would work just like that. In practice auditors are just like every other business in the world: they are all about maximizing their profits. This maximizes the amount of work they recommend and the amount of paper they produce.

Do not mistake me for being anti-audit or anti auditor. I am not. What I am against the abdication of though and responsibility that is becoming standard.

Our Auditors must become our partners in compliance and security, not our masters. We, as IT professionals, need to read our Compliance standards, understand them, and question the plans for compliance. 

 We need to push back against excessive controls that do not actual address the issues. In my time with clients in the field I have seen zealiotious adherence to really odd controls under the blanket logic of “it is required for PCI” or something similar. The compliance standards are invoked like magic words. If I then pull out my copy of the PCI DSS and show that whatever truly odd control is exhibited that day is not in the standards at all I am greeted by an awkward silence. I have seen equally odd reasons for gaping security holes: “We can’t fix that, we have to do it that way for compliance”.

Neither Security nor Compliance is rocket science. We may need to stab some sacred cows and may have to change a few business processes so we aren’t fighting ourselves. Above all we must remember that we as the IT Security industry have a responsibility to design and implement the best solutions we can. Neither Audit nor compliance changes this basic fact.