The Amazing World of Freds Big Data

Hold this thought: There are real golden nuggets of data that many organisations are oblivious to. But first lets look at business process management.

Business process management, what is there not to like. It has revolutionised the way we organise and do business. Right?

Your business is applying, measuring and managing the process, right?

The process is king, its a great process, and your organisation loves it, right?

There is no process other than the known, approved, socialised and proven process, right?

Well, well see about that.

You may think that your best data is in the enterprise data warehouse (Bill or Ralph). I will not argue with that.

You may consider that the gold rush starts and ends in the exploitation of Big Data (what I like to call All Data). You may have a point, or not.

You may speculate that your best data may come from external social media sources? Well, thats a commonly held view, and who am I to challenge a commonly held view, for as erroneous and popularly acclaimed as it may be.

But, if you thought all this were true, would you be right?

Now, consider this

In companies, good, mediocre and bad, people follow processes, they follow these processes as best as they can until they work out how to do something better, faster, cheaper and easier. They may even skip process points completely or simply rearrange them, and in ways that business analysts never thought of. You see, processes can be clever, but people are really smart. But we knew that already.

People can be astute, honestly and innovatively so, and businesses function as well as they do, not only because of the assets, systems or the processes, or even due to the capital of the business, but because of the people who work for them, and conspire together to make things run effectively.

This fact now stated opens up the whole world of hidden data and knowledge, the real data gems of the business world. I mean business world in the widest sense of all the businesses all around the world, whether for profit, non-profit or public.

From the top to the bottom of the organisation, in the matrix, left to the right, there is a wealth of data that refuses to be digitised, evades formal capture and defies measurement.

In my precocious youth I used to refer to this data as the data that falls between the cracks, and would got weird looks from my management as a result.

My first encounter with data that falls between the cracks was with Freds Book.

Freds Book was the mother of parallel processes, and then some. In my youth I was asked to model an industrial process in a factory. My focus was on logistics, production and distribution. In the course of my investigations I discovered some glaring anomalies between the senior management views of the process and the actual interpretation of that process. I dug deeper. Formally, I was told nothing that really veered from the official business operating model. Of course, it was different story after a couple of beers on a Friday night.

It turned out there was a whole sub-process in play, and that it was all being coordinated through Freds Book, an almanac of all things related to the off-book business process, curated by Fred, a shop-steward and supervisor of one of the departments at the core of the process.

I interviewed Fred to find out more of the fascinating process that he was curating, but at the end of a long session he asked me not to reveal his book to senior management.

Freds book was the glue that held part of the business process together, it was the data, information and knowledge that kept things working in spite of inadequacies and contradictions in the formal process. But later I found out there were more books. There was Tommys book in distribution, Lilys book in production, Marias book in quality, and Harrys factory floor management cookbook. The whole business was running on an efficient and effective parallel process, documented in living notebooks (the paper kind), notebooks that underpinned and provided a workable framework for CRM, MIS and ERP.

Over the years I have encountered various artefacts of the Freds Book kind in many organisations. Not only that, sometimes Freds Book is virtual and codified, making it even less tangible to the digital data twitchers and information botherers. But it still has unmeasured value. We know it has value, but we might not be sure as to how much value. Which in general is more than can be said for faddish Big Data. The personal knowledge, experience and wisdom that people bring to bear in their working and private lives is one of the most valuable assets that a business potentially has access to, even if they dont know it or incapable of appreciating what they have and of then encouraging people to share and apply that knowledge and experience, wisely.

So, how do we ensure we dont lose Freds knowledge and experience? How do we stop the good sense from leaking out of organisations? How do we ensure that we dont throw out the knowledge baby and keep the Big Data bath water?

Driving business agility, competitiveness and strength requires far more than Data Warehousing, Big Data, Business Intelligence, Machine Learning, or Analytics and Data Science. It takes far more than supervision by numbers, shapes and colours; decisions based on sophisticated coin flipping; or, the voodoo analytics of the causation-free data bodgers. It needs the data, information and knowledge that resides in peoples heads and the wisdom needed to know when, where and how to apply that knowledge. Thats where the real golden nuggets are.

This is why I am so sceptical of the claims of Big Data gurus and other assorted charlatans, who seem to believe in their Homeric powers to move whole armies across the Alps with the click of a mouse, and just backed up with the invisible safety-net of the promised benefits accruable from exabytes of trashy social media feeds and bloated internet logs.

Historian Paul Kennedy, while acknowledging that no single variable can explain success, maintains the metanarrative that wars are ultimately won by a superior organization imbued with a culture of innovation that actively encourages inquiry, experimentation, and interdisciplinary problem solving.

This to me means that an organisation must focus on what is important, what it is good at, and what it has that is really important, and then apply it.

For many companies diverting the corporate attention from the important to the peripheral is a bad idea, that comes with many risks, and if that marginal and peripheral shiny-thing turns out to be large volumes, varieties and velocities of data with little or no value, then even more reason not to get caught up in the facile, superficial and peripheral issues of little or no value.

I like football, many people do. I support two teams, Real Madrid and Cordoba. When Real Madrid play FC Barcelona what preparations are made in advance? What is the focus of the preparation? Some people say that fan motivation helps the team to perform better, so maybe we should analyse the entire global fan base of Barcelona, Big Data style, in order to work out how to influence the influencers. But would I suggest such a tactic to any of the big clubs? Of course not, they would tell me to go away, and quite rightly so.

Adland legend Dave Trott tells a great story about former Leeds United football club manager Don Revie:

Don Revie had a dossier on all referees.
He used to make his teams study the referee for the upcoming game, so they could ask about his family, his hobbies, etc.
Most teams get 50% of decisions go their way.
Leeds used to get 80% of decisions go their way.

No Big Data Kool-Aid, no Data Science, no voodoo analytics, just good old fashioned knowledge, experience and street smarts.

Many thanks for reading.