A maturity model for Pattern Based Strategy

The model outlined in Table 1 is a useful thinking framework for assessing Pattern Based Strategy maturity within organisations.

Maturity models (misused) can become Anti Patterns, use them to understand the ‘as is’ position and complexities that would prevent (or make difficult) ascent to the next maturity state. Avoid maturity ascent becoming a ‘means in itself’ and always link back to business strategy and business goals. As part of Pattern Based Strategy implementations, consider the cultural and organisational dimensions involved and potential for creation of a specialised function (such as a centre of excellence for Pattern Based Strategy).

Table 1: Pattern Based Strategy (PBS) Maturity Model – [source Steve Nimmons]

A pattern is a “consistent and recurring characteristic or trait that helps in the identification of a phenomenon or problem, and serves as an indicator or model for predicting its future behaviour”.

An effective pattern detection and management capability has significant importance in several areas (abridged list for illustrative purposes):

• Early detection of natural disaster patterns – creation of earthquake probability maps has become a commercial business in the last two decades. Customers are not only governments (national security agenda) but also companies using the maps for corporate and industrial site selection. In these maps a number of patterns are combined: historical (frequency and magnitude of previous quakes), geographical (plate tectonics and movements) and social dimensions (number of inhabitants, demographics etc.)
• Diagnostic medicine – using pattern recognition techniques in medical imaging (and other predictive diagnostics)
• Command and Control Functions for Complex Systems - used to monitor the behaviour of complex systems (such as nuclear plants, power networks, ships, submarines), analysing the input received from different sensors and providing decision support functions to the operator (or potentially executing the actions automatically) with a goal of ensuring that the complex system is maintained in a desired /stable state. In the design of Command and Control Systems, two models should be considered:
  • Frequently, the situation/problem identified by a pattern is a well known/repetitive problem and there is a predefined procedure to manage the situation, either automatically or manually through an operator
  • There are other situations where a predefined procedure is not available (perhaps the evolution of the system cannot be predicted). In these circumstances, operator intervention is needed and the Command and Control System should help by providing all the information available about potential solutions/actions, estimations about the evolution of the system, historical analysis, precedents, etc.

It is critical for Command and Control systems to be as “intelligent” as possible, capable of reacting to multiple scenarios whether previously encountered or not. To achieve this, several techniques are used including simulations and neural network based design.

• Context Aware Computing – contextual information often underpins the key events which define the pattern(s). This is closely liked to Complex Event Processing which can be seen as a technology building block of PBS and Context Aware Computing solutions (and indeed is also a potential input source for Command and Control Systems operating under PBS).

In a commercial context, the early detection of patterns highlighting opportunities, collapsing demand, employee dissatisfaction or negative public image can provide companies with a sustainable competitive advantage and can help them to capitalise on opportunities that exist under varying market conditions.

Some examples of the application of pattern management in business environments include:

• Fraud/Crime detection – For example, monitoring for unusual credit/debit card transactions, or purchasing patterns outside normal parameters for an individual customer
• Recommendation engines in e-commerce sites. Creating patterns from previous user navigations in such a way that can identify affinity between different users and so, propose items acquired by those previous similar users)
• Intelligent content providers that use patterns to identify and classify content coming from different sources that match subscribers’ areas of interest.

Analysing trends, patterns and external developments has always been a part of organisational business intelligence. The baker in Adam Smith’s “Wealth of Nations” (1776) wanted insight into predicted next day sales, ‘event intelligence’ and likely impact on demand.

In the current business environment, reasons to place predictive business intelligence and pattern seeking on the agenda of the CxO include:

• Customers are connected with and active on a lot of social (web) networks on which they discuss the performance and image of organisations as well as potential purchases
• Determining and analysing patterns relating to how organisations are perceived in these networks uncovers corporate image, customer sentiment and may be used to forecast future sales and highlight opportunities and commercial threats
• New and innovative IT solutions make it possible to concurrently data mine more sources rapidly and to find and analyse patterns inherent within the data. Availability of increasing numbers of public data sets also increases analytics potential and patterns can be hypothesised and tested across a rich data landscape
• Following more deliberate and intended strategy based on detected patterns fosters stability and confidence in an organisation. If patterns can be detected quickly and the company reacts with agility, there are opportunities to drive markets through innovation, acquisition and new products.

Fast and accurate pattern detection is also central to a dynamic risk management strategy (which would have obvious relevance in natural disasters, diagnostic medicine and Command and Control Systems as discussed above).

Figure 1 A three-cog model for Pattern Based Strategy

[source: Steve Nimmons]

Pattern Based Strategy

Pattern Based Strategy is as full of complexity as it is of promise. The challenge is the efficient and structured determination of:

1. Area of focus (i.e. what to look for and why)
2. Content and event sources (i.e. where to look and why)
3. Mechanics of pattern and signal detection
4. Reaction to signal and pattern detection
5. Refinement of hypotheses, patterns, signals (i.e. approach to continuous improvement)

Evangelists of Pattern Based Strategy focus on Pattern Detection technologies and say little of the mechanics behind determination of area of focus, content and event sources, reaction to detected patterns (be they weak or strong) and reaction to contradiction.

The ‘three cog model’ in Figure 1 aims to address these deficiencies.

PEST Analysis is suggested as a strategic thinking framework to focus on Political, Economic, Social and Technological forces (be they opportunities or threats).

To help reduce complexity and filter irrelevant forces, a VPEC-T frame (Figure 3) surrounds the PEST analysis activity.

VPEC-T

VPEC-T analysis is a thinking framework comprising a collection of mental filters or guides. It provides a “simplified ‘language’ for preventing loss in translation from business needs to IT solutions” and is used when analyzing the expectations of multiple parties having different views of a system in which they all have an interest in common, but have different priorities and different responsibilities. System, here is used in the broad sense of a set of interacting or interdependent entities, real or abstract, forming an integrated whole. It is applied to ‘systems’ that range from those as small as a performance appraisal, to ones as large as a criminal justice system. [Source: Wikipedia]

VPEC-T (by Carl Bate and Nigel Green) in this context is a useful frame with which to surround PEST analysis to ensure focus and filtering of candidate hypotheses. The first step is to populate a VPEC-T filter, such as that (somewhat simplistically) pictured in Figure 3. Recognising that some analysts extend PEST to PESTEL (including Environmental and Legal forces) the VPEC-T filter includes legal considerations under the Policy dimension and environmental considerations under the Values dimension (in the guise of sustainability).

Figure 2 – The 5 Dimensions of VPEC-T

[source – Lost in Translation by Carl Bate and Nigel Green]

Figure 3 – An Illustrative VPEC-T Filter to frame PEST analysis

[source: Steve Nimmons]

The purpose of the left-most cog in Figure 1 is therefore:

• Define relevance filter using VPEC-T analysis. This constrains the PEST analysis
• Use PEST analysis to determine relevant forces across Political, Economic, Social and Technological domains. There is an argument for introducing Porter’s Five Forces analysis, however for simplicity this is suggested as an extension rather than a foundational element. The key point here is that PEST is about taking a wide view of all potential forces, and the VPEC-T filter is honing in on those of greatest relevance to the specific business
• The output is a filtered PEST analysis which presents a set of forces which the business will likely be subjected to

This feeds into a Scenario Planning cog described later.

PEST Analysis

PEST analysis stands for “Political, Economic, Social, and Technological analysis” and describes a framework of macro-environmental factors used in the environmental scanning component of strategic management.

It is a part of the external analysis when conducting a strategic analysis or doing market research, and gives an overview of the different macro-environmental factors that the company has to take into consideration. It is a useful strategic tool for understanding market growth or decline, business position, potential and direction for operations. [source: Wikipedia]

Figure 4 – PEST Analysis Mind Map (highly simplified for illustration purposes)

[source: Steve Nimmons]

Scenario Planning

There is a significant body of published work on Scenario Planning, however this snippet from Wikipedia is useful for context:

Scenario planning, also called scenario thinking or scenario analysis, is a strategic planning method that some organizations use to make flexible long-term plans. It is in large part an adaptation and generalization of classic methods used by military intelligence.

The purpose of the ‘Scenario Planning’ cog is:

• Based on filtered inputs, create a set of scenario hypotheses
• Prioritise the hypotheses (likelihood of occurrence, timescale, impact on business etc.)
• Define the patterns intrinsic in the scenario set(s)
• Define the signals (positive and negative) that indicate the scenario is developing or likely to develop
• Define the information sources where those signals may be present (again having framed the PEST analysis with VPEC-T, there has been an early opportunity to focus on the Content dimension and hence identify important information sources (which may be internal or external)

Exiting this cog we have a strong definition of ‘patterns to seek’, why (based on Scenario Planning), how we will react if the pattern is detected, the signals that will indicate detection of the pattern and the information sources to monitor for those signals.

The Pattern Detection cog now has an important technical role in correlation or contradiction.

Pattern Detection

Having systematically defined the patterns, related signals and information sources, the algorithmic wizardry of predictive analytics and complex event processing take over. Not to downplay the complexity of scanning significant volumes of real-time, unstructured data and combining data sets in new ways to look for patterns, this area of Pattern Based Strategy is much more widely represented.

I like the idea of triangulation as a means of signal reinforcement, however the strategist must also understand the need for tolerances. For example, reacting to a weak signal in a scenario with potentially disastrous outcomes requires quick intervention. In other cases a ‘wait and see’ approach may be more suitable. This level of sophistication needs to be considered to ensure the right reaction at the right time. The window of strategic advantage may be very small.

My hypothesis is that the Pattern Detection cog also creates an elegant feedback loop to the PEST cog. It is purely theorising, but it seems highly plausible that developing deeper understanding of scenarios, patterns and signals will sharpen the PEST analysis and VPEC-T filtering leading to increasingly sophisticated results.

Pictorial Summary

[source: Steve Nimmons]

Further Reading

For further information on VPEC-T, refer to Lost in Translation.

Article originally published with the BCS in Nov 2008

Panning for gold,  Steve Nimmons looks at complex event processing.

There is an inherent complexity in understanding the relationship between what can appear to be seemingly unconnected events occurring in real or near-real time. I make the temporal distinction as there are sophisticated business intelligence and data mining solutions for pattern or trend discovery in previously captured business information.

These are proven and do a very solid job in specific circumstances. There are also interesting extensions emerging in terms of mash-ups that augment and enrich more fully-featured end-user driven business intelligence solutions. Analysing events in real-time can be exceptionally informative, adding to the overall utility of business intelligence and providing a mechanism for business processes to react advantageously.

I describe complex event processing (CEP) as gold panning the elemental binary soup. CEP is trying to amplify the signal of interesting information against the noise of your event driven architecture. What is interesting is generally termed a ‘concept’ and CEP is all about trying to instantiate concepts from the flow of events.

CEP is typically achieved using forward chaining rules engines, themselves based on the RETE algorithm. There are a number of examples of good COTS products, I myself had the privilege of working on one prototype system that has now been successfully productised.

The business problem we faced was the correlation of events in a distributed mobile telecoms network. Tracing failures throughout the user experience was complex and put a large burden on operations and customer service representatives (CSRs).

The solution was to decode large volumes of real-time data from RADIUS servers, WAP servers, MMS, MML, SMS, micropayment systems et al, and hunt for failure patterns in the datasets (and present these in a simple consumable format). The result was exceptional operational saving, increased user satisfaction and greater insight into systems failures (which were then addressed).

Having ‘done battle’ with early systems I table the following implementation recommendations:

1. Trial candidate CEP systems in a proof of concept unless you are familiar with the solution, product and approach.
2. It is complex and a mutual frustration to produce decent test data with known results. When configuring a CEP system to look for ‘needles in haystacks’ it is very handy to actually know where and what some of them are. False positives, false negatives and correlations that fall inside or outside of configurable tolerances will need to be tested. Start thinking about this very early in the project cycle and engage domain experts that know the respective systems inside out. Make this ‘slick’ and you will get a better result at much lower cost.
3. Tuning is extremely important. Make sure your key attributes can be configured. This often (although not exclusively) relates to timeouts.
4. Ensure, unless by explicit design, that you can handle disasters and fail-over. This may sound obvious, but engines running in memory for ultimate speed may not provide you with necessary state management. An inelegant solution is to run two engines performing the same work. If you need to run at very high throughput volumes this must be part of non-functional testing in a proof of concept.
5. CEP can become an arcane mess and can be quite ‘organic’ in nature. Black-magic systems are only fun for propeller heads with little commercial responsibility, so avoid the ‘allure of mystery’.
6. Make sure the golden nuggets CEP expose are used. This can often be a feedback loop into other business processes. The key point is new information might be disruptive to existing processes, so plan ahead.
7. Depending on the implementation, presenting results to a ‘human’ through a worklist (workflow / BPM) is essential. This might be in the validation of suspect movements, purchases, funds transfers etc. Badly tuned CEP systems with inappropriate thresholds overwhelm the human tasks. Conversely a system that is ‘too loose’ can allow suspect events (potentially of a very serious nature) to pass undetected.

Solutions and industries in which I see increased CEP interest and uptake include: transport, financial services (fraud detection) and fault monitoring systems (cross industries). The latter case is very interesting and systems are being developed that seek to provide early warning of developing faults that generate event streams with tell-tale characteristics.

The definition of ‘tell-tale’ is of course rather slippery, with practically limitless variation between circumstances. This is at the heart of what makes CEP both interesting and challenging.