Friday, December 20, 2019

Introduction to Process-driven Applications

Introduction to Process-driven Applications

Objectives

After completing this section, students should be able to list the advantages in designing and developing applications as a process-driven focused application.

Introducing Process-driven Applications

Any organization is largely dependent on repetitive processes. For example:
  • Creating documents to request a insurance quote.
  • Evaluating scores from a customer to provide a mortgage.
  • Dismissing customer complaints after reviewing them.
Developing and designing applications are ways to automate many of these business processes. Unfortunately, applications usually are not well integrated with each other, causing information to flow from one system to the other without providing a unified data view. This results in an inability to evaluate the existing process, identify bottlenecks, and improve the company's productivity.
A further side-effect of the lack of unified process execution is poor experience for customers who may be required to provide the same information to multiple applications. To minimize this problem, a unified and data-driven process must be created using a process-driven tools. These tools are focused on creating a workflow with all of the required systems and people interaction to give customers an integrated experience and improve the product and business models supported by all applications.
What is Needed?
To support a process-driven tool, there are some requirements that need to be addressed:
  • Processes : Each application or system must be part of a business process. Each business process must be considered as assets that can be adapted, automated, and redesigned. The keys to improving innovation and initiating a digital transformation are mapping business processes to help identify and resolve problems and bottlenecks.
  • Data : Data must be stored in a centralized database where all applications and people supporting the business process have access. The information must flow through all the elements that support each application's execution. In a fragmented environment, it is common to lose information whenever data needs to be transmitted from one system or person to another.

Process-driven Applications

Process-driven applications address the following needs:
  • Orchestrate : engage and collaborate across multiple systems, applications and knowledge workers.
  • Automate : Identify business decisions and automatically choose the correct process or task using rules. These rules are written in a way that can be read by both developers and business users.
  • Enable : Provide mechanisms that allow business users to create their own decision or rules in a process with multiple systems in a low-code environment.
  • Analyze : Evaluate the information captured during the business process that helps improve the process itself. The application engages multiple applications, systems, and people to work together to provide a traceable and unified process.

Key Characteristics

A good process-driven application has the following characteristics:
  • Business-oriented : Provide an intelligent way to solve customer problems with support from both developers and business analysts.
  • Tech-support : Implement best practices to minimize the risk of system unavailability.
Business-Oriented Characteristics
A business process model must support the following business-oriented characteristics:
  • Customer experience : Providing a bad online experience directly impacts the value chain of a business. Most critical business processes expand to include customer interaction points, making business process management part of the product.
  • Intelligent : Optimize processes using automated decisions and tasks with process decision analysis. To accomplish optimization and automation, machine learning and artificial intelligence is an approach for more modern applications to analyze data and make better predictions. Additionally, as the business process is optimized, the tool provides insights to further improve processes an provide value to the business as whole.
  • Omni-Channel : Provide a uniform experience across any channel for seamless switching between interfaces for different devices, such as web browser, mobile application, or customer service center. Furthermore, use home automation platforms and Internet of Things (IoT) enabled appliances to take communication and interaction options to new level. This transparent communication enable organizations to provide omni-channel experience to customers.
Technology Characteristics
The following are technology requirements to support business processes:
  • Distributed : A distributed process-driven application is more reliable to orchestrate the invocation to multiple systems.
  • Cloud Native : As a process-driven application may be invoked by multiple customers, providing an elastic environment where new applications can be deployed under heavy load minimizes the risk of unavailability.
  • Integrated : As multiple systems provide information needed by a process-driven application, an easy to integrate application is fundamental to mitigate integration issues.
Distributed
The distributed process-driven applications enable:
  • Agility : Ability to quickly change processes for experimentation and innovation.
  • Flexibility : Use of smaller, independent but integrated processes to reduce impact on full process when a change is required.
  • Scalability : Ability to individually scale customer-facing components based on usage characteristics.
Cloud-native
Cloud native applications have the following desirable characteristics:
  • Reduce risk of deployment through standardization and management automation.
  • Increase scalability through on-demand auto-scaling features.
  • Facilitate continuous integration (CI) and continuous delivery (CD).
Integrated
Integration with multiple technologies provides the following advantages:
  • Best-of-breed technologies such as software as a service providers and analysis and artificial intelligence engines.
  • Vast array of systems and services such as microservices or data services
  • Open, standard, managed APIs such as those ones provided by personal assistant tools.
  • Hooks to integrate with CI/CD platforms, identity management systems, application servers, and container platforms.

Technologies

The major technologies that need to be provided by a business process automation tool are:
  • Business process management (BPM) : To write down the process in an interchangeable format.
  • Robotic process automation (RPA) : To provide mechanisms to automate manual tasks.
  • Rules : To define decision management capabilities to the tool in a human-readable way.
  • Cognitive : To identify patterns and support decision management capabilities to the tool.
  • Application programming interface (API) : To customize the business process management tool.
  • Low-code : To implement or expand the capability of the business process tool providing extensible mechanisms.
  • Mobile : To provide a simplified access to the application on any mobile device.
  • Integration : To access external systems or services.
  • Analysis : To evaluate metrics on a process for optimization.
  • Containers : To deploy the tool fast on any container-enabled environment.


Introduction to Business Decisions

Objectives

After completing this section, students should be able to list the advantages in using process-driven applications.

Implementing Decision Management Tools

An organization must make business decisions quickly to support customers needs. Developing applications that hard-code business logic into the source code makes these fast changes difficult to accomplish. These changes should be able to be made by someone besides a developer.
A decision management tools provides mechanisms for business analysts to create rules that should support a decision without the need to update the application. Usually, these rules are created using a language that any human can understand, but are evaluated by the engine. Rules generally evaluate a certain condition in the order they are to be executed, which differs from the traditional programming language approach, where you need to chain multiple conditionals to evaluate the correct rule to be processed.
Executing the Correct Rules
Decision manager tools use the following process to select the correct rule:
  • Discovery : Identify and model the decisions that are most important to your operational processes.
  • Services : Design and build independent decision services using business rules and advanced analysis.
  • Analysis : Create a closed loop between operations and analysis to measure results and drive improvement.

Implementing Decision Management

To implement decision management, applications must invoke a decision manager server which stores all of the rules, and passes any parameter needed to evaluate which rule should be used to process the request. These parameters are named facts and they may be manipulated by the rule engine. After processing the content, the manipulated fact return back to the requester.
Modeling Decisions
Decisions made repeatedly by organizations as part of normal business operations should be used to create decisions. Each management level, however, may define different decision definitions to deal with rules.
  • Strategic : Long-term complex decisions usually are made by senior management. They do not address routine decisions as they are company guidelines and they are not focused on a specific task or process objective.
  • Tactical : Decisions that support senior management policies. They are usually less complex than the strategic ones and they define medium term decision.
  • Operational : Decisions based on routines and used on a daily basis to solve regular problems. Due to the nature of operational decisions, they may affect the way tactical and strategic decisions in way that they must be changed. As the operational decisions affect strategic decisions there is a constant feedback needed to define the strategic decisions regularly.

Red Hat Decision Manager

Red Hat Decision Manager implements a decision management tool that may be invoked in any external system using REST APIs. It has a flexible architecture that supports:
  • Embedded : Lightweight runtime that can be embedded in any Java-based application. Rules are executed as part of the application. Rule definitions can be packaged with the application or fetched from a rules repository.
  • Decision server : Lightweight runtime which provides a RESTful and JMS API for rule execution. Supports extension APIs, protocols, and transports.
  • Containerized : Also built on the Decision Server technology, the OpenShift container image provides microservices based rule execution platform for cloud, Platform-as-a-Service, and container environment.

The Future of the Contact Center


Published 25 April 2019 - ID G00383538 - 16 min read


By 2025, the contact center organization will be exploiting the benefits of an application ecosystem and tools to better equip staff to work in teams. Application leaders for customer service must assess the technology and changes for the future of work to provide cutting-edge customer experiences.

Overview

Key Findings

  • Application leaders struggle to buy contact center software that addresses the broader ambitions and challenges of the customer service organization. Contact center vendors often focus too narrowly on getting customers connected, overlooking the other key pillars of customer service.
  • Application leaders for customer service no longer want to invest in a fixed contact center software application. They prefer to be free of the technology refresh cycle and to benefit from innovation on a fluid, ongoing basis.
  • Usage-based software consumption and features — for entirely mobile-activity-based and team-based working — will enable a more digitally dexterous workforce to perform a mix of customer engagement and traditional back-office tasks during their day.
  • Artificial intelligence (AI) will become a foundational element upon which contact center software is created. Almost all key application features will leverage some form of AI.

Recommendations

Application leaders responsible for customer service and support technology should:
  • Adopt a holistic customer service application suite approach to procuring contact center technology, focusing on how the technology improves connecting with customers, orchestrating engagements, access to knowledge and coordinating resources.
  • Use an application ecosystem approach to investing in contact center vendor technology that can power improvements by offering fluid and easy access to features and benefits.
  • Empower greater collaboration in customer service with digital workspaces, by favoring providers that offer the commercial model and mobile productivity features that will support the dissolution of the front-office/back-office divide.
  • Select contact center technology that’s built on a foundational approach to delivering the benefits of AI-based innovation, as well as being able to effortlessly combine with existing AI-based deployments or ongoing projects.

Strategic Planning Assumptions

By 2022, contact center as a service will be the preferred adoption model in 50% of contact centers, up from approximately 10% in 2019.
By 2023, customers will prefer to use speech interfaces to initiate 70% of self-service customer interactions, rising from 40% in 2019.
By 2025, customer service organizations that embed AI in their multichannel customer engagement platform will elevate operational efficiency by 25%.

Analysis

Contact center operations are defined by the technology deployed. Until recently, contact center infrastructure (CCI) supported the operational and physical design of staff sitting in neatly organized rows in selected offices, sporting the latest headsets and helping customers in an isolated way.
Contact center operations of the future will foster greater collaboration between employees with digital workspaces, as well as benefiting from software as a service (SaaS)-based consumption and innovation provided by born-in-the-cloud technology. Activity- and team- based working capabilities in contact center software and services will ensure better support of customer service ambitions as part of the broader customer experience strategy.
Several key technology changes are paving the way for the future of the contact center:
  • Native mobile device agent desktops with in-app experiences. Staff are better engaged and empowered to be productive anywhere.
  • Consumption models to support a complete change in the way contact center staff are organized and structured to support customers.
  • Workstream collaboration and team-hub-native features to offer team-based messaging between customers and subject matter experts.
  • Improved solutions for security and robustness at scale, helping to migrate fragile infrastructure and release large enterprises to fully exploit SaaS-based contact center software.
  • Improvements across all contact center application features based on artificial intelligence (AI), forming an essential part of the core of the software.
  • Use of mobile and home device software development kits (SDKs) to boost in-app experiences beyond surfacing contact information and transferring context.
  • Utilization of mobile devices to speed up the identification and verification processes, and to make engagements easier and more secure.
  • More advanced knowledge management, real-time next-best action and orchestration abilities — beyond having the current “there’s an adapter for that” thinking.
The most impactful areas of change are further described in this note. To fully realize the benefits of upcoming contact center technology, organizations must break free from the shackles of the endless and disruptive technology refresh cycles associated with traditional customer premises equipment and hosted tech platforms, to be able to innovate.
Contact center operations are set to mature — primarily based on a difference of approach with the technology and the degree of focus on broader customer service ambitions. By plotting one against the other, we create four future possible scenarios (see Figure 1).
Figure 1. Future of the Contact Center
Future of the Contact Center
  • Traditional contact centers — Support the largest number of customer service environments today, and, in reality, many could continue to operate as such. Often described as system-of-record solutions in pace layering, these kinds of centers handle large volumes of voice interactions with some support for email and chat. There may be a focus on providing connected customer journeys across channels, but, largely, they are still quite disconnected in terms of technology.
  • Innovation-led contact centers — Realize the benefits of SaaS-based contact center software as part of a cloud-based application ecosystem that’s consumed more fluidly. The technology deployed may be better powered by AI in the core, and offer more connected apps, devices and engagements. Usage-based consumption powers experimental innovation; for example, voice self-service projects can be started with just a credit card.
  • Customer service suite contact centers — Target a specific ambition to address the broader customer service challenges with connected budget and technology suite thinking. Although the depth of the technology in specific technology landscape areas isn’t necessarily the primary focus, better support for a more connected customer service organization is.
  • Optimized customer service contact centers — Reflect both a broader focus on customer service ambitions and the realization of key changes in how technology is used. Typically, a SaaS-based application ecosystem is adopted and applied to a design that incorporates more connected thinking across an entire customer service suite of applications. Staff are inherently more engaged and empowered, breaking the mold of the traditional contact center staff setup.
The future could see any of the above contact center scenarios at the heart of a customer service organization. Application leaders for customer service should consider the recommendations in this note to decide on which of the future contact center scenarios is achievable for their customer service organization of the future.

Adopt a Holistic Customer Service Application Suite Approach to Procuring Contact Center Technology

The contact center is critical to meeting the organizational objectives of delivering consistent, intelligent and personalized customer experiences, irrespective of the customer engagement channel. “Getting connected” is one of four pillars of great customer service (see “Customer Service and Support Technology Primer for 2019”) that contact center vendors provide strong solutions for (see Figure 2).
Figure 2. Four Pillars of Customer Service
Four Pillars of Customer Service
These four pillars of customer service — getting connected, process orchestration, managing resources and knowledge and insights — traditionally reflect discrete investments by different business owners and stakeholders, leveraging different budgets. However, application leaders for customer service are now challenged with balancing departmental requirements with enterprisewide programs. A combined technology strategy — delivering innovation across all four pillars — has become a key focus. As a result, and given that a unified customer service and support (CSS) suite does not exist today, a contact center solution discussion often expands to cover adjacent technology areas. Multiple vendor types — including those for customer engagement center (CEC), workforce engagement management (WEM), knowledge management and analytics — often form part of the conversation.
Contact center software vendors are already broadening their solutions by delivering benefits often surfaced from adjacent technology landscapes. This reflects a future based on vendors continuing to inherit more of the burden of addressing customer service ambitions, with, potentially, a myriad of enabling technologies and best practices. Application leaders for customer service must connect thinking across the customer service organization and challenge contact center vendors to demonstrate their support for all four pillars — to realize the benefits of an optimized customer service contact center of the future.

Invest in SaaS-Based Contact Center Software as Part of a Cloud-Based Application Ecosystem

SaaS has emerged as a preferred way to cost-effectively empower staff to get things done. Subscription consumption benefits, combined with being able to spin up a browser and always have access to the latest productivity tools, has revolutionized the way departments such as HR, finance, sales and marketing run.
Three key factors have led to SaaS-based solutions being the default technology consumption style:
  • Organizations struggle to keep up with, or are constrained in benefiting from, the continual stream of software updates from vendors.
  • Cloud services include capabilities delivered currently by best-of-breed providers offering a lower total cost of ownership (TCO).
  • Opportunities to exploit underused and emerging SaaS applications in support of a stronger alignment with deeper customer experience and digital workplace initiatives.
Contact center as a service (CCaaS) solution adoption is catching up with adoption of SaaS-based solutions elsewhere in the business, as confidence in buying within the contact center operation of larger enterprises grows. Improvements — in CCaaS solutions — in robustness at scale and security challenges are being better addressed. The compelling benefits of a cloud contact center application ecosystem have further driven this change.
Customers can now consume applications and features, as and when required, in a customized way. Contact center software, or the physically defined office, no longer needs to be a fixed thing — it can provide exactly the right benefits for a specific customer service organization. Three key styles of cloud contact center application ecosystem have emerged (see Figure 3).
Figure 3. Contact Center Application Ecosystem
Contact Center Application Ecosystem
  • An application marketplace provides access to (potentially) an infinite number of “click to add” applications or features such as speech analytics or advanced reporting. This solution-first, platform-second approach (to define a more precise feature set) offers an ecosystem that’s flexible and cost-effective. Application leaders in customer service can and will be able to build out the perfect contact center solution for their precise organization, using “apps” that better serve staff and customer needs.
  • An application suite reflects an approach where vendors are incorporating what are typically adjacent marketplace application solutions into a single contact center platform. This approach offers the benefits of incorporating routing, agent desktop, WEM, speech analytics, robotic process automation (RPA) and employee virtual assistant (EVA), for example, into a single solution that pivots around a single user record. This application approach typically delivers more benefits from the core AI engine, by connecting solution features in more advanced ways. Customer service organizations can easily access entire modules or benefits via a simple change in license.
  • A developer-centric contact center reflects a renewed focus on empowering DevOps resources to innovate at the edge. The developer-centric style provides the ability to build capabilities against a more granular set of contact center requirements — to either extend existing contact center software capabilities, build new applications, or better integrate with additional applications. IT organizations with a developer-centric or DevOps approach to sourcing and maintaining contact center technology can benefit from the extended capabilities of CCaaS toolkits or consider a SaaS-based communications platform as a service (cPaaS) solution.
cPaaS vendors are evolving — to build CCaaS solutions based on cPaaS platform capabilities — suggesting a future where customer service organizations may benefit from both approaches in a single solution. This will offer faster time to value through “canned” apps, but with the flexibility to build from there — using an API-based approach — more precise and custom-made cloud-based contact center features.
The vendor landscape is set to demonstrate more offerings across the three styles of cloud contact center application ecosystem (see Figure 3). Fluid and effortless access to an array of continuously evolving capabilities is offering a solution-platform-solution approach that we already see in adjacent areas such as the mobile phone marketplace. Application leaders for customer service should invest in an application ecosystem capable of forming an innovation-led or optimized customer service contact center of the future.

Empower Greater Customer Service Collaboration That Will Support Dissolution of the Front-Office/Back-Office Divide

Technologies associated with digital workplace initiatives can have a transformational impact on business activities and the employee experience. Most work environments include elements of routine and nonroutine work, and all will be impacted by AI (see “Anticipate and Exploit the Top 12 Future Work Trends”). The customer service environment is more routine, and is therefore likely to experience greater disruption through the use of AI to automate routine tasks. At the same time, organizations recognize the benefits of greater collaboration in terms of engaged associates and innovation. As more work in the contact center becomes nonroutine, application leaders in customer service have an opportunity to leverage emerging features and software consumption models to redefine the digital workspace for the contact center.
Software consumption has evolved from perpetual licensing that’s capital expenditure (capex)-based to concurrent or seat-subscription-based licensing. An emerging commercial model is usage-based, effectively providing access to contact center software on a per-hour, per-agent cost basis. This has a profound effect on how contact center staff can be organized, when combined with contact center features fueling mobile activity- and team-based working (see “How We Will Work in 2028”). Instead of contact center staff being occupied for the majority of the day on customer engagements — dependent on occupancy and an employee-surplus setup driven by better first-contact resolution rates, they can also engage in traditional back-office tasks. (For further details, see “Select Consumption-Based Pricing for Cloud Contact Centers to Broaden Scope of Customer Service Applications.”) The office divide is set to disappear, with contact center staff forming part of the broader customer service and overall company customer experience team.
Contact center applications that have advanced engagement features offered natively on mobile devices, and with messaging and collaboration at the core of the design, are empowering agents to break free from working in isolated rows. Contact center staff are equipped to work more productively together, from wherever they are with the device they are carrying. Engagement via video will drive more demand for different working environments than the typical “factory floor” contact center, based on noise volumes and appropriate setting.
We’re beginning to see customer service centers set up for thousands of customers to have access to groups of customer service staff organized in teams. The staff have stand-up meetings, discuss customer challenges or ambitions collectively, and are more aligned to customer problems than traditional key performance indicators. This will become the organizational design for many contact centers of the future. The shifting future of work dynamics, including the gig economy (see “Learn How the Future of Work Is Transforming Employee Experience Globally”), will also be better-supported by both usage-based commercial models and entirely mobile-first, productivity-focused contact center solutions.
Application leaders for customer service should adopt team- and activity-based working that is powered by usage-based consumption and contact center software on mobile devices. This will enable them to develop an innovation-led customer service suite approach, or the optimized customer service contact center of the future.

Assess the Use of AI at the Core of Contact Center Software to Revolutionize Engagement and Interactions

The contact center is, typically, a highly optimized routine work environment, which means it is ripe for automating using AI. It is imperative that application leaders’ investments in contact center software are focused on harnessing the power of AI in customer service interactions.
One of the biggest challenges with implementing AI-based solutions is knowing, firstly, which use cases to apply it to (see “How to Use AI to Improve the Customer Experience”). Customers look to their incumbent contact center vendor for help. This vendor is then pressured to respond with a sense of urgency and, not least, to deliver something that can legitimately be described as an “AI-based” solution. So far, contact center vendors have offered AI technology through the following:
  • Acquisition of something prebuilt — Thereby ensuring alignment with the existing value proposition
  • A bring-your-own-bot approach — So that customers can leverage what they’re already doing, in harmony with the contact center application
  • Technology partnerships — Making it easier for customers to consume and deploy AI-based solutions as part of the vendor’s offering
The kinds of benefits surfaced at this early stage of the market have, inherently, not fulfilled the full potential of what’s possible with AI. The matching of customers and staff instead of skills-based routing, or more intelligent conversational bots over supposedly dumb, rule-based chatbots, are a couple of examples that are powerful — but only scraping the surface of what we’ll see in the future.
Contact center vendors are investing in data science expertise to support engineering solutions that form, natively, part of the communications platform engine and user experience. AI-based technology in the contact center application itself will surface to provide, for example:
  • Consideration of all customer touchpoints across an organization, in order to improve customer and staff engagement as part of customer experience ambitions (organizationwide)
  • Utilization of more analytics data (including social), to prevent customers needing to engage — better-equipping agents when they do need to engage, and freeing up more agent time
  • The use and control of mobile devices — including further exploitation of home and mobile phone virtual personal assistant (VPA) tools, realizing the benefits of “personalized” VPAs, and speeding up easier engagements (see “Predicts 2019: CRM Customer Service and Support”)
  • Evolved self-training of machine learning models to shorten time to AI-based value
  • The distillation of “knowledge everywhere” — reflecting on generation Z in particular as they are sharing more data than ever — to make customer engagements more intelligent or more likely to be addressed first time, every time
  • Better use of proactive or outbound AI-based engagements to prevent customers needing to engage to begin with
  • AI-based entities acting on behalf of customers, which will mean a shift in how contact center vendors have to support new flavors of engagement
The challenge of coordinating both bot- and people-based resources will become both greater and more crucial. If, for example, simpler customer engagements are handled by a bot, this leaves contact center staff in higher occupancy centers with nothing but the challenging and more complex engagements in their day. This then creates an agent churn problem (see “The Future of Customer Service Workforce Engagement Management”).
Application leaders in customer service and support should favor solutions where AI is already a part of, or is planned to be a part of, the core of the contact center software. These solutions will be the foundation on which to build an innovation-led or optimized customer service contact center of the future.

Tuesday, September 3, 2019

5 Concepts That Will Help Your Team Be More Data-Driven



I’ve spent my career helping companies address their data and data quality opportunities. Overall, I rate progress as “slower than hoped.” While there are many contributing factors, one of the most important is the sheer lack of analytic talent, up and down the organization chart. In turn, this lack of talent makes it harder for companies to leverage their data, to take full advantage of their data scientists, and to get in front of data quality issues. Lack of talent breeds fear, exacerbating difficulties in adopting a data-driven culture. And so forth, in a vicious cycle.
Still, progress in the data space is inexorable and smart companies know they must address their talent gaps. It will take decades for the public education systems to churn out enough people with the needed skills — far too long for companies to wait. Fortunately managers, aided by a senior data scientist engaged for a few hours a week, can introduce five powerful “tools” that will help their existing teams start to use analytics more powerfully to solve important business problems. To be sure, these are not the only tools you’ll need — for example, I haven’t included A/B testing, understanding variation, or visualization here.  Nor is my intent to make people experts.  Rather, based on my experiences working with companies on their data strategy, these five concepts offer the biggest near-term bang for the buck.
The first is learning to think like a data scientist. We don’t speak about this often enough, but it is really hard to acquire good data, analyze it properly, follow the clues those analyses offer, explore the implications, and present results in a fair, compelling way.  This is the essence of data science. You can’t read about this in a book — you simply have to experience the work to appreciate it.  To give your team some hands-on practice, charge them with selecting a topic of their own interest (such as “whether meetings start on time”) and then have them complete the exercise described in this article. The first step will lead to a picture similar to the one below, and the rest of the exercise involves exploring the implications of that picture.
Charge that senior scientist you’ve engaged with helping people in completing the exercise, teaching them how to interpret some basic statistics, tables, and graphics, such as a time-series plot and Pareto chart. As they gain experience, encourage your team to apply what they’ve learned in their work everyday. Be sure to make time for people to show others what they’re learning, say by devoting fifteen minutes to the topic in each staff meeting.  Most critically, lead by example — do this work yourself, present your results, and freely discuss the challenges you faced in doing the work.
As you and your team dive into data, you’ll certainly encounter quality issues, which is why pro-actively managing data quality is the next important skill to learn.  Poor data is the norm — fouling operations, adding cost, and breeding mistrust in analytics.  Fortunately, virtually everyone can make a positive impact here.  The first step is to make a simple measurement using the Friday Afternoon Measurement method   (the technique acquired this name because so many teams end up using it on Friday afternoon).

INSIGHT CENTER

To do so, instruct your team members to assemble 10-15 critical data attributes for the last 100 units of work completed by their departments — essentially the last 100 data records.  Then, they should work through each record, marking obvious errors.  They should then count up the error-free records.  The number, which can range from 0 to 100 represents the percent of data created correctly, their Data Quality (DQ) score.  DQ can also be interpreted as the fraction of time the work is completed correctly, the first time.  Most managers are surprised by the results — they expect to score in the high 90s, but DQ = 54 is the median score.
FAM can also point out which data attributes have the biggest error rates, suggesting where improvements can be made, using root cause analysis, described next. Charge each member of your team with making one such improvement.
The third skill is conducting a root cause analysis (RCA) and its pre-requisite, understanding the distinction between correlation and causation. Studying the numbers can point to where most errors occur or demonstrate that two (or more) variables go up and down in tandem, but it cannot fully describe why this is.  For example, studies show that the numbers of live births and storks in the countryside were highly correlated. But storks do not bring babies!
Thus, look to the numbers to understand correlation and to the real-world phenomena to understand causation. Root cause analysis is a structured approach for getting to the real reasons things go wrong — the root causes.  It is important because, too often, managers and teams often accept easy explanations and don’t dig deep enough. And problems remain. RCA can enable them to develop a clearer picture and take actions that are more likely to solve the problem.
To develop this skill with your team, start by discussing “how to explore cause and effect like a data scientist” with your staff.  Then, the next time you find yourself tempted to accept someone’s intuitive reasoning as to why something went wrong, seize the opportunity to conduct a solid root cause analysis. There are many formal means to do so. “The five whys,” which forces you to make sure you’ve gotten to the root cause, and fishbone diagrams, which graphically represent multiple causes, are probably the best known. Have your data scientist pick one, and follow it! Over time, seek to make root cause analysis your standard for all important issues.
The fourth skill stems from the desire all managers have to “be in control.”  My working definition of control is “the managerial act of comparing process to standards and acting on the difference.”  But even the simplest process varies.  How can one distinguish normal day-in, day-out variation from situations that are truly out of control? Fortunately, understanding and applying control charts provides a powerful way to do just that.
Control charts feature a plot of the data, the average, and two “control limits,” (an upper control limit and a lower control limit).  Basic as they are, they reveal so much!  For example, in the Figure below:

  1. Since day 9 falls outside the control limits, a manager can be certain this process is out of control. They should initiate a root cause analysis to figure out why.
  2. There is an uptick at day 4 that looks encouraging. But a manager should not get too excited — the uptick was more likely due to random variation and was not sustained.
  3. It is clear enough that that this process only succeeds 60% of the time. If this is not good enough, the manager must make fundamental changes.
Engage your data scientist in helping you and your team try control charts on a few important processes. Learn as you go, understanding key terms, determining which control charts to use, and striving first to get processes under control — your confidence will grow, as will your ability to manage your team!
Finally, all managers and their teams should learn to understand and apply regression analysis. Regression provides a powerful means to explore the numerical relationships between variables.  To help illustrate this, consider “umbrella sales.”  There are dozens of factors that could increase sales (e.g., rain) or decrease sales (e.g., a competitor’s price cut).  Regression provides a way to determine which variables are most important and their impact on sales.  For example, an analysis may yield:
Monthly sales = 200 + 5*(days of rain) – 10*(competitor price cut in $) + error term
Meaning that:
  • Absent other factors, monthly sales are about 200 units.
  • A day of rain is associated with the sale of five more umbrellas,
  • A competitor cutting its price by one dollar is associated with ten fewer umbrellas sold
The model is not perfect — hence the error term. For example, suppose you sold 250 umbrellas in a month when there were 15 rainy days and a competitor cut its price by $2.  Based on the formula, one would expect umbrellas sales to be 200 + 5*15 – 10*2 = 255 units.  So the error term in that case is 5 umbrellas.
Like all analyses, the more variables, the more complex the analysis, so start by focusing on one independent (e.g., explanatory) variable.  In parallel, read “A Refresher in Regression Analysis,” which uses umbrella sales as an example to explain the terms and underlying concepts. Charge your data scientist with helping your team do the work, and making sure team members don’t get bogged down in details.  Only then should you move onto two, three, or more variables and more complex regression models.
These five tools are powerful, even elegant, in their own ways.  They provide far greater capabilities than the steps described here, which aim only to get you started. You’re certain to take some false steps along the way, but press on.  Work with your data scientist to learn even more. As your team grows more confident in using analytics, the business benefits you gain will more than justify the effort.