Cognitive Computing Systems: Want to build a great MVP? Pick your ingredients carefully

Over the last five articles I have hopefully presented a very strong business case for the value that a cognitive computing system like Herbie can bring to startup founder like Keith and his team. They have successfully collaborated up to this point and have developed a very strong product vision:

We believe that we can build a cognitive computing system that acts both as an early warning system for students “at-risk” of dropping out of high school and as a virtual education “doctor.” The system would be capable of diagnosing in real-time the reasons why a student is likely to drop out and then be able to specifically tell all relevant stakeholders what actions they could and should take to help the student qualify for graduation. Student Lifecycle Management is a key component to solving the graduation problem and it will be powered by a national cognitive computing system.

Now that we know what we want for dinner we need to talk about the recipe we want to use. The recipe I’m referring to in this case is the specific product features and benefits that will bring our solution to life. By leveraging all of the complex analytics completed up to this point and including the lessons learned from our direct customer interaction, Herbie is ready to help Keith develop and finalize one the last important steps before delivering the Minimum Viable Product (MVP): Documenting the product features list.

The recipe for your MVP is a list of the specific product features and benefits to your solution Click To Tweet

The product features list is a one-page document consisting of one or two sentence summaries of the top 10 (or fewer) features of the complete product vision. Essentially, this is the product development team’s contract with the rest of the company. The biggest challenge will be deciding what features will ship in what order. Developing the MVP will start the prioritization process.

The product features list is a 1 page document of the top features in your product vision Click To Tweet

Customers will be critical in guiding the process as they begin interacting with the earliest version of the MVP. You can think of features as the things that the engineering team is building, and the benefits as the problem you are solving for the customer. Your goal is to describe the specific product benefits as seen through the customer eyes (Something new? Better? Faster? Cheaper?) to then develop a “user story.”

A user story is a short narrative that explains what job the product will do. How will it solve a problem that customers are eager to fix, or fulfil a need they have? Ideally, the product solves a mission-critical problem, delivers a compelling, exciting customer benefit, or addresses an unspoken need:

  • Unfortunately, by the time a high-risk student reaches high school as a freshman, many lack both the academic and social skills necessary to see the long-term value of staying in school;
  • We lose as many as 55% by the time their class becomes seniors;
  • Educated workers are the basis of economic growth. They are especially critical as sources of innovation and productivity given the pace and nature of technological progress;
  • Studies show that the typical high school graduate will obtain higher employment and earnings — an astonishing 50% to 100% increase in lifetime income — and will be less likely to draw on public money for health care and welfare. They will also be less likely to be involved in the criminal justice system. Further, because of their increased income potential, the typical graduate will contribute more in tax revenue over their lifetime than if they had dropped out;
  • When the costs of investment to produce a new graduate are taken into account, there is a return of $1.45 to $3.55 for every dollar of investment, depending upon the educational intervention strategy. Under this estimate, each new graduate confers a net benefit to taxpayers of about $127,000 over their lifetime. This is a benefit to the public of nearly $90 billion for each year of success in reducing the number of high school dropouts by 700,000 — or something close to $1 trillion after 11 years;
  • All stakeholders who significantly impact an individual student’s decision to stay in school until graduation will receive real-time advice on how to effectively collaborate to minimize or eliminate the negative environmental elements that influence many to dropout prior to graduation; and
  • This transaction based omni-channel student lifecycle management cognitive computing system will be the tip of the spear that drives behaviors that will enable the US to achieve a national graduation rate of 90%+ by the year 2020.
A user story is a short narrative that explains what job the product will do Click To Tweet

Remember that at this early stage we are still in the process of developing hypotheses that we must test and validate with real customers. All that is required is to follow the customer discovery process with ideas that are reasonable educated guesses. The goal is to quickly move into building an effective MVP that you can put in front of your prospective market for immediate feedback and iteration.

Creating an MVP can increase development productivity between 14 – 95% while reducing costs by 7 – 29%. Being able to quickly bring a product to market, gather crucial feedback, and iterate on it in real-time is a formidable competitive advantage.

Creating an MVP can increase development productivity between 14-95% and reduce costs by 7-29% Click To Tweet

Do you feel comfortable with the process of defining your product features and benefits? Do you believe that Keith and Herbie have collaborated well up this point and come up with a great product vision? With the help of a cognitive computing system like Herbie, do you think that it is possible to achieve a national graduation rate of 90%+ by 2020?

Let me know what you think.

Cognitive Computing Systems: Want to build a great MVP? It’s all about the prescription…

As I discussed in my last article, with Herbie’s assistance Keith and his team determined that being able to proactively predict the likelihood that an at-risk student is going to dropout of high school would be a key component of their product vision. The other half of the equation involves the actions that an educator and the relevant stakeholders should take once a specific problem with an at-risk student has been identified. This involves a brief discussion about the science of prescriptive analytics and how Keith and his team plan to leverage this capability to help complete their product vision.

As a cognitive computing system, Herbie was instrumental in developing the analytical models (called the High School Dropout Propensity Score) needed for early identification of at-risk students. He also took the next step by leveraging prescriptive analytics to provide educators and relevant stakeholders an interactive voice that would offer them iterative advice on how to set the student back on track. This article will define prescriptive analytics, as well as the role it will play as part of our Minimum Viable Product.

Prescriptive analytics builds upon descriptive analytics and predictive analytics; it predicts what will happen, when it will happen, why it will happen and what the best course of action is to optimize outcomes and reduce risk. These solutions combine predictive models, deployment options, localized rules, scoring and optimization techniques to form a powerful foundation for decision management.

Prescriptive analytics predicts what, when & why something will happen and the best course of action Click To Tweet

As a follow-on to the extensive hypothesis development and testing being continually performed by Herbie, he will begin to use those simulated outcomes to develop a prescriptive list of actions that should be taken when those “conditions” occur again in the real world. Machine learning will allow Herbie to build a comprehensive library of real-time actionable “prescriptions” that will directly impact how effectively and efficiently educators and relevant stakeholders serve their at-risk students.

Machine learning can build a comprehensive library of real-time actionable “prescriptions” Click To Tweet

In the process of building this prescriptive library, Herbie and the team will focus on learning the characteristics, attributes and behaviors of the following demographics: 

  1. Low income students: The link between low income and low academic performance is strong, but research shows it is solvable. Among non-low income students, 40 states are above the national average graduation rate of 80%. However, among low-income students, 41 states are below the national average. The good news is states with narrow achievement gaps between low-income and non-low income students appear to be those with the most robust interventions in place to counteract the effects of poverty;
  2. Big city dwellers: While there are nearly 200 fewer dropout factories in urban areas in 2012 than in 2002, more than half of those remaining are located in large urban areas. Most big cities with high concentrations of low-income students still have graduation rates in the 60s, with a few even in the 50s;
  3. Students with disabilities: The national average graduation rate for students with disabilities is 20 percentage points lower than the overall national average. While graduation rates for these students varies greatly by state, these students represent 13% of all students. Without gains nationwide, a 90% graduation rate cannot be reached;
  4. The state of California: As the most populous and most diverse state, California needs to be the focus of national attention and work. With the highest poverty rate in the country, a median household income 20% higher than the nation’s, and a population that is 61% non-Anglo, California is key to reaching the 90% graduation rate nationally, but it also remains a laboratory of innovation in education reform. California has 14% of the nation’s total students, and 20% of the country’s low-income student cohort. The school age population is 52% Latino and 12% Asian/Pacific Islander, with a poverty rate among school age children of 63%; and
  5. Young men of color: Despite gains made by all students of color over the past six years, young men of color continue to lag behind other subgroups of students. In a sub-set of Midwestern and Southern states, which educate a large percentage of African American students, graduation rates for African American males remain in the upper 50s and low 60s.

It is important to think of prescriptive analytics as one of the many successful outcomes possible with a well-designed cognitive computing system. I believe that the real value that Herbie offers the founding team is a single-minded focus on finding better approaches to solving the designated problem based on real data received from real-life attempts to solve the problem with real customers. The only way to reduce bias is to minimize the programmer’s pre-conceived notions of the problem from the equation.

Cognitive systems offer founders a way to solve their problem based on real data from real customers Click To Tweet

The ideal process should look like this:

  • The founding team presents Herbie a problem in the form of a question;
  • Herbie understands the context of the question and performs the necessary analytics to formulate an answer;
  • The founding team considers the answer and makes a decision as a team;
  • The founding team tells Herbie their decision. Herbie then tracks and records the results and impact of that decision; and
  • Based on this data, Herbie learns lessons and adjusts his approach to thinking so that the team will achieve better results and outcomes next time.

With Herbie’s help, Keith and his team have enough information to put forward their first product vision. Their first draft sounds something like this:

  • The United States used to be number one for high school graduation, but times have changed. In 2009, the U.S. ranked 21st out of 26 OECD countries when it came to high school graduation rate according to Andreas Schleicher, Deputy Director for Education for the Organization for Economic Co-operation and Development (OECD);
  • Portugal and Slovenia were tied for first in the rankings, Japan and Finland hold the number two spot, and the Czech Republic ranks 17th;
  • We believe that we can build a cognitive computing system that acts both as an early warning system for students at-risk of dropping out of high school and as a virtual education “doctor” capable of diagnosing in real-time the reasons why a student is likely to dropout, and then be able to tell all relevant stakeholders what specific actions they could and should take to help them qualify for graduation;
  • Millions of students, parents, educators and third party stakeholders will be able to collaborate at a level that was not possible before; and
  • By proactively communicating and executing best practice strategies for reducing dropout rates through the medium of their choice in real-time, we can leverage the entire global community to make sure that every child has a 100% chance of graduating.

This is just an example of the many different ways that Keith and his team could have drafted their product vision, but the documentation process is a crucial first step. It is important during these early stages that everyone agrees on a specific product vision so that the product development team has as clear a mandate as possible before building the MVP.

We can leverage the entire global community to make sure that every child has a 100% chance of graduating Click To Tweet

Now that the team has established a certain level of clarity on the product vision, it is now time to move forward with formally defining the product features and benefits. We will cover that in my next article.

Do you understand the power of prescriptive analytics and how it might be utilized to make sure that the right action is taken at the right time? Do you agree that focusing on the five demographic areas listed above will have the biggest impact on increasing our graduation rates?

Thanks in advance for your continuing feedback.

Cognitive Computing Systems: Want to build a great MVP? Define your Product Vision

In my last post I discussed why it was important for Keith and his team to focus on establishing a repeatable and scientifically sound method of hypothesis development and testing, and reasons why this was a critical step along the path of building an effective Minimum Viable Product (“MVP”). Now it’s time for Keith and his team to formally agree and declare their product vision for solving the graduation problem in America. This step is focused on developing a solid shortlist of product features that will be shared with the product development team.

Fictional startup founder Keith and Herbie, his interactive cognitive computing system, will now dig even deeper to investigate specific aspects of the graduation problem. GradNation Research has shown that there are four factors known to predict or exacerbate dropping out. They are as follows:

  1. Chronic Absenteeism. Missing more than 10% of the school year for any reason is an early indicator of potential dropout. Often associated with lower academic performance, this can be seen as early as first grade;
  2. Lack of Early Intervention. Middle grades are pivotal years that either set a student on the path to high school, college and a career, or a path to disengagement and low achievement in key subjects;
  3. Lack of System Visibility. There are more than six million people between the ages of 18 and 24 who currently are not in school, in possession of a high school diploma or working. These young people cannot be forgotten, and need access to pathways to education, employment and opportunities to take on the jobs of the future; and
  4. Lack of role models. Success in life cannot just come from a classroom education. Students need to develop additional skills, such as self-awareness, self-control, collaboration and conflict resolution. Young people need public, private and nonprofit agencies to work together to provide them with access to positive role models, not just adults, but also by giving them the opportunity to learn from their peers.
Missing more than 10% of a school year as early as 1st grade is likely to lead to dropping out Click To Tweet

On the surface, it is reasonable to assume that the effects of these challenges can be minimized or negated with a proactive and preventative approach. Educators should be able to predict and have sight of the small problems at-risk students are having long before they blossom into big problems.

Keith is exploring the development of iterative predictive models that will give schools the ability to identify these at-risk students very early in their school experience so that proactive actions can be taken to put them back on course. Herbie will leverage his cognitive computing infrastructure to create an effective attribute management program (this defines the data that will be used for analysis). This program will support the construction of industry standard predictive models. These models will feed an executive level scorecard that educators will use to actively and proactively manage the progress of at-risk students.

When it comes to predicting when a student is at-risk of dropping out, the sooner the better Click To Tweet

If having access to effective predictive analytics to an educator represents a major part of Keith’s product vision, then being able to actively provide that same educator with industry best practice advice as to what they should do next to address an identified problem is equally important. What specific actions should an educator take once an at-risk student has been identified?

Being notified of a student problem is not enough. What should an educator do? Click To Tweet

For example, assume that Herbie has built a predictive model that informs an educator that an individual student is falling behind in math skills. Knowing that the next math class will be even more challenging, Herbie executes a predictive model, (the high school dropout propensity score), that calculates the likelihood that the identified student will succeed in their next grade level. Depending on the results, Herbie will proactively send out a threat level notification to the proper educational stakeholder (i.e. principal, teacher or parent).

A realtime high school dropout propensity score would transform education Click To Tweet

Once the proper notifications have been seen, Herbie moves from predictive mode to prescriptive. The ability to offer real-time prescriptive advice in the context of the problem at hand is the biggest value that a cognitive computing system like Herbie can provide to an educator. Prescriptive analytics along with natural language processing is an integral part of the solution and as such will play an equally important role in the makeup of the MVP.

Real-time advice to an educator via prescriptive analytics is a key success factor Click To Tweet

Chronic absenteeism, the lack of early intervention, poor system visibility and lack of good role models are factors that are highly predictive of high school student dropout rates. I believe that a cognitive computing system working in tandem with educators is a key element to solving the graduation problem. Do you share my vision regarding the role that cognitive computing systems can play in significantly speeding up the process of identifying at-risk students, as well as offering consistent real-time advice to all stakeholders involved?

Let me know what you think.

Lean Market Development: How to turn an educated guess into a scalable business model

In my previous article, our fictional startup founder Keith continued his quest to discover a solution that will increase our national high school graduation rate from just over 80% in 2014 to 90%+ by the year 2020. He narrowed the team’s focus to four key elements of the problem and with the assistance of his cognitive computing partner Herbie, the team was able to establish a strong Enterprise Data Architecture (EDA) for the project. All of this puts them in a great position to take the next step towards bringing a solid Minimum Viable Product (MVP) to market: Hypothesis Development & Testing.

An MVP is a concise summary of the smallest possible features that will work as a stand-alone product while still solving the core problem and demonstrating the product’s value. As this is our final destination for this phase of customer discovery we have a few questions we need to answer in the context of the graduation problem before we get there:

  1. What is the long-term product vision for the solution?; and
  2. What are the initial product features & benefits? (Why should people use or buy it?)
Your long-term product vision for your MVP must clearly solve the graduation problem Click To Tweet

According to GradNation, the high school graduation rate, as measured by the Averaged Freshman Graduation Rate, increased from 71.7 % in 2001 to 81 % in 2012. While we are making great progress, minorities, low-income and the handicapped continue to lag behind. Our ability to reach our goal of 90%+ by 2020 will depend almost entirely on our ability to increase our performance in these three demographics. Under the new criteria set by the Every Student Succeeds Act (ESSA), low-graduation-rate high schools (dropout factories) are defined as schools that enrol 100 or more students and have graduation rates of 67 % or less.

The Every Student Succeeds Act addresses the problem of high school dropout factories Click To Tweet

ESSA mandates that these schools use evidence-based reforms that correlate with measurable improvements. There is an endless combination of factors that could account for why these students are dropping out. The only way to be sure is to scientifically test each and every possible theory and to record each and every outcome. This is an extremely time-consuming and resource intensive activity that is often skipped due to either lack of resources or lack of manpower. Herbie can bring his entire cognitive computing infrastructure to bear for Keith in order to alleviate the need he would normally have for these resources. Lower costs and higher profits will be a direct result.

Herbie utilizes some of the latest machine learning algorithms available (i.e. Linear Regression, Logistic Regression, k-Means, Support Vector Machines, Random Forests, Matrix Factorization/ SVD, Gradient Boosted Decision Trees / Machine Naïve Bayes, Artificial Neural Networks, Expectation Maximization etc.) to develop and test the veracity of hundreds and thousands of hypotheses. There are 3 types of predictions typically sought from using these algorithms and they include:

  1. Binary classification which is used to predict the answer to a Yes/No question;
  2. Multiclass classification predicts the correct category from a list; and
  3. Regression predicts the value of a numeric variable.

The standard scientific method of putting forward a testable hypothesis that the team will use is as follows:

“If _____[I do this] _____, then _____[this]_____ will happen.”

For example, “If we assign an adult tutor to an at-risk student the very first time they score below average in a core ‘subject’ test area, then we expect that student to experience a ‘letter grade’ increase over the course of a school term”. This is one of many educated guesses that will need to be made in order to perform the necessary experiments to turn these guesses into verified facts so that real-life decisions can be made in real-time.

Asking the “right” questions is critical if we want cognitive computing to give us the right answers Click To Tweet

The goal of standardizing a specific scientific hypothesis testing methodology is to ensure both data and results integrity throughout the analytical model construction process. Herbie will leverage these models to simulate and test countless variations on Keith’s hypotheses until acceptable patterns of causation begin to emerge. This will allow the team to quickly identify complex business problems that he and his team can profitably address.

Achieving a 90% graduation rate by 2020 will require focus on five key areas:

  1.  Closing the opportunity gap for low-income students;
  2.  Tackling big city ‘dropout factory’ challenges;
  3.  Making handicapped students part of the solution;
  4. Focusing on ‘urban’ California; and
  5. Accelerating the lagging graduation rates for young men of color.

My next article will discuss how Herbie will help Keith the best approach to gaining unique insight into these challenges and how that insight will affect their product vision.

Cognitive Computing can help close opportunity gaps for impoverished students Click To Tweet


Do you see how extensive hypothesis testing very early in the startup development process can drive long-term success? Can you see the potential time and cost savings that a cognitive computing system could offer a founder?

I look forward to hearing your feedback. Stayed tuned for more on this story.


Cognitive Computing Systems: Want to build a great MVP? Start with the data

A key first step in the customer discovery process is building a solid Minimum Viable Product (MVP).

“An MVP is that product which has just those features and no more that allows you to ship a product that early adopters see and, at least some of whom resonate with, pay you money for, and start to give you feedback on.”

In this article I look at how Keith, our fictional startup founder, begins his journey to deliver a disruptive solution to market with the help of an interactive cognitive computing system called “Herbie”. Step one begins with data acquisition and analysis.

As discussed previously, Keith is looking for a solution to increase the graduation rate in America from 80% in 2014 to 90%+ by 2020. He and his team will accomplish this by focusing their efforts on three primary demographics: minorities, impoverished students and the handicapped. They also want to discover if there is a scalable business model that will provide funding for long-term research and development.

Bringing disruptive high-tech solutions to market? Partner with Big Data and a Cognitive System Click To Tweet

Keith begins this particular journey by asking a simple question: What is the nature of the “graduation” problem and how can a cognitive computing system like Herbie help us solve it?

According to Gradnation: America’s Promise Alliance, students who have fallen off track to graduation have been found to be lacking four critical elements:

  1. Positive relationships with caring adults;
  2. Strong and tailored instruction;
  3. Opportunities to engage in learning experiences that connect school to careers and life beyond; and
  4. The support and resources to help them figure out what they want to do once they have earned their diploma.

school dropout cognitive system solution

Addressing these elements will be at the core of any solution that Keith and his team pursues. It also provides some guidance as to the potential benefits and or outcomes that we should expect from the business.

Positive adult relationships plus tailored education equals a higher graduation rate Click To Tweet

Achieving a 90% graduation rate by 2020 will require a thorough understanding of several key challenges:

  1. Closing the opportunity gap: Graduation rates for low-income students ranges from 58% to 85% compared to the national average of 80% for all students;
  2. Tackling the issue of big city concentration: Most “dropout factory” schools with < 60% graduation rates are found in urban areas;
  3. Making special education students part of the solution: Their average graduation rate is 20% below the national average;
  4. Focusing on California: Home to 20% of the nation’s low income students; and
  5. Accelerating graduation rates for young men of color: This is currently only in the upper 50’s and low 60’s.
Urban areas proliferate “dropout factories in America. How do we stop the cycle? Click To Tweet

Getting an objective understanding of each of these is one of the biggest challenges facing Keith and his team. Herbie’s focus is to access, organize and make sense of various forms of big data that will add clarity to the current state of the problem. He will use that as a foundation for ongoing business model strategy analysis.

Herbie begins by reviewing and organizing a shortlist of all data sources relevant to the four critical elements listed above and, under the direction of Keith’s data scientist, he will determine the optimal Enterprise Data Architecture (EDA) framework for this project. This will not only establish a baseline of what we currently know about this problem but also set the stage for the next step in the customer discovery process: Hypothesis Development and Testing.

An Enterprise Data Architecture provides fertile ground for rapid growth. Click To Tweet

In general, an EDA model is comprised of different layers that provide a strong foundation to develop 3 key strategic initiatives, such as:

  • Defining a data strategy that outlines the objectives of the business. This improves data collection and how the data is used in the business process;
  • Facilitating decisions on the potential future of new and modified solutions; and
  • Executing data warehousing, integration and reporting initiatives.

An enterprise data architecture exists on four different levels:

  • High-Level Data Model (HLDM): Constitutes a collection of HLDMs that describe business data through a conceptual viewpoint independent of any present realization by real systems. The HLDM consists of a standard UML class model of the primary data items and their relationships; a superset of business features, such as semantics, universal constraints and syntax;
  • Realization overviews: Describes the relationships between the real vital data objects of the present or planned systems and the conceptual units of the HLDM. This shows the way in which conceptual units are realized by actual units;
  • Source and consumer models: Demonstrates the correlation between various realizations of the same data items, diverse organizational custodians of data elements and the way in which modifications are circulated around different systems; and
  • Transportation and transformation models: Explains the way in which data in the implementation systems changes when moved between systems. They include attribute structure and physical class of system interfaces. This model also depicts the realization of the HLDM within the interface mechanisms, including a backbone or an enterprise application integration (EAI) hub.

Implementing an EDA at such an early stage has clear competitive advantages for Keith:

  • Helps the team accelerate the process of gaining strategic insights from the data;
  • Increases speed to market for new product initiatives;
  • Develops and implements a governance structure that supports an overall data strategy; and
  • Guides developments across systems, such as common reporting, EAI and data warehousing initiatives.

Once the EDA has been developed and the process of data acquisition has begun, Keith and Herbie are now ready to execute the all-important task of hypothesis development and testing. We will discus the nature and value of this process in my next article.

Why do you believe dropout rates are so high for minority, impoverished and handicapped students? As a founder, do you believe that a cognitive computing system will accelerate the problem solving process? Do you agree that designing and deploying an EDA is something that should be done as early as possible?

I look forward to reading your responses