The Pragmatic Marketer:

Volume 7 Issue 5

The journal of technology product management and marketing professionals

In This Issue:

- Updating the Pragmatic Marketing Framework™
By Jim Foxworthy and Steve Johnson
Periodically—and very carefully—we update the Pragmatic Marketing Framework™ to align with current practices for marketing technology solutions. Here is an overview of changes to reflect today’s best practices in product management.

- Job Interview? Use the Pragmatic Marketing Framework!

- The Product Management Triad
By Steve Johnson
Some product managers have a natural affinity for working with Development, others for Sales and Marketing Communications, and others prefer to work on business issues. Finding these three orientations in one person is an almost impossible task. Instead of finding one person with all the skills, perhaps we should find three different people with more specialized skills and have them work as a team.

- Top 10 Tools to Measure User Experience
By Giovanni Calabro
The one fundamental truth vital to web product development success is to know the mind of the user. This article provides a three-staged User Experience Measurement Hierarchy and also outlines a simple approach you can use to determine the best user experience measurement tools and techniques for your web product management needs.

- The Strategic Product Manager and the CFO
By Jon Gatrell and John Mecke
Give your CFO an opportunity to contribute, participate in the day-to-day workings of your product as a business, and you may just get a business partner who can help you and your product be more successful in the marketplace.

Updating the Pragmatic Marketing Framework
By Jim Foxworthy and Steve Johnson

Periodically—and very carefully—we update the Pragmatic Marketing Framework™ to align with current practices for marketing technology solutions. Having trained more than 60,000 product management and marketing professionals, some of these changes are the result of our training efforts and some the evolution of the product management discipline in 21st century technology businesses. Over time some things become more important, some less.

We’ve seen an escalation of the importance of buyer and user personas. Social media channels like Facebook, LinkedIn and Twitter are key elements in today’s marketing efforts. And e-books have become the preferred method of communicating thought leadership with a soft-sell orientation instead of traditional white papers and other “selling-oriented” materials.
So in June 2009, we revised the Pragmatic Marketing Framework—some items were moved, others renamed, a few added, or deleted—to reflect today’s best practices in product management.

This article steps you through the new version of the Framework.

Improved workflow

The horizontal orientation of the Framework has not changed. The activities on the left side are still strategic; the activities on the right still tactical. It’s been that way for more than 15 years.

We’ve streamlined the categories of activities (columns) to better flow from strategic to tactical. They now reflect the way many implement the Framework, from market to strategy to business and planning, to market programs and readiness, to on-going support. These sets of activities are now labeled Market, Strategy, Business, Planning, Programs, Readiness and Support.

Think about that logical structure for a minute. You spend time in the Market trying to uncover unsolved market problems, right? You try to determine whether or not the problems you’ve discovered are pervasive and urgent, and if people are willing to pay to solve them. And then you bring that inside the organization and start a comparison to your existing Strategy. Is this a fit with our strategic initiatives? With our portfolio of products? Is this a business we want to pursue at our company?

Once you get past that test, Business analysis begins. Is this a good investment for us? Do all the financial numbers associated with the investment work? Once you’ve cleared that hurdle, it’s time to begin the process of Planning the actual solution you’re going to create. From there it’s all about go-to-market, so we plan out our Programs, Readiness, and on-going Support.

Let’s cover the Framework changes, one column at a time.

Market column

In the Market column we’ve always had an activity called Market Problems, and as you know if you’ve been to our seminars, that particular activity is the absolute preeminent starting place. So we’ve moved it to the cornerstone of the entire Framework. We combined Market Research with Market Problems since after all, that’s the reason you do market research: to discover market problems. Next to Market Problems is Win/Loss Analysis; a form of discovery research, specifically focused on those who have recently evaluated your products. This activity is becoming more and more important, and Pragmatic Marketing’s 2008 Annual Product Management and Marketing Survey shows 37% of product managers are actively performing win/loss analysis. This has increased significantly from 17% in 2000 when we began surveying product managers. And remember that when you do win/loss analysis, you learn at least as much about the buying process as you learn about the product people were trying to buy.

Strategy column

Market Sizing was renamed Market Definition. Defining the market is much more than just counting the number of opportunities in a segment; it’s about understanding all the different details about that segment and why you should pursue it.

Distribution Strategy is a new activity in the Strategy column. It’s just as important to understand how you’re going to market and deliver to the people who are planning to buy it, as it is to think about the product itself and whether or not it’s a good investment for you. The buyer may want to be sold differently and receive solutions differently. Where once technology companies determined distribution based on their existing channels, it’s now more important to understand how the buyer wants to receive solutions. More and more, product management is determining pricing and packaging, on-premise versus hosted, direct sales versus multi-tier versus web-based sales. How technology solutions are distributed is a part of the strategy, not an afterthought or driven by history and habit.

We moved Product Portfolio and Product Roadmap to this new Strategy column. This is where you compare your new product idea to your existing portfolio and current roadmap to evaluate whether or not this idea, in isolation, fits with the overarching goals of what you’re doing as an organization.

Business column

Once you pass the hurdles of your product strategy, it’s time to run the numbers. In the Business column, Business Case is now called Business Plan. It’s a small change, but Business Case is just an artifact; planning is much more than that. Business planning includes at least two artifacts and a number of inputs. We have always covered these topics: now the activity is more aptly named.

Pricing, Innovation, and Buy, Build or Partner are still part of the Business column.

We’ve always encouraged you to look at Product Performance and Operational Metrics and the reason, of course, is so you can create a profitable product. We’ve combined these into a single activity, Product Profitability, in the Business column. Product Profitability is the ultimate measure of a good business. Development is accountable for functionality and schedules; Sales is accountable for revenue. Product management should obsess about creating profitable products. It’s vitally important that marketing managers understand their role in a “soft sense,” as the president of the product. If you’re the president of the product, the one thing you should be held accountable for is product profitability.

Planning column

Positioning and Use Scenarios are still part of the Planning column—that hasn’t changed.

There’s a new addition to the Framework—the Buying Process.

Many marketing managers worry about the sales process and the associated sales tools, yet our research reveals many sales processes are out of alignment with the buying process. Let’s change that now! Product managers and product marketing managers should understand the way buyers buy and create tools to instrument the selling process accordingly.

What matters is not how you want to sell but how they want to buy. And if you understand how they want to buy, the steps they go through, then you should mirror that with how you sell.

For most vendors, buyers don’t use and users don’t buy. Buyer and user personas have been cornerstones of our seminars and webinars since 2000 and continue to be important for technology companies. A deep understanding of both Buyer Personas and User Personas help companies build products buyers want to buy and users want to use. Buyer personas focus your marketing efforts because a product won’t succeed if you can’t get people to buy it. Not only do you need to understand how they buy, you need to understand who the buyers are and why they buy. Equally important are user personas. Selling your product doesn’t matter if people won’t use it—and recommend it to their colleagues. User personas are critical to the development process of your product because they reveal how users differ from buyers—and also how they differ from your developers, your sales people, and your own executives.

Lastly, in this column, Release Milestones was renamed Status Dashboard. Release Milestones was focused on the key development tasks to deliver products to distribution; Status Dashboard is something more. A status dashboard communicates overall organizational readiness. Not only is the code or device ready, but the organization is ready to deliver it to customers.

As the “single throat to choke” on product status, we encourage product management to maintain the dashboard for their products, whether on the intranet, a wiki or a periodic e-mail, showing the KPIs (Key Performance Indicators) and current project status.

Programs column

There are only three changes to this column.

We added an activity called Program Effectiveness. You’ve probably heard people say “you can’t manage what you can’t measure.” Should you continue going to the big trade shows and conferences? Or should you start doing more webinars, podcasts and social media programs? Should you do both? Program Effectiveness answers that. Once you begin measuring, it’s easy to see what programs make sense and which should be discontinued.

If you’ve attended our Effective Product Marketing™ seminar, you may recall the story about a young marketing professional who got promoted, ran into his senior executive in the hallway and got the following advice. “Don’t focus on your efforts; focus on your results.” Meaning that it really isn’t important how many leads came from a particular marketing program, what matters in the end is how many leads produced how many dollars of revenue.

Thought Leaders was renamed Thought Leadership. It seems like a small change in words, but it reveals that you’re no longer just working with thought leaders; many of you now are the thought leaders. You’re providing thought leadership directly to your markets with blogging and other social media like LinkedIn, Facebook and Twitter. With today’s new rules of marketing, marketers don’t have to rely on the media and industry analysts to be the thought leaders to tell their story; marketers can now tell their own stories directly to buyers.

The last activity in the Programs column is Referrals & References. It replaces Success Stories as a broader definition of how to leverage customers in your selling process, helping prospects evaluate your solution with people who already have experience with your organization.

Readiness column

We moved Sales Process from the Product Planning column to the Readiness column. The sales process should mirror the buying process. Collateral, such as white papers and e-books, are external documents that your sales teams provide to the buyer persona; Sales Tools, such as competitive write-ups and ROI calculators, are internal documents your channel uses to be more effective in selling.

Support column

The Support column remains the same with the exception of renaming Answer Desk to the broader Channel Support.

So there it is: the Pragmatic Marketing Framework, updated for 2010 and beyond, based on enhancements that came from the market (you). We’ve tested this revision with current and potential customers, and received great responses. We know you’ll find it aligns with the reality of product management in your organization.
If you’re new to us, come to one of our seminars to learn how you can use the Pragmatic Marketing Framework as a systematic approach to define and deliver products to market. To create products that people want to buy.

If it’s been a few years since you attended a Pragmatic Marketing seminar, come back for a refresher (New slides, new tools, new stories!).

Want an updated copy of the Framework?
Go to PragmaticMarketing.com to see the new Framework
with definitions for all the activities.
Download a copy to use in your
internal presentations.
Buy the poster to hang on your cubicle wall.

Jim Foxworthy is an industry veteran with more than 30 years in the computer industry. Prior to joining Pragmatic Marketing, Jim was president of a consulting firm focused exclusively on implementations of the Pragmatic Marketing Framework. Contact Jim at jfoxworthy@pragmaticmarketing.com

Steve Johnson is a recognized thought-leader on the strategic role of product management and marketing. Broadly published and a frequent keynote speaker, Steve has been a Pragmatic Marketing instructor for more than 10 years and has personally trained thousands of product managers and hundreds of company senior executive teams on strategies for creating products people want to buy. Steve is the author of the Product Marketing blog. Contact Steve at sjohnson@pragmaticmarketing.com

Job Interview? Use the Pragmatic Marketing Framework!

Here’s a great tip: when you’re part of a product management job interview, take along your Pragmatic Marketing Framework. Whether you are a hiring manager or job candidate, referring to the Framework during the interview process can be a highly effective way to structure the conversation.

Hiring Managers. One approach to an effective interview is to refer to the Pragmatic Marketing Framework and use it to explain the open position.

Perhaps you used the activities in the Framework to divide work amongst multiple job titles. Using the Framework in an interview, you can identify the activities this candidate will perform, along with any hand-offs to other team members.
Or perhaps the ideal candidate needs to handle all 37 activities on the Framework. In that case, use the Framework to have the candidate talk about the various activities and identify their strengths and weaknesses.

Candidates. It is always a great idea to have the Pragmatic Marketing Framework with you when you interview for a new position. Convey to a hiring manager that you use a structured process for your work, to show both competence and confidence. If they know the framework, you’ll build instant rapport; if they’ve never seen the framework, then you look even smarter.

The laminated version of the Framework is also an excellent tool to illustrate your view of a particular role in a company. For example, if you are interviewing for a position that will require you to be the “messenger of the market,” the Pragmatic Marketing Framework can be used to talk about how you would fulfill that responsibility. Perhaps you’d point at Market Problems or Win/Loss Analysis and illustrate with a prior set of job experiences how you used data from market interviews to drive product direction.

Summary: Don’t be afraid to use the Pragmatic Marketing Framework during the interview. If the person across the table is also familiar with the model, you immediately have context for the balance of the interview. Maybe they will pull out their copy, and it will work like a secret handshake!

The Product Management Triad

By Steve Johnson

Some product managers have a natural affinity for working with Development, others for Sales and Marketing Communications, and others prefer to work on business issues. Finding these three orientations in one person is an almost impossible task. Instead of finding one person with all the skills, perhaps we should find three different people with more specialized skills and have them work as a team.

How do you organize product management when there are multiple people involved with varying skill sets? How any product managers do you need? What are their roles in the company? Is product management a support role or a strategic one? How do you use the various product management titles such as product manager, product marketing manager, program manager, or product owner?

Titles are poorly understood and defined differently by many organizations. Every year, participants in Pragmatic Marketing’s Annual Product Management and Marketing Survey identify hundreds of different titles for those conducting product management activities. An ideal solution for many companies is the “product management triad.”

Some product managers have a natural affinity for working with Development, others for Sales and Marketing, and some prefer to work on business issues. Finding these three orientations in one person is very difficult. Instead, perhaps we should find three different people who each possess one or more of these skills and have them work as a team.

The product management triad includes a strategist, a technologist, and a marketer. Start with a business-oriented senior product manager responsible for product strategy. Make this person a director of products or product line manager (PLM). Now add a technology-oriented technical product manager (TPM) and a marketing oriented product marketing manager (PMM).

Let’s look at an example of how applying the triad had success for a company of nine product managers and nine products, one product manager per product. The salespeople disliked some of the product managers and loved others. The ones the salespeople loved were hated by developers. Applying the triad, they created three product lines with a PLM for each and then assigned a TPM and PMM to each product line. Now, for each product line, one person concentrates on product strategy and the business of the product line, while another works with Development to build the best product, and another takes the product message to the channel by working with Marketing Communications and the sales team.

Warning: Some companies attempt to put these three people in three different departments. They put the PLM into Sales to do business development; they put the TPM in Development and the PMM in Marketing Communications. This always fails. To work as a team, they must actually be a team. Having the TPM and PMM report to the same person, the PLM, minimizes conflict and overlap, giving the team a common objective. It has the added benefit of giving a new director the chance to learn to be a good manager of two people before getting five or ten people to manage.

Product management teams provide career paths from entry-level positions to director, all within the product line.

Execution vs. ownership

As shown in the graphic on the previous page, these three positions overlap. This is deliberate. Execution of these tasks must be collaborative in order to succeed. For example, Win/Loss Analysis is an excellent data source for Positioning and the Buying Process. Your PLM and PMM ought to perform win/loss visits together to ensure you gain the most value.

But do not confuse execution with ownership. Ownership of a task equates to accountability. As the executive leader of a team structured this way, the PLM is held accountable for win/loss analysis even when the TPM and PMM gather the win/loss data.

Does this model make sense for you?

Director, Product Strategy

The director of product strategy has a business-orientation and is responsible for the development and implementation of the strategic plan for a specific product family. They maintain close relationships with the market (customers, evaluators, and potentials) for awareness of market needs. This includes identification of appropriate markets and development of effective marketing strategies and tactics for reaching them. This person is involved through all stages of a product family’s lifecycle.

The director of product strategy must:

- Discover and validate market problems (both existing and future customers)

- Seek new market opportunities by leveraging the company’s distinctive competence

- Define and size market segments

- Conduct win/loss analysis

- Determine the optimum distribution strategy

- Provide oversight of strategy, technical, and marketing aspects of all products in the portfolio

- Analyze product profitability and sales success

- Create and maintain the business plan including pricing

- Determine buy/build/partner decisions

- Position the product for all markets and all buyer types

- Document the typical buying process

- Approve final marketing and go-to-market plans

Technical Product Manager

The technical product manager is responsible for defining market requirements and packaging the features into product releases. This position involves close interaction with development leads, product architects, and key customers. A strong technical background is required. Job duties include gathering requirements from existing and potential customers as well as recent evaluators, writing market requirements documents or Agile product backlogs, and monitoring the implementation of each product project.

The technical product manager must:

- Conduct technology assessments

- Analyze the competitive landscape

- Maintain the product portfolio roadmap

- Monitor and incorporate industry innovations

- Define user personas for individual products

- Write product requirements and use scenarios

- Maintain a status dashboard for all portfolio products

Product Marketing Manager

The product marketing manager provides product line support for program strategy, operational readiness and on-going sales support. This position requires close interaction with Marketing Communications and sales management. Strong communication skills are a must. Duties include converting technical positioning into key market messages and launching new products into market.

The product marketing manager must:

- Define buyer personas and determine market messages

- Create the marketing plan including methods for customer acquisition as well as customer retention

- Measure effectiveness of product marketing programs

- Maintain product launch plans

- Deliver thought-leading content via events, blogs, e-books, and other outlets

- Identify best opportunities for lead generation

- Create standard presentations and demo scripts

- Identify product references for industry and customer referrals

- Align sales tools and the ideal sales process to the typical buying process

- Facilitate channel training including competitive threats and related industry news

How you implement the product management triad depends on your organization and the skills of your team. Also consider having a role for your base technology or architecture for issues that span product lines. The “architecture” product manager can own acquisitions, third-party partnerships, and common tools needed across all product lines.

Take inventory of the skills of each of the product managers. Create an organization chart of one triad per product line with no names assigned. Now try to move the business-oriented staff (usually your senior product managers) to the PLM positions, development-oriented product managers to TPM and sales-oriented ones to PMM. The remaining holes in your organization chart represent your new hiring profiles.

Steve Johnson is a recognized thought-leader on the strategic role of product management and marketing. Broadly published and a frequent keynote speaker, Steve has been a Pragmatic Marketing instructor for more than 10 years and has personally trained thousands of product managers and hundreds of company senior executive teams on strategies for creating products people want to buy. Steve is the author of the Product Marketing blog. Contact Steve at sjohnson@pragmaticmarketing.com

Top 10 Tools to Measure User Experience

By Giovanni Calabro

The one fundamental truth vital to web product development success is to know the mind of the user. Why?

Because there is a direct correlation between techniques used to measure an online customer’s experience and the ability to persuade him or her to act. As more of our daily activities, both personal and professional, are managed online, most organizations are responsible for multiple websites and web-based tools that serve a broad range of audiences: customers, prospects, employees, channel partners, regulatory agencies, etc. That means online success depends on the ability to accurately measure a website or web application’s performance for unique target audiences on multiple levels across multiple dimensions.

However, relying on assumptions drawn from rudimentary measurements of user-centered design can’t provide sufficient data to tailor a website or web application to your users’ unique needs.

What’s more, when determining whether a website or web-based application is meeting the needs of its audience(s), it’s important to separate aesthetics from function. For example, your product may be very well-designed and visually appealing to your users but at the same time, can be functionally “unusable.”
We’ve all experienced unusable sites and applications. Whether it’s the online help site provided by your cable or satellite provider, or the online banking system you use to pay bills. Beautiful design can be painfully frustrating for users. Alternatively, poorly designed sites and applications may be quite functional, but fall far short in reinforcing the brand. Finding the appropriate balance between aesthetics and function is a difficult task. Understanding and measuring the user experience is the first step to achieving that balance. To remain competitive and retain customers, product managers need tools to optimize both aesthetics and function.

This article provides a three-staged User Experience Measurement Hierarchy and also outlines a simple approach you can use to determine the best user experience measurement tools and techniques for your web product management needs.

User Experience Measurement Hierarchy

The majority of organizations sit at the bottom of the pyramid, having only achieved Stage One: General Knowledge of their users’ online experience. Due to the perceived cost and complexity of measuring user engagement, the widespread availability of user-centered design best practices, and significant cost pressures, this basic level is understandable. However, leading organizations in industries such as insurance, manufacturing, healthcare, travel/hospitality and financial services, have made the migration to Stage Two: Understand User Behavior, and are seeing significant return on investment by way of increased revenue, reduced support costs and improved customer satisfaction. Even rarer are firms that have achieved Stage Three: Influence Your Users. While we’ve yet to see these tools widely adopted, they represent the future of user experience measurement.

Stage One

General Knowledge

A basic sense of site or web application performance

The first stage of user experience measurement involves very little measurement at all. At Stage One, organizations are typically working with “heuristic analyses” or best practices to create assumptions about users. Whether performed by an internal usability expert or a usability consultant, the result is an “informed guess” as to what users need.

Tools to find out general knowledge about the user include:

1. Heuristics
Heuristics are de facto web standards and research from the cognitive sciences that reveal a number of best practices in interaction styles, page layout, and visual design. Usability problems found through a heuristic evaluation are typically linked to aspects of the interface that are reasonably easy to demonstrate. For example, use of colors, layout and information structure, consistency of the terminology, or consistency of the interaction mechanism.

2. Expert Review
An expert review is a rapid survey of a website/application from specialists in user-centered design. Expert reviews can take you to the next level of user experience measurement by systematically rating the performance of your product on several relevant dimensions: navigation effectiveness, content selection, visual presentation, branding, and interaction simplicity. An expert review can often be done with a short turnaround and can be used to develop a measure of all your web assets as well as identify areas that need the most attention.

3. Web Hits/Usage Analysis
Web hits/usage analysis uses tools like Google Analytics, Omniture and WebTrends to study the hit pattern of your site or application. They can give you a sense of the number of pages visited and viewed, the typical navigation flow, and key drop-off locations. You can also track statistics such as bounce rates and conversion metrics. When combined with customer relationship management data, web hits/usage analysis can be highly effective in mapping and understanding user behavior.

It’s important to remember, web analytics/usage metrics rely on assumptions about user behavior and how those assumptions correspond to your web goals. So while these tools can help you determine the relative “success” of your website in meeting business goals, it’s difficult to know whether you are truly capturing the needs of your key target audience(s). The knowledge you gain through web analytics tools may be too general to understand individual user’s thinking and ultimately persuade them to act or change their online behavior.

The clear advantages of Stage One user experience measures are they are inexpensive, easy-to-use, can be quickly deployed and applied to many sites and applications.

Stage Two

Understand User Behavior

Identify what users are doing and where problems exist

The user-centered behavior metrics of Stage Two can provide an in-depth analysis of obstacles to accomplishing tasks on your site(s) and applications.
In order to truly understand what your target audience is doing online, you need to study a representative sample of individual users first hand. In-depth analysis typically is accomplished through user (usability) test sessions performed remotely or in person. These sessions will reveal much more about how users are anticipating the site or how the application works and what design issues might stop a user from accomplishing key tasks.

The best tools for understanding user behavior include:

4. User Testing
User testing is highly effective in designing for specific user groups. User testing can be done in person, in a lab or focus group or even remotely online, and involves interviewing individuals who are representative of your target audience. By asking them to accomplish specific, representative tasks to determine ease of use and recommend improvements, you can gather first-hand information about how users are behaving and what they are saying about your site or application. User testing is often most effective when it’s done early in the design process, before development and visual design.

The advantage of user testing is you learn what the target audience can and can’t accomplish, understand why users are having trouble, how to correct it, and receive direct feedback. The disadvantages are obvious: it can be more expensive to test individuals, especially when you’re dealing with multiple user types and a large number of test participants. In addition to a testing facility and moderator costs, there are also costs to recruit and compensate test participants. Participants typically receive a small stipend and refreshments. Remote testing can serve as a more cost-effective alternative; however, additional web conferencing costs apply.

5. Session Analysis
Session Analysis involves studying multiple single-user transactions, live or replayed based on certain criteria (e.g., error message present, drop-off at buy button). They can reveal a great deal of information about the user’s path and how to correct flow problems. Session analysis tools like TeaLeaf® are good tools for transactional sites and web applications.

These tools provide detailed naturalistic web session analysis, monitoring and replaying individual sessions and capturing the user experience either live or replayed for specific individuals. Error messages, broken links, or users clearly deviating from the expected path reveals critical system weaknesses. The ability to produce a recording of an individual user’s sessions—including the page sequence, form inputs, button selections and actual HTML page served to the customer’s browser—can help find and correlate an individual user’s behavior to application errors.

6. Online Surveys
Collecting larger population samples through online surveys is a quick and inexpensive way to capture likes and dislikes, most common feature requests, and summary information about a user persona. It is important to collect larger population samples to ensure that in-depth studies are representative of the target audience.
Companies like ForeSee Results provide a variety of intercept survey options integrating best practices from web analytics, market research and other tools, such as the University of Michigan’s American Customer Satisfaction Index (ACSI), to help companies convert satisfaction data into user-driven web development strategies.

7. A/B and Multivariate Testing
A/B and Multivariate Testing allows alternative designs to be tested and measured with a small segment of users. Nearly all high-transactional sites, such as, Yahoo, Google and Amazon.com, use this methodology which enables a comparison between the existing site and the prototype site for conversion optimization. Tools like SiteSpect and OnDialog are just a few of the options available.
The beauty of A/B and Multivariate Testing is they offer the ability to isolate and assess the performance of virtually every element of a web site, landing page, or application. From page layout, to headline text, to complete color palette variations, A/B and Multivariate Testing can be highly effective in identifying what works and what doesn’t from a user’s perspective. Plus, when managed through a marketing automation tool such as SiteSpect or OnDialog, it can also be easily scaled and deployed with no regard to internal technology infrastructure, a significant benefit for organizations with complex or legacy infrastructure issues.

Stage Two User Experience Measurement tools such as User Testing, Session Analysis, Online Surveys and A/B and Multivariate Testing are ideal for understanding what users are doing and where they are experiencing difficulty. Each has its own advantages and disadvantages. While all can be used, each solves a unique problem for marketers and product managers. Fortunately for product managers, each of these tools is also widely available at varying price points that can be customized to meet your specific requirements.

Stage Three

Influence Your Users

Determine whether a website or application is compelling

It has never been enough to “put something out there” that basically works. In order to influence users, you must directly measure what users think. Websites and applications must be compelling to the users both to be successful and for key influencers to spread the word about their positive experience. Increasingly, sites are ranked on their persuasiveness and influence on user thinking rather than just the ability to accomplish a goal.

The following tools are ideal for understanding user experience and satisfaction:

8. Eye Tracking
Eye tracking is a well-established vision science technology that can be readily applied to user testing. It can record where users are looking first, what is capturing their attention, and what they are choosing to act on. Eye tracking goes beyond measuring overt behavior and provides a window into the user’s thinking.
Eye tracking studies are particularly effective when applied in an online publishing environment; for example, news and entertainment sites. By testing a subset of a target audience, product managers can gain powerful insight into critical data unavailable through any other means:

- What first captures readers’ attention when visiting the page

- How often readers see the diversity of news and content the publication offers

- Where readers are looking versus where they actually click

9. Emotion/Trust Measurement
Several early prototypes are using various measurements (e.g., skin conductance, heart rate, facial emotion detection, mood survey) to determine how users feel while using sites and tools. There is an increasing consensus that both explicit (survey) and implicit (emotion detection) tools are needed to really understand user motivations and experience.

10. Neuro-marketing
While only a few tools have made their way from neuroscience laboratories, there are some applications of Electroencephalograms (EEG) technology and functional Magnetic Resonance Imaging (fMRI) which are being used in the commercial arena to measure engagement and emotional response to user experience. These tools provide a window into user satisfaction.

The critical feature of these technologies is to measure the experience of your target audience and understand what they find compelling and appealing. Using these techniques, you can also identify which messages are most effective and what dictates trust and positive emotion from your website or web application.
The disadvantages of these “higher level” measurement techniques are many. They require more equipment and analysis. Given their lack of widespread adoption, it can be difficult to achieve buy-in within your organization to pursue such an initiative. Finally, due to the higher expense, product managers are cautioned to develop a realistic return on investment model before diving in.

Measuring and managing experience

Although time and cost limitations exist in measuring user experience, there is a clear progression in user experience measurement sophistication: from general knowledge to influencing user behavior. No matter what level of sophistication you choose, knowing as much information about your users as possible helps craft a compelling experience for your highest use sites and applications.

Giovanni Calabro has more than 10 years of experience leading interactive research and design efforts for a wide range of business sectors. Giovanni currently serves as Vice President of User Experience at Siteworx, a Reston, Virginia-based interactive agency and Web strategy consultancy.
Giovanni is an avid writer and presenter on all topics related to the business aspects of user experience and user interface design. Contact Giovanni at giovanni@siteworx.com, or visit the Siteworx blog www.siteworx.com/blog for deeper insights into user experience design and measurement techniques.

The Strategic Product Manager and the CFO

By Jon Gatrell and John Mecke

Where are you from in the company again?

Too many technology organizations promote from within—product managers, solutions consultants or outstanding support persons. This behavior has pluses and minuses. The pluses are evident—those employees know the product, the market, and they understand how the product is actually used versus how the product is marketed.

While technology DNA abounds in most product management organizations, the soft spots often emerge when it comes to managing a business. Ultimately, product management isn’t about delivering features, it’s about bringing profitable products to market.

Marketing metrics

How often is a product manager asked about average unit price of a given product, profitability of the product group or cost of support? How many product managers are forced to piece together anecdotal data from multiple systems in order to {fill in the blank}?

Practicing product management with partial data sets is the norm. In many organizations, Product Management’s view of the data is different than Support, which is different than Sales, and definitely different than Finance. So where do you start?

The best place to start is Finance because they typically use the same numbers the board of directors use to validate your product’s performance. While the finance team may not have all the data a product manager needs, their numbers are what your organization ties back to, not that of Sales and Support.

Got to know your numbers

Microsoft Excel spreadsheets are everywhere; what’s a product manager to do? It’s not just the numbers that you work from, but also the way you work the numbers to deliver value for your CFO. There are effectively two kinds of CFO’s—strategic and non-strategic. One’s a leader; one’s a bean counter. It’s a shame. Finance is supposed to be about the strategic use of money, but too often, only focus on counting the top-line dollars.

Do you have a strategic CFO or a tactical bean counter? This very much depends not only on how the CFO behaves, but also the leadership team and the rest of the business. If you expect a bean counter that creates pretty Excel tables, which are essentially scorekeeping for the CEO and board of directors, then that is what you get. Give the CFO an opportunity to contribute, participate in the day-to-day workings of your product as a business, and you may just get a business partner who can help you and your product be more successful in the marketplace.

Litmus test

Conducting a litmus test to determine whether your CEO is a bean counter or a strategic leader is fairly simple. The following questions and answers directionally identify which type you might be working for:

Q How many customers abandoned the product last year versus new buyers?
Bean Counter We lost 2% revenue.
Strategic CFO You lost 4% of your customers, which is 2% of the revenue. Can you help find out how we might accelerate this trend with other customer segments?

Q What was the average deal size last quarter? Is it lower than the previous quarter?
Bean Counter We did better last quarter than the previous quarter in number of deals and bookings.
Strategic CFO The average deal size has gone down, but we closed more deals and bookings. Can you find out why and what is the change in the deals?

Q What was the gross margin of new customers last year?
Bean Counter Your P&L was running at 68% gross margin.
Strategic CFO Your new customers are at ~62% gross margin or dilutive to the rest of the business. Check with Support to see if there are new things driving up costs and work with Sales to see if we are still selling that product? It has a steep learning curve for customers and drags down the business.

Yes, the strategic CFO wants to know more and if you bring back the right data, you gain a partner at the company. The challenge is if you have a strategic partner, you need some basic skills, not just so you don’t get buried in the boardroom, but so you can speak less marketing and more finance and even more business. In this article, we use every-day concepts and regular speak to demystify the wonders of finance, at least enough to help a product manager gain the basic modeling skills so you aren’t discredited when you go back with the answers.

Product management is all about getting your organization to make the best business decisions—to achieve monthly, quarterly, and annual objectives. Good decisions are based on information—not data. A spreadsheet containing summary product line bookings by month is data. Understanding the key drivers behind monthly sequential changes or year-over-year changes is information. Technology companies tend to get lost in their data. Differing political factions in the company often try to use point pieces of data to justify their positions or tear down their opponents’ positions. Strategic CFOs have seen this game played before and recognize that at the end of the day, it rarely moves the business forward.

The strategic CFO plays the role of translating data into actionable information. Help your CFO and the entire organization become more strategic by adopting strategies and techniques that automatically translate data into information that can be applied on a daily basis.

Strategic product managers should add the following four proficiencies to their repertoires.

- Financial literacy

- Flux analysis

- Tier analysis

- Money wheel analysis

Increasing your skills in these four areas enables you to help the CFO and your company become more strategic and effective in the marketplace.

Financial literacy

Typically, at best, most product managers have a superficial understanding of basic financial concepts. While product managers may know a few of the financial terms, they generally struggle to apply the terms in business situations to explain or justify the financial value of a specific product, service, or proposal. How many times in the past quarter have you heard your sales team say a prospect “just didn’t see the value” in acting now. How many times have you heard your marketing team say “we’re surprised that the campaign generated such little interest—you’d think in these tough economic times any company would want to save money by using our solutions.”? CFOs see the world through numbers, financial statements, and cash flow. If you cannot speak their language, how will you ever become a strategic asset to them? How can you honestly deliver a compelling ROI statement for your product if you can’t find your way around an income statement or a balance sheet? In fact, financial illiteracy costs technology companies far more than they care to admit.

Here’s a simple case in point. A technology company had a long-standing relationship with a customer in which they provided a value-added outsourced solution that cost over $1.5M a year. After a change in management, the customer decided to re-examine its investment in the outsourced solution. A new executive wanted to bring the solution in-house. To do so, the company would have to invest over $750,000 in equipment and software, plus set up a group of employees to do the processing/value-added services that the service provider had been doing. The company believed that bringing the business in-house would save $500K a year. The salesperson on this deal was afraid the customer would take the solution in-house. As a result, he offered to cut the monthly fees by 50% so the customer would not even consider taking the solution in-house. With the price cut, the new executive would be able to claim $750,000 in savings. He would be a hero and the company would still have a customer.

Fortunately, a product manager supporting the deal did a little research about the customer. Even though it was a large company with greater than $200M in revenue, it had serious financial challenges. At the time they considered in-sourcing the solution, the company only had $5M in cash on their balance sheet and a limited revolving credit facility. Once the sales team realized the company would have to invest about 20% of their existing cash balance to make the project work, they realized the customer had not been serious about in-sourcing. Instead, they were really looking to get a massive price reduction. Armed with this knowledge, both parties were able to structure a win-win deal that did not result in a 50% revenue reduction. This is a classic example of how financial illiteracy can directly impact your top and bottom lines.

Product managers should be able to:

- Locate financial statements and public SEC filings on the web

- Read and interpret, at a high level, income statements, balance sheets, and cash flow statements

- Extract key tidbits from 10-Qs, 10-Ks, and proxy statements of public companies

- Interpret key statistics about company performance and valuation

Apply this information on a daily basis to help better position the value of your products in context of your customer’s business. If your financial literacy is a little low, admit it, and get some help. Talk with your CFO and see if you can get a few key members of his/her staff to create a basic financial literacy program. Having them teach the fundamental language of finance ensures you are all using the same vocabulary and are on the same page.

Flux Analysis

turn basic revenue data into actionable information

Flux analysis

Flux analysis is an example of how you take basic revenue data and turn it into actionable information. Almost all software technology companies have at least one or more software as a service (SaaS) offerings. SaaS requires different analytical techniques than traditional licensed software products. Flux analysis helps organizations understand the month-to-month revenue changes in a SaaS product line from a customer behavioral perspective. If you understand the dynamics of your customer base’s behavior, you can identify and implement tactics to reinforce positive trends and mitigate the impact of negative trends. Take a look at the model above.

There are dozens of actionable takeaways from this analysis. The key thing is that the analysis puts the complex dynamics of monthly SaaS customer behavior into a context that the entire team understands. Strategic CFOs intuitively discern these changes based on their experience. By adopting this type of value-added analysis, you can elevate the analytical skills of your team and the company. Now when you talk about growing your business, you can focus on real drivers and not spend hours arguing about data.

A word of extreme caution: Anytime you present financial information—revenues, COGS, operating expenses, EBITDA—you must ensure your numbers exactly match the numbers in the finance reports. Finance is the official scorekeeper and you must reconcile your numbers with theirs. Otherwise, you’ll never be able to defend the accuracy of your analysis and the validity of your recommendations. Everyone else in the business will lob hand grenades in your direction, simply because you failed to spend 15 minutes ensuring your total revenue number matched what finance reported.

Tier Analysis

categorize customers into groups based on their annual revenue contribution to your business

Tier analysis

Tier analysis is a tool that should be in every product manager’s toolkit. Tier analysis categorizes customers into groups based on their annual revenue contribution to your business. The reality is not all customers are created equal—some contribute significantly more to the business while others contribute significantly less. You need to protect, defend, and grow the critical customers while providing the smaller customers with an appropriate level of service. Take a look at the tier analysis below.

This type of analysis helps a product manager understand and justify providing differentiated service to specific customer segments. You can extend this type of analysis by combining it with the flux analysis mentioned above.

Money Wheel Analysis

summary of a product’s quarterly sales to discover the most effective, repeatable sales transactions in the market

Money wheel analysis

Strategic CFOs understand that the most successful technology sales forces leverage repeatable sales transactions to scale revenue. The money wheel is an analytical tool that technology companies can use to discover the most effective, repeatable sales transactions in their markets. Take a look at the table below that summarizes a quarter’s sales transactions for a particular product.

A money wheel has two major components: Spokes & Spoke-lets. Money wheel spokes identify the categories of repeatable sales transactions that occur within a product/service line:

- Net New Customers organizations that have never purchased from your company

- Add-On Sales existing customer buying an add-on product/service to support use of an existing product/service

- Expansion Sales existing customer buying more seats/copies/users

- Migration Sales existing customers paying to migrate from one product version to another

- Financial Transactions financially driven sales transactions. Examples include 3-year prepaid maintenance deals with significant discounts or an add-on product bundled so that some license revenue can be recognized. Includes conversion from term to perpetual software licenses, disaster recovery licenses, etc.

Spoke-lets are the types of repeatable sales transactions within a specific money wheel spoke. A spoke-let answers the question: “Why did someone wake up this morning and decide they needed to buy a new software solution?” There are lots of ways to refine a money wheel analysis. If you sort the spokes/spoke-lets by salesrep and region, you can obtain a much better understanding of why certain reps and regions are consistently more successful than others. You can also sorts the spokes and spoke-lets vertically to see where your ‘horizontal’ solution is getting the most traction. By understanding the distribution and drivers of sales transactions, you can identify patterns that your sales team can leverage.

Real numbers, real knowledge

We recognize the above analyses may not be considered the baseline requirements for financial literacy, but they should be close. If you have this type of information at hand, not only will operational reviews go better, you will improve your product’s position in the marketplace and your strategic partnership with the CFO.

Jon Gatrell is vice president of product management at Stonebranch, a provider of secure communications and data exchange solutions. Jon is a published author and frequent speaker. Jon blogs on best practices in product management and marketing at spatiallyrelevant.org. Contact Jon at jon.gatrell@stonebranch.com

John Mecke is Managing Director of DevelopmentCorporate, an Atlanta-based corporate development advisory firm. He specializes in mergers, acquisitions, & divestitures as well as working with startups/late stage technology companies to transform their businesses. John blogs at developmentcorporate.com on private equity, product management, and the monetization of social media. Contact John at john.mecke@developmentcorporate.com

Volume 7 Issue 5 • 2009

The Pragmatic Marketer™
8910 E. Raintree Drive • Scottsdale, AZ 85260
Offers techniques for developing, consolidating and communicating product plans and vision to multiple audiences—both inside and outside the company. Designed for members of the product management, marketing and development teams tasked with planning product direction and communicating product strategy.

In addition to the extensive published schedule, training can be conducted onsite at your office, saving travel time and costs for attendees, and allowing a much more focused discussion on internal, critical issues.

Pragmatic Marketing’s seminars have been attended by more than 45,000 product management and marketing professionals.

Visit

www.PragmaticMarketing.com

or call (800) 816-7861

Founder and CEO
Craig Stull

Editor-in-Chief
Kristyn Benmoussa

Editor
Linda Sowers

—————————————————
Interested in contributing an article? Visit www.PragmaticMarketing.com/submit

No part of this publication may be reproduced, stored in any retrieval system, or transmitted, in any form or by any means, electronic, mechanical photocopying, recording or otherwise, without the prior written permission of the publisher. For advertising rates, call (480) 515-1411.
Other product and/or company names mentioned in this journal may be trademarks or registered trademarks of their respective companies and are the sole property of their respective owners. The Pragmatic Marketer, a Pragmatic Marketing publication, shall not be liable regardless of the cause, for any errors, inaccuracies, omissions, or other defects in, or untimeliness or unauthenticity of, the information contained within this magazine. Pragmatic Marketing makes no representations, warranties, or guarantees as to the results obtained from the use of this information and shall not be liable for any third-party claims or losses of any kind, including lost profits, and punitive damages. The Pragmatic Marketer is a trademark of Pragmatic Marketing, Inc.
Printed in the U.S.A.
All rights reserved.
ISSN 1938-9752 (Print)
ISSN 1938-9760 (Online)
About Pragmatic Marketing®
Pragmatic Marketing provides training seminars, onsite workshops, consulting services and an online resource center for technology product managers, product marketers and the executives who lead them.

Over 60,000 product management and marketing professionals at 4,500 companies in the United States, Australia, Canada, England, Finland, France, Germany, Hong Kong, India, Japan, Singapore and South Africa have been trained with the Pragmatic Marketing Framework, a practical, market-driven approach to creating and delivering technology products. More than 10,000 have become Pragmatic Marketing Certified, the world’s leading product